PoliBa-DeepLearning/NanoSocrates
2
0
Fork 0
Code Issues 3 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Added attention_mask

Browse Source
This commit is contained in:
GassiGiuseppe
2025-10-05 17:49:01 +02:00
parent b303affd18
commit 6f219f634f
5 changed files with 20 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
Project_Model/Libs/Transformer/Utils/attention_mask.py Normal file
View File

@@ -0,0 +1,4 @@
import torch
def get_attention_mask(embedding_dimension: int) -> torch.Tensor:
return torch.triu(torch.ones(embedding_dimension, embedding_dimension, dtype=torch.bool), diagonal=1)