PoliBa-DeepLearning/NanoSocrates
2
0
Fork 0
Code Issues 3 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

updated special token

Browse Source
This commit is contained in:
GassiGiuseppe 2025-10-08 11:02:18 +02:00
parent 159266a603
commit e353c200d7
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
Scripts/Libs/CleaningPipeline/special_token.py
View File

@ -10,6 +10,7 @@ class SpecialToken(Enum):
RELATIONSHIP = "<PRED>"
OBJECT = "<OBJ>"
ABSTRACT = "<ABS>"
END_OF_SENTENCE = "<EOS>"
CORPUS_END = "<END>"
## Tasks' Token