PoliBa-DeepLearning/NanoSocrates
2
0
Fork 0
Code Issues 3 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Fixed JSON incompatibility

Browse Source
This commit is contained in:
Christian Risi 2025-10-01 00:32:43 +02:00
parent 9a8e726d74
commit 97bac464f3
1 changed files with 8 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
Scripts/Training/bpe_trainer.py
View File

@ -86,7 +86,14 @@ def train(args: ProgramArgs):
) )
VOCABULARY = BPE_ENCODER.vocabulary VOCABULARY = BPE_ENCODER.vocabulary
VOCABULARY_JSON = json.dumps(VOCABULARY)
JSON_VOCABULARY: dict[str, int]= {}
for key, item in VOCABULARY.items():
TUPLE_STR = f"{key}"
JSON_VOCABULARY[TUPLE_STR] = item
VOCABULARY_JSON = json.dumps(JSON_VOCABULARY)
print(f"Saving Vocabulary in {VOCABULARY_PATH}") print(f"Saving Vocabulary in {VOCABULARY_PATH}")