Christian Risi
|
3b274ad807
|
Added a way to take the default special token list
|
2025-10-04 19:43:02 +02:00 |
|
Christian Risi
|
03cdca1f00
|
Modified imports for BPE
|
2025-10-04 19:42:02 +02:00 |
|
Christian Risi
|
9c5f42153f
|
fixed typos
|
2025-10-03 15:17:44 +02:00 |
|
Christian Risi
|
51f491d033
|
fixed typos
|
2025-10-03 13:27:17 +02:00 |
|
Christian Risi
|
6b9cb7cd35
|
Modified imports
|
2025-10-03 13:26:42 +02:00 |
|
GassiGiuseppe
|
845d645348
|
added some stubs on special_regex_maker
|
2025-10-03 10:38:35 +02:00 |
|
Christian Risi
|
fbbe6226bb
|
Finished uploading stubs for TokeNano
|
2025-10-01 18:56:53 +02:00 |
|
Christian Risi
|
dbf1d99408
|
Added json utils to save and load json files
|
2025-10-01 12:20:59 +02:00 |
|
Christian Risi
|
7020c9e683
|
Added utils to make regexps and iterators that check for last element
|
2025-09-30 13:33:12 +02:00 |
|
Christian Risi
|
2fe1ce9e9a
|
Updated Inits
|
2025-09-30 13:32:37 +02:00 |
|