Christian Risi b09bd4acba Created trainer to train BPE

2025-09-30 13:33:40 +02:00

..

__init__.py

Updated Inits

2025-09-30 13:32:37 +02:00

Encoder.py

Added Special Encoder

2025-09-28 18:03:47 +02:00

NanoSocratesBPE.py

Added fit method

2025-09-30 13:33:28 +02:00

NanoSocratesChunker.py

Added Chunker

2025-09-28 18:02:06 +02:00

NanoSocratesSpecial.py

Added Special Encoder

2025-09-28 18:03:47 +02:00

NanoSocratesSplitter.py

Added Splitter to divide tokens from text

2025-09-28 18:03:16 +02:00

NanoSocraTrainer.py

Created trainer to train BPE

2025-09-30 13:33:40 +02:00