Christian Risi 2e595a3a23 Changed training phase to take directly data instead of its encode

2025-10-02 09:56:44 +02:00

..

__init__.py

added multithreaded training to package

2025-10-02 01:31:05 +02:00

Encoder.py

Added Special Encoder

2025-09-28 18:03:47 +02:00

NanoSocratesBPE.py

Fixed decoding phase

2025-10-02 09:33:58 +02:00

NanoSocratesChunker.py

Added Chunker

2025-09-28 18:02:06 +02:00

NanoSocratesSpecial.py

Added Special Encoder

2025-09-28 18:03:47 +02:00

NanoSocratesSplitter.py

Added Splitter to divide tokens from text

2025-09-28 18:03:16 +02:00

NanoSocraTrainer.py

Added a way to recover iteration work

2025-10-01 12:21:42 +02:00

NanoSocraTraineRam.py

Refactored to remove tokens that can't be compressed anymore

2025-10-01 19:42:22 +02:00

NanoSocraTrainerPool.py

Changed training phase to take directly data instead of its encode

2025-10-02 09:56:44 +02:00

TokeNano.py

Finished uploading stubs for TokeNano

2025-10-01 18:56:53 +02:00

TokeNanoCore.py

Finished uploading stubs for TokeNano

2025-10-01 18:56:53 +02:00