Christian Risi
|
a1d143187d
|
corrected test to reflect changes in BPE trainer
|
2025-10-02 20:11:43 +02:00 |
|
Christian Risi
|
2194cc7b4f
|
Changed test to use pool trainer
|
2025-10-02 09:56:05 +02:00 |
|
Christian Risi
|
eadba1fb82
|
Corrected test to reflect changes in NanoSocratesBPE
|
2025-10-02 09:33:47 +02:00 |
|
Christian Risi
|
76f24d4eb0
|
Renamed file
|
2025-09-30 23:58:43 +02:00 |
|
Christian Risi
|
ccacea18d8
|
Created files to test BPE training
|
2025-09-30 13:33:54 +02:00 |
|
Christian Risi
|
e433941405
|
Added BPE
TODO:
- complete the fit method
|
2025-09-28 18:04:44 +02:00 |
|
Christian Risi
|
d179e01971
|
Added Splitter to divide tokens from text
|
2025-09-28 18:03:16 +02:00 |
|
Christian Risi
|
3e8b5c5579
|
Added test for chunker
|
2025-09-26 18:50:32 +02:00 |
|
Christian Risi
|
3f48b5c428
|
Added text files to test a chunker
|
2025-09-26 18:48:44 +02:00 |
|