PoliBa-DeepLearning/NanoSocrates
2
0
Fork 0
Code Issues 3 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 1eae8582b2 Fixed decoding phase Christian Risi 2025-10-02 09:33:58 +02:00
  • eadba1fb82 Corrected test to reflect changes in NanoSocratesBPE Christian Risi 2025-10-02 09:33:47 +02:00
  • aa765b4555 Added time checking Christian Risi 2025-10-02 08:48:45 +02:00
  • 17d82f0a4e Added support to resume workload Christian Risi 2025-10-02 08:48:28 +02:00
  • 0975c19e69 added nwew method to encode from list of tokens Christian Risi 2025-10-02 08:48:13 +02:00
  • 3fe4e45ceb Fixed a bug while joining frequencies Christian Risi 2025-10-02 01:50:37 +02:00
  • d19426fa62 added multithreaded training to package Christian Risi 2025-10-02 01:31:05 +02:00
  • 63baf29805 Added multithreaded training Christian Risi 2025-10-02 01:30:24 +02:00
  • b80b4e4112 Fixed returning type hints Christian Risi 2025-10-02 01:29:57 +02:00
  • 7cfaf601b4 Refactored to remove tokens that can't be compressed anymore Christian Risi 2025-10-01 19:42:22 +02:00
  • fbbe6226bb Finished uploading stubs for TokeNano Christian Risi 2025-10-01 18:56:53 +02:00
  • b3d444979f Added flag to resume work correctly Christian Risi 2025-10-01 12:22:09 +02:00
  • 66bcf6e55f Added a way to recover iteration work Christian Risi 2025-10-01 12:21:42 +02:00
  • dbf1d99408 Added json utils to save and load json files Christian Risi 2025-10-01 12:20:59 +02:00
  • 97bac464f3 Fixed JSON incompatibility Christian Risi 2025-10-01 00:32:43 +02:00
  • 9a8e726d74 Added cdebug configuration Christian Risi 2025-10-01 00:22:22 +02:00
  • 7ab9b0358e Added script to run BPE Christian Risi 2025-09-30 23:59:09 +02:00
  • 30c2938d29 Fixed typing Christian Risi 2025-09-30 23:58:54 +02:00
  • 76f24d4eb0 Renamed file Christian Risi 2025-09-30 23:58:43 +02:00
  • 89a0a1f4bb Fixed bug for utf-8 conversion Christian Risi 2025-09-30 23:58:31 +02:00
  • 64e355e80c Added regex to delete new lines and * from ObjectURI GassiGiuseppe 2025-09-30 15:00:07 +02:00
  • 397e29742a minor update of settings GassiGiuseppe 2025-09-30 13:58:20 +02:00
  • ccacea18d8 Created files to test BPE training Christian Risi 2025-09-30 13:33:54 +02:00
  • b09bd4acba Created trainer to train BPE Christian Risi 2025-09-30 13:33:40 +02:00
  • c9032cab09 Added fit method Christian Risi 2025-09-30 13:33:28 +02:00
  • 7020c9e683 Added utils to make regexps and iterators that check for last element Christian Risi 2025-09-30 13:33:12 +02:00
  • 2fe1ce9e9a Updated Inits Christian Risi 2025-09-30 13:32:37 +02:00
  • 18fc2ba9d8 Added Exceptions Christian Risi 2025-09-30 13:32:24 +02:00
  • 5acee1d1a5 Merge branch 'dev' into dev.bpe Christian Risi 2025-09-30 11:35:27 +02:00
  • 2e36753da4 Merge pull request 'dev.etl' (#5) from dev.etl into dev Giuseppe Gassi 2025-09-30 11:28:57 +02:00
  • 007f1e9554 minor updates GassiGiuseppe 2025-09-29 18:53:33 +02:00
  • c319398ca0 little update to UML pipeline GassiGiuseppe 2025-09-29 17:03:31 +02:00
  • 255d8a072d First implementation of the cleaning pipeline UML GassiGiuseppe 2025-09-29 16:59:52 +02:00
  • 8167c9d435 Added Toy Dataset entry point into the Pipeline class Before it was forced into the sql_endpoint, now all the pipeline can be managed in the Pipeline class GassiGiuseppe 2025-09-29 16:03:49 +02:00
  • bd72ad3571 Added file to execute the complete cleaning pipeline GassiGiuseppe 2025-09-29 15:21:26 +02:00
  • 6ddb7de9da Added sqlAlchemy to requirements GassiGiuseppe 2025-09-29 15:19:19 +02:00
  • 564b0d712e Modified UML diagram Christian Risi 2025-09-28 18:05:03 +02:00
  • e433941405 Added BPE Christian Risi 2025-09-28 18:04:44 +02:00
  • b46df4f91a Added Special Encoder Christian Risi 2025-09-28 18:03:47 +02:00
  • d179e01971 Added Splitter to divide tokens from text Christian Risi 2025-09-28 18:03:16 +02:00
  • b071145f6e Added Chunker Christian Risi 2025-09-28 18:02:06 +02:00
  • ed0255e99b Updated imports Christian Risi 2025-09-28 18:01:35 +02:00
  • 3e8b5c5579 Added test for chunker Christian Risi 2025-09-26 18:50:32 +02:00
  • 8db35732f9 Added Chunker to restrict our domains Christian Risi 2025-09-26 18:50:23 +02:00
  • 9552d61f8d Added Excetption for when we don't find a delimiter Christian Risi 2025-09-26 18:49:56 +02:00
  • be8a87ce01 Modified the architecture for BPE Christian Risi 2025-09-26 18:49:29 +02:00
  • 5801a819e9 Added vars to make it easier to work here Christian Risi 2025-09-26 18:49:06 +02:00
  • 3f48b5c428 Added text files to test a chunker Christian Risi 2025-09-26 18:48:44 +02:00
  • 9972ab8a51 Added imports Christian Risi 2025-09-26 18:48:23 +02:00
  • 650b37c586 Added vscode setting to execute jupyternotebook from root dir GassiGiuseppe 2025-09-26 11:24:34 +02:00
  • 90012285b5 UML Diagram to explain bpe workflows Christian Risi 2025-09-25 20:18:21 +02:00
  • 1bbb4a0999 Added new paper Christian Risi 2025-09-25 20:17:48 +02:00
  • e521b0704e deleted TODO in path_splitter_tree, as it was already resolved GassiGiuseppe 2025-09-25 19:19:11 +02:00
  • ee0aa583d5 Added Docs for BPE research Christian Risi 2025-09-25 19:10:45 +02:00
  • 0a698e9837 Added schema to extract from DB for BPE Christian Risi 2025-09-25 19:09:52 +02:00
  • 9440a562f2 Merge branch 'dev.etl' of https://repositories.communitynotfound.work/PoliBa-DeepLearning/NanoSocrates into dev.etl dev.report GassiGiuseppe 2025-09-25 18:33:51 +02:00
  • 5eda131aac Fixed creation query to be unique even with movieID in RDFs Christian Risi 2025-09-25 17:58:09 +02:00
  • 57884eaf2e CSV support added to path_splitter_tree Also resolved a minor bug to print also leaf nodes GassiGiuseppe 2025-09-25 17:57:46 +02:00
  • 4548a683c2 Fixed DB Christian Risi 2025-09-25 17:57:45 +02:00
  • 3eec49ffa5 WIP: added test file: clean_relationship.jupyter to create a first cleaning pipeline GassiGiuseppe 2025-09-25 16:28:24 +02:00
  • 0bc7f4b227 Fixed Typos Christian Risi 2025-09-25 12:37:52 +02:00
  • f28952b0a2 Added todo Christian Risi 2025-09-25 12:00:26 +02:00
  • 0b626a8e09 Modified query to take all data Christian Risi 2025-09-25 11:53:12 +02:00
  • b254098532 Added views to count for subjects and objects Christian Risi 2025-09-25 11:40:44 +02:00
  • ee88ffe4cf Added View to filter over relationship counts Christian Risi 2025-09-25 11:32:03 +02:00
  • 70b4bd8645 Added Complex query Christian Risi 2025-09-25 11:31:34 +02:00
  • 6316d2bfc4 Added queries to take data from SQL for dataset Christian Risi 2025-09-25 11:27:19 +02:00
  • 87ca748f45 Updated DB to reflect new changes Christian Risi 2025-09-24 19:29:57 +02:00
  • 4315d70109 Merged abbreviation_datawarehouse into datawarehouse Christian Risi 2025-09-24 19:29:43 +02:00
  • 9a5d633b5e Fixed Typos Christian Risi 2025-09-24 19:29:07 +02:00
  • a6760cd52d Updated SQL Queries to support parsing in DB Christian Risi 2025-09-24 19:28:55 +02:00
  • a7eb92227d Moved all db queries file in their own folder GassiGiuseppe 2025-09-24 16:44:55 +02:00
  • 9f221e31cd Merge branch 'dev.etl' of https://repositories.communitynotfound.work/PoliBa-DeepLearning/NanoSocrates into dev.etl GassiGiuseppe 2025-09-24 16:32:52 +02:00
  • 47197194d5 WIP abbrevietion_datawarehouse to creat an abbreviation system GassiGiuseppe 2025-09-24 16:32:09 +02:00
  • 0cdbf6f624 Added query to retrieve a dirty dataset from SQLite DB Christian Risi 2025-09-24 16:15:47 +02:00
  • 3e30489f86 Updated Queries for DB Christian Risi 2025-09-24 14:44:53 +02:00
  • 8a22e453e4 Fixed csv Christian Risi 2025-09-24 14:44:25 +02:00
  • 7feb4eb857 Fixed URI generation Christian Risi 2025-09-24 14:44:07 +02:00
  • 70af19d356 Removed unused imports and added trailing slashes Christian Risi 2025-09-24 14:04:48 +02:00
  • a4b44ab2ee Fixed Typos Christian Risi 2025-09-24 14:04:27 +02:00
  • 74b6b609dd Fixed typos Christian Risi 2025-09-24 13:59:19 +02:00
  • 59796c37cb Added script to take dbpedia uris Christian Risi 2025-09-24 13:49:29 +02:00
  • f696f5950b Added uri-abbreviations Christian Risi 2025-09-24 13:48:53 +02:00
  • 605b496da7 Added barebone UML diagram for a Cleaning Pipeline Christian Risi 2025-09-23 19:49:01 +02:00
  • 7d693964dd Added new directories to tree structure Christian Risi 2025-09-23 19:47:56 +02:00
  • 25f401b577 Fixed bug for parsing and added CLI functionalities dev.splitter Christian Risi 2025-09-23 17:58:08 +02:00
  • 14c5ade230 Added CLI functionalities Christian Risi 2025-09-23 17:57:38 +02:00
  • 4c9c51f902 Added barebone to have a splitter chris-admin 2025-09-23 15:34:53 +02:00
  • 63c1a4a160 added little snippet to rebuild db from db_creation.sql GassiGiuseppe 2025-09-22 17:52:23 +02:00
  • 51114af853 DataRetrivial deleted since it does the same thing as datawarehouse.py GassiGiuseppe 2025-09-22 17:51:35 +02:00
  • 3a6dca0681 Infos about Dataset contruction from csv moved from python file to markdown GassiGiuseppe 2025-09-22 17:39:44 +02:00
  • 346098d2b7 Added query.sql , file with the query used to populate the Dataset GassiGiuseppe 2025-09-22 17:21:32 +02:00
  • 64f9b41378 Built datawarehouse.py which populate the dataset GassiGiuseppe 2025-09-22 17:17:22 +02:00
  • ac1ed42c49 Folder DataCleaning renamed to DatasetMerging since it doesn't clean nothing and instead Build the dataset GassiGiuseppe 2025-09-22 17:11:49 +02:00
  • edd01a2c83 Dataset updated, the new one is built with the new method ( 50 new rows found ... upon 13 milion ) GassiGiuseppe 2025-09-22 16:57:06 +02:00
  • 5aa9e3fcf3 Added in DBPEDIA the query to get Film \ wiki page ID plus some editing GassiGiuseppe 2025-09-22 15:42:57 +02:00
  • 0970cabf92 reverse.csv grammar correction of the header it seemed to have missplaced the header also in the middle of the csv GassiGiuseppe 2025-09-22 13:47:20 +02:00
  • a26d92750f Update movie-pageid.csv : grammar correction of the header GassiGiuseppe 2025-09-22 12:59:35 +02:00
  • 34c4782232 Dataset.db update. it seems to be correct GassiGiuseppe 2025-09-20 23:33:56 +02:00
  • c5439533e6 DataRetrivial update, without df GassiGiuseppe 2025-09-20 23:32:08 +02:00