original code, current compression for 100KB and 1MB: 28,081 240,741 ------ with new ability 27,865 = *0.14MB DROP* 239,436
original code with preprocessor 27,875 --- 60,800 230,708 --- 608,000 ------ original code with preprocessor and new ability 27,242 --- 60,800 *0.63MB DROP* 226,517 --- 608,000 I guess it likes working on word level/ BPE level more. I have not exhausted it yet either, I might be able to get off 1MB-1.8MB. ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Tca9507420a7bfdfd-Mbbf1036b16ad58c23eb70d06 Delivery options: https://agi.topicbox.com/groups/agi/subscription
