Re: [agi] The limitations of the validity of compression.

[immortal . discoveries]

Indeed, if someone removes a dozen char/word types in the training/test set if 
they use Perplexity as evaluation, they can get a higher score. To work/compare 
with enwiki8 you must either decompress it losslessly or train on it in full 
and in full on test set
------------------------------------------
Artificial General Intelligence List: AGI
Permalink: 
https://agi.topicbox.com/groups/agi/T2a0cd9d392f9ff94-Mad032fa4e019de2c87d66d51
Delivery options: https://agi.topicbox.com/groups/agi/subscription