"*Q: In a contest to produce the best model of the data in evidence where contestants are highly motivated by monetary prize awards, what are some reasons for using the size of its executable archive the best loss function?*"
@Matt, GPT-4 yes seems to failed this Q above. Also, what in the world was I saying lol. Hmm, no, it did not say Lossless Compression is better than Perplexity, and another correction to my reading is James also brought up executables in each question, it didn't on its own. Furthermore (not in LC favor here though lol) it seems to say LC *is* too intense and approximations instead are used in the last Q/A if I'm reading this correct. Am I wrong? I know it's useful and interesting and etc but it does seem to cost more resources to "run"? But wait doesn't using 1GB for the training dataset work as good as 1TB? Hence maybe it is not much cost to use LC. Though it at least got this question below right about that: On Tuesday, August 15, 2023, at 2:15 PM, James Bowery wrote: > *Q: Relative to a given set of data, why is the size of its executable > archive the most principled loss function?* On Tuesday, August 15, 2023, at 2:15 PM, James Bowery wrote: > In other words, the size of the executable archive represents the minimum > number of bits required to represent or “describe” the data, and therefore > provides a fundamental lower bound on the complexity of the data. On Tuesday, August 15, 2023, at 2:15 PM, James Bowery wrote: > The aim of most machine learning algorithms is to learn a model from the data > that captures this underlying complexity without overfitting. Overfitting > would be akin to storing the data as opposed to learning or capturing its > patterns. Thus, an optimal learning algorithm should have its loss function > directly related to this measure of complexity. On Tuesday, August 15, 2023, at 2:15 PM, James Bowery wrote: > In conclusion, the size of the executable archive bearing the data can be > seen as a principled choice for a loss function since it directly measures > the intrinsic complexity in the data, which is what most learning algorithms > strive to capture. However, in practice, employing this as a loss function > can be computationally challenging and instead approximations or alternative > loss functions might be used. ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T8e755985a1fa3a8b-M4a45b3cfd1111e164ee6bea7 Delivery options: https://agi.topicbox.com/groups/agi/subscription
