I was reading some years ago on encode.su about using PAQ to generate text, similar to what this one does with gzip/zlib. The paper mentions having to exclude its own output from the context window or else it degenerates to repeating the same character over and over. PAQ has the same problem unless you turn off updates to the model while generating text.
-- Matt Mahoney, [email protected] On Fri, Jul 3, 2026, 8:25 PM John Rose via AGI <[email protected]> wrote: > On Wednesday, May 13, 2026, at 2:23 AM, Alan Grimes wrote: > > I think I'm starting to trend towars being a compressionist > > > Can gzip be a language model? > > https://nathan.rs/posts/gzip-lm/ > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/T671cabb63b7c8c5e-M7413f676d0359e023a9f28b9> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T671cabb63b7c8c5e-M71fb2aa3139005b8e58d4d2a Delivery options: https://agi.topicbox.com/groups/agi/subscription
