--- Vladimir Nesov <[EMAIL PROTECTED]> wrote:

> The same probably goes for text
> compression: clever (but not intelligent) statistics-gathering
> algorithm on texts can probably do a much better job for compressing
> than human-like intelligence which just chunks this information
> according to its meaning.

That is only true because there is a 3 way tradeoff between speed, memory, and
compression ratio.  On a 1 GB input the best text compressors improve rapidly
as memory is increased to 2 GB, which is as far as I can test.  At this point,
simple algorithms like BWT and PPM do almost as well as more sophisticated
programs that mix lexical, syntactic and semantic constraints.  These programs
would use a lot more memory if they could, unlike the simpler models which
have most or all of the memory they need.  On smaller input, the memory
pressure is reduced and the simpler algorithms can't compete.

And text is the only data type with this property.  Images, audio, executable
code, and seismic data can all be compressed with very little memory.

-- Matt Mahoney, [EMAIL PROTECTED]

This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:

Reply via email to