--- Russell Wallace <[EMAIL PROTECTED]> wrote: > On 9/23/07, Matt Mahoney <[EMAIL PROTECTED]> wrote: > > I realize that a language model must encode both the meaning of a text > string > > and its representation. This makes lossless compression an inappropriate > test > > for evaluating models of visual or auditory perception. The tiny amount > of > > relevant information in a picture would be overwhelmed by incompressible > pixel > > noise. > > I don't think that's a showstopper. Clearly the entropy of video is > higher, as a percentage of the uncompressed file size, than is the > case for text, but tests are relative. Suppose the best lossless video > compression achieved at a given time is only 10% (though I think we > could do better than this). A program that improved this to 11% would > still be measurably, objectively better than the competition.
It is true that lossless compression can be measured very precisely. But there are three problems. First, the data set needs to be huge, 20 years of video, or 1 PB of MPEG-2. (It has to be large enough to train the model, the reason I use 1 GB of text). Second, video has only a few bits per second of perceptible features out of 10^7 bits per second (and remember, MPEG-2 is already compressed). But the third I think is a showstopper. Lossless video compression does not model the visual perception system. It models the physics of the video source. This is the difference between video and text compression. The source of text is the human brain. The probability distribution of language coming out through the mouth is the same as the distribution coming in through the ears. But there is no equivalent for vision. > There is also the consideration that text compression is of no real > value, because frankly text is already small enough that it doesn't > need to be compressed. Better lossless image and video compression, on > the other hand, apart from being of more potential relevance to AI > would also be of value in its own right - there are a lot of > situations where it would be better to not have to throw away any of > the information. My goal is not to compress text but to be able to compute its probability distribution. That problem is AI-hard. Lossless video compression would not get far. The brightness of a pixel depends on the number of photons striking the corresponding CCD sensor. The randomness due to quantum mechanics is absolutely incompressible and makes up a significant fraction of the raw data. -- Matt Mahoney, [EMAIL PROTECTED] ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=49570768-a5da72
