Re: Another slightly OT q...
On 09/05/07, Ted Mittelstaedt [EMAIL PROTECTED] wrote: -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Gary Kline Sent: Tuesday, May 08, 2007 7:19 PM To: [EMAIL PROTECTED] Cc: Gary Kline; FreeBSD Mailing List Subject: Re: Another slightly OT q... So it *was* a hoax? Rats. Some weeks ago on Public Broadcasting, a few sentences were spoken on the potential of fractal geometry to achieve [I'm guessing] data-compression on the order of what Sloot was claiming. So far, no one has figured it out. It may be a dream... . There's some cool math out there that explains all of this but I never liked math, but it isn't necessary to know the math to understand the issue. Just consider the problem for a while and you will realize that the compression ratio of a specific data stream varies dependent on the amount of repetition in the input datastream. A perfectly unrandom datastream, like a constant series of logical 1's, carries no information, but has a compression ratio that is infinite. A perfectly random datastream, on the other hand, also carries no information, but has a compression ratio that is zero. I believe that a datastream that is 50% of the way between either extreme carries the most information, and I believe your typical datastream is much closer to the perfectly unrandom side than the perfectly random side, compression is merely the process of pushing the randomness of the stream closer to the random side. Actually, the more information (as such) the closer the data stream is to perfectly random. The relation- ship might be asymptotic, but I am no maths major. Thus, if the input datastream is very close to the perfectly unrandom side - meaning it has a very high amount of repetition in it, you can get some pretty spectacular compression ratios. But as you move closer to unrandom, you carry less data. So, the better applications emit datastreams that are less unrandom, therefore compression does not work as well on them. I suppose this leads to the discussion about what data and information really are. Imagine a can. The can is data. Imagine tha can is full of worms. This of course is completely ignoring the other data issue, is the application data efficient to begin with? For example, you can transfer about a page of information in ASCII that consumes about 1K of data, that same page of information in a MS Word file consumes a hundred times that amount of space - Word is therefore extremely inefficient with data. In this case, since word has to replace typesetting, layout, and formatting software, in addition to being a word processor the header and meta information tend to bloat the files quite a lot. Every few years someone comes along who makes some mad claims about some new buzzword-enhanced compression technology. Obviously, if there is ever a radical leap forward in that area the theory will have to follow, since modern theory cannot accomodate (lossless) compression past the point of randomness (generally less than 16:1 even for Danielle Steele). mp3, avi, real media mpeg, et al are a different story entirely, sicne they are lossy and optimised for their respective information. -rw-r--r-- 1 1705 1705 7826420 May 9 10:58 ssion_i_really_fuckin_care_about_you.rm -rw-r--r-- 1 1705 1705 7791691 May 9 10:58 ssion_i_really_fuckin_care_about_you.rm.bz2 In this case, very slightly compressible: with some data your resulting file will be slightly larger, yet the raw datastream (and it looks like it was filmed from a cameraphone here (though most likely an 8mm digicam (these, I believe, compress on the fly, so the raw datastream never touches tape))) would probably have been many tens, if not several hundreds, of megabytes. Remember life before the tweel? -- -- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Another slightly OT q...
At 23:22 08/05/2007, you wrote: see: http://nl.wikipedia.org/wiki/Jan_Sloot about the Sloot Digital Coding System. great stuff. Danke. that seems to be German, in Dutch Bedankt would be appropiate. i appreciate the effort though. I found the english wkik entry so I could understand the piece. Since Sloot is dead, no way of knowing. actually there is. the Dutch wiki-article has much more detail about the case. this Sloot guy claimed he could compress any movie into 1kb ( 1024 bytes ). he stored this 1kb movies on smartcards, which he would feed into his magic machine to show the movies. in The Netherlands there was press-coverage about this, and he was able to attract investors and even a ( up to then ) reputable IT-guru assiocated with Philips to back him. they found silicon-valley investors who were interested. he died / killed himself / was murdered before his hoax could be uncovered. It's easily covered. Check usenet comp.compression faq. It's the Counting Theorem. For a sequence of n-bits there are 2^n possible files (f.e. there are 256 files of 8 bits). Unfortunately there are only (2^n)-1 possible files lesser than the original (f.e. there are 128 files of 7 bits, 64 of 6, 32 of 5, 16 of 4, 8 of 3, 4 of 2 and 2 of 1 bits, total, 255 files) so using an algorithm you can't compress all n-bits sequences. For this case, you can easily check that using this guy algorithm, you can have only 2^8192 movies. Perhaps you think they are a lot, but nearly all are white noise movies. There are 1-2 monthly of this claims on comp.compression. Normally they are beginners to Information Theory / Compression but others are simply cheaters. sounds a lot like the perpetuum mobile stories. In fact, it is. Information Theory uses entropy concept too and claims like this breaks the 2nd and 3rd principle. But this is the kind of leap forward that would save, oh, a few measley $Billions. And give millions of us faster and broader access. HTH ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Another slightly OT q...
At 19:05 08/05/2007, you wrote: Hey Guys, Does anybody have any websites where I can look up the latest in compression technology? I've got a very thin ISDL link so pulling streams over is like looking at postage-stamp images. Depends on what you are trying to compress. You can look these pages for lossless compression: www.maximumcompression.com prize.hutter1.net www.cs.fit.edu/~mmahoney/compression/text.html datacompression.info If you are looking for zip/deflate try www.7-zip.com , it has the best zip/deflate engine, you can get zip standard files smaller than based zlib ones. It's for windows, but there is a unix port inside the web pages. There were whispers about fractal-compression as being the golden goal, but I can't find much about this one. Anybody have any clues here? Depends on what you want to compress. Fractal compression for text can't work, text has no autosimetry. For images it is being developed, but results are bad. You can check citeseer for . Currently for video/image wavelets are the best (dirac, snow, jpeg2000 and more) codecs but there is no standard codec that uses them (no ISO-ITU, no mpeg consortium). For text, the best compressors are neural nets (paq family) or ppm. thanks up front, gary HTH ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Another slightly OT q...
On Wed, May 09, 2007 at 07:50:02PM +0200, Eduardo Morras wrote: At 23:22 08/05/2007, you wrote: he died / killed himself / was murdered before his hoax could be uncovered. It's easily covered. Check usenet comp.compression faq. It's the Counting Theorem. For a sequence of n-bits there are 2^n possible files (f.e. there are 256 files of 8 bits). Unfortunately there are only (2^n)-1 possible files lesser than the original (f.e. there are 128 files of 7 bits, 64 of 6, 32 of 5, 16 of 4, 8 of 3, 4 of 2 and 2 of 1 bits, total, 255 files) so using an algorithm you can't compress all n-bits sequences. For this case, you can easily check that using this guy algorithm, you can have only 2^8192 movies. Perhaps you think they are a lot, but nearly all are white noise movies. There are 1-2 monthly of this claims on comp.compression. Normally they are beginners to Information Theory / Compression but others are simply cheaters. sounds a lot like the perpetuum mobile stories. In fact, it is. Information Theory uses entropy concept too and claims like this breaks the 2nd and 3rd principle. Thanks for these tidbits of insight! I figured there were some laws or principles that told what the upper bound could be. I've always been pretty good at math, but since Life is bounded, I'll take your word for it :-) gary PS: I'll check out the URL's in your last email. They'll probably be overkill, but for reference -- Gary Kline [EMAIL PROTECTED] www.thought.org Public Service Unix ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Another slightly OT q...
On 5/8/07, Gary Kline [EMAIL PROTECTED] wrote: Hey Guys, Does anybody have any websites where I can look up the latest in compression technology? I've got a very thin ISDL link so pulling streams over is like looking at postage-stamp images. There were whispers about fractal-compression as being the golden goal, but I can't find much about this one. Anybody have any clues here? thanks up front, gary see: http://nl.wikipedia.org/wiki/Jan_Sloot about the Sloot Digital Coding System. great stuff. regards, usleep ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Another slightly OT q...
On Tue, May 08, 2007 at 09:16:11PM +0200, [EMAIL PROTECTED] wrote: On 5/8/07, Gary Kline [EMAIL PROTECTED] wrote: Hey Guys, Does anybody have any websites where I can look up the latest in compression technology? I've got a very thin ISDL link so pulling streams over is like looking at postage-stamp images. There were whispers about fractal-compression as being the golden goal, but I can't find much about this one. Anybody have any clues here? thanks up front, gary see: http://nl.wikipedia.org/wiki/Jan_Sloot about the Sloot Digital Coding System. great stuff. Danke. I found the english wkik entry so I could understand the piece. Since Sloot is dead, no way of knowing. But this is the kind of leap forward that would save, oh, a few measley $Billions. And give millions of us faster and broader access. cheers! gary regards, usleep -- Gary Kline [EMAIL PROTECTED] www.thought.org Public Service Unix ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Another slightly OT q...
Gary, On 5/8/07, Gary Kline [EMAIL PROTECTED] wrote: On Tue, May 08, 2007 at 09:16:11PM +0200, [EMAIL PROTECTED] wrote: On 5/8/07, Gary Kline [EMAIL PROTECTED] wrote: Hey Guys, Does anybody have any websites where I can look up the latest in compression technology? I've got a very thin ISDL link so pulling streams over is like looking at postage-stamp images. There were whispers about fractal-compression as being the golden goal, but I can't find much about this one. Anybody have any clues here? thanks up front, gary see: http://nl.wikipedia.org/wiki/Jan_Sloot about the Sloot Digital Coding System. great stuff. Danke. that seems to be German, in Dutch Bedankt would be appropiate. i appreciate the effort though. I found the english wkik entry so I could understand the piece. Since Sloot is dead, no way of knowing. actually there is. the Dutch wiki-article has much more detail about the case. this Sloot guy claimed he could compress any movie into 1kb ( 1024 bytes ). he stored this 1kb movies on smartcards, which he would feed into his magic machine to show the movies. in The Netherlands there was press-coverage about this, and he was able to attract investors and even a ( up to then ) reputable IT-guru assiocated with Philips to back him. they found silicon-valley investors who were interested. he died / killed himself / was murdered before his hoax could be uncovered. sounds a lot like the perpetuum mobile stories. But this is the kind of leap forward that would save, oh, a few measley $Billions. And give millions of us faster and broader access. good luck! regards, usleep ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Another slightly OT q...
On Tue, May 08, 2007 at 11:22:31PM +0200, [EMAIL PROTECTED] wrote: Gary, On 5/8/07, Gary Kline [EMAIL PROTECTED] wrote: On Tue, May 08, 2007 at 09:16:11PM +0200, [EMAIL PROTECTED] wrote: On 5/8/07, Gary Kline [EMAIL PROTECTED] wrote: Hey Guys, Does anybody have any websites where I can look up the latest in compression technology? I've got a very thin ISDL link so pulling streams over is like looking at postage-stamp images. There were whispers about fractal-compression as being the golden goal, but I can't find much about this one. Anybody have any clues here? thanks up front, gary see: http://nl.wikipedia.org/wiki/Jan_Sloot about the Sloot Digital Coding System. great stuff. Danke. that seems to be German, in Dutch Bedankt would be appropiate. i appreciate the effort though. Yes. As far as I know, on my father's side, I'm .5 German and .5 Hollander. But I'm also a linguistic moron. So I'm infinitely grateful that so many folk are speak English.. I found the english wkik entry so I could understand the piece. Since Sloot is dead, no way of knowing. actually there is. the Dutch wiki-article has much more detail about the case. this Sloot guy claimed he could compress any movie into 1kb ( 1024 bytes ). he stored this 1kb movies on smartcards, which he would feed into his magic machine to show the movies. Aha! Were these smartcards like microprint? One of my favorite philsophy texts has 400+ pages and is compressible into one small thinfilm. *Or*, by smartcard do you mean something non-optical? in The Netherlands there was press-coverage about this, and he was able to attract investors and even a ( up to then ) reputable IT-guru assiocated with Philips to back him. they found silicon-valley investors who were interested. he died / killed himself / was murdered before his hoax could be uncovered. sounds a lot like the perpetuum mobile stories. So it *was* a hoax? Rats. Some weeks ago on Public Broadcasting, a few sentences were spoken on the potential of fractal geometry to achieve [I'm guessing] data-compression on the order of what Sloot was claiming. So far, no one has figured it out. It may be a dream... . But this is the kind of leap forward that would save, oh, a few measly $Billions. And give millions of us faster and broader access. good luck! regards, usleep Same! and bedankt; ciao, gary -- Gary Kline [EMAIL PROTECTED] www.thought.org Public Service Unix ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Another slightly OT q...
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Gary Kline Sent: Tuesday, May 08, 2007 7:19 PM To: [EMAIL PROTECTED] Cc: Gary Kline; FreeBSD Mailing List Subject: Re: Another slightly OT q... So it *was* a hoax? Rats. Some weeks ago on Public Broadcasting, a few sentences were spoken on the potential of fractal geometry to achieve [I'm guessing] data-compression on the order of what Sloot was claiming. So far, no one has figured it out. It may be a dream... . There's some cool math out there that explains all of this but I never liked math, but it isn't necessary to know the math to understand the issue. Just consider the problem for a while and you will realize that the compression ratio of a specific data stream varies dependent on the amount of repetition in the input datastream. A perfectly unrandom datastream, like a constant series of logical 1's, carries no information, but has a compression ratio that is infinite. A perfectly random datastream, on the other hand, also carries no information, but has a compression ratio that is zero. I believe that a datastream that is 50% of the way between either extreme carries the most information, and I believe your typical datastream is much closer to the perfectly unrandom side than the perfectly random side, compression is merely the process of pushing the randomness of the stream closer to the random side. Thus, if the input datastream is very close to the perfectly unrandom side - meaning it has a very high amount of repetition in it, you can get some pretty spectacular compression ratios. But as you move closer to unrandom, you carry less data. So, the better applications emit datastreams that are less unrandom, therefore compression does not work as well on them. This of course is completely ignoring the other data issue, is the application data efficient to begin with? For example, you can transfer about a page of information in ASCII that consumes about 1K of data, that same page of information in a MS Word file consumes a hundred times that amount of space - Word is therefore extremely inefficient with data. Probably the worst offender of this are the news websites like www.cnn.com. They insist on putting more and more news articles into videos rather than just a couple screens of text. I just do not see any benefit to the consumer of a video of an interview with someone like George Bush, when the video consists of 2 sentence fragments. The entire story could be written on a webpage, sans video. Do they really think the typical reader doesen't know what he looks like already? I see this a lot with audio files, also. For example, how many times have you come across an .mp3 file that was of speech only - perhaps a professor's lecture - that's been recorded in CD quality full stereo? A .wav file recorded at the lowest sampling rate in mono, which is perfectly acceptable for speech, would be smaller. Ted ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
