Abram, There is a large literature on the topic of time series prediction (an area I've been doing some work in lately). There are many different methods, applicable in various different contexts. I've been working fairly short, noisy numerical time series...
The methods one ends up using, in practice, when one wants to do effective time series prediction of various sorts, are generally NOT based on compression.... You could maybe theoretically interpret them in terms of weird ways to do compression, with a lot of hand-waving, but that's not really the convenient way to think about them... I guess this is the sort of thing Russell means by the disconnect between theory and engineering/science practice, regarding the connection between compression and prediction. You could argue that everyone in the time series prediction field is doing it wrong, and they should be using compressors instead of their current algorithms. It's possible. But I'm not so sure... In my own experience, for instance, if you're using an automated program learning algorithm to do time series prediction, the Occam's Razor bias in the program learning algorithm's fitness function is just one among many aspects to tune to get good results. In this case, compression is one ingredient to pay attention to in order to get prediction to work well, but focusing solely on compression isn't the best path.... practically speaking... -- Ben G On Tue, Sep 4, 2012 at 7:07 PM, Abram Demski <[email protected]> wrote: > Matt tried precisely that experiment awhile ago, using a very good, >> state-of-the-art text compression program. The fact that it could do text >> prediction at all was an interesting validation of the mathematical theory, >> but its performance as a predictor was unsurprisingly abysmal. > > > Russel, > > What are you referring to here? PAQ? PAQ uses arithmetic compression, so > the compression performance varies precisely with the predictive ability; > the only way to get it to compress better is to make it predict better, and > the only reason it compresses well right now is because it predicts well. > Furthermore, if we had some better prediction method, we could directly > convert it to a better compression method. > > This is why I feel the compression<->prediction link is quite solid: > because there is already an engineering-level solution here. > > There is one weakness here: predictions should be given as probabilities. > If you give me a module which generates hard predictions, I can't so easily > plug it into an arithmetic compressor. > > If we take a real compression program running on real hardware and try to >> use it as a predictor let alone a general intelligence, does it perform >> well? > > > This paper: > > http://arxiv.org/abs/0909.0801/ > > uses a prediction framework similar (but, somewhat different) from PAQ, > plugs it into a generic planner, and gets decent results. I find it > unlikely that PAQ would do much worse in the same experimental set-up. (I > find it likely that PAQ would do just a bit better... could be wrong, > though.) > > So, the experimental answer seems to be "yes" to me. What experiments to > the contrary are you thinking of? > > On Tue, Sep 4, 2012 at 3:06 PM, Russell Wallace <[email protected] > > wrote: > >> For my part, the link I question is the implicit one between mathematical >> principle and engineering practice. >> >> If we had an infinitely powerful computer on which we could run a >> Kolmogorov compression oracle, would that give us AGI? Yes. >> >> If we take a real compression program running on real hardware and try to >> use it as a predictor let alone a general intelligence, does it perform >> well? No. Matt tried precisely that experiment awhile ago, using a very >> good, state-of-the-art text compression program. The fact that it could do >> text prediction at all was an interesting validation of the mathematical >> theory, but its performance as a predictor was unsurprisingly abysmal. >> >> Even the reverse doesn't hold. We haven't got AGI to test it with, but we >> can test it with NGI which we do have. Are humans good at compression? No. >> Matt observes one reason for this is the lack of deterministic reset >> capability, but if we imagine solving that problem by running an uploaded >> human on digital hardware, thus enabling the mind in question to perform >> lossless compression, it's clear the result would be very inefficient >> compared to a specialized compression program. >> >> >> On Tue, Sep 4, 2012 at 9:06 PM, Abram Demski <[email protected]>wrote: >> >>> Jim, >>> >>> What about the connection between prediction and compression? This is >>> tightly argued. The basic argument for compression as an AGI test is: >>> >>> compression<-prediction<-AGI >>> >>> That is, AGI should be able to do prediction well, and predicting well >>> allows compression. A stronger version goes like this: >>> >>> compression<->prediction<->AGI >>> >>> That is, compressing well also allows prediction, and prediction is >>> central enough to AGI that we can reasonably say, predicting well would >>> allow AGI. >>> >>> The various links and directions in this argument have different >>> strengths and weaknesses. My question to you is: which links are you >>> questioning here? >>> >>> Best, >>> >>> Abram >>> >>> On Mon, Sep 3, 2012 at 9:08 AM, Jim Bromer <[email protected]> wrote: >>> >>>> I would agree that any practical AGI program that I could imagine would >>>> tend to compress some data. However, that does not mean that any good >>>> compressor is practical AGI program. >>>> >>>> Now do all AGI programs have to be compressors? I would say that on a >>>> theoretical basis no. For instance, if there was an effective AGI program >>>> that compressed some data through common references, I could offer another >>>> AGI program that worked as well but simply duplicated all the common >>>> references as they were needed to be used in some specific circumstance. >>>> This shows that, on a theoretical basis, some AGI programs would not need >>>> to act as compressors. >>>> >>>> Now, let's see if all good effective AGI algorithms would effectively >>>> compress data in all cases. We know that search time is one of the main >>>> contemporary problems and so this suggests that duplication - at least in >>>> some circumstances- might be more effective than compression. >>>> >>>> So to say that compression is the same as AGI is an over generalization. >>>> To say that any theoretical AGI program would have to be a compressor >>>> is an over generalization. >>>> To say that all effective AGI algorithms will effectively compress data >>>> in all circumstances (as it learns) is an over generalization. >>>> >>>> Although I agree that any practical AGI program that I could imagine >>>> will tend to compress some data, the imagined theoretical correlation >>>> between compression and AGI is just not strong enough to argue that >>>> compression is a good test of knowing. >>>> >>>> Jim Bromer >>>> >>>> >>>> >>>> >>>> On Mon, Sep 3, 2012 at 11:23 AM, Jim Bromer <[email protected]>wrote: >>>> >>>>> On Sat, Sep 1, 2012 at 1:35 PM, Abram Demski <[email protected]>wrote: >>>>> >>>>>> This is *why* compression is a good test! It is a harsh test for >>>>>> "knowing". >>>>> >>>>> >>>>> >>>>> Abram, >>>>> That doesn't make any sense. You would have to substantiate that >>>>> there is a high correlation between compression with "knowing" in order to >>>>> make that statement. So far, the only correlation is of the nature of >>>>> statements like those which say that calculators can do something that had >>>>> previously only been possible for human beings to do. Although that kind >>>>> of achievement can be taken as evidence that computers are able to do some >>>>> kinds of thinking, that kind of thinking has been repeatedly repudiated in >>>>> this group as being "narrow AI" and not AGI. So your claim that >>>>> compression is a good harsh test for knowing has to be absolutely >>>>> repudiated as well. >>>>> >>>>> I am glad that I am able to categorize the compression-is-intelligence >>>>> approach as "narrow AI" even though I am not totally happy with that >>>>> label. Because now, for the first time, I will be able to chip away at >>>>> the >>>>> basis of the approach (at least in this group) without having to go >>>>> through >>>>> some long winded reasoning that will be mostly ignored in this group (as >>>>> it >>>>> obviously has been in the past.) >>>>> >>>>> Compression is not a test for "knowing", except possibly for some >>>>> special cases of narrow AI methods. >>>>> >>>>> Jim Bromer >>>>> >>>>> >>>>> >>>>> >>>>> On Sat, Sep 1, 2012 at 1:35 PM, Abram Demski <[email protected]>wrote: >>>>> >>>>>> Jim, >>>>>> >>>>>> This is *why* compression is a good test! It is a harsh test for >>>>>> "knowing". >>>>>> >>>>>> --Abram >>>>>> >>>>>> On Fri, Aug 31, 2012 at 6:26 PM, Jim Bromer <[email protected]>wrote: >>>>>> >>>>>>> One of the mistakes that the compressors-is-intelligence guys make is >>>>>>> that an ideal compression of knowledge cannot be made before knowing >>>>>>> occurs. This is so obvious that some people will go to some lengths >>>>>>> to point it out. But some of the implications are also really >>>>>>> obvious >>>>>>> as well. >>>>>>> >>>>>>> If an ideal compression cannot be made before knowing has occurred >>>>>>> then an ideal compression can never be made for a program that is >>>>>>> capable of learning because it will always be able to learn something >>>>>>> new. Isn't this obvious? >>>>>>> Jim Bromer >>>>>>> >>>>>>> >>>>>>> ------------------------------------------- >>>>>>> AGI >>>>>>> Archives: https://www.listbox.com/member/archive/303/=now >>>>>>> RSS Feed: >>>>>>> https://www.listbox.com/member/archive/rss/303/7190161-766c6f07 >>>>>>> Modify Your Subscription: https://www.listbox.com/member/?&; >>>>>>> Powered by Listbox: http://www.listbox.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Abram Demski >>>>>> http://lo-tho.blogspot.com/ >>>>>> >>>>>> *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> >>>>>> <https://www.listbox.com/member/archive/rss/303/10561250-164650b2> | >>>>>> Modify <https://www.listbox.com/member/?&;> Your Subscription >>>>>> <http://www.listbox.com> >>>>>> >>>>> >>>>> >>>> *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> >>>> <https://www.listbox.com/member/archive/rss/303/7190161-766c6f07> | >>>> Modify <https://www.listbox.com/member/?&;> Your Subscription >>>> <http://www.listbox.com> >>>> >>> >>> >>> >>> -- >>> Abram Demski >>> http://lo-tho.blogspot.com/ >>> >>> *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> >>> <https://www.listbox.com/member/archive/rss/303/1658954-f53d1a3f> | >>> Modify <https://www.listbox.com/member/?&;> Your Subscription >>> <http://www.listbox.com> >>> >> >> *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> >> <https://www.listbox.com/member/archive/rss/303/7190161-766c6f07> | >> Modify <https://www.listbox.com/member/?&;> Your Subscription >> <http://www.listbox.com> >> > > > > -- > Abram Demski > http://lo-tho.blogspot.com/ > > *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> > <https://www.listbox.com/member/archive/rss/303/212726-11ac2389> | > Modify<https://www.listbox.com/member/?&;>Your Subscription > <http://www.listbox.com> > -- Ben Goertzel, PhD http://goertzel.org "My humanity is a constant self-overcoming" -- Friedrich Nietzsche ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-c97d2393 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-2484a968 Powered by Listbox: http://www.listbox.com
