The problem with Occam's Razor or algorithmic information theory is that simplicity is language dependent. Any object can be described using one bit if the language is complex enough. We should prefer simple languages to avoid this problem, but defining simple languages just leads to a circular definition. It seems there is no getting around picking a language arbitrarily. And as long as we are doing so, why not pick one that is easy to compute in our observed universe, and penalizes run time. This is what Ben did. As opposed to e.g. Wolfram's 3 color 2 state Turing machine, where even the proof of Turing completeness was far from trivial.
In spite of its problems, Occam's Razor is well established in every branch of science and is the foundation of every application of machine learning. I would really like to see a definition of simplicity that is grounded in experimental data, but I understand the difficulties of doing so. In addition to covering such a broad area of science, it is difficult to test any model that isn't easy to compute. For example, there may be a description of the observable universe that requires only a few hundred bits, but there is obviously no way to simulate a universe (estimated at 10^120 quantum operations) on any computer contained within it. On Sun, Jun 28, 2020 at 1:28 AM Ben Goertzel <[email protected]> wrote: > > Hi James, etc., > > That paper sat on my hard drive for about a decade because I wasn't so > happy with the way I'd phrased things in the introduction... but > finally I decided to just post it on Arxiv anyway because I felt the > basic formalization of simplicity measures was OK, and I wanted to use > it in some other papers I was going to publish or post... > > Anyway the general framing discussion at the start of that paper is > probably not how I'd choose to frame things today, but the key point > there is that I wanted to have a characterization of "what is a > simplicity measure" that was more abstract and axiomatic rather than > committing intrinsically to a particular measure (such as algorithmic > information, or mixes of runtime and program length as in > Schmidhuber's frontier search, etc.)... > > This was useful to me in thinking about combinatorial decision dags > (which I'm looking at as a potential representational underpinning for > Atomese 2.0 language) and also in thinking about Occam's Razor in the > context of hypercomputation and nonwellfounded sets (which are beyond > the Turing level of computation), which is relevant to my recent blog > post on preservation of goal systems under self-modification.... (in > that case I am looking at goal systems defined in terms of > nonwellfounded sets as a potential way of thinking about computable > goal systems, in teame way that we can look at real number math as a > way of thinking about practical calculations involving > finite-precision numbers) ... > > I don't expect you to read through it all, but this formalization of > simplicity for me is part of an overall attempt to come to a > fundamental theoretical understanding of what is general intelligence, > which has been written up in bits and pieces in various papers over > the years, as roughly listed out in > > http://multiverseaccordingtoben.blogspot.com/2020/05/gtgi-general-theory-of-general.html > > This theoretical work has been only off-and-on correlated w/ practical > work I've done w/ openCog, SingularityNET etc. but is playing a > slightly greater role in recent work aimed at formulating an effective > meta-representational and programming language framework for a new > majorly improved/different version of openCog some colleagues and I > are now working on... > > ben > > On Sat, Jun 27, 2020 at 7:09 AM James Bowery <[email protected]> wrote: > > > > I don't hold that against Goertzel. Solomonoff's 2 seminal papers on > > algorithmic induction are "complex" as well. It's just that I'm not very > > motivated by a complaint that universal computation is "very specialized" > > without a "general" context stated in an incisive, concise and intuitive > > manner. The complaint is absurd on its face. > > > > On Sat, Jun 27, 2020 at 7:15 AM stefan.reich.maker.of.eye via AGI > > <[email protected]> wrote: > >> > >> It's a little funny when a paper on defining simplicity is a highly > >> complex read... :) > > > > Artificial General Intelligence List / AGI / see discussions + participants > > + delivery options Permalink > > > -- > Ben Goertzel, PhD > http://goertzel.org > > “The only people for me are the mad ones, the ones who are mad to > live, mad to talk, mad to be saved, desirous of everything at the same > time, the ones who never yawn or say a commonplace thing, but burn, > burn, burn like fabulous yellow roman candles exploding like spiders > across the stars.” -- Jack Kerouac -- -- Matt Mahoney, [email protected] ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T37756381803ac879-M1823b0fb590483515507ae1e Delivery options: https://agi.topicbox.com/groups/agi/subscription
