>>When you apply Bayes rule, you are assuming that future outcomes will be
>>drawn from the same distribution as past outcomes, i.e. you are assuming
>>that induction works.
>
>Not true! ALL I am assuming is basic, timeless Bayes Rule.
The rule makes a statement about distributions. To apply the rule
correctly, we must make sure that we are talking about the same
distribution.
Whenever you condition, you are assuming that evidence was drawn from
the distribution about which you are making a probability statement.
For example, if you are trying to determine the bias of a coin, you
update your posterior for the bias under the assumption that your data
actually came from the distribution of the coin. If you are told that
the evidence you have is not from the coin in question, you will not use
your evidence to update the bias for the coin.
>I am beginning with a prior distribution which places some mass on "simple
>universe," some mass on "unlearnable universe," and some mass on "edge of
>chaos universe evolving a self-understanding." Then I am conditioning on
>data from what is observable to me in the present, which is all the data we
>ever have. This gives me a posterior distribution on all three hypotheses.
>"Simple universe" gets thrown out immediately, leaving me with
>non-negligible mass on "unlearnable" and "edge of chaos." For me (maybe
>not for you) there is far more mass on "edge of chaos."
If you are simply choosing between models in a world without time, then
there is no point in talking about induction and no point in talking
about the future.
If you believe that time exists and are attempting to make a scientific
statement about how rules in the world hold over time, then you are
interested in the question of induction. Looking at a mass of prior
data can give you a hypothesis about how rules held in the world in the
past. If you try to update your posterior on induction being true in
the present or future, you are assuming stationarity, i.e. that evidence
you have about induction in the past is germane to the present. This
means that you are assuming an underlying distribution that does not
change across time, which is equivalent to the assumption that induction
is valid. This is circular.
>In the very most unlearnable of the
>unlearnable hypotheses, the past, present and future have absolutely
>nothing to do with each other. The present materialized exactly as it is
>right now, including my memories of my childhood and this quaint idea I
>seem to have that I'm learning, and is going to dissolve again in an
>instant. For me (maybe not for you) this hypothesis has very low posterior
>probability, because the likelihood of my seeing something I can understand
>so well as a temporally evolving universe is quite high under the "edge of
>chaos" hypothesis and miniscule under this "very, VERY unlearnable"
>hypotheiss.
This is a classic example in which, by *construction*, evidence cannot
influence your posterior. Any attempt to construct a posterior based on
evidence violates the premise.
>>Therefore, you cannot use Bayes rule to justify
>>induction without falling into the trap of circular reasoning.
>
>This is not circular at all. It's just Bayes rule. I'm starting with a
>prior, conditioning on what I see, and ending up with a posterior.
>
>>More generally, Bayes rule is not an escape from the requirement that we
>>reason non-circularly. If you construct a hypothesis that is
>>self-reinforcing, then you have constructed template which can be used to
>>justify anything.
>
>I cannot justify anything.
>
>I can't justify belief in the tooth fairy, or that I will wake up tomorrow
>morning with purple spots all over my body, or that the sun won't rise
>tomorrow. I can't prove them wrong, but they have very low evidential
>support.
Sorry - What I should have said is that by using such reasoning, one can
justify anything that fits the template.
The template for the particular form of circular reasoning in question here
requires a premise about an inference rule (or precondition thereof) and
then the use of that rule to establish the premise. The classic example
of how to fill this template is the "counter-induction" rule. We assume
that things that have worked in the past will *not* work in the future
(and vice versa). We apply this rule in a universe where induction
holds and observe that the rule has failed. From this, we conclude, due
to the counter-inductive principle, that the rule must work in the
future. Each new failure only strengthens our belief in
counter-induction.
To make this a more Bayesian argument, we would need to replace the
stationarity assumption we normally make when making predictions about
the future with one that replaces the probabilities with their
complements, i.e P(A) -> P(~A), etc. Call this "counter-Bayes" rule.
Now we entertain the hypothesis, H, that our counter-stationarity
assumption is valid. We examine our data and see that in the past, the
counter-stationarity assumption did terribly; P(H) is very low. We
apply our non-stationarity assumption to the hypothesis and we conclude
that it has great evidential support for holding in the present and
future since 1-P(H) is very high. As before, each new failure only
strengthens the posterior for H. Why not? We are just using timeless,
reliable counter-Bayes rule.
--
Ron Parr email: [EMAIL PROTECTED]
--------------------------------------------------------------------------
Home Page: http://robotics.stanford.edu/~parr
