>> Whenever you condition, you are assuming that evidence was drawn from
>> the distribution about which you are making a probability statement.
>
>No, at most you are simply assuming one large joint probability distribution
>over all the variables of interest (including data variables). There is no
>i.i.d. assumption required. Although the assumption that the data are i.i.d. is
>essential to frequentist methods, with their reliance on sampling distributions
>and central limit theorems, such an assumption is completely unnecessary with
>Bayesian methods (although it does simplify the problem).
As I said earlier, the whole freqentist/subjectivist issue is a total
red herring. If the term distribution bothers you, simply replace my
assertion that the observation came from the correct distribution with
an assertion that the observations were actually generated in a manner
that gives you evidence about variables connected to the proposition in
question.
Regarding the discussion about coin tosses: If you assume that the bias
of the two coins is correlated, then of course, information from one
flip gives you information about the other. I suppose the point of this
digression is that you aim to suggest that one might entertain a weaker
version of induction in which the past is perhaps noisily correlated
with the future. None of this matters. The difficulty is in the
assumption of any correlation. You can obfuscate things by hiding the
correlation in various ways, but once you have assumed that the past
gives you information about the future, you have assumed your premise.
>> If you are simply choosing between models in a world without time, then
>> there is no point in talking about induction
>
>That's a pretty strong statement; would you care to elaborate? I can point to a
>number of examples of induction where there is no inherent temporal ordering on
>the data and the hypothesis, or where the data may be temporally ordered *after*
>the hypothesis. For exmple, mathematicians make conjectures based on analogies
>and patterns their minds perceive in known proofs and theorems, then try to
>prove these conjectures; but these theorems are timeless, lacking any inherent
>temporal ordering. As another example, historical science (paleontology,
>archaeology, etc.) is all about making inferences about the past from data in
>the present -- precisely the opposite of the flow of causality.
The topic of discussion here is scientific induction. Mathematical
induction, which is formally grounded, is a separate topic. Inferences
made about the past are not what is typically referred to as scientific
induction and they do not have the same difficulty since we typically
view the past as static.
>That's one possibility, but you don't have to do it that way. If you suspect
>that the rules might change over time, then the thing to do is to propose one or
>more models of how they might change over time. You can then use Bayesian model
>comparison techniques to compare these models with each other and with a model
>in which the rules are invariant over time.
You've just pushed the circularity back one layer. The question of
whether your observations give you information about the future still
remains and the circularity still results when you try to conclude
something about the future.
>I don't think you can make this argument at all rigorous. The moment you try to
>translate this argument into mathematics you're going to run into trouble. The
>self-referential nature of the argument itself should be a big red warning flag
>to you -- it's a cousin to Russell's paradoxical "set of all sets that do not
>contain themselves."
[counter-induction]
Of course, I'm not arguing for counter-induction. My point is that the
argument is as rigorous as the argument in favor of induction. The red
warning flags pop up in both arguments in the same places for the same
reasons.
>Several things to note here:
[Bayesian counter-inductionism]
>1. You can't use Bayes' Rule to compute P(H) unless you have some alternative to
> compare it to.
Actually, I do not need this since my claim is that the inference is
invalid regardless.
>2. Again you're getting self-referential, by having your counter-stationarity
> hypothesis H say something about itself. I doubt you can formalize
>this.
No - and this is actually very important - the counter-stationarity
assumption is simply a different assumption about the relationship
between the past and the future. The is nothing self-referential about
stationarity or counter-stationarity per se. When we apply a rule that
uses one of these assumptions in order to prove the assumption, the
self-referential difficulties emerge. We run into this problem whether
we are using counter-bayes rule or bayes-rule.
>3. It's not at all clear what your counter-stationarity hypothesis
>really
>*means*.
> From what you've said so far it sounds like it might be self-contradictory,
>in
> which case you can't apply Bayes' rule -- you get the probabilistic
>equivalent of
> dividing by zero when you take a probability conditional on a known false
>hypothesis.
I'm not sure where you're seeing these contradictions and lack of
clarity. It's a simple assumption about how the world changes over time
that just happens to be different from the standard one that people seem
to be unwilling to admit that they are making.
>In conclusion, you need to translate this argument into rigorous mathematics
>before it will be at all convincing. As it stands, it is far too vague.
>English is simply too slippery and ambiguous to use for discussing this kind of
>thing.
Perhaps I should review the point of this exercise. The circularity of
the proposed arguments justifying induction is quite plain; it's first
year reading for a philosophy major. Some people have trouble
recognizing their own circular reasoning. This is because some
assumptions are so central to their way of thinking that they don't
realize when they are making them. Fortunately, philosophers have a
nice tool for making such circularity manifest. They replace the
cherished hypothesis with a less-cherished one, then carry through the
same line of reasoning, employing the same circularity and show that the
less-cherished hypothesis is support equally well. I didn't invent this
tactic and I didn't invent counter-induction.
Usually this exercise helps make the places where the assumptions crept
into the original argument more salient. Sometimes people get confused
and think that the less-cherished hypothesis is actually what the
argument is about. I hope that in all of this discussion, I haven't led
anybody to believe that I am trying to justify counter-induction or,
more importantly, that the acceptance of the counter-inductionist
argument is central to recognizing the circularity of the original
argument. It's simply a tool to help people see the flaws in their own
reasoning. If the counter-induction thing is not working for you, then
go back to the basics and think about how one could possibly hope to
justify the hypothesis that past evidence is relevant to the future by
using a rule which requires this hypothesis as a precondition.
So, I don't feel any particular need to make the counter-induction
argument more rigorous. However, if I am presented with a rigorous
argument justifying induction, then I might be tempted to match this
with a counter-induction version - if anybody remains unconvinced after
attempting to go through the effort of producing a rigorous
_non-circular_ justification of induction.
--
Ron Parr email: [EMAIL PROTECTED]
--------------------------------------------------------------------------
Home Page: http://robotics.stanford.edu/~parr
