Robert Dodier wrote:
> [Discussion of Hume's argument that induction is impossible.]
>
> This is especially disconcerting because a formal theory of induction
> is easily stated in terms of probability: induction is the computation
> of the posterior for an unobserved phenomenon conditioned on
> observed phenomena. [...]
>
> But if there is no such thing as induction, how can we make use of
> probabilistic models? Doubtless I am missing something: on the one
> hand, Hume's anti-inductive arguments seem unanswerable, but on the
> other, we all seem to reason inductively every day with great success.
There have been several results that show that if you start with no knowledge at
all, you can't learn anything beyond your raw data. For example, you can apply
Bayesian probability theory to the problem of inferring an unknown Boolean
function f:{0,1}^n -> {0,1} from samples of input and corresponding output, but
if you have no prior information at all (corresponding to a uniform prior over
the space of Boolean functions), your data give you no information at all about
the function values on unseen inputs. But if you have a non-uniform prior --
for example, you know that the function has to be in a sufficiently restricted
functional form -- then your data can give you useful information about unseen
cases.
As another example, suppose you have a sequence x_1, x_2, x_3, ... where each
x_i must be either H or T. If this is all you know about the sequence, knowing
x_1, ..., x_n gives you no information at all about x_(n+1). (Your prior is a
uniform distribution over (x_1,...,x_n,x_(n+1)).) But if you know that there is
some systematic, common influence affecting each of the x_i -- for example, you
have the probabilistic model
P(x_i=H | theta = t) = t;
P(theta = t) = f(t);
the x_i are independent given theta --
then knowing x_1,...,x_n can give you considerable information about x_(n+1)
(especially if n is large and (x_1+...+x_n)/n is near 0 or 1.)
Note that in both of these cases having some restricted parametric form -- even
if you are completely ignorant as to the values of its parameters -- is often
sufficient information to allow induction.
So the upshot is this: Hume is right in that if you are completely ignorant you
can't learn anything beyond memorizing data. But once you have some systematic
prior knowledge induction does become possible.
Since we as humans do engage in successful induction all throughout our lives
(learning language, learning to understand what we see and hear, etc.), we must
therefore be born with significant prior knowledge wired into our brains. The
interesting question now is, what exactly is that prior knowledge with which we
are born, and where does it come from?
-- Kevin S. Van Horn
