On 13 Mar 2016, at 20:29, Dan wrote:
I would be interested in any proof relating a formal link between
the variational principle of minimum Fisher information and the
incompleteness theorems.
I think of Fisher information as a measure of how much any one
particular channel of information transmission can possibly describe
some source phenomenon. In a sense, the Cramer-Rao bound provides a
theoretical limit to how much can be known about the source
phenomenon using any one particular information channel. You see why
this makes me think of Goedel's theorems?
I am not sure. Gödel's incompleteness is very general and says that
machines or theories which are ideally correct (or just consistent)
are unable to prove some truth about themselves, like their own
consistency. The key along that type of incompleteness is the self-
reference phenomenon, with the intensional (modal) view, or without
(it shows no machine can prove all true sentences in arithmetic).
May be look at this: "ASYMPTOTIC DENSITY AND COMPUTABLY ENUMERABLE
SETS" by Downey, Jockusch Schupp.
http://www-old.newton.ac.uk/preprints/NI12039.pdf
The idea is to approximate a recursively enumerable (semi-computable)
set A by a computable (recursive) set B such that A \ B has a low
density, for various notion of density. This use tools coming from
self-reference. That might inspires you.
But like I said, this is more elated to simplicity/immunity (Post)
than with creativity/productivity (Gödel)
Theories or machine's beliefs are creative, that is recursively
enumerable (re), with a complement which is not recursively
enumerable, but in some constructive sense: there is a computable
function f which for any re set w_i included in the complement can
produced a "counterexemple" f(i) belonging to the complement but not
in w_i. Such set are called productive, and a set is creative (which
is the set view of a Turing Universal machine, by a result by Myhill)
if it is re and its complement is productive.
A set A is simple if it is RE and its complement is Immune, which
means that there is no RE subsets in the complement. All the w_i
attempting to enumerate the complement will cross the set A. This
notion can be related to Chaitin incompleteness result, and is more
information theoretic like.
You need to study a bit of recursion theory, if you want make a clear
relationship between Fisher theorem, and computability.
Any one formal axiomatic system or language which is non-
contradictory cannot prove all truths about something.
In the same way
That is what you need to show.
that multi-sensor fusion works around the limitation imposed by
Cramer-Rao bound (by increasing Fisher information... how much can
be known about the source), multiple programming languages can be
combined to prove more truths than any one language alone.
Programming languages do not prove. Theories and theorem provers prove
things, but some set of numbers like the set of true arithmetical
proposition cannot be enumerated by any machine, nor machine with
halting oracle, of totality oracle, etc. Chaitin form of
incompleteness is different. It is that a machine cannot prove the
randomness of a string more complex than itself. Mathematically it is
related to Post simple/immune notions.
Adding new axioms (undecidable sentences) augment the range of
provability. In the theoretical abductive inference it is different,
by a non union theorem it can be shown that for natural identification
criterion two inductive machine are more powerful than one. So again,
you might also study "theoretical learning learning". Google on this
and name like Gold, Blum, Case & Smith, Royer, etc. There is the book
by Weinstein, Stob,
Fisher theorem seems to me far away of both Gödel, and of Post-
Chaitin, but nearer to Post and Chaitin than to Gödel. You might
search on the name Cristian Calude who wrote interesting papers around
this, including a good book.
Ah, I found my exemplar: Cristian Calude, information and randomness,
an Algorithmic Perspective. A Springer book, 1994.
Bruno
--
You received this message because you are subscribed to the Google
Groups "Everything List" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/everything-list.
For more options, visit https://groups.google.com/d/optout.
http://iridia.ulb.ac.be/~marchal/
--
You received this message because you are subscribed to the Google Groups
"Everything List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/everything-list.
For more options, visit https://groups.google.com/d/optout.
