On 2/26/2012 5:58 AM, Evgenii Rudnyi wrote:
I have written a summary for the discussion in the subject:
http://blog.rudnyi.ru/2012/02/entropy-and-information.html
No doubt, this is my personal viewpoint. If you see that I have missed something, please
let me know.
I think you are ignoring the conceptual unification provided by information theory and
statistical mechanics. JANAF tables only consider the thermodynamic entropy, which is a
special case in which the macroscopic variables are temperature and pressure. You can't
look up the entropy of magnetization in the JANAF tables. Yet magnetization of small
domains is how information is stored on hard disks, c.f. Donald McKay's book "Information
Theory, Inference, and Learning Algorithm" chapter 31.
Did you actually read E. T. Jaynes 1957 paper in which he introduced the idea of basing
entropy in statistical mechanics (which you also seem to dislike) on information? He
wrote "The mere fact that the same mathematical expression -SUM[p_i log(p_i)] occurs in
both statistical mechanics and in information theory does not in itself establish a
connection between these fields. This can be done only by finding new viewpoints from
which the thermodynamic entropy and information-theory entropy appear as the same
/concept/." Then he goes on to show how the principle of maximum entropy can be used to
derive statistical mechanics. That it *can* be done in some other way, and was
historically as you assert, is not to the point. As an example of how the information
view of statistical mechanics extends its application he calculates how much the spins of
protons in water would be polarized by rotating the water at 36,000rpm. It seems you are
merely objecting to "new viewpoints" on the grounds that you can see all that you /want/
to see from the old viewpoint.
Your quotation of Arnheim, from his book on the theory of entropy in art, just shows his
confusion. The Shannon information, which is greatest when the system is most disordered
in some sense, does not imply that the most disordered message contains the greatest
information. The Shannon information is that information we receive when the *potential
messages* are most disordered. It's a property of an ensemble or a channel, not of a
particular message.
Brent
--
You received this message because you are subscribed to the Google Groups
"Everything List" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/everything-list?hl=en.
