Re: Entropy and information

```Dear Brent,

```
First thank you for your time. Your answers as well as answers of other participants has helped me to understand better the opposite viewpoint and better to organize my thoughts.
```
```
You are right that the best is to study a good book but first I have already a stack of books to read and second reading a book alone is usually boring. I like much more to clear a question in a discussion. It is more enjoyable.
```
```
What I will probably do is read more about the history development with Maxwell's demon that John has mentioned. Somehow I have missed it.
```
```
I believe that it is normal when we have different opinions but I hope that my emails has helped you to understand the opposite viewpoint better.
```
```
As for engineers and physicists, I do not know. Let us take for example a landfill. There are too many of them in the modern society and engineers are trying to find a solution to reuse waste. An open question: Does your statement that all physical processes are not irreversible will help engineers to find a better solution?
```
```
Finally the JANAF Tables assume that the magnetic field is zero. If it is not, then one has to add a corresponding term.
```
Best wishes,

Evgenii

On 27.02.2012 20:43 meekerdb said the following:
```
```On 2/27/2012 10:59 AM, Evgenii Rudnyi wrote:
```
```On 27.02.2012 00:13 meekerdb said the following:
```
```On 2/26/2012 5:58 AM, Evgenii Rudnyi wrote:
```
```I have written a summary for the discussion in the subject:

http://blog.rudnyi.ru/2012/02/entropy-and-information.html

No doubt, this is my personal viewpoint. If you see that I have
missed something, please let me know.
```
```
I think you are ignoring the conceptual unification provided by
information theory and statistical mechanics. JANAF tables only
consider the thermodynamic entropy, which is a special case in which
the macroscopic variables are temperature and pressure. You can't
look up the entropy of magnetization in the JANAF tables.
```
```
I do not get your point. JANAF Tables have been created to solve a
particular problem. If you need change in concentration, surface
effects, magnetization effects, you have to extend the JANAF Tables.
And this has been to solve particular problems. Experimental
thermodynamics is not limited to JANAF Tables. For example, the
databases in Thermocalc already include dependence on concentration.
```
```
And you don't get my point. Of course all forms of entropy can be
measured and tabulated, but the information theory viewpoint shows how
they are unified by the same concept.

```
```
```
```Yet
magnetization of small domains is how information is stored on hard
disks, c.f. Donald McKay's book "Information Theory, Inference, and
Learning Algorithm" chapter 31.
```
```
Do you mean that when we consider magnetization, then the entropy
become subjective, context-dependent, and it will be finally filled
with information?
```
```
It is context dependent in that we consider the magnetization. What does
the JANAF table assume about the magnetization of the materials it
tabulates?

```
```
```
```Did you actually read E. T. Jaynes 1957 paper in which he introduced
the idea of basing entropy in statistical mechanics (which you also
seem to dislike) on information? He wrote "The mere fact that the
same mathematical expression -SUM[p_i log(p_i)] occurs in both
statistical mechanics and in information theory does not in itself
establish a connection between these fields. This can be done only by
finding new viewpoints from which the thermodynamic entropy and
information-theory entropy appear as the same /concept/." Then he
```
```
I have missed this quote, I have to add it. In general, the first
Jaynes's paper is in a way reasonable. I wanted to better understand
it, as I like maximum likelihood, I have been using it in my own
research a lot. However, when I have read in Jaynes's second paper the
following (two quotes below), I gave up.

“With such an interpretation the expression “irreversible process”
represents a semantic confusion; it is not the physical process that
is irreversible, but rather our ability to follow it. The second law
of thermodynamics then becomes merely the statement that although our
information as to the state of a system may be lost in a variety of
ways, the only way in which it can be gained is by carrying out
further measurements.”

“It is important to realize that the tendency of entropy to increase is
not a consequence of the laws of physics as such, … . An entropy
increase may occur unavoidably, due to our incomplete knowledge of the
forces acting on a system, or it may be entirely voluntary act on our
part.”

This I do not understand. Do you agree with these two quotes? If yes,
could you please explain, what he means?
```
```
Yes. The physical processes are not irreversible. The fundamental
physical laws are time reversible. The free-expansion of a gas is
*statistically* irreversible because we cannot follow the individual
molecules and their correlations, so when we consider only the
macroscopic variables of pressure, density, temperature,... it seems
irreversible. In very simple systems we might be able to actually follow
the microscopic evolution of the state, but we can choose to ignore it
and calculate the entropy increase as though this information were lost.
Whether and how the information is lost is the crux of the measurement
problem in QM. Almost everyone on this list assumes Everett's multiple
worlds interpretation in which the information is not lost but is
divided up among different continuations of the observer.

```
```
```
```goes on to show how the principle of maximum entropy can be used to
derive statistical mechanics. That it *can* be done in some other
way, and was historically as you assert, is not to the point. As an
example of how the information view of statistical mechanics extends
its application he calculates how much the spins of protons in water
would be polarized by rotating the water at 36,000rpm. It seems you
are merely objecting to "new viewpoints" on the grounds that you can
see all that you /want/ to see from the old viewpoint.
```
```
```
```Your quotation of Arnheim, from his book on the theory of entropy in
art, just shows his confusion. The Shannon information, which is
greatest when the system is most disordered in some sense, does not
imply that the most disordered message contains the greatest
information. The Shannon information is that information we receive
when the *potential messages* are most disordered. It's a property of
an ensemble or a channel, not of a particular message.
```
```
It is not a confusion of Arnheim. His book is quite good.
```
```
Then why doesn't he know that Shannon's information does not refer to
particular messages?

```
```To this end, let me quote your second sentence in your message.

> I think you are ignoring the conceptual unification provided by
> information theory and statistical mechanics.

You see, I would love to understand the conceptual unification.
```
```
I find that doubt this. If you really loved to understand it there is
lots of material online as well as good books which Russell and I have
suggested.

```
```To this end, I have created many simple problems to understand this
better. Unfortunately you do not want to discuss them, you just saying
general words but you do not want to apply it to my simple practical
problems. Hence it is hard for me to understand you.

If to speak about confusion, just one example. You tell that the
seems to be unwilling to employ this knowledge in practice. Why is
that? Why engineers seem not to be impressed by the conceptual
unification?
```
```
I'm not an expert on this subject and it has been forty years since I
studied statistical mechanics, which is why I prefer to refer you to
experts. Engineers are generally not impressed with conceptual
unification; they are interested in what can be most easily and reliably
applied. RF engineers generally don't care that EM waves are really
photons. Structural engineers don't care about interatomic forces, they
just look yield strength in tables. Engineers are not at all concerned
with 'in principle' processes which can only be realized in carefully
contrived laboratory experiments. But finding unifying principles is the
job of physicists.

Brent

```
```
--
You received this message because you are subscribed to the Google Groups
"Everything List" group.
To post to this group, send email to everything-list@googlegroups.com.
To unsubscribe from this group, send email to