On 18.02.2012 23:37 Russell Standish said the following:
On Sat, Feb 18, 2012 at 04:49:44PM +0100, Evgenii Rudnyi wrote:
On 09.02.2012 07:49 meekerdb said the following:

There's an interesting paper by Bennett that I ran across, which
discusses the relation of Shannon entropy, thermodynamic entropy,
and algorithmic entropy in the context of DNA and RNA



I have browsed the paper. It is nice indeed. A couple of comments.

1) Reversible computation

The author seems not to reject the idea of reversible computation.
This, in my view, shows that the first statement from the paper

"Computers may be thought of as engines for transforming free
energy into waste heat and mathematical work."

just does not work literally. If reversible computation is
possible, then we do not have any thermodynamic limits in this
respect. What is left is just a thermal noise in form of kT.

My understanding is that is possible to perform a reversible
computation with arbitrarily small amounts of energy provided you do
the computation slowly enough. The only way to do it for zero energy
expenditure is to not do it at all.

2) Maxwell demon

I have never understood a problem with the Maxwell's demon. Why it
is not enough to say that it does not exist? Why for example
Maxwell's demon touches the imagination of physicists and
engineers and the idea of the God not?

Its a thought experiment. Its quite well-defined (formalisable)
whereas the notion of God is not.

If one well defines a thought experiment with the Maxwell's demon, then it is quite clear that such thing does not exist. Why then to spend on it so much time?

3) Reversible chemical reactions and reversible thermodynamic

I think that the author misuses the term reversible in a sense
that the word has completely different meaning in thermodynamics
and in chemistry. In thermodynamics, the reversible process implies
that the entropy of the system and surrounding does not change
(the entropy of the Universe remains constant). In chemistry, a
term reversible reaction means we have two reactions (forward and
backward) running in parallel. Thereafter, by playing with
conditions we could transform A to B and then B back to A.
However, when a reversible chemical reaction takes place it is
impossible to implement it as a reversible thermodynamic process.
Hence a reversible chemical reaction is not thermodynamically

A reversible computation has the same meaning of reversible as in
thermodynamics. Change of entropy is zero. Information is conserved.
Reversible computations can never erase memory locations, for
instance, or implement assignment.

It is hard to say for sure what the author meant. Let me first quote him.

p. 912(8) "It is well known that all chemical reaction are in principle reversible: the same Brownian motion that accomplishes the forward reaction also sometimes brings product molecules together, pushes them backward through the transition state, and lets them emerge as reactant molecules."

p. 934(30) "As indicated before, the synthesis of RNA by RNA polymerase is a logically reversible copying operations, and under appropriate (nonphysiological) conditions, it could be carried out at an energy cost of less than kT per nucleotide."

My understanding was that in the first quote reversible has the meaning from chemistry. Let us consider for example a reaction

A = B

with the forward reaction rate of 1000 and the backward reaction rate of 1. Then we can imagine two different initial states

1) C(A) = 1, C(B) = 0
2) C(A) = 0, C(B) = 1

The equilibrium state will be the same, but we reach it from different sides. In both cases however the process will be thermodynamically irreversible.

My point was that one word has different meanings and it would be good to understand what has been meant.

4) Algorithmic entropy

I have missed the point on the connection between the algorithmic
entropy and thermodynamic entropy. Here would be good to be back
to the Jason's example from about his work on secure pseudo-random
number generators


What a thermodynamic system should be considered at all here?

In my view, the algorithm is independent of implementation
details. It seems that this is one of the points at this list when
people claim that it could be possible to make a conscious robot.
Yet, how then the thermodynamic entropy could be connected with
the algorithmic entropy?

That seems like a non-sequitur. Could you expand on your thinking

I have read once more the section 6 "Algorithmic entropy and thermodynamics" (p. 936 (30)) from the paper. I should confess that I do not know exactly what the author meant with the algorithmic entropy. My reading was

algorithmic entropy == entropy of an algorithm

and I have considered and will stick to this meaning.

In my understanding, when we consider an algorithm, this is a pure IT construct, that does not depend whether I will implement it with an abacus or some Turing machine, with Intel or PowerPC processor. From this follows that the algorithm and hence its entropy does not depend on temperature or pressure of a physical system that does the computation. In my view it makes sense.

Let us consider consciousness now. Our brains produces it and our brain has some thermodynamic entropy. If we assume that the same effect could be achieved with some robot, does it mean that the thermodynamic entropy of the robot must be the same as that of the brain?

5) DNA, RNA and information

I have recently read

Barbieri, M. (2007). Is the cell a semiotic system? In:
Introduction to Biosemiotics: The New Biological Synthesis. Eds.:
M. Barbieri, Springer: 179-208.

I'm afraid semiotics leaves me cold. I've never seen one useful
conjecture come out of it. Apologies to all those Pearceans out

Everything is in comparison. Recently I got interested in Artificial Life and people have recommended me Christoph Adami “Introduction to Artificial Life“. He for example claims

p. 5 "An even more general approach is the thermodynamic one, which attempts to define living systems in terms of their ability to maintain low levels of entropy, or disorder, only".

One could say something like this but then the question what is the entropy. And this is what Adami writes about the entropy

p. 94 “Entropy is a measure of the disorder present in a system, or alternatively, a measure of our lack of knowledge about this system.”

p. 96 “If an observer gains knowledge about the system and thus determines that a number of states that were previously deemed probable are in fact unlikely, the entropy of the system (which now has turned into a conditional entropy), is lowered, simply because the number of different possible states in the lower. (Note that such a change in uncertainty is usually due to a measurement).

p. 97 “Clearly, the entropy can also depend on what we consider “different”. For example, one may count states as different that differ by, at most, del_x in some observable x (for example, the color of a ball drawn from an ensemble of differently shaded balls in an urn). Such entropies are then called fine-grained (if del_x is small), or course-grained (if del_x is large) entropies.”

I am a thermodynamicist and frankly speaking I was just shocked after reading it. For me it was clear that Adami does not know what the experimental thermodynamics is (and presumably he is unaware of the experimental thermodynamics at all).

I understand now that Adami's viewpoint is quite common among physicists but I do not think that this brings us "useful conjecture come out of it". I had discussion about this on the biotaconv list, see summary at


but no one there could explain me what is the differences in consequences in artificial life research between the two statements

1) The thermodynamic and information entropies are equivalent.

2) The thermodynamic and information entropies are completely different.

It seems that either 1) or 2) does not influence artificial life research at all.

In this sense, I like the Barbieri's paper much more. At least I could follow his logic.


You received this message because you are subscribed to the Google Groups 
"Everything List" group.
To post to this group, send email to everything-list@googlegroups.com.
To unsubscribe from this group, send email to 
For more options, visit this group at 

Reply via email to