Jerry, Steven, John, Bob, lists,
I want to thank Jerry for bringing to my attention Miller's impressive
book, "Living Systems" [1], which I thought I had "thumbed through" once
but did not: I simply conflated it with another book.
Miller's book is the first biology book that I have seen so far that
provides an extensive discussion on the meaning of "information", which I
have collected below as Items (1) through (17). I agree with most of these
items except a few.
(A) In Item (9), Miller indicates that Schroedinger coined the term
"negentropy" but it was Brillouin who coined the word as an abbreviation of
Schroedinger's expression "negative entropy". As I pointed out in [2], the
concept of "negative entropy" violates the Third Law of Thermodynamics (but
that of "negative entropy change" does not).
(B) In Item (9), Miller assumes that the negentropy is the same as
information, which assumption being referred to as the Negentropy Principle
of Information (NPI) by Brillouin (1951, 1953, 1956). I refuted NPI in
[2], in agreement with Wicken [3] based on the thought experiment called
the "Bible test", which was designed to demonstrate the fundamental
difference between Shannon entropy, S_S (also called informational entropy,
S_I) and thermodynamic entropy, S_T: The S_T of the Bible increases with
temperature but S_I does not. See Item (28). Wicken's argument against NPI
is summarized in Items (18) through (27) extracted from [3].
(C) In Item (10), Miller assumes that the Second Law of Thermodynamics
stated as "a system tends to increase in entropy over time" applies to all
physical systems but it does not: It applies only to isolated systems which
cannot exchange any energy or matter with their environment and not to
closed (only energy can be exchanged; e.g., a refrigerator) or open systems
(both energy and matter can be exchanged; e.g., living cells). For example,
the thermodynamic entropy content of a living cell can decrease while its
Shannon entropy can increase when growing and differentiating.
(D) My conclusion would be that it is impossible to define the relation
between information and thermodynamic entropy without knowing whether the
thermodynamic system under consideration is open, closed or isolated. In
other words, the relation between *Shannon entropy* (also called
*information*) and *thermodynamic entropy* depends on the nature of
thermodynamic system involved.
(E) As I indicated in my previous email of today, I believe that
"information" is an irreducibly triadic relation (as shown in Figure 1
below) but "entropy" is a pat of the "free energy" that drives semiosis in
which "information" is processed. Free energy is a function of both energy
and thermodynamic entropy. In other words, "information" is the whole
system of 3 nodes and 3 arrows whereas entropy is a part of the "energy"
that drives the processes indicated by the 3 arrows.
f g
VARIETY ------------> MESSAGE -------------> FUNCTION
| ^
| |
|______________________________________|
h
f = selection
g = action
h = information flow
Figure 1. An "irreducibly triadic definition of information", an instance
of the irreducibly triadic semiosis and the sign.
(1) The technical sense of "information (H) . . . . is not the same thing
as meaning or quite the same as information as we usually understand it."
[1, p. 11].
(2) "Meaning is the significance of information to a system which possesses
it: it constitutes a change in that system's processes elicited by the
information, often resulting from associates made to it on previous
experience with it." [1, p. 11].
(3) Information is ". . . .the degree of freedom that exists in a given
situation to choose among signals, symbols, messages, or patterns to be
transmitted." [1, p. 11].
(4) The amount of information is measured as the logarithm to the base 2 of
the number of alternative patterns, forms, organizations, or messages."
[1, p. 11].
(5) The unit of information, bit, ". . .is the amount of information which
relieves the uncertainty when the outcomes of a situation with two equally
likely alternatives is known." [1, p. 11].
(6) "The term marker was used by von Neumann to refer to those observable
bundles, units, or changes of matter-energy whose patterning bears or
conveys the informational symbols from the ensemble or repertoire. These
might be the stones of Hammurabi's day which bore cuneiform writings,
parchments, writing paper, Indian,s smoke signals, a door key with notches,
punched cards, paper or magnetic tape, a computer's magnetized ferrite core
memory, an arrangement of nucleotides in a DNA molecule, the molecular
structure of a hormone, pulses on a telegraph wire, or waves emanating
from a radio station. If a marker can assume n different states of which
only one is present at any given time, it can represent at most log_2(n)
bits of information." [1, p. 11].
(7) "Communication of almost every sort requires that the marker move in
space, from the transmitting system to the receiving system, and this
movement follows the same physical laws as the movement of nay other sort
of matter-energy." [1, pp. 11].
(8) "The disorder, disorganization, lack of patterning, or randomness of
organization of a system is known as its entropy (S). This is the amount
of progress of a system from improbable to probable states. The unit in
which it is measured empirically is ergs or joules per degree absolute."
[1, p. 13].
(9) "It was noted by Wiener and by Shannon that the statistical measure for
the negative of entropy is the same as that for information, which
Schroedinger has called negentropy." [1, p. 13].
(10) "Since, according to the second law, a system tends to increase in
entropy over time, it must tend to decrease in negentropy or information."
[1, p. 13].
(11) "There is therefore no principle of the conservation of information
as there are principles of the conservation of matter and energy." [1, p.
13].
(12) "The total information can be decreased in any system without
increasing it elsewhere, but it cannot be increased without decreasing it
elsewhere." [1, p. 13].
(13) "Making one or more copies of a given information pattern does not
increase information overall, though it may increase the information
in the system which receives the copied information." [1, p. 13].
(14) "Matter-energy and information always flow together. Information is
always borne on a marker." [1, p. 13]
(15) "Conversely there is no regular movement in a system unless there is
a difference in potential between two points, which is negative entropy or
information." [1, p. 13]
(16) "Which aspect of the transmission is most important depends upon how
it is handled by the receiver. If the receiver responds primarily to the
material energetic aspect, I shall call it, for brevity, a matter-energy
transmission; it the response is primarily to the information, I shall
call it an information transmission. For example, the banana eaten by a
monkey is a nonradndom arrangement of specific molecules, and thus has its
informational aspect, but its use to the monkey is chiefly to increase the
energy available to him." [1, p. 13].
(17) Referring to Table 2-1 in his book, James Miller states "It indicates
that there are several pairs of antonyms used in the section, one member of
which is associated with the concept of information (H) and the other
member of which is associated with its negative, entropy (S)." [1, p. 13].
(18) "The concept of entropy has had a long and interesting history,
beginning with its implicit introduction by Carnot to its explicit
formalization as a state function by Clausius to its statistical treatment
by Boltzmann and Gibbs to its application to communications theory by
Shannon (Shannon and Weaver 1949). The latter achievement has seemed to
several scientists a true generalization of the entropy conception, its
freeing from the particular disciplinary framework of thermodynamics for
application to probability distributions generally (Gatlin 1972; Yockey
1977). This mistaken belief is a major impediment to productive
communication between thermodynamics and information theory, and we will
examine it carefully." [3, p. 177].
(19) "entropy is defined to express constraints on the directions natural
processes can take, according to the equation dS = dQ/T where dS is the
differential change in entropy resulting from an infinitesimal flow of heat
dQ at temperature T. In any spontaneous or irreversible process, entropy
always increases. Any concept claiming to be a generalization of classical
entropy would have to share this essential property." [3, p. 177].
(20) "Boltzmann gave thermodynamics its first, rough statistical
interpretation, introducing the microstate-macrostate distinction on which
the explanation of irreversibility hangs. Boltzmann used as a model a gas
with N particles and having a total kinetic energy of E (see Brush 1983).
This energy was divided into J discrete pieces and assigned to these N
particles in all possible combinations. Each combination constituted one of
W equiprobable microstates of the system. Given these assumptions,
Boltzmann was able to define his entropy function by the equation
H = k ln W or
H = -k ln P
where k is Boltzmann's constant and P = 1/W." [3, p. 177].
(21) "Natural, entropy producing processes are those that increase the
microscopic possibilities of the system-environment supersystem. A
nonequilibrium system can therefore be regarded as compressed in
probability space Wicken 1981), accessing only a small fraction of the
microstates available to it. Irreversible processes are expansions in
probability space, from macrostates having relatively few microscopic
complexions to macrostates having relatively many such complexions." [3, p.
178].
(22) "Microstates are only equiprobable if they have the same energy. For
systems that are open to energy exchanges with their environments, this is
not usually the case. A more general formula for the entropy of a system,
derived from Gibbs' systematic development of statistical thermodynamics,
is
H = - k E Pi ln Pi
where Pi refers to the energy-dependent probabilities of the various i
microstates. Under isolated conditions, or under conditions where kinetic
barriers to reaction keep the system in a single, nonequilibrium
macrostate, this reduces to Boltzmann's equation." [3, 178].
(23) "The Shannon equation is
H = - K E Pi log2 Pi,
where K is generally taken as unity. Since proportionality constants and
logarithm bases are more matters of convenience and scaling than of
substance, the relationships among the variables in the two equations are
identical. Gibbs circumspectly referred to his statistical formulations as
"entropy analogues" rather than "entropies" (Denbigh 1982). The question is
then whether the Shannon equation generalizes the entropy analogues of
statistical mechanics." [3, 178].
(24) "The answer to this question depends on whether Shannon entropies
have properties consistent with thermodynamic entropies. Issues relevant to
this question are: (a) Do both entropies behave the same way? and (b) are
they both based on the same kinds of probabilistic assumptions? Neither is
the case." [2, 178].
(25) "The starting point for the Shannon entropy is an alphabet of symbols
which have the capacity to convey information because they can be
transmitted in alternative sequences. The entire ensemble of possible
sequences can be abstractly assigned an "entropy", which measures the
uncertainty connected with knowing a priori the sequence of elements in any
given message. Shannon initially suggests (Shannon and Weaver 1949, p. 49)
that information is carved from that entropic space. This much can at least
be borne in a spirit of suspended judgment. But he slips immediately (p.
50) to assigning entropies to the symbols and messages themselves. Here he
permanently parts company with statistical entropy." [2, p. 180-181].
(26) "What allows us to assign a thermodynamic system an entropy is that
any measurable macrostate in which it resides can be expressed in a variety
of alternative microstates. Since these are all accessible by the system,
there is an essential uncertainty in knowing its microstate at any instant.
I would concur with Denbigh (1982) that there is nothing "subjective" about
this uncertainty, that it belongs to the macrostate by virtue of its
ensemble of microstates. A message, in contrast, cannot possess entropy. It
is what has been said, a *fait accompli*." [3, p. 181].
(27) "To appreciate the importance of restricting entropy to thermodynamic
applications -- or, more broadly to applications in which a
macrostate-microstate relationship obtains, one need only reflect on
Weaver's remarks about the Shannon formulation making contact with a
universal law. It does no such thing. Yet, as long as the term "entropy"
buttresses the Shannon formula, the second law remains a steady source of
justification for ideas that must find their own grounds of support. If
there is a single generalized entropy concept manifoldly expressing itself,
one might expect all exemplifications of it to have the property of
increasing in time " [3, p. 187].
(28) "When one heats up a book such as the Bible, the thermodynamic
entropy associated with molecular motions of the paper constituting the
pages of the Bible will increase but the informational entropy associated
with the arrangement of letters in the Bible will not be altered until the
temperature increases high enough to burn the Bible. This thought
experiment may be conveniently referred to as the Bible test." [4, Footnote
c on p. 100].
All the best.
Sung
--
Sungchul Ji, Ph.D.
Associate Professor of Pharmacology and Toxicology
Department of Pharmacology and Toxicology
Ernest Mario School of Pharmacy
Rutgers University
Piscataway, N.J. 08855
732-445-4701
www.conformon.net
References:
[1] Miller, J. G. (1978). Living Systems, McGraw-Hill Book Company,
New York.
[2] Ji, S. (2012). The Third Law of Thermodynamics and “Schroedinger’s
Paradox” <http://www.conformon.net/?attachment_id=1033>. In:*Molecular
Theory of the Living Cell: Concepts, Molecular Mechanisms, and Biomedical
Applications.* Springer, New York. pp. 12-15. PDF available at
http://www.conformon.net under Publications > Book Chapters.
[3] Wicken, J. S. (1985). Entropy and Information: Suggestions for
Common Language. Phil. Sci. 54: 176-193.
[4] Ji, S. (2012). The Information-Entropy Relations. In:*Molecular
Theory of the Living Cell: Concepts, Molecular Mechanisms, and Biomedical
Applications.* Springer, New York. pp. 97-101.
-----------------------------
PEIRCE-L subscribers: Click on "Reply List" or "Reply All" to REPLY ON PEIRCE-L
to this message. PEIRCE-L posts should go to [email protected] . To
UNSUBSCRIBE, send a message not to PEIRCE-L but to [email protected] with the
line "UNSubscribe PEIRCE-L" in the BODY of the message. More at
http://www.cspeirce.com/peirce-l/peirce-l.htm .
