Re: [Fis] Info Theory
Hi there Stan SS: Info theory presumably applies to everything and anything. GR: It was never intended to apply to anything but communication instruments. That is sending English language down a pipe. GR: In my opinion it still only does, I cant get my head around how say information theory actually applies to direct human communication or organic sensing systems. All our sensing systems are energy transduction systems, once inside the individual it 's moved via Na/K pumps aided by ADP to ATP conversions to the brain all electrical, chemical energy. So in the environment it's just a sound (phonon) or light (photon) or chemical or heat energy where are the bits (information theory part) or markers. They are just not there. Unless this information is what underlies energy and is what makes up the rest of the universe including dark matter and dark energy. And is also what underlies the theory of Geometricdynamics.(Relativity theory).?? How so I would not know. Gavin On Sun, Jan 23, 2011 at 7:39 AM, Gavin Ritz garr...@xtra.co.nz wrote: Are you saying Karl that Information theory is the glue that binds energy and entropy production? or the the fabric behind these two concept? If so what is the bridging qualitative and quantitative propositions and formulae for this binding? It's quite something to say this, because one of the qualitative foundations of information theory is word frequency of English from Zipfs law. John Pierce (Information Theory) Regards Gavin From: joe.bren...@bluewin.ch joe.bren...@bluewin.ch To: karl.javors...@gmail.com; Pedro C. Marijuan pcmarijuan.i...@aragon.es; fis@listas.unizar.es Sent: Sat, 22 January, 2011 7:32:24 AM Subject: Re: [Fis] Info Theory Dear Karl, The assumption I would like to check that we share is that existence and energy are primitive and numbers something derived. When one moves from the quantum vacuum or singularity into the thermodynamic world, as soon as change occurs, something is no longer totally itself; there is something new along side of it in 4D space-time. The number of entities has increased, and this is the situation is the reality of which addition is the model. Iteration, which also occurs in reality, does the rest. If I understand you correctly, you feel that numbers, once available and manipulated in more complex ways, can model many other things, especially, of course, aspects of information. If a numerical perspective is convenient and even necessary for an understanding of nature, I would still like to know if it is sufficient. Are you able to capture, in your information theory, for example, the informational processes involved in: · emotions · creativity · anti-social behavior (rational and irrational) · complex political processes · your own theory? I think it would make for a more interesting and productive discussion if you were to tell us where your theory does NOT apply, rather than let us raise naïve objections to which you already have clear answers. I would like to know, for example, which of several possible approaches to the definition of a logical object are involved; at what point the limitations of machines become determining; and under what conditions one should seek to maximize (because valuable) heterogeneity as opposed to homogeneity. Very interesting discussions can then be envisaged at the “boundaries” between different approaches. Thank you and best wishes, Joseph Ursprüngliche Nachricht Von: karl.javors...@gmail.com Datum: 20.01.2011 21:03 An: Jerry Chandlerjerry_lr_chand...@mac.com, Joseph Brennerjoe.bren...@bluewin.ch, Pedro C. Marijuanpcmarijuan.i...@aragon.es Betreff: Info Theory Hope that the FIS server will eventuially accept this, too.. For you, individually: Information Theory: Let me answer the points raised so far: Joe Brenner: My hope is that this discussion will have a good deal to do with qualitative as well as quantitative aspects of information. Perhaps people should state clearly what the primary interests and objectives are of their remarks. Jerry Chandler: The unspoken premise of many discussants appears to me to be a view of information theory as a universal glue, a universal predicate, a universal code. The assertion is outspoken, explicit and apodictically declaratory: information theory IS a universal glue, a universal predicate, a universal code Yet, any effort to use quantum logic to describe inheritance requires the construction of semantic bridges between messages before the encoding occurs. The existence of such semantic links or connections is intrinsic to the logical premise or assertion lies in the encoding process, not the experimental science that generates the information. The concepts and procedures underlying quantum logic and inheritance root BOTH in a common concept of
Re: [Fis] Info Theory
Karl I cant fault your thinking, ( with some very important things you mention) but your comment about Shannon's view doesn't make a whole lot of sense to me, fire- or not fire is just one logical connective form. Just One aspect of proposition logic connective. Imperative Logic which is actually more pervasive in organic systems is still unaccounted by your comments below. In fact Imperative Logic (fire neuron now) is how organic systems communicate with itself and others. And totally unaccounted for in most models. Shannon's communication theory is about getting data down a pipe and reading it at the other end. Not really intended to be anything else. If you say Limits of Glue then Information Theory does not underlie entropy production (or energy) or the Non-equilibrium free energy (combination of the two), and in fact may only be one qualitative aspect of entropy production. Then cannot be used to explain the Reality of Nature. Regards Gavin From: karljavorszky karl.javors...@gmail.com To: Gavin Ritz garr...@xtra.co.nz Cc: joe.bren...@bluewin.ch; fis@listas.unizar.es; pcmarijuan.i...@aragon.es Sent: Mon, 24 January, 2011 1:58:28 AM Subject: Re: [Fis] Info Theory Limits of Glue Joe:...that existence and energy are primitive and numbers something derived. Yes of course. We know that Nature exists and has manifold properties. (Thomas Aquinas). We speak about our experiences with Nature. To make certain that we understand each other clearly, we use words with progressive degrees of formal meaning. The extreme of this is that we use the public language, i.e. numbers, - where no person has (should have) subjective connotations, and the denotations of the words are clear. The imagery built up by this method has the shortcomings that it is a very abstract, detached, idealised way of speaking about Nature. It has the advantages that we each know that we mean the same as we say in this model Nature is in a constant change as we refer to the fact that the Euclid spaces which give mass a localisation are derived from the concept that a reordering always takes place, no side of a logical argument having any innate, intrinsic claim of being more true than other aspects. It is a continuous reordering which brings forth the convoys of objects moving together (“strings”) and one of the readings yields coordinates in two perfectly rectangular spaces. So the basic principle is that it moves, as Heraclit said it should. What I say in normal, subjectively colored language is that space is actually two spaces which are merged into each other. The fabric of space is made up of the undecided logical (sub-)questions of the relevance of aspects. If it is more descriptive of a+b=c that 2a-3b is in such and such way more related to b-2a than to 2b-3a (just to mention an example), then space either constricts or expands or the strings going thru the truth points of this debate have to carry more fillings or less. The stuff must be somewhere. The 4D space you ask about is perfectly there, with strings attached, twice. Yes, physiology is a science of accounting and maintaining very strict limits. This is even more true of neurology. That we humans have funny ideas is built into the mechanism and can be seen e.g. on wolves, bears, apes as they play and chase imaginary prey (which is strictly speaking a hallucination). The translation sequenced-commutative is what we see in the DNA and in the functions of the brain. The electrical discharges which we call thoughts are sequenced and come from specific places, but are otherwise uniform. The cells fire or fire not. They have two logical states. This is the Shannon way of doing things. Then, interdependent with this, we have multiform material which is displaced. The fluids are only generally somewhere in the region, they can lose their place, and quite importantly they are of several varieties. The anti-Shannon idea is that there are more forms in Nature (which we can speak about in a formalized fashion) than this one and not this one. The model presented is not an explanation for everything and all. It is a tool to play with. We have 16 kinds of building blocks in two sets, black and white. We pair the blocks and order them. Then we reorder them again. We then discuss which pair goes with which other pairs together in a convoy. This appears at first sight very complicated but is extremely logical. The glue in question connecting and partly fusing concepts in our brain and between sciences and societies and among particles and galaxies is well pictured in the formal language by the strings that show the (possibly irrelevant) spatial coordinates of the convoys. It is not the accountant’s job to give names to amounts systematically under way and partly misplaced. It is the scientists’ prerogative to decide what they call a string, a field, a force, a molecule. Accounting processes connect points in Euclid spaces with extents. We
Re: [Fis] Info Theory
Hi there Loet If the information exchange is provided with meaning, then this is probably just a qualitative aspect of entropy production. However this is not Information Theory. The Relation between energy and information theory is 0.693kT (k is the Boltzmann constant and T is the temp in kelvin) Joules/bit this however is for a machine not a living organism. Knowledge transfer systems are Imperative Logic Systems hereto totally uncounted for. Regards Gavin Dear colleagues, It seems to me that the relation between information and energy provides the special case that the entropy is thermodynamic entropy. The relation is S = k(B) H. H is dimensionless, but S is not because k(B) adds the dimensionality of Joule/Kelvin. H can also be considered as probabilistic entropy. S is relevant in the case that the system of reference is the chemico-physical one based on collisions among particles. This level – the exchange of momenta and energy – is always involved in higher-order exchange processes, but the next-order ones emerge on top of the lower-order ones. For example, when specifically molecules are exchanged, life can emerge (Maturana). The self-organization may also reduce the uncertainty locally (“negentropy”). The system of reference, however, then is different from the chemico-physical one. The information exchange is provided with meaning. Things change dramatically when meaning can again be communicated because then models can be entertained at a more rapid speed than the underlying (that is, modeled) systems. The redundancy generation can then prevail over the entropy generation and a knowledge-based economy, for example, maintained. The discursive models proliferate options other than the ones which occurred historically. This cultural system incurs on the historical manifestations and thus counteracts upon their following of the entropy law. The social system, for example, can be based on other premises than the lower-order ones. For example, the “survival of the fittest” can be replaced by universal human rights. In other words: the specification of the system of reference provides the information exchanges with meaning. This meaning can again be communicated reflexively in the respective disciplines. The systems can be expected to gain in their capacity to process complexity insofar as these different layers become more nearly decomposable. This expansion spans the different dimensionalities and thus can be expected to enlarge the space for knowledge-based interventions. Best wishes, Loet LoetLeydesdorff Professor, University of Amsterdam AmsterdamSchoolof Communications Research (ASCoR), Kloveniersburgwal 48, 1012 CXAmsterdam. Tel.: +31-20- 525 6598; fax: +31-842239111 l...@leydesdorff.net ; http://www.leydesdorff.net/ From:fis-boun...@listas.unizar.es [mailto:fis-boun...@listas.unizar.es] On Behalf Of Gavin Ritz Sent: Monday, January 24, 2011 12:42 PM To: Stanley N Salthe Cc: fis@listas.unizar.es Subject: Re: [Fis] Info Theory Hi there Stan SS: Info theory presumably applies to everything and anything. GR: It was never intended to apply to anything but communication instruments. That is sending English language down a pipe. GR: In my opinion it still only does, I cant get my head around how say information theory actually applies to direct human communication or organic sensing systems. All our sensing systems are energy transduction systems, once inside the individual it 's moved via Na/K pumps aided by ADP to ATP conversions to the brain all electrical, chemical energy. So in the environment it's just a sound (phonon) or light (photon) or chemical or heat energy where are the bits (information theory part) or markers. They are just not there. Unless this information is what underlies energy and is what makes up the rest of the universe including dark matter and dark energy. And is also what underlies the theory of Geometricdynamics.(Relativity theory)..?? How so I would not know. Gavin On Sun, Jan 23, 2011 at 7:39 AM, Gavin Ritz garr...@xtra.co.nz wrote: Are you saying Karl that Information theory is the glue that binds energy and entropy production? or the the fabric behind these two concept? If so what is the bridging qualitative and quantitative propositions and formulae for this binding? It's quite something to say this, because one of the qualitative foundations of information theory is word frequency of English from Zipfs law. John Pierce (Information Theory) Regards Gavin From:joe.bren...@bluewin.ch joe.bren...@bluewin.ch To: karl.javors...@gmail.com; Pedro C. Marijuan pcmarijuan.i...@aragon.es; fis@listas.unizar.es Sent: Sat, 22 January, 2011 7:32:24 AM Subject: Re: [Fis] Info Theory Dear Karl, The assumption I would like to check that we share is that existence and energy are primitive and
Re: [Fis] Info Theory
At 02:05 PM 1/24/2011, Loet Leydesdorff wrote: Content-Type: multipart/alternative; boundary==_NextPart_000_00C9_01CBBBC7.684CC9E0 Content-Language: en-us Dear colleagues, It seems to me that the relation between information and energy provides the special case that the entropy is thermodynamic entropy. The relation is S = k(B) H. H is dimensionless, but S is not because k(B) adds the dimensionality of Joule/Kelvin. H can also be considered as probabilistic entropy. S is relevant in the case that the system of reference is the chemico-physical one based on collisions among particles. This level the exchange of momenta and energy is always involved in higher-order exchange processes, but the next-order ones emerge on top of the lower-order ones. I think that joules are energy, and temperature is energy per degrees of freedom. If you cancel out the energy part you get degrees of freedom, which is dimensionless (a number), and is also a good measure for things like information in physical terms. So I don't see a problem. W For example, when specifically molecules are exchanged, life can emerge (Maturana). The self-organization may also reduce the uncertainty locally (ânegentropyâ). The system of reference, however, then is different from the chemico-physical one. The information exchange is provided with meaning. Only because it is interpreted (biosemiotics). Things change dramatically when meaning can again be communicated because then models can be entertained at a more rapid speed than the underlying (that is, modeled) systems. The redundancy generation can then prevail over the entropy generation and a knowledge-based economy, for example, maintained. The discursive models proliferate options other than the ones which occurred historically. This cultural system incurs on the historical manifestations and thus counteracts upon their following of the entropy law. The social system, for example, can be based on other premises than the lower-order ones. For example, the âsurvival of the fittestâ can be replaced by universal human rights. In other words: the specification of the system of reference provides the information exchanges with meaning. Quite. This meaning can again be communicated reflexively in the respective disciplines. The systems can be expected to gain in their capacity to process complexity insofar as these different layers become more nearly decomposable. This expansion spans the different dimensionalities and thus can be expected to enlarge the space for knowledge-based interventions. Dimensionalities add degrees of freedom, and thus information capacity. So information capacity can emerge (or even be created) by the sort of process you mention. It is all physically grounded, though. One more thing: GR: That's thermodynamically impossible. Any organic system requires to convert and transduce energy so you may think it does no work but the relationship is like Ostwald's Ripening there is always an energy cost Sorry Gavin, but you are mistaken. The entropy budget is made at the expense of information loss in these cases. Incidentally, you are posting too often. The rules say two a week. This allows people to check sources, etc. Google is good, plus archives of the fis list. Schroedinger, What is Life? (1945). The connection is via negentropy, and then to biological information in the DNA (he called it a nonperiodic crystal). The 1929 Szillard, SZILARD L., Z. Phys., 53 (1929) 840-856, paper is in German. An English translation can be found in Leff and Rex, Maxwell's Demon (1990, Princeton University Press). It is generally regarded as the first explicit connection between information and physics. John Professor John Collier, Acting HoS and Acting Deputy HoS colli...@ukzn.ac.za Philosophy and Ethics, University of KwaZulu-Natal, Durban 4041 South Africa T: +27 (31) 260 3248 / 260 2292 F: +27 (31) 260 3031 http://collier.ukzn.ac.za/ ___ fis mailing list fis@listas.unizar.es https://webmail.unizar.es/cgi-bin/mailman/listinfo/fis
Re: [Fis] Info Theory
Dear Gavin, With all due respect, I would like to strongly disagree. I believe the communications format is an historical artifact that were best put aside. It unnecessarily constrains the application of information theory for many useful purposes. Most especially, the calculus that is built upon the Shannon formula has incredibly wide application. It is useful anywhere constraint enters the picture. Of course, as a thermodynamicist myself, I understand your desire for purity of definition. So if you were to insist on constraining information to communication, then something like Collier's enformation needs to be coined to handle the myriad of other productive ways that information theory can be of use to us. I'll warn you, however, it's going to be difficult to draw a clear line between information and enformation. :) The best, Bob - Robert E. Ulanowicz| Tel: +1-352-378-7355 Arthur R. Marshall Laboratory | FAX: +1-352-392-3704 Department of Biology | Emeritus, Chesapeake Biol. Lab Bartram Hall 110 | University of Maryland University of Florida | Email u...@cbl.umces.edu Gainesville, FL 32611-8525 USA | Web http://www.cbl.umces.edu/~ulan -- Quoting Gavin Ritz garr...@xtra.co.nz: Hi there Stan SS: Info theory presumably applies to everything and anything. GR: It was never intended to apply to anything but communication instruments. That is sending English language down a pipe. GR: In my opinion it still only does, I cant get my head around how say information theory actually applies to direct human communication or organic sensing systems. All our sensing systems are energy transduction systems, once inside the individual it 's moved via Na/K pumps aided by ADP to ATP conversions to the brain all electrical, chemical energy. So in the environment it's just a sound (phonon) or light (photon) or chemical or heat energy where are the bits (information theory part) or markers. They are just not there. Unless this information is what underlies energy and is what makes up the rest of the universe including dark matter and dark energy. And is also what underlies the theory of Geometricdynamics.(Relativity theory).?? How so I would not know. Gavin On Sun, Jan 23, 2011 at 7:39 AM, Gavin Ritz garr...@xtra.co.nz wrote: Are you saying Karl that Information theory is the glue that binds energy and entropy production? or the the fabric behind these two concept? If so what is the bridging qualitative and quantitative propositions and formulae for this binding? It's quite something to say this, because one of the qualitative foundations of information theory is word frequency of English from Zipfs law. John Pierce (Information Theory) Regards Gavin From: joe.bren...@bluewin.ch joe.bren...@bluewin.ch To: karl.javors...@gmail.com; Pedro C. Marijuan pcmarijuan.i...@aragon.es; fis@listas.unizar.es Sent: Sat, 22 January, 2011 7:32:24 AM Subject: Re: [Fis] Info Theory Dear Karl, The assumption I would like to check that we share is that existence and energy are primitive and numbers something derived. When one moves from the quantum vacuum or singularity into the thermodynamic world, as soon as change occurs, something is no longer totally itself; there is something new along side of it in 4D space-time. The number of entities has increased, and this is the situation is the reality of which addition is the model. Iteration, which also occurs in reality, does the rest. If I understand you correctly, you feel that numbers, once available and manipulated in more complex ways, can model many other things, especially, of course, aspects of information. If a numerical perspective is convenient and even necessary for an understanding of nature, I would still like to know if it is sufficient. Are you able to capture, in your information theory, for example, the informational processes involved in: · emotions · creativity · anti-social behavior (rational and irrational) · complex political processes · your own theory? I think it would make for a more interesting and productive discussion if you were to tell us where your theory does NOT apply, rather than let us raise naïve objections to which you already have clear answers. I would like to know, for example, which of several possible approaches to the definition of a logical object are involved; at what point the limitations of machines become determining; and under what conditions one should seek to maximize (because valuable) heterogeneity as opposed to homogeneity. Very interesting discussions can then be envisaged
[Fis] Foundational Views of Shannon Information Theory
List: My responses to recent posts by Karl, Stan, Joe, Loet, Gavin, John, and Bob by the number of the digest that I rec’d. I seek to address several basic issues. First, I would repeat my assertion from my post of Jan. 20, 2010, along with Karl’s denial and my comments about his denials: JLRC: The unspoken premise of many discussants appears to me to be a view of information theory as a universal glue, a universal predicate, a universal code. KJ: The assertion is outspoken, explicit and apodictically declaratory: information theory IS a universal glue, a universal predicate, a universal code Karl: Out-spoken? JLRC: Yes, I spoke-out. :-) Karl: Explicit? JLRC: Yes. Rosen argues that biology requires a separate symbol system, that is outside of mathematical category theory. My explicit response to the category theory approach to information theory is contained in three recent papers – Axiomathes, Discrete Applied Math, and a chapter in a book by Vrobel and Otto Rossler. If desired, I will forward copies of these papers to list members. Karl: apodictically declaratory: JLRC: Yes! By design. ;-) JLRC: Perhaps you have not considered the reasons why Shannon information lacks universality. So, Precisely what is it that you are denying about the appearances of information theory? · That category theory is applicable to biology? · By inference, that set theory / predicate logic is sufficient to describe optical isomers? · That the simple “yes/No” choice essential to Shannon information is a universal code for human knowledge? (The notion of a binary encoding of all information is denied by Dalton’s premise – the ostensive source of chemical codes.) · Or, is it that you believe that addition is a universal operation of mathematics? JLRC: Your numerous posts on your decade-long mediations on the nature of arithmetic remain unpersuasive. The consistency of group theory and ring theory provide an adequate explanation for all iterative arithmetic operations. Your persistence is admirable, your intelligence is substantial, your logic questionable and your conclusions lack extension. Stan (545:10) Re: [Fis] Ostension and the Chemical / Molecular Biological Science, …It is this translation from material observations into logical form, in particular into fully explicit, crisp logical form that I am questioning. Yes, it can lead to short term triumphs, via engineering,… JLRC: Hu, I think you miss the point. The abstract symbol systems of Dalton, Lavoisier, and Coulomb underly the foundations of thermodynamics as well as the Shannon theory of information as well as our concept of such abstractions as “energy” and “entropy.” These symbol systems are now firmly embedded in the logic of scientific communications. Perhaps you wish to infer that concept of ostension is not useful in the natural sciences? Or, is it that in your world view, “utility” is a bad word? BTW, Lavoisier / Daltonian logical forms are not fully explicit in the usual sense of mathematics. They are closer to codes with an exact syntax. Joe (245:11) …”that existence and energy are primitive and numbers something derived.” JLRC: Are you putting the cart before the horse? As a consequence of the international system of units, number takes priority over all other scientific and economic proper names. Number is the antecedent to expressing quantity of most any sort. JLRC: Are you attempting to substitute semantics for syntax in your view of information theory? In your view of symbolic logic? In your view of the concept of order? JLRC: The order of the atomic numbers of the chemical elements stands in one:one correspondence with the any list of objects, with the listing of elements of a group, in a listing of the roots of a polynomial, in a listing of a vector, in a listing of the nodes of a graphs, and so forth. The existence of a listing is essential to the basic attributes of a message. It is essential to communication. Joe (245:11) …and under what conditions one should seek to maximize (because valuable) heterogeneity as opposed to homogeneity. JLRC: What fundamental classes of informational variables can be used to express heterogeneity? Homogeneity? How do such classes relate to Rosen’s postulates of separate and distinct symbol systems? Or, Aristotelian causal structures? Or, Descarte’s “clear and distinct” ideas Gavin (245:12) …one of the qualitative foundations of information theory is word frequency of English from Zipfs law. JLRC: Minor technical point. Perhaps you mean the frequency of usage of different alphabet symbols in a linguistic message? Stan (245:12) … Put otherwise, does anyone know of data about natural things that would not deliver a power law? JLRC: Power laws are the exception, not the rule in the natural sciences. For example, catalysis, the source of nearly all of biological
