On 09/08/2015, Sampo Syreeni <de...@iki.fi> wrote: > > In limine you could have *any* signal *at all*, but always sent with > 100% certainty: when you saw it, its surprisal would be precisely zero. > That's the trivial case of the underlying probability space for the > entire signal being composed of a single message with probability 1, > *whatever* that one signal might be.
Exactly. And in *all* other cases except that this very single corner case, "surprisal" is nonzero, hence entropy is nonzero (whatever the probabilities). Why do you guys keep repeating this single corner case of 1 signal with 100% probability having a Shannonian probabilistic entropy of zero? Yes, as discussed ad nauseum, in that case, (Shannon) entropy is zero, as it is a special corner case. Has been discussed several times. And in virtually *all* other cases, surprisal is nonzero, hence Shannon entropy is *nonzero*, irregardless of the actual probabilities of the ensemble. It would be entirely pointless to repeat it again, I showed this in various ways. (And algorithmic entropy is always nonzero, even when the Shannon entropy is zero. To understand that, you need to understand algorithmic entropy.) > Peter's problem then seems to be that he doesn't specify that underlying > probability space nor state his assumptions fully. You do not need to specify the probability space 'fully' to know that when you have a probabilistic ensemble of at least two different 'words' (as you call them) from the symbol space with probability different from one, then the Shannon entropy of a message (average information content per message in bits) will be nonzero, without exception. It comes straight from Shannon's entropy formula. Since I also showed this in detail with formulas, there's no point in repeating it like the 3rd time. If you don't understand it, read Shannon's paper. If you cannot apply logic to a simple mathematical formula, that is not my problem. > But between the line > he assumes that signals coming from an entirely different kind of, far > more correlated-between-his-chosen-partition were an equal option. I never assumed that. Your reading sounds flawed. Maybe first re-read what I wrote. Best regards, Peter S _______________________________________________ music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
