Don't use either method. In PAQ6 I did weighted addition of counters. But in later versions I did weighted addition of probabilities in the logistic domain.
For example suppose your bit counters were (1, 1) = 0.5 and (1, 100) = 0.99. Then with equal weights PAQ6 predicts (2, 101) = 0.98. Later versions calculate squash(stretch(0.5) + stretch(0.99)) = 0.98, where squash(x) = 1/(1+e^-x)) and stretch(x) = ln(x)/ln(1-x) are inverses of each other. This is better because renormalization after counter overflow doesn't affect the weight of evidence. You still have the problem that a sequence like 00001 should not always make the same bit prediction as 10000. You solve this using indirect context models that map the sequence to a probability (initially 0.2) and adjust according to the next bit. On Tue, May 25, 2021, 8:15 PM <[email protected]> wrote: > In my code I turn counters for predictions into probabilities (%s), but do > it for EACH order/context used. Is that wrong? The 2nd calculation below > doesn't seem to work little own well and it's resulting set of predictions > doesn't exactly match really. Any idea to make it work so it is less > computations? > > 2, 6 <<< counters > 0.25%, 0.75% <<< probabilities > *0.25% <<< weight for this order > 0.0625, 0.1875 <<< result > + > 3, 6 > 0.33%, 0.66% > *0.75% > 0.2499999999975, 0.499999999995 > 0.3124999999975, 0.687499999995 <<< both results added together > > VERSUS cheaper way below: > > 2, 6 > *0.25 <<< doesn't do probabilities, only at last step, 1 time > 0.5, 1.5 > + > 3, 6 > *0.75 > 2.25, 4.5 > 2.75, 6 > 0.3142857142857143, 0.685714286 <<< probabilities > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/T4ae6b3d15d72b22e-Mdb156a30561bb93cef36c4d7> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T4ae6b3d15d72b22e-M473953cc1689661d09c9cdac Delivery options: https://agi.topicbox.com/groups/agi/subscription
