Oops, math error in stretch() in my last post. x = stretch(p) = ln(p/(1-p)). p = squash(x) = 1/(1+e^-x) = stretch^-1(x).
In zpaq I implement both functions as lookup tables using 12 bits for x in (-8, 8) and 15 bits for p (odd multiples of 2^-16 in (0, 1)). For small numbers it gives geometric averaging. Mixing 0.5 and 0.99 is about 0.9. The mixer weights start at 1/n for n inputs and are updated w += .001 x input x output error. Most of the compression comes from byte orders 0-8 and word orders 0-2 and a match model for long contexts that predicts the bit following the last match with p = (n-1)/n. The modeling can be done in parallel but is serialized at coding each bit: predict, code, update. On Wed, May 26, 2021, 2:16 AM <[email protected]> wrote: > On Wednesday, May 26, 2021, at 1:38 AM, Matt Mahoney wrote: > > or example suppose your bit counters were (1, 1) = 0.5 and (1, 100) = > 0.99. Then with equal weights PAQ6 predicts (2, 101) = 0.98. > > Equal weights you say, but I don't believe giving ex. orders0-16 the same > weight...some get ignored in the "judge court room"...do you mean something > else? > > I think because you only look at the last 5 letters (lol :p) that you are > able to give the last 5 contexts the same weight, cuz they are all pretty > sure, whereas orders 6 and up are immensely rare contexts. Right? > > Maybe a batch (pattern mechanism) would help, orders0-5 get the same > weight, orders6-8 get same own weight, and so on. ? > > This would also allow parallization I think too, not just less % > calculations. > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/T4ae6b3d15d72b22e-M12c0d320343a444585de0635> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T4ae6b3d15d72b22e-M4f20b235920bdafb2f07f305 Delivery options: https://agi.topicbox.com/groups/agi/subscription
