They might understand you better on encode.su if you post some short pieces of code. I assume you are talking about ppmd. What Shkarin probably did was experiment with different ways of mixing statistics to get the best compression. Theory is nice but you need to test it. I didn't read the code but saw it explained there. I forgot by who, probably Shelwein.
On Wed, Mar 10, 2021, 7:57 PM <[email protected]> wrote: > I tried encode.su many times but shelwien is reading my post like chinese > bro haha even after being clear, I better ask you for once ok. > > http://mattmahoney.net/dc/dce.html#Section_58 > > See where you say: > "They use partial update exclusion. When a character is counted in some > context, it is counted with a weight of 1/2 in the next lower order > context. Also, when computing symbol probabilities, it performs a weighted > averaging with the predictions of the lower order context, with the weight > of the lower order context inversely proportional to the number of > different higher order contexts of which it is a suffix." > > Please let me explain. What I'm talking about is the idea of windowing the > prompt ex. [i[ [w[a[s[ [e[a[t[i[n[g[ [a[ ]]]]]]]]]]]]]]] and when you > window shorter views you get lots more stats right, so, some of the ones in > the longer higher order are IN the shorter ones, just a few, but still, so > I think others are saying to give these few "sames" in the lower order > windows 0.5% the weight since we already seen the predictions from longer > windows on the prompt. BUT, if we apply global weights to orders - or > handle the weighting using an automatic criteria instead of manual static > mix weights, we need not remove 0.5% weight on shorter windows's same > predictions if give the higher order less weight correctly in the first > place. Seeing this, I don't have to code this idea, since it is the same > thing... > > For example what I do in mine is if I search for the last 17 letters and > get only 3 predictions (it doesn't know other letters can follow yet), the > roof for counts needed is lower for example, hence, if i have just 66 > counts and 3 unique predictions, I may have captured the distribution - > perhaps only 3 different kinds of predictions come next! The rest are so > rare it's not required to see them. Maybe we are confident here. Hence, I > know how much weight to give the 16th order predictions. It's all about > order 16, not order 16's that appear IN order 15 counts predicted, we > simply finish it all in order 16. > > Right? > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + delivery > options <https://agi.topicbox.com/groups/agi/subscription> Permalink > <https://agi.topicbox.com/groups/agi/Ta4523bc1ad2012c8-M35a1e220a64919da7c58a67a> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Ta4523bc1ad2012c8-M4349004479aa99a21d6e4cb5 Delivery options: https://agi.topicbox.com/groups/agi/subscription
