I tried encode.su many times but shelwien is reading my post like chinese bro haha even after being clear, I better ask you for once ok.
http://mattmahoney.net/dc/dce.html#Section_58 See where you say: "They use partial update exclusion. When a character is counted in some context, it is counted with a weight of 1/2 in the next lower order context. Also, when computing symbol probabilities, it performs a weighted averaging with the predictions of the lower order context, with the weight of the lower order context inversely proportional to the number of different higher order contexts of which it is a suffix." Please let me explain. What I'm talking about is the idea of windowing the prompt ex. [i[ [w[a[s[ [e[a[t[i[n[g[ [a[ ]]]]]]]]]]]]]]] and when you window shorter views you get lots more stats right, so, some of the ones in the longer higher order are IN the shorter ones, just a few, but still, so I think others are saying to give these few "sames" in the lower order windows 0.5% the weight since we already seen the predictions from longer windows on the prompt. BUT, if we apply global weights to orders - or handle the weighting using an automatic criteria instead of manual static mix weights, we need not remove 0.5% weight on shorter windows's same predictions if give the higher order less weight correctly in the first place. Seeing this, I don't have to code this idea, since it is the same thing... For example what I do in mine is if I search for the last 17 letters and get only 3 predictions (it doesn't know other letters can follow yet), the roof for counts needed is lower for example, hence, if i have just 66 counts and 3 unique predictions, I may have captured the distribution - perhaps only 3 different kinds of predictions come next! The rest are so rare it's not required to see them. Maybe we are confident here. Hence, I know how much weight to give the 16th order predictions. It's all about order 16, not order 16's that appear IN order 15 counts predicted, we simply finish it all in order 16. Right? ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Ta4523bc1ad2012c8-M35a1e220a64919da7c58a67a Delivery options: https://agi.topicbox.com/groups/agi/subscription
