This is an interesting phenomenon. Are you running checksums? On 19 Aug 2021 21:13, "Matt Mahoney" <[email protected]> wrote:
> I've run into that problem too, that text prediction degrades to repeating > characters. There are no English words that repeat the same character 3 > times in a row, but this pattern is still common in many files so we have > to model for it. > > On Wed, Aug 18, 2021, 5:08 PM <[email protected]> wrote: > >> BTW do you know why Alex and Byron have the only scores of 15MB but can't >> generate completions? Byron says his is complex and can't stop it from >> repeating itself. Byron does use others algorithms plus his own, so it's a >> freak a thing to start with. Alex, how big is his code in lines....many >> questions here.... He doesn't seem to share much about his code. I'll have >> to ask him why not or where is his text completions. As for NNCP which is >> basically GPT, it too reaches ~15MB, but apparently I think he can't >> generate easily because he much decode or something 16 bytes or something. >> This should really be corrected, it should be important like LC I feel. >> > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/Td13a829978c4c9f3-Mf8d35ac47fa75a3d4645d8d8> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Td13a829978c4c9f3-Mef52c6db9c8a86259e9379f9 Delivery options: https://agi.topicbox.com/groups/agi/subscription
