I've run into that problem too, that text prediction degrades to repeating characters. There are no English words that repeat the same character 3 times in a row, but this pattern is still common in many files so we have to model for it.
On Wed, Aug 18, 2021, 5:08 PM <[email protected]> wrote: > BTW do you know why Alex and Byron have the only scores of 15MB but can't > generate completions? Byron says his is complex and can't stop it from > repeating itself. Byron does use others algorithms plus his own, so it's a > freak a thing to start with. Alex, how big is his code in lines....many > questions here.... He doesn't seem to share much about his code. I'll have > to ask him why not or where is his text completions. As for NNCP which is > basically GPT, it too reaches ~15MB, but apparently I think he can't > generate easily because he much decode or something 16 bytes or something. > This should really be corrected, it should be important like LC I feel. > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/Td13a829978c4c9f3-M92ddac36cb461eb515b7f084> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Td13a829978c4c9f3-Mf8d35ac47fa75a3d4645d8d8 Delivery options: https://agi.topicbox.com/groups/agi/subscription
