So I decided/ thought wait, my program that learns related words should store the relations for ex. each of the 50,000 vocab words to each the 50K vocab words, which comes to at least 1 billion relations, or 1GB.
Each relation being at least 1 or 2 bytes. While word2vec just stores embeds....Embeds are given to each of the 50K vocab words, so you only have 50K embeds. Each the 50K vocab words has in GPT2 I think 1024 dimension numbers, so that's 50,000 numbers each about maybe 3-4 bytes. So theirs, in RAM, maybe 150MB....mine would have been at least 1GB.... I'm thinking about just working with GPT...I need to learn it fast, like in 1 day, and make the code as small as 300 lines, even the ADAM. Currently it's approx. 6000 lines in C++. I hate, big code, absolutely hate it. All those files may help the modification process but in the end all that code means a LOT of scrolling etc, and you GET lost, IMO. IMO having files like int, and etc are in the way, so the folder could look nicer, but I see this in every project, everything is as dirty as can be. Also SOMEHOW DALL-E is also got the hole/ delay/ pattern of it abilities because that is the only way to recognize and complete the rest of a big car in an image if only had seen small car before, this would take a while to get right. It also seems to have the ghosting ability, like if you see a line (mirror/reflective wood/floor/anything) then that means the whatever is no left should be what is on the other side of the line/context in the middle, upside down, being the reflection. This doesn't mean my theory is lost. Everything in my view is in GPT, in fact embeds are better cuz they (I THINK) use less RAM, maybe they don't. And I know how to make GPT into AGI, mostly, and learnt a ton about the future and evolution. Simply my project is a lot of work and it is slower than the GOOGLE parade of thousands of openAI workers, so it is very intimidating. I may have semi-wasted a few years but hey Transformers and DALL-E just came out, so it was not easy to see just how good they are. It should have been easy for others to explain to me why adding more data improves AI, and Embeds, etc, but they did not, maybe because they thought their project was the successful way and new little how GPT works, or that I am useless and why help me with any more than a teaspoon of effort. Still very poor no one explains it clear, but everyone is that bad. ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Tc124b3d00b83e897-M50d76a3100eae385ff7d4530 Delivery options: https://agi.topicbox.com/groups/agi/subscription
