I'm going with a vocab size of 50,000 like GPT-2 had.
How many relational connections should it learn? 50,000*50,000 is a lot of rare words trying to relate to other rare words. 5K*5K? ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T01fa5e447808d368-M93a40b837a74a106e544270b Delivery options: https://agi.topicbox.com/groups/agi/subscription
