Follows is everything I got out of that long eyed post: *"This is really, really weird in terms of the way human mind approach arithmetic, right? For a human who knows how to do 2-3 digit arithmetic, the error rate at 4-5 digit arithmetic — when given time and motivation for doing the arithmetic problems — is going to be either 0% or very close to 0%, or else way closer to 100%. Once a human learns the basic algorithms of arithmetic, they can apply them at any size, unless they make sloppy errors or just run out of patience."* Thank you. I agree. BTW when humans don't know the answer, they use small things they do know to carry over numbers and solve bigger equations. This requires a Task Pattern learnt. GPT learns Task Patterns, not sure why it didn't here :) I think nowhere in the dataset was how to do arithmetic and/or nor the correct mechanisms to carry out sequences of complex tasks.
You mention: Q: Reverse the following array: [1, 3, 5, 6, 10, 4, 2, 77] A: [10, 6, 4, 2, 77, 3, 5, 1] and Q: How many eyes does my foot have? A: Your foot has two eyes. Again, this is just Tasks. The latter question is a simple pattern question thing that is a sad fault, this can be fixed by some simple trick surely. "Given all the ridiculous wastes of resources in modern society, it’s hard to get too outraged at the funds spent on GPT3" Totally agree. "if one focuses on the fairly limited pool of resources currently being spent on advanced AI systems without direct commercial application, one wonders whether we’d be better off to focus more of this pool on fundamental innovations in representation, architecture, learning, creativity, empathy and human-computer interaction, rather than on scaling up transformers bigger and bigger." This is true. And we can see the accuracy curve, we knew 100x bigger gpt2 would result in little improvement, why they do this. They're selling an API is why maybe. However GPT-3 definitely is close-ish to AGI, many of the mechanisms under the illusive hood are AGI mechanisms. Like turtle > man, the limbs are there, the eyes, the head, the but, the spine, the lungs, just doesn't look like man so much....but it's so frikin close! ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T21c073d3fe3faef0-Mda7392e1ee037e7902f6672d Delivery options: https://agi.topicbox.com/groups/agi/subscription
