Taking another look closely at IGPT, it seems indeed that other ANNs are better AND use less compute too. If you trained them on as much compute as OpenAI did, they'd be even better. And they work on larger images. However, I have yet to find a paper showing as-diverse (completes it with various new correct futures, and jungle, buildings, art, faces etc) image-completion results like theirs, I'm only finding fairly-good less-diverse in-painting, which is easier to get right. It's "possible" GPT-2 is better/more diverse completing. https://arxiv.org/pdf/1903.04227.pdf
Also, if these "other authors" cannot apply it to text to surpass GPT-2, then this means GPT-2/Transformers for text/image really is a more general understanding of AI. Even if it's the best general understanding, there could still remain some accelerator-ness to it i.e. if you want to use image data then change the architecture a bit to optimize for that. It is however not explained like i'm 5 and appears to do odd architectural choices, it is a blackbox as far as I'm concerned. ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T48eb73fe225c230b-M652c6830b74912c7787113d4 Delivery options: https://agi.topicbox.com/groups/agi/subscription
