I should note that as it digests new data in 800GB it will end up throwing away
more more often as it already stores something similar to it already or doesn't
favorite what it read or didn't read it enough in the data to learn it. As
said, it can't store all of Earth data losslessly, only some; lossy. As it
trains on the given small 2GB dataset, it starts off saving on space by using
general patterns, it basically 'knows it all', hence performance is high after
training a man for 20 years but performance shrinks if you store every data of
Earth. Of course their paper says error lowers, rises, lowers, so it is clearly
'storing things' that aren't that re-used. It could be that as it learns a
higher level layer, it piles up cost to learn all the alphabet or all the
words, but then it just slashes all the rest of the training by pointing to its
stored nodes now. So rule of thumb is find as much diverse data as you can,
store all different_enough patterns to 'know it all' without storing the full
universe and don't store rare/ unfavorited,/ unrelated features and don't store
features that have few parents utilizing them except for higher layer nodes.
You won't know all data of Earth losslessly, only that your and Tom's cat drink
milk, only that you have blonde hair, only that there is 7 billion humans on
Earth. The low memory is all that's needed to understand the universe.
------------------------------------------
Artificial General Intelligence List: AGI
Permalink:
https://agi.topicbox.com/groups/agi/Tc1f2c133ae3e4762-Mdf9620b27c84a83edbaa6f9b
Delivery options: https://agi.topicbox.com/groups/agi/subscription
