I could be wrong but I thought Transformers have layers of self/attention, so like it clarifies what ex 'it' is (the cat) and then in the next pass / layer it uses that to figure out what 'the thing i just mentioned' refers to (it (the cat)).
? ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T22ce813ce07d9b1a-M450271af8965958352874108 Delivery options: https://agi.topicbox.com/groups/agi/subscription
