Deep Learning from what I know is many layers that result in certain outputs
based on prior layer outputs. In my network, I currently use a tree, no
branch-reconnections yet. The idea of branches is to store contexts ex. walk
wal wa w, so you can feed in wal and get k as output for context wal, and store
them all efficiently (overlapping contexts). I'm going to use branch
reconnections ("net-work") as lookaheads for hole and delay match searching ex.
the 'a' in the branch 'walking' knows what letters come after 'wa_P' (where P
is) regardless of what letter is at "_". This only needs to be done for early
layers where the trunk has many breadth to search through otherwise. Another
thing about tree branch reconnections is, a tree with no reconnections (tree,
not net) is same, because w>a>l>k is a branch that stores all context
conditions to get >Prediction, while a net-work is meant for parallelism and
lookaheads. For a branch with no order ex. ingwalk or walking or ginwlak, you
could also implement such as a branch and not as a network.
I still want to know what part of PRE-trained GPT/ BERT model/code looks far
back on the user prompt regardless of what is in the network, because in the
pre-trained model I tested I can feed it a new word like T5yo6G and say I told
my friends they too should buy the new > and then it predicts "T5yo6G". What
this is is predicting things seen recently at high levels, more if more recent
and seen more times (and more if more related or same word). This is not
training online, so what mechanism is predicting new things seen recently after
matching contexts? Is it simply just taking network output predictions logits
and upping them by simply scanning the last ex. 10,000 tokens? That's be a
simple fix, just like what I do. Though I do it before exponential functions
etc etc.
A test I tried shows it does it at end look:
*I sold the C8e0W4 to my friend, and when I went home I saw my own* MacbookPro
on my desk. I had become the best friend of my most trusty laptop.
So, my next laptop was the C8e0W4.
------------------------------------------
Artificial General Intelligence List: AGI
Permalink:
https://agi.topicbox.com/groups/agi/Tb5526c8a9151713b-M03ea2079c6555579e80f2af5
Delivery options: https://agi.topicbox.com/groups/agi/subscription
