Yeah it is a lot to wrap your head around, the data isn't really stored per se, it's trained on the text word-by-word and it tries to guess what word comes next after the previous word, and that shapes its parameters, the text itself is never stored per se, it just tries to answer your queries based on what it thinks the response is, the training data can't cover every possible niche topic because they havent fed it every bit of text there is, that's probably why it didnt understand a question around cathode poisoning. In some cases it does spit out articles largely verbatim, but that's undesired and unintended, and usually occurs when there isnt a lot of data on the subject (called overfitting).
On Tue, Jan 24, 2023, 2:42 PM gregebert <[email protected]> wrote: > I'm having a hard time grasping how a system could store and index so much > information locally. That in itself is daunting due to the sheer volume of > data. The other thing I'm curious about is how much of the response is > plagiarized/reworded from existing information, versus true AI (collated > from various sources, analyzed for consistency, and then cherry-picked for > the best info without being redundant or missing important tidbits). > > On Tuesday, January 24, 2023 at 11:03:01 AM UTC-8 wyager wrote: > >> All of GPT-3's world-knowledge is embedded into its parameters. Whatever >> it tells you is the equivalent of a human telling you something "from >> memory". It takes only a few seconds for it to respond to a prompt. >> >> Connecting AIs to external data sources is, I imagine, under active >> research, but none of the current LLMs use external data. >> >> > On Jan 24, 2023, at 13:24, gregebert <[email protected]> wrote: >> > >> > John - How long did it take for 'HAL' to compose the response ? That >> would give a lot of clues about how far-and-wide it searches for info. I'm >> guessing it took at least several minutes. >> > >> > -- > You received this message because you are subscribed to the Google Groups > "neonixie-l" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web, visit > https://groups.google.com/d/msgid/neonixie-l/360c505b-7b50-4230-879d-0a31b1bf6b85n%40googlegroups.com > <https://groups.google.com/d/msgid/neonixie-l/360c505b-7b50-4230-879d-0a31b1bf6b85n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "neonixie-l" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web, visit https://groups.google.com/d/msgid/neonixie-l/CAPDJQ6Cm-c6%2Bs0L8cxqOqbABSzd0LMWG3yGut2DnS5UkM9sd%3Dg%40mail.gmail.com.
