Nei LLM, il concetto di "emergent ability” ha una definizione precisa:
An ability that is “not present in small models but is present in large
models.”
https://www.jasonwei.net/blog/emergence
Non mi pare che in questo contesto sia mai stata considerato di:
"use it as a hypothesis to predict the outcome of a complex, unknown system,
with the hope that a desired property will emerge;”
Che il comportamento sorprendente e non facilmente spiegabile dei LLM, vada
oltre le capacità per cui sono stati allenati, è un fatto appurato anche se
controverso.
Ne ho parlato con Giorgio Parisi, che sul tema dei sistemi complessi ha vinto
il premio Nobel, e ha concordato con me che il fenomeno possa essere appunto
spiegato come l’applicazione su larga scala di una semplice legge di
probabilità: in questo caso la probabilità delle prossima parola in una
sequenza.
— Beppe
> On 19 Jul 2023, at 10:37, [email protected] wrote:
>
> Date: Wed, 19 Jul 2023 14:36:59 +0000
> From: Daniela Tafani <[email protected]
> <mailto:[email protected]>>
> To: "[email protected] <mailto:[email protected]>"
> <[email protected] <mailto:[email protected]>>
> Subject: [nexa] “Emergence” isn’t an explanation, it’s a
> prayer. A critique of Emergentism in Artificial Intelligence
> Message-ID: <[email protected]
> <mailto:[email protected]>>
> Content-Type: text/plain; charset="windows-1252"
>
> “Emergence” isn’t an explanation, it’s a prayer
> A critique of Emergentism in Artificial Intelligence
> <https://ykulbashian.medium.com/?source=post_page-----ef239d3687bf-------------------------------->
> [cardboarddreams]
>
> Emergence is the notion that in a complex system, the interactions of the
> whole may exhibit properties that are not present in the individual parts. It
> is most often applied to examples in physics and nature, such as the
> collective behaviours of ant colonies, the self-organizing principles of
> social groups, or the macro properties of molecules. In the last few decades,
> it has given rise to emergentist perspectives of human
> cognition<https://onlinelibrary.wiley.com/doi/10.1111/j.1756-8765.2010.01116.x>,
> and even of
> consciousness<https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597170/>. These
> are based on the recognition that the complexities and mysteries of the human
> mind, being a part of nature, may be characterized as emergent phenomena.
>
> This approach has an intuitive appeal. It is supported by the superficial
> facts: the brain — the source of intelligence and consciousness— is most
> certainly a complex of interconnected neurons. Emergentist interpretations of
> human behaviour may also boast some recent wins—the proliferation of LLMs
> (e.g. ChatGPT) may be seen as one such success. This has reignited the
> discussion of whether emergence is the best way to frame intelligence.
>
> You may have noticed that the last paragraph switched between two subtly
> different uses of “emergence”. The first use was to describe an observed
> emergent property; consciousness, we have seen it, likely emerges out of
> neuronal interactions. The second was to use it as a hypothesis to predict
> the outcome of a complex, unknown system, with the hope that a desired
> property will emerge; e.g. intelligence will arise from the interactions of
> artificial neurons at scale. The latter example is the driving motivation
> behind the multi-million dollar Human Brain
> Project<https://en.wikipedia.org/wiki/Human_Brain_Project>. The project
> justifies its cost by leaning on evidence from observed instances of
> emergence. Doing so paints emergence as a theory; that is, existing
> observations can be used to justify future predictions.
>
> But emergence is not a theory. Emergence can only be ascribed to a phenomenon
> in retrospect, once you already know what has “emerged”. The higher-level
> properties that emerge are qualitatively different from those at the
> lower-level — otherwise it wouldn’t be “emergence”. So by necessity they
> could not have been predicted from the lower-level ones. The properties of
> “intelligence” could not have been logically foreseen from the properties of
> neurons unless you had already observed that property emerge in a similar
> substrate. And even then it’s just a guess that is likely to be wrong given
> the complexity of the interactions involved; small differences can easily
> invalidate the hypothesis. In both cases emergence gives no new information:
> when explaining existing examples it gives you no new insights about the
> processes except that they happen; and when predicting unknown behaviours it
> gives very poor guarantees that anything you expect to happen will do so.
>
> Emergence is only really valid as a general metaphysical classification of
> certain phenomena. It’s a metaphysical category, like “cause”, “effect” or
> “change”. Using the word when explaining cognition is not wrong per se, it
> just has no real meaning or explanatory force. It’s like having a theory of
> “thing-happened-ness” — it’s correct, but void of content. Take, for example,
> the following quotes from a review article on emergence:
>
> This process gives rise to an emergent tendency to facilitate perception of
> items consistent with the patterns of English orthography, without explicitly
> representing this knowledge in a system of rules, as in other approaches.
>
> …
>
> However, such modes of thought themselves might be viewed as emergent
> consequences of a lifetime of thought-structuring practice supported by
> culture and education.
>
> Emergence in Cognitive Science,
> McClelland<https://onlinelibrary.wiley.com/doi/10.1111/j.1756-8765.2010.01116.x>
>
> If you removed the word “emergent” from the above two sentences, would
> anything important change? Indeed any sentence that includes “emergent” would
> give the same information if you removed it; “it gives rise to emergent
> properties” means the same as “it gives rise to properties”, or “there is an
> emergent tendency” is not substantially different from “there is a tendency”.
>
> Adding “emergent” to any sentence doesn’t increase its useful information
> content.¹
>
> Emergence has no information that fundamentally differentiates it from a
> “miracle”. If I were to say that applying
> transformers<https://arxiv.org/abs/1706.03762> to Neural Networks creates
> intelligence through a miracle, I would be ridiculed. Were I to say that they
> create intelligence through emergent interactions, suddenly they gain an air
> of scientific credibility — but what have I added to the conversation with
> the use of that word? What quantifiable scientific facts are entailed in the
> term “emergent”? There are none.
>
> In cognitive science, emergence is regularly used to “explain” the connection
> between two phenomena, when it is otherwise complex and difficult to predict:
> e.g. how neuronal firing gives rise to consciousness, or transformers to the
> appearance of language comprehension. Where there may be a connection, but
> nothing more is known or can be proved, emergence is a placeholder that fills
> the gap. The word gives weight and gravitas to what is essentially a blank
> space.
>
> Despite emergence contributing nothing of substance to the discussion, as a
> concept it admittedly has a compelling intuitive appeal. There is a wonderful
> feeling about the notion of emergence. It does seem to be adding something
> valuable, as if you’ve discovered a magical ingredient by which you can
> explain mysterious phenomena. That’s the reason it continues to be popular,
> and gets inserted into scientific discussions. It convinces the listener that
> something has been explained with scientific rigour when all we’ve done is to
> say “it’s complicated”.
>
> Besides the good feeling, however, emergence is void of any explanatory
> power. And so it has no scientific value in a predictive capacity. You can’t
> use it to say anything about what an unknown system will do; only what you
> hope it will do. When applied to pie-in-the-sky AI futurism, emergence has
> become synonymous with “I’m sure the system will work itself out”. It
> indicates that the author has a feeling that a complex system will align at
> some point, but no clear sense of how, why, or when. Insofar as intelligence
> does manifest in a specific instance, “emergence” doesn’t tell us anything
> interesting about how it happened. And insofar as intelligence hasn’t yet
> manifested, emergence doesn’t tell us when it will or what direction to take
> to get there.
>
> In the field of AI development, emergence is invoked whenever someone
> encounters a phenomenon in the human mind and has no idea how to even start
> explaining it (e.g. art, socialization, empathy, transcendental aesthetics,
> DnD, etc). If said researcher already has a working theory of AI, this
> realization is disheartening. So they look deeper into the matter, find some
> point of overlap between the existing theory and the missing behaviour, and
> assume that with enough time and complexity the missing pieces will emerge.
>
> Emergence is attractive in such cases because it puts the author’s mind at
> ease, by making it seem like they have a viable mechanism that only needs
> more time to be vindicated. It placates their inner watchdog, the one that
> demands concrete, scientific explanations. Emergence, being related to
> complexity and superficially validated by experiments such as Conway’s Game
> of Life, is enough to lull that watchdog back to sleep.
>
> This justifies continuing to ignore any shortcomings in a theoretical model,
> and persisting on the current path. Like the proverbial man who searches for
> his lost keys under the lamplight, because that is where the light is, he
> hopes that with enough persistence his keys will “emerge”. The only other
> alternative is to admit failure, and to give up any hope of accomplishing
> what you want within this lifetime.
>
> Scientists, it seems, can have superstitions too. And emergence has a
> powerful narcotic effect: it feels so reasonable and credible on a gut
> level². There are many factors that prevent a given researcher from
> investigating emergence too deeply and realizing that it lacks any substance.
> First, there appears to be a lot of external evidence to back it up in the
> natural world. This, as was pointed out, equivocates between retrospective
> and prospective uses of the term, and so legitimate uses are being
> conscripted to justify the illegitimate ones. Secondly, the fact that
> emergence exclusively concerns itself with intractably complex systems means
> anything behind its curtain by definition can’t be studied. So it
> conveniently excludes itself from exactly that analysis which would reveal it
> to be hollow.
>
> In the end emergence isn’t an explanation; it’s an observation combined with
> a recognition of ignorance. Wherever emergence shows up there is an implicit
> acceptance that everyone involved is at a loss for how to approach the topic.
> It’s not that properties like intelligence won’t emerge from neural activity,
> it’s that emergence is a placeholder that justifies and promotes a lack of
> interest in exploring the details behind the connection. It discourages
> investigation. By invoking the term, we are merely thanking the nature gods
> for granting us this emergent property (aka property), and trying not to
> examine their gifts too profanely or with ingratitude. This impulse is
> understandable, since we don’t think we’ll discover an answer if we were to
> dig in. But we shouldn’t allow our insecurities to masquerade as science, or
> else they may become ingrained to the extent that they are difficult to
> uproot. A false answer stands in the way of a true one.
>
> ¹ This used to say ‘You can remove “emergent” from any sentence and it would
> mean the same thing’, but that has caused some confusion, so to clarify: the
> word “emergent” when used as an adjective doesn’t add new or useful
> information; you won’t know any more about the subject than you did before.
>
> ² A self-aware researcher should notice if they have a strong intuitive or
> emotional reason for holding on to the idea. If you ever feel that emergence
> is so self-evident that it can never be disproved, that should give you pause
> — perhaps you have strayed outside the bounds of scientific inquiry and into
> metaphysical expositions. Not that there’s anything wrong with the latter…
>
>
> https://ykulbashian.medium.com/emergence-isnt-an-explanation-it-s-a-prayer-ef239d3687bf
_______________________________________________
nexa mailing list
[email protected]
https://server-nexa.polito.it/cgi-bin/mailman/listinfo/nexa
