On Thu, Feb 3, 2022 at 4:27 PM John Clark <[email protected]> wrote:
> On Thu, Feb 3, 2022 at 2:11 PM Terren Suydam <[email protected]> > wrote: > > > *the code generated by the AI still needs to be understandable* > > > Once AI starts to get to be really smart that's never going to happen, > even today nobody knows how a neural network like AlphaZero works or > understands the reasoning behind it making a particular move but that > doesn't matter because understandable or not AlphaZero can still play > chess better than anybody alive, and if humans don't understand how that > can be than that's just too bad for them. > With chess it's clear what the game is, what the rules are, how to win and lose. In real life, the game constantly changes. AlphaCode can potentially improve its code, but to what end? What problem is it trying to solve? How does it know? Even in domains with seemingly simple goals, it's a problem. Imagine an AI tasked with making as much money in the stock market as it can. Pretty clear signals for winning and losing (like chess). And perhaps there's some easy wins there for an AI that can take advantage of e.g. arbitrage (this exists already I believe) or other patterns that are not exploitable by human brains. But it seems to me that actual comprehension of the world of investment is key. Knowing how earnings reports will affect the stock price of a company, relative to human expectations about that earnings report. That's just one tiny example. You have to import a universe of knowledge of the human domain to be effective... a universe we take for granted since we've acquired it over decades of training. And I'm not talking about mere information, but models that can be simulated in what-if scenarios, true understanding. You need real AGI. I think that's true with AIs that would supplant human programmers for the reasons I said. > > *The hard part is understanding the problem your code is supposed to >> solve, understanding the tradeoffs between different approaches, and being >> able to negotiate with stakeholders about what the best approach is.* > > > You seem to be assuming that the "stakeholders", those that intend to use > the code once it is completed, will always be humans, and I think that is > an entirely unwarranted assumption. The stakeholders will certainly have > brains, but they may be hard and dry and not wet and squishy. > To get to the point where machines are the stakeholders, we're already past the singularity. > > *> It'll be a very long time before we're handing that domain off to an >> AI.* > > > I think you're whistling past the graveyard. > Of course, nobody can know what the future holds. But I think the problem of AGI is much harder than most assume. The fact that humans, with their stupendously parallel and efficient brains, require at least 15-20 *years *on average of continuous training before they're able to grasp the problem domain we're talking about, should be a clue. Terren > > John K Clark See what's on my new list at Extropolis > <https://groups.google.com/g/extropolis> > -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/everything-list/CAMy3ZA9Lv0u_or9Y5zswnNhUyW%2BdBLKiePvhnRevAtqD%2BAPNnQ%40mail.gmail.com.
