Matt, See inline comment:
On Sat, Jan 27, 2024 at 11:18 AM Matt Mahoney <[email protected]> wrote: > I'm not sure what you are asking about judgment. Do you mean deciding what > is true or false, or deciding what is right or wrong? > > There is no such thing as objective truth. We believe certain things to be > true either because other people said they are true, or because your senses > said so, or because your brain was programmed to believe so. But people can > lie, your senses can lie, and evolution only cares about reproduction, not > truth. People tell you that the world is round or that God exists. > Evolution gave you senses of consciousness and free will, something you > want to preserve by not dying, which results in more offspring. > > In case you missed the message from the Flat Earthers (who secretly know > about time zones), it is not about the shape of the world. The message is > about how you know what you believe to be true. > > An AGI trained on human knowledge is going to believe the same things that > humans believe. *The same goes for judgements about right and wrong.* > > > I guess I don't want to get too deep with this philosophically, but at the same time I don't disagree with you. The benchmarks that are used to test AI would fall mostly under what Kahneman describes as matters of fact. For a matter of fact we should all agree, eg., some mathematical formula follows the rules, some drug causes some side effects, some person with an interest in jazz is likely to listen to Miles Davis. And of course, it is cloned from a mountain of human data. Kahneman also describes "matters of judgment." In this, we expect to differ in opinion, so long as the opinion falls within BOUNDARIES. What are the acceptable boundaries? How does an AGI determine such boundaries? If someone ventures beyond the "acceptable boundaries" in judgment, we think they are stupid or crazy. So that leaves a progression from fact -> judgment -> out-of-bounds-stupid-judgment. It's the second part that interests me, matters of judgment. To me, a true judgment is firstly based on a situation with a lack of evidence, there is dispute about what evidence is important, the situation is novel, a human would likely have intuitions and biases, exercise the capacity of free will, use "gut feeling" etc. Training an AGI for such a judgment seems tricky, and I doubt it magically falls out of the mountain of input data, but that's what I'm wondering about: if we venture beyond the training data, beyond the benchmarks, how is a true judgment handled, how is it computered? or just simply can it be handled? Is there a limit? I'm looking for opinions of a more practical nature. We can dispute the deep philosophy behind "objective" and "facts" but for practical purposes we tend to regard some issues like that (approve or not, I can't change the world, there is typically a prevailing "objective"). I'm trying to build up some material for a meetup event. So thanks guys. MIke PS., There was a mistake in my prior messages, which should read "A true judgment would be unique to an AGI" if anyone cares! > On Tue, Jan 23, 2024, 5:55 PM Mike Archbold <[email protected]> wrote: > >> Hey everybody, I've been doing some research on the topic of judgments in >> AI. Looking for some leads on where the art/science of decision making is >> heading in AI/AGI. Note: by "judgment" I mean situations which have a >> decision that is open to values within boundaries, not that can be >> immediately and objectively correct or incorrect. >> >> Lately I have been studying LLM-as-a-Judge theory. I might do a survey or >> such, not sure... looking for leads, comments etc. >> >> Thanks Mike Archbold >> > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/T5edfab21647324f7-Mafaf2b831d3d61d112457d6a> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T5edfab21647324f7-M01bb7d28deb3ae1dc8dc4cb3 Delivery options: https://agi.topicbox.com/groups/agi/subscription
