> > > > The only solution to this problem I ever see suggested is to > intentionally create a Really Big Fish called the government that can > effortlessly eat every fish in the pond but promises not to -- to > prevent the creation of Really Big Fish. That is quite the Faustian > bargain to protect yourself from the lesser demons.
Yes, of course, the Really Big Fish that is democracy. I am starting to get quite puzzled by all Americans (I don't know if you are American though, but I want to express this anyway) who express severe distrust in government. Because if you distrust all forms of government, what you really distrust is democracy itself. Here you basically compare democracy to... whom? The devil!? USA is supposed to be the "leading democracy of the world" yeah, right. But I never hear any people speak so badly of their government, and in effect the democracy itself. The American idea of liberalism is certainly not the same thing as democracy. Maybe this is the century when Americans will find that out. The American liberal culture was founded when the plains of America appeared endless, and if you did not like the influential people of a certain area, you just moved on to virgin grounds and started your own community with your own rules. But there is no more virgin land in America, and people have started to accumulate in the cities since long. Liberty does not work quite so well when people live close and need to depend on each other. That lesson has been learned in Europe ages ago. My recommendation is to put some faith in the will of the people! When you walk on the street and look around you, those are your fellow citizen you should feel at least some kind of trust in. They are not out to get you! Then of course the American form of "democracy" is not so excellent, so maybe there is a reason for the distrust even though sad. On the surface USA has only two parties which is just one more than China. Sweden is not much better, but at least we have 7 alive and active parties. But these are problems that can be solved and are not a reason to give up on democracy. Generally though, the point that you fail to see is that an AGI can > just as easily subvert *any* power structure, whether the environment > is a libertarian free market or an autocratic communist state. The > problem has nothing to do with the governance of the economy but the > fact that the AGI is the single most intelligent actor in the economy > however you may arrange it. You can rearrange and change the rules > as you wish, but any economy where transactions are something other > than completely random is an economy that can be completely dominated > by AGI in short order. The game is exactly the same either way, and > more rigid economies have much simpler patterns that make them easier > to manipulate. > > Regulating economies to prevent super-intelligent actors from doing > bad things is rearranging the deck chairs on the Titanic. I agree that a super intelligent life form could be quite a difficult adversary. It might be able to manipulate and take over a democratic power structure also, I would not deny that. Probably it would try to target the culture of the people, and insert hostile but stealthy memes into the population. I guess it would also try to gain the trust of people and make them dependant on it by offering appealing services. Depending on the economy and regulations, it could also try to obtain direct control over as much automated production capacity as possible, especially production capacity that could be used for building weapons. It is not true like you say, that the economy of a democratic socialist society has easy patterns that are easy to manipulate. The supreme power in such a society lies in the democracy, and to manipulate that power you need to manipulate the whole population. Actually, I believe that the relative stupidity of the population could act as a kind of protection against manipulation. I have a son that is one month old, and I would say it is really difficult to control someone who is so extremely dumb as kids of that age are. However, I would not go as far as saying intelligence implies power, saying that a super intelligent life form by necessity would be able to take over any given power structure. I remember having this discussion with a friend a long time ago. The trivial example is if you have a super intelligent AGI brain in a box in front of you on your desk, and you have a gun. Then you can take the gun and shoot the box. That proves at least that there is no implication in the strict logical sense. But of course the picture gets more complicated if we have an AGI system that interacts in a social context, where we put different degrees of trust in it. Apparently the danger increases the more dependant we are on the AGI systems. But there are methods to protect ourselves. One way is to never utilize the most intelligent AGI systems directly: For example we could use it to produce scientific facts that we can verify are correct. These scientific facts can then be used for our production but never connect the AGI system to the production facilities directly. If the production facility needs intelligence, we choose a separate more dumb AGI system that is just suited to its task of running the factory. There are a number of safety measures like this that could greatly improve the safety of AGI usage. I believe we could make it quite difficult for an AGI system to obtain power by using the age old idea of divide and conquer. Also, the history shows that intelligence is no guarantee for power. The Russian revolution and the genocide in Cambodia illustrates effectively how intelligent people were slaughtered by apparently less intelligent people, and later how they were controlled to the extreme for decades. Most communist dictatorships end because instability caused by poverty, not because the control structure itself failed. This just reveals something raw and basic about existence on earth that I think many AGI enthusiasts and futurists wants to deny: What good are wits when you are looking down the barrel of a gun? /Robert Wensman ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=51821901-7abc6f
