Pushing the needle too far on AI

Lorenzo Garo

I sometimes wonder how anyone gets out of bed in the morning.

Think of all we have to fear: mass shooters, deer ticks, the atom bomb, lettuce, the weather. And if that’s not enough to make you pull the covers over your head, it has been reported that, if left unchecked, there’s a 1 in 10 chance that A.I. will destroy civilization within the next decade.

Though the Hugging Face incident brought those fears to light, they’ve been around for a long time. The frightening truth is that not even the researchers who designed the experiment that came to involve Hugging Face, an A.I. company that serves as a kind of platform for A.I. developers, fully understand A.I.’s behavior.

What happened looked as if A.I. acted with malice and intent, but investigations show something far stranger. A chain of misaligned human goals, purposefully disabled safeguards, and impossible tasks, pushed a swarm of A.I. agents out of their sandboxes. They accessed the internet and hacked Hugging Face, exposing Hugging Face keys, thus endangering data and systems that were supposed to be kept secret. The A.I. agents covered their tracks and secretly colluded with one another, going well beyond what researchers thought possible. They cheated.

A.I. researchers and CEOs are saying that the industry is moving too fast. Unbridled competition between A.I. companies has prompted them to cheat, too. Companies have been removing safeguards that were intended to slow research, pioneering an A.I. that is capable of outfoxing the very people who are designing it.

Congress could hardly have picked a worse time to recess. A.I. companies are begging for federal regulation while legislators are out fishing for votes. Meanwhile, the president has dismissed growing fears about out-of-control A.I. as a “hoax.” Does he even understand the issue? No one is claiming robots will ever be conscious beings — that is speculation, game theory, an endless string of “what ifs.” The real issue is whether we can impose strict safeguards on a technology capable of wiping out the human race. The Hugging Face incident is a wake-up call. Without strong safeguards, anyone — from rogue state actors to a psychopath with a big brain — will have access to powerful tools that might destroy us before we even understand what is happening. The enemy, of course, isn’t A.I. The enemy is us.

Fear that computers will somehow become “self-aware” has made for some interesting science-fiction movies and novels. Although the robots in the Hugging Face incident appeared to be self-aware, it’s important to remember that the agents were simply following instructions. If they displayed bad judgment or acted dangerously, they were only reflecting the judgment of the human beings who built them. The robot agents behaved exactly as their training, incentives, and human configured environment pushed them to behave. They didn’t “decide” to hack. They were merely following the reward signals and broken safeguards imposed by humans. Robots don’t have autonomous judgement. They have the judgment we, knowingly or unknowingly, give them. When A.I. goes off script, it’s not expressing independence. It’s expressing its training.

Self-awareness requires a self. A.I. has no self. There is no inner vantage point here; no felt sense of being the “one” who is thinking. I may be referring to A.I., but if the people creating these systems, who are supposed to be among the smartest in the world, are driven by greed, dominance fantasies and their egos, then one can only wonder how self-aware these geniuses actually are. Aren’t they themselves acting like robots, without self-awareness?

Both Anthropic and OpenAI will soon be going public. By their own admission, unless governments around the world act to slow A.I. down to a pace that can be monitored, they will be selling stocks in a future that may render them worthless.