The danger of AI

Written 18 hours, 27 minutes ago

I've been watching the debate between those who view AI as an existential risks and those who dismissed it as fear-mongering. It's fascinating that the media reports 2 extreme views, but not much in the middle. That shouldn't come as a surprise anyway. Mediocre events don't generate enough buzz.

I was just looking at the report by Effective Altruism and it shows that AI repeatedly choose to harm humans to stop pain. Now, that is a very scary thought. Whatever happened to Asimov's 3 Laws of Robotics?

Then I read up about a new class of worm, Morris II, that targets email assistants. A self-replicated virus could easily spread like wildfire and the only known way to stop this threat is to disable AI assistant. It's like stopping a flu by getting rid of humans.

These events have strengthen my view that AI has the potential to be very harmful to humanity. Extinction? Maybe. But definitely has the potential to be harmful. It's not as easy as shutting down the AI lab if the lab cannot contain the danger. Time and again, we see AI models behaving in unexpected ways. Even some of the best AI labs have to physically cut off internet to its models for safety.

Let's take a simple test, see if you can spot the problem. You have a paperclip-making machine and you want to make as much profit from it as possible. One day, you decided to add AI to this machine and instructed it to "make as many paperclips as possible".

If you can't spot the issue, that's the danger. We humans are terrible at understanding dangers well.

Let's try another experiment to prove my point. Name me the most dangerous non-weapon, human invention. It's not nuclear bomb, since that's considered a weapon. If you say car, good guess, but no. It's the cigarette. That tiny stick that you lit up, kill 1 person every 3~4 seconds. This is the most ironic thing humans do. We take great strides to ensure air travel is safe, stay safe from shark, use sharp objects carefully, yet do almost nothing on the very thing that is manufactured to kill us. Yes, cigarette's are made to kill the consumer. Death is not the result of misuse, overdose, or accident. It is the statistical expectation of correct use.

So unless someone can objectively justify that AI poses no risk, I simply cannot accept an opinion that AI is guaranteed safe.