https://www.theguardian.com/books/2019/oct/24/human-compatible-ai-problem-control-stuart-russell-review

Creating machines smarter than us could be the biggest event in human history – and the last

Here’s a question scientists might ask more often: what if we succeed? That is, how will the world change if we achieve what we’re striving for? Tucked away in offices and labs, researchers can develop tunnel vision, the rosiest of outlooks for their creations. The unintended consequences and shoddy misuses become afterthoughts – messes for society to clean up later.

Today those messes spread far and wide: global heating, air pollution, plastics in the oceans, nuclear waste and babies with badly rewritten DNA. All are products of neat technologies that solve old problems by creating new ones. In the inevitable race to be first to invent, the downsides are dismissed, unexplored or glossed over.