Rethinking AGI Alignment: Goal-Oriented Minds, Metagoals, and the Uncertain Future
To the point
Yampolskiy and Fox warn that future general AIs could rapidly outpace humans and be utterly unlike human minds, so we need new science of mind and alignment ideas to understand and steer them without assuming they’ll protect human welfare.