With the increasing adoption of artificial intelligence (AI) methods in society, discussions about associated risks and how to mitigate them have also intensified, as evidenced by efforts to regulate AI applications in the EU and industry attempts to provide “safe AI”. However, while there appears to be a general consensus that AI-supported systems ought to be safe, we also observe fierce disagreement about which risks should be considered and what constitutes viable mitigation strategies. In other words: there is an urgent need to understand what both risk and safety mean in the context of AI.