It was just me being a dumbass. In ACF Pro (and possibility newer versions of ACF free?), when “Allow Null?” option is set to “Yes” for your field a small cross appears at the right-hand side of the field, which allows the user to clear the value.
Welcome to the Advanced Custom Fields community forum.
Browse through ideas, snippets of code, questions and answers between fellow ACF users