My final solution ended up being as you suggested, and I have it fire on save_post. Basically when the post saves, it checks for the state of the two conditional objects. Depending on what’s set for those two conditionals, it does a check against each of the fields that could have data in them, and if it’s present, clears the data from that ACF Field, and then deletes the object term relationship via WordPress directly.
It appears to be working fine so far!
