Found that it wasn’t fully deleted. Found it in the database and then realised there was a Trash area within the plugin.
Once it was deleted from the Trash, I was able to add the export.
I don’t really agree with the UX though – it would be much better if it replaced the existing field groups, or at least moved a copy to trash when it did so.
