I faced the same issue while adding the field group via PHP using the current_user action hook and generating the field value dynamically, based on the post_id.
By its nature the AJAX call does not recognize the current_user action hook (and others). Switching to admin_init did the trick for me. The field group will then remain visible when assigning a taxonomy term.
