You could use an acf/prepare field filter to remove the layouts that you don’t want to show https://www.advancedcustomfields.com/resources/acfprepare_field/. May be a little complicated. I think you’d need to loop though the layouts in the flex field to find the ones you want to remove. I think that the developer might consider this an “edge” case, but you could try submitting your feature request by opening a new ticket https://support.advancedcustomfields.com/new-ticket/ (the developer really does not read these forums very often)
Welcome to the Advanced Custom Fields community forum.
Browse through ideas, snippets of code, questions and answers between fellow ACF users
Helping others is a great way to earn karma, gain badges and help ACF development!