No, there isn’t any way to add setting to flex fields and adding a field to every layout is the only way to do what you want. I generally do this by creating a “layout settings” group that I clone into all flex layouts.
The issue issue is that that when editing there are no controls on a flex field other than “add new”. Adding an input here would be like adding an input to a text field or a radio field for the user to enter data into. This is not really possible for any type of field.
Those controls are just JS that does those things, they are not inputs that allow values to be changed. They do allow control of any display on the front end, for example to show it or not on mobile or desktop.