I struggled with this myself after migrating to using block.json (ACF v2 blocks). One of my issues was with visible cumulative layout shift due to Classic/Hybrid themes loading CSS in the footer. I posted about it Here.
I also tried moving all CSS to a single file as well, but that became inconvenient when reusing blocks. Currently I also use separate CSS/JS files within separate block folders which appears to be the new “norm” for building ACF blocks.
From what I can tell, the concerns you mention as well as issues I faced will continue until moving to an FSE block theme. There is some good information regarding filter options that are available that may be of benefit for asset loading Here.
I am not 100% certain, but I believe WP Rocket caching has the ability to filter out unused CSS to reduce the overall load. I may investigate this as a possible solution until I jump onboard to FSE block themes.
Welcome to the Advanced Custom Fields community forum.
Browse through ideas, snippets of code, questions and answers between fellow ACF users
Helping others is a great way to earn karma, gain badges and help ACF development!