I’ve seen ACF slow down the admin of a site with complex field groups. However, let’s face it, if you do anything complex in the admin you’re going to slow it down, at least that’s my experience, and I’m don’t get overly concerned about the admin. I’m more concerned about the front end.