Here is how I solved it. I was missing a filter for my custom post type which I added to my “custom_setup” function.
add_filter('rest_prepare_custom_nav', function($response) {
$response->data['acf'] = get_fields($response->data['id']);
return $response;
});
So the filter is a wild card filter, applied like
apply_filter("rest_prepare_YOUR-CUSTOM-POST-TYPE-NAME-HERE", ...