ACF stores dates as ‘YYYMMDD’ Sorting on this field as either a number or as text should work.
couple of questions
You said that this is a date field, but is this date field a sub field of a repeater or in a flexible content field?
Can you successfully make this filter sort by anything else? For example do these change the order?
'orderby' => 'title',
'order' => 'ASC'
'orderby' => 'title',
'order' => 'DESC'