The code you have sorts the field according to their ID value, I’m going to be completely honest, I have never had much luck using array_multisort() and when I need to sort things differently than ACF sorts them then I will generally do sorting in another way.
Anyway, the first thing you need to do is to get the date to sort by instead of sorting by the ID
foreach( $gallery as $i => $image ) {
$order[ $i ] = get_field('date', $image['id'], false);
}
the above code gets the date field attached to the image. The 3rd parameter is false to get the unformatted value of the field to sort by.
