OK… After a day of messing around and testing everything I found the fix. I dont quite understand why it works in this way but it does. With ID it would display the field with no entries but with key it displays both.
I had to change
$fields['images']['ID']
to
$fields['images']['key']