Check out the documentation for get_field()
, specifically the section titled “Get a value from different objects”. You should be able to get the value from a field assigned to a category by passing the category ID as second parameter:
$color = get_field( 'my_field', $category[0]->cat_id );
…
echo($color);