The problem in the OP is likely due to the fact that artists can share categories, so while one may not have anything in a category others might making them non empty.
There isn’t really any way to do what you want to do directly.
What you would need to do is get all of the media posts related to the artist then loop over all of them to get collect a list of categories and then use this in the “include” argument for getting the terms.
