I’m not sure what is causing your issue. I just set up a test field group containing a group field with a relationship field as a sub field. The name of the relationship field is simply “relationship”. I then added an acf/fields/relationship/query/name=relationship and this filter runs as expected.
Welcome to the Advanced Custom Fields community forum.
Browse through ideas, snippets of code, questions and answers between fellow ACF users