Hopefully this will help, but I think I understand what you’re doing. I’m not exactly sure on the details of how to accomplish it, but I can give you a general outline.
Enqueue a JavaScript.
When your field is loaded or changes the script is run.
The script will look at the field and get the font that needs to be include then creates a new <style> element in the head of your document with the link to the correct CSS file.
You may also want to delete any previous <style> elements that were created when the user switches fonts.
