My rule of thumb is that anything that can be a single object is a post. It’s not how I’m going to relate them that’s important but what they represent. A taxonomy is uses when multiple descriptors can be applied to an object.
An event is the single top level object. It can have multiple topics, another post type. Topics can have categories, that’s a taxonomy. For something like you are talking about I would probably also create a “user” post type of some kind, maybe call it “speaker” that may or may not be related to actual site users and instead of relating events to speakers I would relate topics to speakers. I may or may not use my own plugin to create bidirectional relationships for these. The plugin is really meant for simple relationships and I’d have to evaluate it’s use or building something custom depending on how complex the relationships ended up being.
