So your biggest concern doesn't seem to be a problem: You can just do query("file.ext") and get back the format without loading the file. I see that in the way you want to use it, a hierarchy would be a nice addition. You could just do this: myload(path) = myload(query(path)) myload{T <: VectorGraphic}(::File{T}) = ... Right now, VectorGraphic would need to be a union of the formats, though.
We do dynamically load libraries to completly decouple the loading from the IO library, so that a new user doesn't need to know what the best IO library is for his format. Since he doesn't know, FileIO must know. When you have an IO library already loaded, FileIO might use it. So far it depends on the priority of that library, if FileIO loads another one. Maybe I should change that, so that FileIO determines first if any of the IO candidates is already loaded and if so, it just uses that library.