Hi, Marc, Thanks for your thoughtful response.
On Dec 21, 2012, at 2:17 AM, Marc Welz wrote: > So we have been thinking about this (and actually the kcs code does > use dlopen), but two things are giving us pause: > > 1 - The .so files will have to be compiled, and if the toolchain and libraries > used to build the .so files differ substantially from what was used to > build > tcpborphserver itself, we are likely to encounter interesting failures. And > given that some people will use cross compilers while others native ones, I > expect that this will occur more often than in normal environments I think this is less of a worry than you think. Anybody building plug-ins would be certainly capable and well advised to build their own tcpborphserver binary so potential tool chain mismatch issues would be non-existent. Likewise, building a plug-in for the "stock" tcpborphserver binary would be result in a binary plug-in that is useful for anyone running the stock tcpborphserver. It's not like there's a plethora of incompatible tcpborphservers out there. > 2 - Similarly, if the katcp headers (defines, prototypes) differ between > server > build and module build, we are looking at support issues. Consider a data > structure which acquires an extra field between server and module build - > accessing that data structure will result in core dumps in the best case This potential problem exists all the time when linking against a shared library. In the case of a plug-in, one could mitigate the potential issue by requiring the plug-in to indicate (via some TBD mechanism) which version it supports. If the tcpborphserver executable loading the plug-in detects an incompatible version number, then it would refuse to call the plug-in's "register commands" function. > Using big software engineering words: I think the codebase isn't yet mature > enough and userbase isn't yet large enough to tilt the cost/benefit balance > toward using shared objects. Now is the perfect time to implement such functionality: *before* people start glomming onto tcpborphserver and it becomes too big/unmanageable to add plug-in capabilities and before people get too entrenched in the non-plug-in way of extending tcpborphserver that they ignore plug-ins. > Having said this: I think adding module loading support is quite easy, Agreed. > and you are welcome to add it: Thanks. Unfortunately my current duties do not afford me the time to do this right now. Hopefully someone who wants/needs to extend the functionality of tcpborphserver will do so by adding at plug-in functionality at some point. It would also be a great way to allow sites to add "experimental" features to tcpborphserver without having to expose them to the world (or the world to them). > At the end of raw.c add a register_flag_mode_katcp call to a new function, > (say "load_request_cmd"), which then does the dlopen on a given > filename, extracts a symbol, maybe even computes it from the filename, say > filename_cmd, and then does a register_flag_mode_katcp on that symbol - > that should allow anybody to load an additional command by going > "?load-request name", where name.so is an object implementing name_cmd > to provide a ?name request. That's basically the model I was envisioning except that I think it would be nice for a single plug-in to provide multiple commands. Perhaps the server would call a "get_command_names" function than returns a list of commands provided. It would also be nice if the new commands integrated in with the ?help output so a little more information would be needed from the plug-in besides just names. > Easy, and good enough to use within a single organisation. With git push pull > magic anybody who thinks this is useful should be able to grab it. Agreed. I envision most plug-ins will be geared to specific gateware designs so they will probably be developed in parallel with the gateware. I'm thinking of somewhat complex things like fringe rotation for correlators as well as not-so-complex-but-pre-canned ADC initialization code. > The more demanding work is to make shared objects code useful for third > parties: Making sure that there is logic/magic to test that incompatible APIs > or API revisions are not being used, that all data accesses happen via calls > to > opaque pointers, not directly to data structure members (see point 2) and > that the API is sane ... I'd expect that I'll end up reworking said > API once or > twice before we are happy with it and can declare it stable (which is what > shared objects rely upon). These concerns are legitimate and would be nice to address, but I think they will sort themselves out fairly easily once the plug-in functionality is added. I hope they are not seen as a deterrent to adding the plug-in functionality in the first place. If we wait for everything to be perfect before we add plug-in functionality, we'll never add it! :-) Thanks, Dave
