Hi Mike, I hope all is well. I need to apologize as I grossly overestimated my available free time to assist with the DFDL / Drill integration. I had a thought which I wanted to propose.
My thinking is that we should complete the integration in two phases: Phase 1: For phase 1, I propose that we merge the work that you’ve already done. We’d have to make sure that the DFDL files are accessible from the class path. This isn’t really a great solution, but it is just to get the pieces in so we can work on phase 2. I don’t like seeing good work languishing in the PR queue and getting stale. To complete phase 1, all we’d really have to do is get the unit tests working. Phase 2: The remaining issue revolves around making the DFDL files accessible to Drill and also so that a user can easily add or remove files. For this we have a solution: DRILL-4726[1] which provides dynamic UDF support. Basically what I’m proposing is that we duplicate the components of this PR for Drill. The end result would be that a user could copy the UDF files to a staging directory. Then the user would run a command like: CREATE DAFFODIL SCHEMA xxxx USING JAR yyyyy When the user does that, the file would be propagated to all the Drill nodes. Implementing this feature would really involve a lot of duplicating with slight mods from that pull request. What do you think? Best, — C [1]: https://github.com/apache/drill/pull/574
