paul-rogers commented on pull request #2359: URL: https://github.com/apache/drill/pull/2359#issuecomment-979494770
@dzamo, @cgivre: we should certainly encourage people to add plugins (connectors) to everything under the sun. The question really is: does the rather limited Drill core team want to support all of them at the same level as the main ones that Just Gotta Work. For huge files. On local, S3, HDFS. With splits. With metadata. With excellent error reporting. And on and on. The PDF thing seems like the prototypical "not in the core, but a great add-on" example. Yes, some one user might want to scan zillions of PDF files on an HDFS system. But, the core Drill team can't spend its resources on that when there are still basics to address. Let's do this: let's get this into Drill for now. Then: * Ensure we have a good way to build plugins separate from the Drill code. (Last time I tried, I couldn't get it to work.) * Explain how to create a plugin in a users own repo, built against Drill. * Explain how to drop the plugin into a running Drill to add that functionality. This way, even without a marketplace, folks can build their own connectors separate from the Drill repo. A next step might be to split Drill's monorepo into multiple parts, as do many other projects. The current repo would be the core. Then, the contrib stuff could move to its own repo(s) under the Drill project. And, users can create their own extensions in their own repos. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
