Sorry, somehow Github “flagged” my account and shut down public access. Should be back to normal now.
- Paul > On Mar 16, 2017, at 4:10 AM, Muhammad Gelbana <[email protected]> wrote: > > That's very helpful Paul ! But I can't access the wiki page you referenced. > May be it's set to private ? > > *---------------------* > *Muhammad Gelbana* > http://www.linkedin.com/in/mgelbana > > On Thu, Mar 16, 2017 at 1:50 AM, Paul Rogers <[email protected]> wrote: > >> Hi Muhammad, >> >> I know of no tutorial. I recently updated the “mock” storage plugin and so >> have some experience with this interface. You can find my notes at [1]. >> >> Unfortunately, creating a storage plugin seems to require a significant >> commitment of time and effort because you must understand: >> >> * The storage plugin structure. Some bits are a bit unusual (such as >> bindings between the various bits and pieces.) >> * Enough about Calcite to provide it with the required plan-time >> information. >> * The Jackson-serialization structure for various components. >> * The rather complex process by which you work with ScanBatch and value >> vectors to get data from your data source into value vectors. >> * Drill as a whole so you can build Drill and debug it. The only way to >> test a plugin is by running it inside Drill. >> >> Your best approach is to carefully study existing plugins. You can start >> simple, say with the mock plugin. Replace the bits of the mock >> implementation with your own. Try to get it to work for a single table. >> Then, add other functionality gradually. >> >> By the time you are done you will be well on your way to being an expert >> in some of Drill’s internals. >> >> Thanks, >> >> - Paul >> >> [1] https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Model >> >>> On Mar 15, 2017, at 10:35 AM, Muhammad Gelbana <[email protected]> >> wrote: >>> >>> Everyone, >>> >>> Is there a tutorial on how to write a custom storage plugin to support >> some >>> sort of a proprietary data source ? >>> >>> I understand I can configure a storage plugin based on pre-shipped >> storage >>> plugins such as the one for MongoDB, MySQL\JDBC, HBase, Hadoop HDFS..etc, >>> but that's not what I need. >>> >>> I need to write a new plugin to support a storage that is not publicly >>> available. >>> >>> The best that I've got so far is the information on these pages: >>> >>> - >>> https://github.com/apache/drill/blob/master/exec/java- >> exec/src/main/java/org/apache/drill/exec/store/StoragePlugin.java >>> - >>> https://github.com/apache/drill/tree/master/contrib/ >> storage-kudu/src/main/java/org/apache/drill/exec/store/kudu >>> >>> >>> But this isn't enough to understand what needs to be done or troubleshoot >>> errors while developing the plugin. >>> >>> *---------------------* >>> *Muhammad Gelbana* >>> http://www.linkedin.com/in/mgelbana >> >>
