Hi Muhammad, I know of no tutorial. I recently updated the “mock” storage plugin and so have some experience with this interface. You can find my notes at [1].
Unfortunately, creating a storage plugin seems to require a significant commitment of time and effort because you must understand: * The storage plugin structure. Some bits are a bit unusual (such as bindings between the various bits and pieces.) * Enough about Calcite to provide it with the required plan-time information. * The Jackson-serialization structure for various components. * The rather complex process by which you work with ScanBatch and value vectors to get data from your data source into value vectors. * Drill as a whole so you can build Drill and debug it. The only way to test a plugin is by running it inside Drill. Your best approach is to carefully study existing plugins. You can start simple, say with the mock plugin. Replace the bits of the mock implementation with your own. Try to get it to work for a single table. Then, add other functionality gradually. By the time you are done you will be well on your way to being an expert in some of Drill’s internals. Thanks, - Paul [1] https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Model > On Mar 15, 2017, at 10:35 AM, Muhammad Gelbana <[email protected]> wrote: > > Everyone, > > Is there a tutorial on how to write a custom storage plugin to support some > sort of a proprietary data source ? > > I understand I can configure a storage plugin based on pre-shipped storage > plugins such as the one for MongoDB, MySQL\JDBC, HBase, Hadoop HDFS..etc, > but that's not what I need. > > I need to write a new plugin to support a storage that is not publicly > available. > > The best that I've got so far is the information on these pages: > > - > > https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePlugin.java > - > > https://github.com/apache/drill/tree/master/contrib/storage-kudu/src/main/java/org/apache/drill/exec/store/kudu > > > But this isn't enough to understand what needs to be done or troubleshoot > errors while developing the plugin. > > *---------------------* > *Muhammad Gelbana* > http://www.linkedin.com/in/mgelbana
