Actually I managed to get past that error with an educated guess that if I created a BatchCreator class, it would automagically be picked up somehow. I'm now at the point where my RecordReader is being invoked!
On Sun, Jan 12, 2020 at 2:03 PM Andy Grove <[email protected]> wrote: > Between reading the tutorial and copying and pasting code from the Kudu > storage plugin, I've been making reasonable progress with this but am I but > confused by one error I'm now hitting. > ExecutionSetupException: Failure finding OperatorCreator constructor for > config com.mydb.MyDbSubScan > Prior to this, Drill had called getSpecificScan and then called a few of > the methods on my subscan object. I wasn't sure what to return for > getOperatorType so just returned the kudu subscan operator type and I'm > wondering if the issue is related to that somehow? > > Thanks. > > > On Sat, Jan 11, 2020 at 10:13 PM Andy Grove <[email protected]> wrote: > >> Thank you both for the those responses. This is very helpful. I have >> ordered a copy of the book too. I'm using Drill 1.17.0. >> >> I'll take a look at the Jdbc Storage Plugin code and see if it would be >> feasible to add the logic I need there. In parallel, I've started >> implementing a new storage plugin. I'll be working on this more tomorrow >> and I'm sure I'll be back with more questions soon. >> >> Thanks again for your help! >> >> Andy. >> >> >> >> >> >> >> >> On Sat, Jan 11, 2020 at 6:03 PM Charles Givre <[email protected]> wrote: >> >>> HI Andy, >>> Thanks for your interest in Drill. I'm glad to see that Paul wrote you >>> back as well. I was going to say I thought the JDBC storage plugin did in >>> fact push down columns and filters to the source system. >>> >>> Also, what version of Drill are you using? >>> >>> Writing a storage plugin for Drill is not trivial and I'd definitely >>> recommend using the code from Paul's PR as that greatly simplifies things. >>> Here is a tutorial as well: >>> https://github.com/paul-rogers/drill/wiki/Create-a-Storage-Plugin >>> >>> If you need additional help, please let us know. >>> -- C >>> >>> >>> On Jan 11, 2020, at 5:57 PM, Andy Grove <[email protected]> wrote: >>> >>> Hi, >>> >>> I'd like to use Apache Drill with a custom data source that supports a >>> subset of SQL. >>> >>> My goal is to have Drill push selection and predicates down to my data >>> source but the rest of the query processing should take place in Drill. >>> >>> I started out by writing a JDBC driver for the data source and >>> registering >>> that with Drill using the Jdbc Storage Plugin but it seems to just pass >>> the >>> whole query through to my data source, so that approach isn't going to >>> work >>> unless I'm missing something? >>> >>> Is there any way to configure the JDBC storage plugin to only push >>> certain >>> parts of the query to the data source? >>> >>> If this isn't a good approach, do I need to write a custom storage >>> plugin? >>> Can these be added on the classpath or would that require me maintaining >>> a >>> fork of the project? >>> >>> >>> >>> I appreciate any pointers anyone can give me. >>> >>> Thanks, >>> >>> Andy. >>> >>> >>>
