Agree with Ted and Julian's comments and would add one more point: The Planner registers the Storage plugin optimizer rules from all the plugins [1]. The assumption is that a single query could be querying multiple data sources, joining them in Drill etc, so it is the rule's responsibility to have the proper constructor and matches() method to specialize it. For example, if you have a logical planning rule and have a DrillScanRel which is supposed to have a GroupScan of JDBCGroupScan, you could check that in your matches() and return False if otherwise.
[1] https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/PlannerPhase.java#L197 Aman On Sun, Jul 2, 2017 at 11:47 AM, Julian Hyde <[email protected]> wrote: > What Ted said. > > But also, conversely, you should know that in Calcite you can write a > general-purpose rule. Or better, re-use a general-purpose rule that someone > else has written. There are logical rules, for example constant reduction > and logic simplification, that work regardless of the data source. And > there are common patterns, for example pushing down projects and filters, > that apply to many different data sources. You would want to push down > projects and filters to Parquet just as would would want to push them into > a JDBC source, albeit that Parquet cannot handle as rich an expression > language. > > General-purpose rules often take classes as constructor parameters, so you > can instantiate the rule to look for a FooProject.class rather than a > JdbcProject.class or LogicalProject.class. > > Julian > > > > On Jul 2, 2017, at 11:08 AM, Ted Dunning <[email protected]> wrote: > > > > It all depends on how you write your rules. If you write them so that > they > > apply too generally, then the rules themselves are at fault. > > > > If you write rules that only optimize for your input format, then you > > should be fine. > > > > > > On Jul 2, 2017 9:41 AM, "Muhammad Gelbana" <[email protected]> wrote: > > > >> I wrote a plugin for a custom JDBC datasource. This plugin registers a > >> couple of rules. > >> > >> When I ran an SQL query that uses parquet files, I found that a rule of > my > >> JDBC plugin was invoked to optimize the query ! > >> > >> I believe this is a mistake. Please correct me if I'm wrong. > >> > >> I'm saying this is a mistake because a rule registered by a plugin that > >> utilizes a specific datasource should only be concerned about queries > >> executed by that plugin. > >> > >> Query optimizations for a JDBC datasource won't probably work for > queries > >> targeted for parquet files ! > >> > >> What do you think ? > >> > >> Gelbana > >> > >
