Re: Improvements to storage plugin planning integration support

Hi Aman Sinha, Yes, Phoenix uses materialization in Calcite to model its secondary index querying. But it's not right to say "In that sense, it would seem to fit into physical planning phase rather than logical, since indexes are a faster physical access mechanism for a scan. The logical

Re: Improvements to storage plugin planning integration support

I do not know how Phoenix's planning works. For Drill, my understanding is during logical planning, "collation" trait is only used in SortRemoveRule, to remove the redundant sort operator. (Those "sort" operators are the one created by Calcite for user-explicit "ORDER BY" / "LIMIT", not the

Re: Improvements to storage plugin planning integration support

Hi Aman, Thought these entries might be related to our discussion about non-covering indices just now: https://issues.apache.org/jira/browse/CALCITE-772 https://issues.apache.org/jira/browse/CALCITE-773 Thanks, Maryann On Thu, Oct 22, 2015 at 2:48 PM, Aman Sinha wrote:

Re: Improvements to storage plugin planning integration support

Thanks Maryann. On Thu, Oct 22, 2015 at 1:04 PM, Maryann Xue wrote: > Hi Aman, > > Thought these entries might be related to our discussion about > non-covering indices just now: > > https://issues.apache.org/jira/browse/CALCITE-772 >

Re: Improvements to storage plugin planning integration support

Catching up on this thread. Jacques, if I understand correctly, you are proposing that instead of the single point of initialization of rules when we instantiate FrameworkConfig (in DrillSqlWorker), we would have more entry points to plug into different phases of planning and storage plugins

Re: Improvements to storage plugin planning integration support

The dead air must mean that everyone is onboard with my recommendation PlannerIntegration StoragePlugin.getPlannerIntegrations() interface PlannerIntegration{ void initialize(Planner, Phase) } Right :D -- Jacques Nadeau CTO and Co-Founder, Dremio On Fri, Oct 9, 2015 at 7:03 AM, Jacques

Re: Improvements to storage plugin planning integration support

I would +1 (1-3) for sure. I do not have much understanding of programs however additional flexibility for storage plugin devs sounds cool in general when used responsibly =) so +0 for (4) -H+ On Mon, Oct 12, 2015 at 4:12 PM, Jacques Nadeau wrote: > The dead air must mean

Re: Improvements to storage plugin planning integration support

Dead air means I’ve had ‘flu and been busy… can you give me another day to think about this? > On Oct 12, 2015, at 4:12 PM, Jacques Nadeau wrote: > > The dead air must mean that everyone is onboard with my recommendation > > PlannerIntegration

Re: Improvements to storage plugin planning integration support

Of course! This was just my tongue-in-cheek way of saying *bump* -- Jacques Nadeau CTO and Co-Founder, Dremio On Mon, Oct 12, 2015 at 5:37 PM, Julian Hyde wrote: > Dead air means I’ve had ‘flu and been busy… can you give me another day to > think about this? > > > On

Improvements to storage plugin planning integration support

A number of us were meeting last week to work through integrating the Phoenix storage plugin. This plugin is interesting because it also uses Calcite for planning. In some ways, this should make integration easy. However, it also allowed us to see certain constraints who how we expose planner