I think the main difference between the two is that Druid's built-in SQL uses Druid classes internally to model things, rather than just sticking to the HTTP API like the Druid rules in Calcite. If you think it makes sense for Calcite's Druid adapter to depend on Druid jars to get those classes too, then I think it makes sense to collaborate to try to keep the rule sets in sync. Possibly by having Calcite depend on druid-sql and get the rules that way, or vice-versa by having Druid depend on calcite-druid. But if not, I think close collaboration would be tough since rules would need to be rewritten to be ported from Calcite to Druid or vice versa.
Gian On Mon, Dec 19, 2016 at 11:40 AM, Julian Hyde <jh...@apache.org> wrote: > Gian, > > It’s off-topic for this thread, but I wanted to ask whether you have any > plans to keep the Calcite rules in Druid in sync with the Druid rules in > Calcite (the “druid adapter”). > > My goal is to enable two modes of operation: (a) native SQL for Druid, (b) > hybrid systems that combine Druid rules with rules for other stores (e.g. > Hbase) and other processing engines (e.g. Drill, Spark, Hive). We have > something of a fork right now, and that’s not bad per se, but it will be > bad if one of the copies of the code starts to fall behind in functionality. > > Julian > > > > On Dec 19, 2016, at 10:57 AM, Gian Merlino <g...@imply.io> wrote: > > > > Fwiw, Druid has a built-in set of Calcite rules now, and it does support > > pushing down order by and limit. It can even re-order the grouping > > dimensions for better performance, and choose timeseries, topN, or > groupBy > > as appropriate. Depending on what you're trying to do, this may work for > > you. The code is in master, but not yet in a release. > > > > Gian > > > > On Mon, Dec 19, 2016 at 1:43 AM, Gupta, Shuchi <shuchi.gup...@ca.com> > wrote: > > > >> I could see that if order by is present in the query, Calcite is > handling > >> this sorting on its own. Can you tell me the class where this sorting is > >> happening. > >> > >> I tried to debug in SqlParserImpl class , in the OrderBy(boolean accept) > >> method. But while debugging it appears this code is not being used. > >> > >> -----Original Message----- > >> From: Julian Hyde [mailto:jh...@apache.org] > >> Sent: Monday, December 19, 2016 12:22 PM > >> To: dev@calcite.apache.org > >> Subject: Re: Pushing order by to source > >> > >> The solution is simple — someone needs to fix CALCITE-1206. > Contributions > >> welcome. > >> > >>> On Dec 18, 2016, at 10:45 PM, Gupta, Shuchi <shuchi.gup...@ca.com> > >> wrote: > >>> > >>> Hi Julian > >>> > >>> The image is a clip from : https://urldefense.proofpoint. > >> com/v2/url?u=https-3A__calcite.apache.org_docs_druid- > >> 5Fadapter.html&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxf > >> kkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m= > >> nESklzzFnQyTkJm40O_uL3ZyMWQfnBNuQyU_AZBBNhs&s=D3wrMo9-XzMRmZl1OI_ > >> bGEQncVEtyaDWyD4MgJNrrXQ&e= <https://urldefense. > >> proofpoint.com/v2/url?u=https-3A__calcite.apache.org_docs_ > >> druid-5Fadapter.html&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxf > >> kkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m= > >> nESklzzFnQyTkJm40O_uL3ZyMWQfnBNuQyU_AZBBNhs&s=D3wrMo9-XzMRmZl1OI_ > >> bGEQncVEtyaDWyD4MgJNrrXQ&e= >which discusses issue : [CALCITE-1206] > >>> > >>> "That plan shows that Calcite was able to push down the GROUP BY part > of > >> the query to Druid, including the COUNT(*) function, but not the ORDER > BY > >> ... LIMIT. (We plan to lift this restriction; see [CALCITE-1206].)" > >>> > >>> -----Original Message----- > >>> From: Julian Hyde [mailto:jh...@apache.org <mailto:jh...@apache.org>] > >>> Sent: Monday, December 19, 2016 12:07 PM > >>> To: dev@calcite.apache.org <mailto:dev@calcite.apache.org> > >>> Subject: Re: Pushing order by to source > >>> > >>> The image didn’t come through in your email. Could you convert to text, > >> or post an image hosted elsewhere? > >>> > >>>> On Dec 18, 2016, at 10:24 PM, Gupta, Shuchi <shuchi.gup...@ca.com> > >> wrote: > >>>> > >>>> Dear Calcite Developers > >>>> > >>>> I am referring to Calcite-Druid and could see an open issue to support > >> pushdown order by. > >>>> > >>>> > >>>> > >>>> > >>>> My custom database(based on elastic search) will return 100 records by > >> default at a time, unless a limit is specified. > >>>> Here, for a use case with order by and limit in place, limit > definitely > >> would have to be pushed to source. Of course, order by needs to be > applied > >> before limit, we need to pass it to source too. > >>>> > >>>> Could you please suggest a way out. > >>>> Your suggestions in this regard would be highly appreciated. > >>>> > >>>> Thanks and Regards > >>>> Shuchi Gupta > >>>> Software Engineer (CA) > >>>> > >>>> CA Technologies | Plot 115/1, 115/16-115/21 | Hyderabad, 500032 > >>>> Office: +71262 | shuchi.gup...@ca.com > >>>> > >>>> <mailto:shuchi.gup...@ca.com <mailto:shuchi.gup...@ca.com>> < > >> http://www.ca.com/us/default.aspx <http://www.ca.com/us/default.aspx>> > < > >> https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter. > com_CAInc&d= > >> DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_ > >> BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=AbKoujKDKHK07fU7vUzCTVq1jh- > >> hu70PQgQ6btO15h0&s=NKfWiWlYFTnagSsZ3eHKWOzBNoQkTpcO-8OdlF2nhfI&e= < > >> https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter. > com_CAInc&d= > >> DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_ > >> BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=AbKoujKDKHK07fU7vUzCTVq1jh- > >> hu70PQgQ6btO15h0&s=NKfWiWlYFTnagSsZ3eHKWOzBNoQkTpcO-8OdlF2nhfI&e=> > < > >> https://urldefense.proofpoint.com/v2/url?u=http- > >> 3A__www.slideshare.net_cainc&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxf > >> kkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m= > >> AbKoujKDKHK07fU7vUzCTVq1jh-hu70PQgQ6btO15h0&s=0PSRyw5KQS6JsRWn9- > >> bIrM4uoep1OPfJNE7v5FL8-5I&e= <https://urldefense. > >> proofpoint.com/v2/url?u=http-3A__www.slideshare.net_cainc&d=DwIFaQ&c=_ > >> hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_ > >> BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=AbKoujKDKHK07fU7vUzCTVq1jh- > >> hu70PQgQ6btO15h0&s=0PSRyw5KQS6JsRWn9-bIrM4uoep1OPfJNE7v5FL8-5I&e=> > < > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.facebook.com_ > >> CATechnologies&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxf > >> kkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m= > >> AbKoujKDKHK07fU7vUzCTVq1jh-hu70PQgQ6btO15h0&s= > >> d4IxMhqg2zJ7qb3TujVJen6cJPr7mifND9ibWnD-JmA&e= <https://urldefense. > >> proofpoint.com/v2/url?u=https-3A__www.facebook.com_ > >> CATechnologies&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxf > >> kkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m= > >> AbKoujKDKHK07fU7vUzCTVq1jh-hu70PQgQ6btO15h0&s= > >> d4IxMhqg2zJ7qb3TujVJen6cJPr7mifND9ibWnD-JmA&e=> > <https://urldefense. > >> proofpoint.com/v2/url?u=http-3A__www.youtube.com_user_ > >> catechnologies&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxf > >> kkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m= > >> AbKoujKDKHK07fU7vUzCTVq1jh-hu70PQgQ6btO15h0&s= > >> JPbD4L92sDXh3qOx83yFKMR2xMhX2x9_fMlI-aY16L4&e= <https://urldefense. > >> proofpoint.com/v2/url?u=http-3A__www.youtube.com_user_ > >> catechnologies&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxf > >> kkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m= > >> AbKoujKDKHK07fU7vUzCTVq1jh-hu70PQgQ6btO15h0&s= > >> JPbD4L92sDXh3qOx83yFKMR2xMhX2x9_fMlI-aY16L4&e=> > <https://urldefense. > >> proofpoint.com/v2/url?u=http-3A__www.linkedin.com_company_ > >> 1372-3Fgoback-3D.cps-5F1244823420724-5F1&d=DwIFaQ&c=_ > >> hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_ > >> BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=AbKoujKDKHK07fU7vUzCTVq1jh- > >> hu70PQgQ6btO15h0&s=Y5qwUboGHA_4Em2wQlcmcsvQrCQPLPb5ubMlUqwqIBw&e= < > >> https://urldefense.proofpoint.com/v2/url?u=http- > >> 3A__www.linkedin.com_company_1372-3Fgoback-3D.cps- > >> 5F1244823420724-5F1&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxf > >> kkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m= > >> AbKoujKDKHK07fU7vUzCTVq1jh-hu70PQgQ6btO15h0&s=Y5qwUboGHA_ > >> 4Em2wQlcmcsvQrCQPLPb5ubMlUqwqIBw&e=> > <https://urldefense. > >> proofpoint.com/v2/url?u=https-3A__plus.google.com_- > >> 2BCATechnologies&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxf > >> kkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m= > >> AbKoujKDKHK07fU7vUzCTVq1jh-hu70PQgQ6btO15h0&s= > >> Aet4z5ajwvD5r7049sMVcnxREu4ET88HmWE1I7sYaRg&e= <https://urldefense. > >> proofpoint.com/v2/url?u=https-3A__plus.google.com_- > >> 2BCATechnologies&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxf > >> kkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m= > >> AbKoujKDKHK07fU7vUzCTVq1jh-hu70PQgQ6btO15h0&s= > >> Aet4z5ajwvD5r7049sMVcnxREu4ET88HmWE1I7sYaRg&e=> > < > >> http://www.ca.com/us/rss.aspx?intcmp=footernav < > http://www.ca.com/us/rss. > >> aspx?intcmp=footernav>> > >> > >> > >
