As I said before and in the bug the reason for Hive¹s dependency on names is due to the fact that Calcite plan gets translated to Hive AST. Hive Gen Plan enforces constraints on certain ops (like in the case of union, the field names should match).
Once Hive translates Calcite Ops to Hive Ops directly, this problem would go away. The work is scheduled to be completed in another 3 months or so. On 1/28/15, 11:27 AM, "Julian Hyde" <[email protected]> wrote: >John, > >First of all, let me say I feel your pain. > >But a client should not make assumptions about the names that Calcite >assigns. (It¹s like using a Java HashMap and making assumptions about >iteration order. If the iteration order changes from one version of the >JDK to the next, it is not a bug in the JDK; it is a bug in your app.) >There have been discussions about this issue on this list recently. > >Hive needs to use Calcite in the way it was designed ‹ where you identify >fields by position, not by name. How long would you need to fix Hive to >do this? If you can commit to fixing Hive by a particular date, then I >will accept the patch short-term. The modifications would go in >deprecated (so no one else starts using them) and I would log an issue to >remove them. > >By the way, I have fixed the other issues. As soon as this is resolved, >we can make another RC. > >Julian > > >On Jan 28, 2015, at 10:17 AM, John Pullokkaran ><[email protected]> wrote: > >> I filed CALCITE-575 to keep track of this. >> I have attached a patch to it. >> >> Patch fixes many of the issues for Hive (~ 25). >> >> >> On Tue, Jan 27, 2015 at 3:12 PM, John Pullokkaran < >> [email protected]> wrote: >> >>> Unfortunately this requires a patch to Calcite. >>> I am testing the patch right now. >>> >>> So far changes are to RelFieldTrimmer, RexUtil, ProjectRemoveRule. >>> >>> On Tue, Jan 27, 2015 at 10:55 AM, John Pullokkaran < >>> [email protected]> wrote: >>> >>>> Many of Hive issues seems to be the result of CALCITE-92. >>>> >>>> I don¹t think there is anything wrong with the patch itself, but it >>>> breaks Hive¹s usage of Calcite (due to Calcite OP tree getting >>>>converted >>>> back to Hive AST). >>>> >>>> I am going to try disabling it in Hive¹s usage of Calcite; will update >>>> soon. >>>> >>>> John >>>> >>>> On Jan 27, 2015, at 9:38 AM, Julian Hyde <[email protected]<mailto: >>>> [email protected]>> wrote: >>>> >>>> On Mon, Jan 26, 2015 at 10:11 PM, John Pullokkaran >>>> <[email protected]<mailto:[email protected]>> >>>> wrote: >>>> >>>> On Mon, Jan 26, 2015 at 1:42 PM, John Pullokkaran < >>>> [email protected]<mailto:[email protected]>> >>>>wrote: >>>> >>>> We seems to have issues with union, subquery, and constants. >>>> >>>> Currently we are debugging these to find more details. >>>> >>>> Would update EOB today. >>>> >>>> So far 2 issues been discovered with Calcite RC; CALCITE-570, >>>>CALCITE-571. >>>> >>>> John, >>>> >>>> Is that your final list? I have a fix for >>>> https://issues.apache.org/jira/browse/CALCITE-570, am working on >>>> https://issues.apache.org/jira/browse/CALCITE-571. I intend to produce >>>> a new RC including these and >>>> https://issues.apache.org/jira/browse/CALCITE-567 and >>>> https://issues.apache.org/jira/browse/CALCITE-568 (both issues that >>>> Vladimir found in RC1). If your list is final RC2 could be as early as >>>> end of today. >>>> >>>> When there is an RC2, would it help to have a few days to review it >>>> before I start a vote? >>>> >>>> Julian >>>> >>>> >>> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or >>entity to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the >>reader >> of this message is not the intended recipient, you are hereby notified >>that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender >>immediately >> and delete it from your system. Thank You. >
