John, I think the reason we haven't been bit as much by this is that Drill rewrites the entire tree right before going back to names. With most operators, this will regenerate and propagate the names in the row types to be consistent. This would be guaranteed if you enforce that requirement it in your own rels (which I assume you're at when you're about to translate back to your internal representation). I don't know the conversion to AST code well enough to know if this would help you but it might get you around this issue short term.
On Wed, Jan 28, 2015 at 11:27 AM, Julian Hyde <[email protected]> wrote: > John, > > First of all, let me say I feel your pain. > > But a client should not make assumptions about the names that Calcite > assigns. (It’s like using a Java HashMap and making assumptions about > iteration order. If the iteration order changes from one version of the JDK > to the next, it is not a bug in the JDK; it is a bug in your app.) There > have been discussions about this issue on this list recently. > > Hive needs to use Calcite in the way it was designed — where you identify > fields by position, not by name. How long would you need to fix Hive to do > this? If you can commit to fixing Hive by a particular date, then I will > accept the patch short-term. The modifications would go in deprecated (so > no one else starts using them) and I would log an issue to remove them. > > By the way, I have fixed the other issues. As soon as this is resolved, we > can make another RC. > > Julian > > > On Jan 28, 2015, at 10:17 AM, John Pullokkaran < > [email protected]> wrote: > > > I filed CALCITE-575 to keep track of this. > > I have attached a patch to it. > > > > Patch fixes many of the issues for Hive (~ 25). > > > > > > On Tue, Jan 27, 2015 at 3:12 PM, John Pullokkaran < > > [email protected]> wrote: > > > >> Unfortunately this requires a patch to Calcite. > >> I am testing the patch right now. > >> > >> So far changes are to RelFieldTrimmer, RexUtil, ProjectRemoveRule. > >> > >> On Tue, Jan 27, 2015 at 10:55 AM, John Pullokkaran < > >> [email protected]> wrote: > >> > >>> Many of Hive issues seems to be the result of CALCITE-92. > >>> > >>> I don’t think there is anything wrong with the patch itself, but it > >>> breaks Hive’s usage of Calcite (due to Calcite OP tree getting > converted > >>> back to Hive AST). > >>> > >>> I am going to try disabling it in Hive’s usage of Calcite; will update > >>> soon. > >>> > >>> John > >>> > >>> On Jan 27, 2015, at 9:38 AM, Julian Hyde <[email protected]<mailto: > >>> [email protected]>> wrote: > >>> > >>> On Mon, Jan 26, 2015 at 10:11 PM, John Pullokkaran > >>> <[email protected]<mailto:[email protected]>> > >>> wrote: > >>> > >>> On Mon, Jan 26, 2015 at 1:42 PM, John Pullokkaran < > >>> [email protected]<mailto:[email protected]>> > wrote: > >>> > >>> We seems to have issues with union, subquery, and constants. > >>> > >>> Currently we are debugging these to find more details. > >>> > >>> Would update EOB today. > >>> > >>> So far 2 issues been discovered with Calcite RC; CALCITE-570, > CALCITE-571. > >>> > >>> John, > >>> > >>> Is that your final list? I have a fix for > >>> https://issues.apache.org/jira/browse/CALCITE-570, am working on > >>> https://issues.apache.org/jira/browse/CALCITE-571. I intend to produce > >>> a new RC including these and > >>> https://issues.apache.org/jira/browse/CALCITE-567 and > >>> https://issues.apache.org/jira/browse/CALCITE-568 (both issues that > >>> Vladimir found in RC1). If your list is final RC2 could be as early as > >>> end of today. > >>> > >>> When there is an RC2, would it help to have a few days to review it > >>> before I start a vote? > >>> > >>> Julian > >>> > >>> > >> > > > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > >
