> > This is old code, hard to attribute to jiras > Okay, HIVE-2337 was way back in release 0.8.0 so I think we can safely omit version information. If anyone objects, I can link the wikidoc to that ticket. (And I've marked HIVE-2337 as being related to HIVE-279 in the JIRA.)
Thanks for all the help. -- Lefty On Wed, Dec 11, 2013 at 7:03 PM, Harish Butani <hbut...@hortonworks.com>wrote: > I see. Let's leave it in. > > This is old code, hard to attribute to jiras: > - The PPD code comes from: HIVE-279, HIVE-2337 > - I cannot tell when the join condition parsing code was added. > > regards, > Harish. > > On Dec 11, 2013, at 5:17 PM, Lefty Leverenz <leftylever...@gmail.com> > wrote: > > > Maybe we should remove the section on Hive Implementation here. > > It is in the Design doc; this information only concerns developers. > > But this is the Design doc (unless there's another one somewhere -- maybe > attached to a JIRA ticket?) and it's in the Resources for Contributors part > of the wiki, so it seems appropriate to me. I'll delete the implementation > section if that's your preference. > > Here are the links again, with fixes: > > - Design Docs<https://cwiki.apache.org/confluence/display/Hive/DesignDocs> > (bottom > of list) > - Predicate Pushdown > Rules<https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-PredicatePushdownRules> > > > Speaking of JIRA tickets, is there one for this and should I add any > version information? > > -- Lefty > > > On Wed, Dec 11, 2013 at 7:59 AM, Harish Butani <hbut...@hortonworks.com>wrote: > >> getQualifiedAliases is a private method in JoinPPD. >> >> Maybe we should remove the section on Hive Implementation here. It is in >> the Design doc; this information only concerns developers. >> >> regards, >> Harish. >> >> >> On Dec 11, 2013, at 3:05 AM, Lefty Leverenz <leftylever...@gmail.com> >> wrote: >> >> Happy to fix the sentence and the link. I pointed out the name change >> just so you would review it, so please don't apologize! >> >> One more question: why am I not finding getQualifiedAliases() in the >> SemanticAnalyzer class? It turns up in OpProcFactory.java with javadoc >> comments, but I can't find it anywhere in the API docs -- not even in the >> index (Hive 0.12.0 API <http://hive.apache.org/docs/r0.12.0/api/>): >> >> *getQMap()*<http://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/ql/QTestUtil.html#getQMap()> >> - >> Method in class >> org.apache.hadoop.hive.ql.QTestUtil<http://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/ql/QTestUtil.html> >> >> *getQualifiedName()*<http://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.html#getQualifiedName()> >> - >> Method in class >> org.apache.hadoop.hive.serde2.typeinfo.TypeInfo<http://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.html> >> String >> representing the qualified type >> name.*getQualifiers()*<http://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html#getQualifiers()> >> - >> Method in class >> org.apache.hive.service.cli.thrift.TTypeQualifiers<http://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html> >> >> *getQualifiersSize()*<http://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html#getQualifiersSize()> >> - >> Method in class >> org.apache.hive.service.cli.thrift.TTypeQualifiers<http://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html> >> Most mysterious. >> >> -- Lefty >> >> >> On Tue, Dec 10, 2013 at 2:35 PM, Harish Butani >> <hbut...@hortonworks.com>wrote: >> >>> I can see why you would rename. >>> >>> But this sentence is not correct: >>> 'Hive enforces the predicate pushdown rules by these methods in the >>> SemanticAnalyzer and JoinPPD classes:' >>> >>> It should be: >>> Hive enforces the rules by these methods in the SemanticAnalyzer and >>> JoinPPD classes: >>> >>> (The implementation involves both predicate pushdown and analyzing join >>> conditions) >>> Sorry about this. >>> >>> So the link should say 'Hive Outer Join Behavior' >>> >>> regards, >>> Harish. >>> >>> >>> On Dec 10, 2013, at 2:01 PM, Lefty Leverenz <leftylever...@gmail.com> >>> wrote: >>> >>> How's this? Hive >>> Implementation<https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-HiveImplementation> >>> >>> Also, I moved the link on the Design Docs >>> page<https://cwiki.apache.org/confluence/display/Hive/DesignDocs>from >>> *Proposed* to *Other*. (It's called SQL Outer Join Predicate Pushdown >>> Rules<https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior> >>> which >>> doesn't match the title, but seems okay because it's more descriptive.) >>> >>> -- Lefty >>> >>> >>> On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani >>> <hbut...@hortonworks.com>wrote: >>> >>>> You are correct, it is plural. >>>> >>>> regards, >>>> Harish. >>>> >>>> On Dec 10, 2013, at 4:03 AM, Lefty Leverenz <leftylever...@gmail.com> >>>> wrote: >>>> >>>> Okay, then monospace with "()" after the method name is a good way to >>>> show them: parseJoinCondition() and getQualifiedAlias() ... but I >>>> only found the latter pluralized, instead of singular, so should it be >>>> getQualifiedAliases() or am I missing something? >>>> >>>> trunk> *grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'* >>>>> >>>>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221: >>>>>> * the comments for getQualifiedAliases function. >>>>> >>>>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230: >>>>>> Set<String> aliases = getQualifiedAliases((JoinOperator) nd, owi >>>>> >>>>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242: >>>>>> // be pushed down per getQualifiedAliases >>>>> >>>>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471: >>>>>> private Set<String> getQualifiedAliases(JoinOperator op, >>>>>> RowResolver rr) { >>>>> >>>>> >>>>> >>>> -- Lefty >>>> >>>> >>>> On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani >>>> <hbut...@hortonworks.com>wrote: >>>> >>>>> Looks good. Thanks for doing this. >>>>> >>>>> Minor point: >>>>> >>>>> *Rule 1:* During *QBJoinTree* construction in Plan Gen, the parse >>>>> Join Condition logic applies this rule. >>>>> *Rule 2:* During *JoinPPD* (Join Predicate Pushdown) the get >>>>> Qualified Alias logic applies this rule. >>>>> >>>>> FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the >>>>> SemanticAnalyzer and JoinPPD classes respectively. >>>>> Writing these as separate words maybe confusing. You are better judge >>>>> of how to represent this(quoted/bold etc.) >>>>> >>>>> regards, >>>>> Harish. >>>>> >>>>> >>>>> On Dec 9, 2013, at 1:52 AM, Lefty Leverenz <leftylever...@gmail.com> >>>>> wrote: >>>>> >>>>> The Outer Join Behavior >>>>> wikidoc< >>>>> https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior>is >>>>> >>>>> done, with links from the Design >>>>> Docs <https://cwiki.apache.org/confluence/display/Hive/DesignDocs> >>>>> page and >>>>> the Joins doc< >>>>> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization >>>>> > >>>>> . >>>>> >>>>> Harish (or anyone else) would you please review the changes I made to >>>>> the definition >>>>> for "Null Supplying >>>>> table"< >>>>> https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-Definitions >>>>> > >>>>> >>>>> ? >>>>> >>>>> -- Lefty >>>>> >>>>> >>>>> On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair <the...@hortonworks.com> >>>>> wrote: >>>>> >>>>> :) >>>>> >>>>> >>>>> On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz < >>>>> leftylever...@gmail.com> >>>>> wrote: >>>>> >>>>> Easy as 3.14159.... (I can take a hint.) >>>>> >>>>> -- Lefty >>>>> >>>>> >>>>> On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair <the...@hortonworks.com> >>>>> >>>>> wrote: >>>>> >>>>> >>>>> FYI, Harish has a written a very nice doc describing predicate push >>>>> down rules for join. I have attached it to the design doc page. It >>>>> will be very useful for anyone looking at joins. >>>>> >>>>> >>>>> >>>>> https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html >>>>> >>>>> >>>>> (any help converting it to wiki format from html is welcome!). >>>>> >>>>> -- >>>>> CONFIDENTIALITY NOTICE >>>>> NOTICE: This message is intended for the use of the individual or >>>>> >>>>> entity to >>>>> >>>>> which it is addressed and may contain information that is confidential, >>>>> privileged and exempt from disclosure under applicable law. If the >>>>> >>>>> reader >>>>> >>>>> of this message is not the intended recipient, you are hereby notified >>>>> >>>>> that >>>>> >>>>> any printing, copying, dissemination, distribution, disclosure or >>>>> forwarding of this communication is strictly prohibited. If you have >>>>> received this communication in error, please contact the sender >>>>> >>>>> immediately >>>>> >>>>> and delete it from your system. Thank You. >>>>> >>>>> >>>>> -- >>>>> CONFIDENTIALITY NOTICE >>>>> NOTICE: This message is intended for the use of the individual or >>>>> entity to >>>>> which it is addressed and may contain information that is confidential, >>>>> privileged and exempt from disclosure under applicable law. If the >>>>> reader >>>>> of this message is not the intended recipient, you are hereby notified >>>>> that >>>>> any printing, copying, dissemination, distribution, disclosure or >>>>> forwarding of this communication is strictly prohibited. If you have >>>>> received this communication in error, please contact the sender >>>>> immediately >>>>> and delete it from your system. Thank You. >>>>> >>>>> >>>>> >>>>> CONFIDENTIALITY NOTICE >>>>> NOTICE: This message is intended for the use of the individual or >>>>> entity to which it is addressed and may contain information that is >>>>> confidential, privileged and exempt from disclosure under applicable law. >>>>> If the reader of this message is not the intended recipient, you are >>>>> hereby >>>>> notified that any printing, copying, dissemination, distribution, >>>>> disclosure or forwarding of this communication is strictly prohibited. If >>>>> you have received this communication in error, please contact the sender >>>>> immediately and delete it from your system. Thank You. >>>>> >>>> >>>> >>>> >>>> CONFIDENTIALITY NOTICE >>>> NOTICE: This message is intended for the use of the individual or >>>> entity to which it is addressed and may contain information that is >>>> confidential, privileged and exempt from disclosure under applicable law. >>>> If the reader of this message is not the intended recipient, you are hereby >>>> notified that any printing, copying, dissemination, distribution, >>>> disclosure or forwarding of this communication is strictly prohibited. If >>>> you have received this communication in error, please contact the sender >>>> immediately and delete it from your system. Thank You. >>>> >>> >>> >>> >>> CONFIDENTIALITY NOTICE >>> NOTICE: This message is intended for the use of the individual or entity >>> to which it is addressed and may contain information that is confidential, >>> privileged and exempt from disclosure under applicable law. If the reader >>> of this message is not the intended recipient, you are hereby notified that >>> any printing, copying, dissemination, distribution, disclosure or >>> forwarding of this communication is strictly prohibited. If you have >>> received this communication in error, please contact the sender immediately >>> and delete it from your system. Thank You. >>> >> >> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> > > > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
