Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

://reviews.apache.org/r/2001/#review4912 --- On 2012-01-29 17:56:48, Yin Huai wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2001

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2001/ --- (Updated 2012-02-10 20:49:01.177796) Review request for hive. Changes ---

Re: [ANNOUNCE] New Hive Committer - Brock Noland

Congrats Brock! On Tue, Jul 16, 2013 at 1:26 PM, Lefty Leverenz le...@hortonworks.comwrote: Congratulations hooray, another committer to share the load! – Lefty On Tue, Jul 16, 2013 at 12:49 PM, Mark Grover grover.markgro...@gmail.comwrote: Congrats, Brock! On Tue, Jul 16, 2013

Review Request 12767: [HIVE-4877] In ExecReducer, remove tag from the row which will be passed to the first Operator at the Reduce-side

mac and tests passed. Thanks, Yin Huai

Re: Trying to run TestRCFile.java

Hi Mainak, Can you try 'ant test -Dtestcase=TestRCFile'? It should work. Thanks, Yin On Fri, Jul 19, 2013 at 10:16 AM, Mainak Ghosh gho...@us.ibm.com wrote: Hello, I am new to Hive codebase and was hoping to run some of its test code specifically TestRCFile.java. Unfortunately I am not

Re: Trying to run TestRCFile.java

] private static ColumnarSerDe serDe; Any idea what these are? How to resolve them? I personally do not have a lot of idea on remote debugging Hive. So a step by step manual if present would be of great help. Thanks and Regards, Mainak. [image: Inactive hide details for Yin Huai ---07/19/2013 10

Re: Review Request 12767: [HIVE-4877] In ExecReducer, remove tag from the row which will be passed to the first Operator at the Reduce-side

://reviews.apache.org/r/12767/#review23527 --- On July 19, 2013, 5 p.m., Yin Huai wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12767

Re: Review Request 12767: [HIVE-4877] In ExecReducer, remove tag from the row which will be passed to the first Operator at the Reduce-side

are hbase_stats_empty_partition.q and ppd_key_ranges.q in TestHBaseCliDriver. I manually tested these two in my mac and tests passed. Thanks, Yin Huai

Re: Review Request 12767: [HIVE-4877] In ExecReducer, remove tag from the row which will be passed to the first Operator at the Reduce-side

Errors Success rateTime 26882 0 99.93% 43249.945 Two failures are hbase_stats_empty_partition.q and ppd_key_ranges.q in TestHBaseCliDriver. I manually tested these two in my mac and tests passed. Thanks, Yin Huai

Re: Trying to run TestRCFile.java

. Any idea how I can achieve that? The problem here is my debugger attaches to the live instance of hive and testcase runs using ant. Thanks and Regards, Mainak. [image: Inactive hide details for Yin Huai ---07/19/2013 11:25:20 AM---Hi Mainak, Here are steps:]Yin Huai ---07/19/2013 11:25:20

Re: Review Request 12767: [HIVE-4877] In ExecReducer, remove tag from the row which will be passed to the first Operator at the Reduce-side

p.m., Yin Huai wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12767/ --- (Updated July 19, 2013, 5 p.m.) Review request

Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

, Yin Huai

Re: [ANNOUNCE] New Hive Committer - Gunther Hagleitner

Congratulations Gunther! On Sun, Jul 21, 2013 at 7:55 AM, Jarek Jarcec Cecho jar...@apache.orgwrote: Congratulations Gunther! Jarcec On Sun, Jul 21, 2013 at 01:00:31AM -0700, Carl Steinbach wrote: The Apache Hive PMC has voted to make Gunther Hagleitner a committer on the Apache Hive

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

: https://reviews.apache.org/r/12795/diff/ Testing --- Running tests. Thanks, Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

generated e-mail. To reply, visit: https://reviews.apache.org/r/12795/#review23671 --- On July 22, 2013, 4:19 a.m., Yin Huai wrote: --- This is an automatically generated e-mail

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

. Will report what I get later. - Yin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12795/#review23673 --- On July 22, 2013, 4:19 a.m., Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

. Thanks, Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

/clientpositive/multi_join_union.q.out 5182bdf ql/src/test/results/clientpositive/union34.q.out 166062a Diff: https://reviews.apache.org/r/12795/diff/ Testing --- Running tests. Thanks, Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

/clientpositive/multiMapJoin2.q.out PRE-CREATION ql/src/test/results/clientpositive/multi_join_union.q.out 5182bdf ql/src/test/results/clientpositive/union34.q.out 166062a Diff: https://reviews.apache.org/r/12795/diff/ Testing --- Running tests. Thanks, Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

/multi_join_union.q.out 5182bdf ql/src/test/results/clientpositive/union34.q.out 166062a Diff: https://reviews.apache.org/r/12795/diff/ Testing --- Running tests. Thanks, Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

/test/results/clientpositive/multi_join_union.q.out 5182bdf ql/src/test/results/clientpositive/union34.q.out 166062a Diff: https://reviews.apache.org/r/12795/diff/ Testing --- Running tests. Thanks, Yin Huai

Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

/multiMapJoin2.q.out PRE-CREATION ql/src/test/results/clientpositive/multi_join_union.q.out 5182bdf ql/src/test/results/clientpositive/union34.q.out 166062a Diff: https://reviews.apache.org/r/12795/diff/ Testing --- Running tests. Thanks, Yin Huai

Re: [ANNOUNCE] New Hive Committer - Thejas Nair

Congrats Thejas!!! On Tue, Aug 20, 2013 at 4:54 PM, Thejas Nair the...@hortonworks.com wrote: Thanks everybody! This is a great honor. I hope to live up to the expectations! It is great to see the fresh momentum in hive and I am glad to be able to work with the community on it. Now I will

Re: DISTRIBUTE BY works incorrectly in Hive 0.11 in some cases

Seems ReduceSinkDeDuplication picked the wrong partitioning columns. On Fri, Aug 23, 2013 at 9:15 PM, Shahansad KP s...@rocketfuel.com wrote: I think the problem lies with in the group by operation. For this optimization to work the group bys partitioning should be on the column 1 only. It

Re: DISTRIBUTE BY works incorrectly in Hive 0.11 in some cases

Created a jira https://issues.apache.org/jira/browse/HIVE-5149 On Sun, Aug 25, 2013 at 9:11 PM, Yin Huai huaiyin@gmail.com wrote: Seems ReduceSinkDeDuplication picked the wrong partitioning columns. On Fri, Aug 23, 2013 at 9:15 PM, Shahansad KP s...@rocketfuel.com wrote: I think

Re: DISTRIBUTE BY works incorrectly in Hive 0.11 in some cases

forgot to add in my last reply To generate correct results, you can set hive.optimize.reducededuplication to false to turn off ReduceSinkDeDuplication On Sun, Aug 25, 2013 at 9:35 PM, Yin Huai huaiyin@gmail.com wrote: Created a jira https://issues.apache.org/jira/browse/HIVE-5149

Review Request 13862: ReduceSinkDeDuplication can pick the wrong partitioning columns

Diff: https://reviews.apache.org/r/13862/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 13862: ReduceSinkDeDuplication can pick the wrong partitioning columns

ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 3297ebb Diff: https://reviews.apache.org/r/13862/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 13862: [HIVE-5149] ReduceSinkDeDuplication can pick the wrong partitioning columns

a52f4eb ql/src/test/results/clientpositive/groupby_rollup1.q.out f120471 ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 3297ebb Diff: https://reviews.apache.org/r/13862/diff/ Testing --- Thanks, Yin Huai

Review Request 13910: [HIVE-5186] Remove JoinReducerProc from ReduceSinkDeDuplication

c380a2d ql/src/test/queries/clientpositive/reduce_deduplicate_extended.q a5e9cdf ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 3297ebb Diff: https://reviews.apache.org/r/13910/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 13862: [HIVE-5149] ReduceSinkDeDuplication can pick the wrong partitioning columns

On Sept. 2, 2013, 5:39 a.m., Yin Huai wrote: Another sanity check. Done. - Yin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13862/#review25819

Re: Review Request 13862: [HIVE-5149] ReduceSinkDeDuplication can pick the wrong partitioning columns

On Sept. 2, 2013, 5:29 a.m., Yin Huai wrote: Thanks for adding comments! We can have a query like ... explain select * from (select * from src1 cluster by key) tmp sort by key, value; In this case, at first, we have two MR jobs. Since the second job is used for sort

Re: Review Request 13862: [HIVE-5149] ReduceSinkDeDuplication can pick the wrong partitioning columns

.q.out f120471 ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 3297ebb Diff: https://reviews.apache.org/r/13862/diff/ Testing --- Thanks, Yin Huai

Re: [ANNOUNCE] New Hive Committer - Yin Huai

! On Sep 4, 2013, at 6:49 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Yin Huai a committer on the Apache Hive project. Please join me in congratulating Yin! Thanks. Carl -- Alexander Alten-Lorenz http

Re: [jira] [Commented] (HIVE-5149) ReduceSinkDeDuplication can pick the wrong partitioning columns

Sure. Let me take a look at 0.11 branch. On Wed, Sep 4, 2013 at 2:53 PM, Eric Chu e...@rocketfuel.com wrote: Hi Yin Huai, This patch doesn't apply cleanly in Hive 11. For example, ReduceSinkDeDuplication.java is in ql/src/java/org/apache/hadoop/hive/ql/optimizer/ in Hive 11 while the patch

Re: [jira] [Commented] (HIVE-5149) ReduceSinkDeDuplication can pick the wrong partitioning columns

Hi Eric, Can you try https://issues.apache.org/jira/secure/attachment/12601446/HIVE-5149-branch-0.11.patch? Thanks, Yin On Wed, Sep 4, 2013 at 3:14 PM, Yin Huai huaiyin@gmail.com wrote: Sure. Let me take a look at 0.11 branch. On Wed, Sep 4, 2013 at 2:53 PM, Eric Chu e

Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarStruct.java 0317024 serde/src/test/org/apache/hadoop/hive/serde2/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

--- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

/ColumnProjectionUtils.java https://reviews.apache.org/r/14221/#comment51371 ok will remove it. serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java https://reviews.apache.org/r/14221/#comment51372 ok, will also remove other null check - Yin Huai On Sept. 19, 2013, 5:48

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

/serde2/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

/hadoop/hive/serde2/columnar/LazyBinaryColumnarStruct.java 0317024 serde/src/test/org/apache/hadoop/hive/serde2/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

/columnar/LazyBinaryColumnarStruct.java 0317024 serde/src/test/org/apache/hadoop/hive/serde2/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

/columnar/LazyBinaryColumnarStruct.java 0317024 serde/src/test/org/apache/hadoop/hive/serde2/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc

/TestColumnProjectionUtils.java PRE-CREATION Diff: https://reviews.apache.org/r/14221/diff/ Testing --- Thanks, Yin Huai

Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2001/ --- Review request for hive. Summary --- This optimizer exploits intra-query

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2001/ --- (Updated 2011-12-29 18:50:12.277210) Review request for hive. Summary ---

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2001/ --- (Updated 2012-01-29 17:56:48.704757) Review request for hive. Changes ---

Re: Custom SerDe: Initialize() passes a null configuration to my Custom SerDe

Can you try to set serde properties? https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AddSerDeProperties I have not tried it, but seems it is the right way to pass configurations to serde class. Thanks, Yin On Mon, Oct 14, 2013 at 8:20 AM, Rui Martins

Re: VOTE: Remove phabricator instructions from hive-development guide (wiki), officially only support Apache's review board.

+1 On Thu, Oct 17, 2013 at 5:51 PM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: +1 Thanks, Gunther. On Thu, Oct 17, 2013 at 2:18 PM, Owen O'Malley omal...@apache.org wrote: Ed, I didn't remember being unable to see revisions without a login. That is uncool. I'll

Review Request 15101: HIVE-5697: Correlation Optimizer may generate wrong plans for cases involving outer join

/src/test/results/clientpositive/correlationoptimizer1.q.out dc46a77 Diff: https://reviews.apache.org/r/15101/diff/ Testing --- HiveQA Thanks, Yin Huai

Re: [ANNOUNCE] New Hive Committer - Xuefu Zhang

Congrats Xuefu! On Mon, Nov 4, 2013 at 7:11 AM, Lianhui Wang lianhuiwan...@gmail.comwrote: Congrats Xuefu! 2013/11/4 Thejas Nair the...@hortonworks.com Congrats Xuefu! On Sun, Nov 3, 2013 at 11:11 PM, Mohammad Islam misla...@yahoo.com wrote: Congrats Xuefu! --Mohammad

Does the trunk still have an eclipse formatter?

Hi all, After importing the trunk to eclipse, I found that eclipse did not load the hive formatter. Does trunk still have one? If so, where is the location of it? Thanks, Yin

Re: Review Request 16172: ql.plan.ConditionalResolverCommonJoin.resolveMapJoinTask also sums those tables which are not used in the child of this conditional task.

/ConditionalResolverCommonJoin.java https://reviews.apache.org/r/16172/#comment58602 Let's change it to log the exception instead of printing the stack trace. - Yin Huai On Dec. 11, 2013, 2:12 a.m., Navis Ryu wrote: --- This is an automatically

Re: Review Request 16172: ql.plan.ConditionalResolverCommonJoin.resolveMapJoinTask also sums those tables which are not used in the child of this conditional task.

://reviews.apache.org/r/16172/#comment58622 Seems it is not an error? If so, let's not put it in the ErrorMsg. ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverCommonJoin.java https://reviews.apache.org/r/16172/#comment58623 Is this one necessary? - Yin Huai On Dec. 18, 2013, 5

Re: Timeline for the Hive 0.13 release?

Guys, Seems ConditionalResolverCommonJoin is not working correctly? I created https://issues.apache.org/jira/browse/HIVE-6668 and set it as a blocker. thanks, Yin On Fri, Mar 14, 2014 at 11:34 AM, Thejas Nair the...@hortonworks.comwrote: Can you also add HIVE-6647

Re: [ANNOUNCE] New Hive PMC Member - Gunther Hagleitner

Congrats Gunther! On Fri, Dec 27, 2013 at 10:13 AM, Brock Noland br...@cloudera.com wrote: Congratulations Gunther! On Fri, Dec 27, 2013 at 9:00 AM, Xuefu Zhang xzh...@cloudera.com wrote: Many congratulations, Gunther! --Xuefu On Thu, Dec 26, 2013 at 8:59 PM, Carl Steinbach

Re: Review Request 16172: ql.plan.ConditionalResolverCommonJoin.resolveMapJoinTask also sums those tables which are not used in the child of this conditional task.

/ConditionalResolverCommonJoin.java https://reviews.apache.org/r/16172/#comment59277 For the example in the description of HIVE-5945, the same alias $INTNAME can actually refer to different intermediate tables. So, at here, we will not update the correct size for an alias $INTNAME. - Yin Huai On Dec. 30, 2013, 2

Review Request 16531: HIVE-6083: User provided table properties are not assigned to the TableDesc of the FileSinkDesc in a CTAS query

/ Testing --- Thanks, Yin Huai

Re: Review Request 23270: Wrong results when union all of grouping followed by group by with correlation optimization

Do we need this? ql/src/test/queries/clientpositive/correlationoptimizer16.q https://reviews.apache.org/r/23270/#comment83870 I think correlationoptimizer8 is for cases with UNION ALL. Can we add test queries in that file? - Yin Huai On July 4, 2014, 12:15 a.m., Navis Ryu wrote

Do we support == operator?

Hi, Based on our language manual ( https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF), seems we do not support == operator. However, in FunctionRegistry, we treat == as =. I guess we want to make == invalid and throw an exception when a user uses it? Thanks, Yin

configuring log4j for unit tests

Hello All, When I run hive unit tests, I always can find an error saying the class of org.apache.hadoop.log.metrics.EventCounter can not be found. Seems the reason is that in hadoop 0.20.2, the location of EventCounter is org.apache.hadoop.metrics.jvm.EventCounter, so log4j should load the old

Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

relevant to the patch. Please see https://issues.apache.org/jira/browse/HIVE-2206?focusedCommentId=13456171page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13456171 for details. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

://issues.apache.org/jira/browse/HIVE-2206?focusedCommentId=13456171page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13456171 for details. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

, and skewjoinopt19.q to skewjoinopt20.q, I cannot test these queries on my machine either. All other tests pass. Thanks, Yin Huai

Review Request: HIVE-3495 elements in aggParameters passed to SemanticAnalyzer.getGenericUDAFEvaluator are generated in two different ways

to skewjoinopt17.q, and skewjoinopt19.q to skewjoinopt20.q, I cannot test these queries on my machine either. All other tests pass. Thanks, Yin Huai

Review Request: HIVE-3493 aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

, and skewjoinopt19.q to skewjoinopt20.q, I cannot test these queries on my machine either. All other tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

, and skewjoinopt19.q to skewjoinopt20.q, I cannot test these queries on my machine either. All other tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

to skewjoinopt20.q, I cannot test these queries on my machine either. All other tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

to do that? Thanks. - Yin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7126/#review11858 --- On Sept. 24, 2012, 3:53 p.m., Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

, skewjoinopt10.q, skewjoinopt15.q to skewjoinopt17.q, and skewjoinopt19.q to skewjoinopt20.q, I cannot test these queries on my machine either. All other tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

://reviews.apache.org/r/7126/diff/ Testing (updated) --- All tests pass. Thanks, Yin Huai

Re: Review Request: For UDFAs, when generating a plan without map-side-aggregation, constant agg parameters will be replaced by ExprNodeColumnDesc

/clientpositive/nullgroup4.q.out 3dd3c66 ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out 9ac4bbc Diff: https://reviews.apache.org/r/7221/diff/ Testing (updated) --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3493 aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

) --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3495: For UDFAs, when generating a plan without map-side-aggregation, constant agg parameters will be replaced by ExprNodeColumnDesc

/results/clientpositive/nullgroup4.q.out 3dd3c66 ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out 9ac4bbc Diff: https://reviews.apache.org/r/7221/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3495: For UDAFs, when generating a plan without map-side-aggregation, constant agg parameters will be replaced by ExprNodeColumnDesc

/results/clientpositive/nullgroup4.q.out 3dd3c66 ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out 9ac4bbc Diff: https://reviews.apache.org/r/7221/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3493 aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

/clientpositive/create_view.q 0af3442 ql/src/test/results/clientpositive/create_view.q.out 3b78f1e Diff: https://reviews.apache.org/r/7222/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3493 aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 3f2 ql/src/test/queries/clientpositive/create_view.q 0af3442 ql/src/test/results/clientpositive/create_view.q.out 3b78f1e Diff: https://reviews.apache.org/r/7222/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3495: For UDAFs, when generating a plan without map-side-aggregation, constant agg parameters will be replaced by ExprNodeColumnDesc

/clientpositive/nullgroup4_multi_distinct.q.out 9ac4bbc ql/src/test/results/clientpositive/udaf_percentile_approx.q.out 4f85a82 Diff: https://reviews.apache.org/r/7221/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-3495: For UDAFs, when generating a plan without map-side-aggregation, constant agg parameters will be replaced by ExprNodeColumnDesc

/test/results/clientpositive/nullgroup4_multi_distinct.q.out 9ac4bbc ql/src/test/results/clientpositive/udaf_percentile_approx.q.out 4f85a82 Diff: https://reviews.apache.org/r/7221/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

://reviews.apache.org/r/7126/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

://reviews.apache.org/r/7126/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

/groupby1.q.xml e1eb209 ql/src/test/results/compiler/plan/groupby2.q.xml 268eb90 ql/src/test/results/compiler/plan/groupby3.q.xml 3e02d15 ql/src/test/results/compiler/plan/groupby5.q.xml 89e1207 Diff: https://reviews.apache.org/r/7126/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

/plan/groupby2.q.xml 7b07f02 ql/src/test/results/compiler/plan/groupby3.q.xml a6a1986 ql/src/test/results/compiler/plan/groupby5.q.xml 25e3583 Diff: https://reviews.apache.org/r/7126/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

/groupby5.q.xml 632d88e Diff: https://reviews.apache.org/r/7126/diff/ Testing --- All tests pass. Thanks, Yin Huai

Re: using the Hive SQL parser in Spark

Let me add Reynold to the thread. On Fri, Dec 18, 2015 at 12:36 PM, Gopal Vijayaraghavan wrote: > > >We have looked into various options, and it looks like the best option is > >to copy the ANTLR grammar file from Hive into Spark. Because the grammar > >file is tightly

[jira] [Updated] (HIVE-2206) add a new optimizer for query correlation discovery and optimization

[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-2206: --- Attachment: Queries Two queries (TPC-H Q17 and TPC-H Q18) can be used for testing this optimizer. Q17

[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization

[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13054816#comment-13054816 ] Yin Huai commented on HIVE-2206: The current optimizer can identify correlations with query

[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization

[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13276409#comment-13276409 ] Yin Huai commented on HIVE-2206: @anders, I will try to make this patch more concise

[jira] [Created] (HIVE-4572) ColumnPruner cannot preserve RS key columns in columnExprMap

Yin Huai created HIVE-4572: -- Summary: ColumnPruner cannot preserve RS key columns in columnExprMap Key: HIVE-4572 URL: https://issues.apache.org/jira/browse/HIVE-4572 Project: Hive Issue Type: Bug

[jira] [Updated] (HIVE-4572) ColumnPruner cannot preserve RS key columns in columnExprMap

[ https://issues.apache.org/jira/browse/HIVE-4572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-4572: --- Attachment: HIVE-4572.replay.patch To see the problem, you can apply the patch HIVE-4572.replay.patch

[jira] [Updated] (HIVE-4572) ColumnPruner cannot preserve RS key columns corresponding to un-selected join keys in columnExprMap

[ https://issues.apache.org/jira/browse/HIVE-4572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-4572: --- Summary: ColumnPruner cannot preserve RS key columns corresponding to un-selected join keys in columnExprMap

[jira] [Updated] (HIVE-4572) ColumnPruner cannot preserve RS key columns corresponding to un-selected join keys in columnExprMap

[ https://issues.apache.org/jira/browse/HIVE-4572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-4572: --- Attachment: HIVE-4572.1.patch.txt Add a patch

[jira] [Updated] (HIVE-4572) ColumnPruner cannot preserve RS key columns corresponding to un-selected join keys in columnExprMap

[ https://issues.apache.org/jira/browse/HIVE-4572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-4572: --- Attachment: HIVE-4572.2.patch.txt Update a new patch. With this one, HIVE-2206 can work

[jira] [Commented] (HIVE-4572) ColumnPruner cannot preserve RS key columns corresponding to un-selected join keys in columnExprMap

[ https://issues.apache.org/jira/browse/HIVE-4572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13663236#comment-13663236 ] Yin Huai commented on HIVE-4572: [~hagleitn] Yes, it is correct. I have tested

  1   2   3   4   5   6   7   >