[jira] [Updated] (HIVE-2388) Facing issues while executing commands on hive shell. The system throws following error: only on Windows Cygwin setup
[ https://issues.apache.org/jira/browse/HIVE-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth tiwari updated HIVE-2388: --- Attachment: start.sh Facing issues while executing commands on hive shell. The system throws following error: only on Windows Cygwin setup - Key: HIVE-2388 URL: https://issues.apache.org/jira/browse/HIVE-2388 Project: Hive Issue Type: Bug Components: CLI, Query Processor Affects Versions: 0.7.1 Environment: Cygwin Windows Reporter: Siddharth tiwari Priority: Critical Labels: patch Fix For: 0.7.1 Attachments: start.sh, start.sh Original Estimate: 456h Remaining Estimate: 456h DDL runs well but the following command describes throw error pls help with resolution and how to get about it hive show tables ; FAILED: Hive Internal Error: java.lang.IllegalArgumentException(java.net.URISyntaxException: Relative path in absolute URI: file:C:/cygwin/tmp//siddharth/hive_2011-08-18_ 03-11-05_208_1818592223695168110) java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: file:C:/cygwin/tmp//siddharth/hive_2011-08-18_03-11-05_208_181859222369516 8110 at org.apache.hadoop.fs.Path.initialize(Path.java:140) at org.apache.hadoop.fs.Path.init(Path.java:132) at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:142) at org.apache.hadoop.hive.ql.Context.getLocalScratchDir(Context.java:168) at org.apache.hadoop.hive.ql.Context.getLocalTmpFileURI(Context.java:282) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:205) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) Caused by: java.net.URISyntaxException: Relative path in absolute URI: file:C:/cygwin/tmp//siddharth/hive_2011-08-18_03-11-05_208_1818592223695168110 at java.net.URI.checkPath(URI.java:1787) at java.net.URI.init(URI.java:735) at org.apache.hadoop.fs.Path.initialize(Path.java:137) ... 16 more -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2413) BlockMergeTask ignores client-specified jars
[ https://issues.apache.org/jira/browse/HIVE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar updated HIVE-2413: Attachment: HIVE-2413.v1.patch Empty string not handled correctly in JC so handling it here... BlockMergeTask ignores client-specified jars Key: HIVE-2413 URL: https://issues.apache.org/jira/browse/HIVE-2413 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Krishna Kumar Assignee: Krishna Kumar Priority: Minor Attachments: HIVE-2413.v0.patch, HIVE-2413.v1.patch User-specified jars are not added to the hadoop tasks while executing a BlockMergeTask resulting in a ClassNotFoundException. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2420) partition pruner expr is not populated due to some bug in ppd
[ https://issues.apache.org/jira/browse/HIVE-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095410#comment-13095410 ] Amareshwari Sriramadasu commented on HIVE-2420: --- Will not be able to look into it this week. Will try and do in the next week. partition pruner expr is not populated due to some bug in ppd - Key: HIVE-2420 URL: https://issues.apache.org/jira/browse/HIVE-2420 Project: Hive Issue Type: Bug Reporter: He Yongqiang Assignee: Amareshwari Sriramadasu Attachments: HIVE-2420.reproduce.diff -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1545) Add a bunch of UDFs and UDAFs
[
https://issues.apache.org/jira/browse/HIVE-1545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095417#comment-13095417
]
Jonathan Chang commented on HIVE-1545:
--
Sure. Can you let me know which functions are not included? I think part of
the resolution of this issue will be to port some of the UDFUtils-specific
stuff to the UDF development package on github.
Add a bunch of UDFs and UDAFs
-
Key: HIVE-1545
URL: https://issues.apache.org/jira/browse/HIVE-1545
Project: Hive
Issue Type: New Feature
Components: UDF
Reporter: Jonathan Chang
Assignee: Jonathan Chang
Priority: Minor
Attachments: core.tar.gz, ext.tar.gz, udfs.tar.gz, udfs.tar.gz
Here some UD(A)Fs which can be incorporated into the Hive distribution:
UDFArgMax - Find the 0-indexed index of the largest argument. e.g., ARGMAX(4,
5, 3) returns 1.
UDFBucket - Find the bucket in which the first argument belongs. e.g.,
BUCKET(x, b_1, b_2, b_3, ...), will return the smallest i such that x b_{i}
but = b_{i+1}. Returns 0 if x is smaller than all the buckets.
UDFFindInArray - Finds the 1-index of the first element in the array given as
the second argument. Returns 0 if not found. Returns NULL if either argument
is NULL. E.g., FIND_IN_ARRAY(5, array(1,2,5)) will return 3. FIND_IN_ARRAY(5,
array(1,2,3)) will return 0.
UDFGreatCircleDist - Finds the great circle distance (in km) between two
lat/long coordinates (in degrees).
UDFLDA - Performs LDA inference on a vector given fixed topics.
UDFNumberRows - Number successive rows starting from 1. Counter resets to 1
whenever any of its parameters changes.
UDFPmax - Finds the maximum of a set of columns. e.g., PMAX(4, 5, 3) returns
5.
UDFRegexpExtractAll - Like REGEXP_EXTRACT except that it returns all matches
in an array.
UDFUnescape - Returns the string unescaped (using C/Java style unescaping).
UDFWhich - Given a boolean array, return the indices which are TRUE.
UDFJaccard
UDAFCollect - Takes all the values associated with a row and converts it into
a list. Make sure to have: set hive.map.aggr = false;
UDAFCollectMap - Like collect except that it takes tuples and generates a map.
UDAFEntropy - Compute the entropy of a column.
UDAFPearson (BROKEN!!!) - Computes the pearson correlation between two
columns.
UDAFTop - TOP(KEY, VAL) - returns the KEY associated with the largest value
of VAL.
UDAFTopN (BROKEN!!!) - Like TOP except returns a list of the keys associated
with the N (passed as the third parameter) largest values of VAL.
UDAFHistogram
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-1989: recognize transitivity of predicates on join keys
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-09-01 17:58:25.969663) Review request for hive. Changes --- Fix unit tests Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1163909 http://svn.apache.org/repos/asf/hive/trunk/conf/hive-default.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ppd_transitivity.q PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join8.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join8.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_transitivity.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml 1163909
[jira] [Updated] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charles Chen updated HIVE-1989:
---
Attachment: HIVE-1989v8.patch
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Fix For: 0.8.0
Attachments: HIVE-1989v1.patch, HIVE-1989v4.patch,
HIVE-1989v5-WITH-HIVE-2382v1.patch, HIVE-1989v6-WITH-HIVE-2383v1.patch,
HIVE-1989v8.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2 via
transitivity. The current lack places a burden on the user to add a
redundant predicate, and makes impossible (at least in strict mode) join
views where both underlying tables are partitioned (the join select list has
to pick one of the tables arbitrarily).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[ https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095473#comment-13095473 ] jirapos...@reviews.apache.org commented on HIVE-1989: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-09-01 17:58:25.969663) Review request for hive. Changes --- Fix unit tests Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1163909 http://svn.apache.org/repos/asf/hive/trunk/conf/hive-default.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ppd_transitivity.q PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join8.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join8.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_transitivity.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1163909
[jira] [Commented] (HIVE-2417) Merging of compressed rcfiles fails to write the valuebuffer part correctly
[ https://issues.apache.org/jira/browse/HIVE-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095608#comment-13095608 ] He Yongqiang commented on HIVE-2417: Committed, thanks Krishna Kumar! Merging of compressed rcfiles fails to write the valuebuffer part correctly --- Key: HIVE-2417 URL: https://issues.apache.org/jira/browse/HIVE-2417 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Krishna Kumar Assignee: Krishna Kumar Attachments: HIVE-2417.v0.patch, HIVE-2417.v1.patch The blockmerge task does not create proper rc files when merging compressed rc files as the valuebuffer writing is incorrect. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2417) Merging of compressed rcfiles fails to write the valuebuffer part correctly
[ https://issues.apache.org/jira/browse/HIVE-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-2417: --- Resolution: Fixed Status: Resolved (was: Patch Available) Merging of compressed rcfiles fails to write the valuebuffer part correctly --- Key: HIVE-2417 URL: https://issues.apache.org/jira/browse/HIVE-2417 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Krishna Kumar Assignee: Krishna Kumar Attachments: HIVE-2417.v0.patch, HIVE-2417.v1.patch The blockmerge task does not create proper rc files when merging compressed rc files as the valuebuffer writing is incorrect. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2337) Predicate pushdown erroneously conservative with outer joins
[
https://issues.apache.org/jira/browse/HIVE-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095607#comment-13095607
]
John Sichi commented on HIVE-2337:
--
+1. Will commit when tests pass.
Predicate pushdown erroneously conservative with outer joins
Key: HIVE-2337
URL: https://issues.apache.org/jira/browse/HIVE-2337
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Charles Chen
Assignee: Charles Chen
Fix For: 0.9.0
Attachments: HIVE-2337v1.patch, HIVE-2337v2.patch, HIVE-2337v3.patch,
HIVE-2337v4.patch, HIVE-2337v5.patch, HIVE-2337v6.patch, HIVE-2337v7.patch
The predicate pushdown filter is not applying left associativity of joins
correctly in determining possible aliases for pushing predicates.
In hive.ql.ppd.OpProcFactory.JoinPPD.getQualifiedAliases, the criteria for
pushing aliases is specified as:
{noformat}
/**
* Figures out the aliases for whom it is safe to push predicates based on
* ANSI SQL semantics For inner join, all predicates for all aliases can
be
* pushed For full outer join, none of the predicates can be pushed as
that
* would limit the number of rows for join For left outer join, all the
* predicates on the left side aliases can be pushed up For right outer
* join, all the predicates on the right side aliases can be pushed up
Joins
* chain containing both left and right outer joins are treated as full
* outer join. [...]
*
* @param op
* Join Operator
* @param rr
* Row resolver
* @return set of qualified aliases
*/
{noformat}
Since hive joins are left associative, something like a RIGHT OUTER JOIN b
LEFT OUTER JOIN c INNER JOIN d should be interpreted as ((a RIGHT OUTER
JOIN b) LEFT OUTER JOIN c) INNER JOIN d, so there would be cases where joins
with both left and right outer joins can have aliases that can be pushed.
Here, aliases b and d are eligible to be pushed up while the current criteria
provide that none are eligible.
Using:
{noformat}
create table t1 (id int, key string, value string);
create table t2 (id int, key string, value string);
create table t3 (id int, key string, value string);
create table t4 (id int, key string, value string);
{noformat}
For example, the query
{noformat}
explain select * from t1 full outer join t2 on t1.id=t2.id join t3 on
t2.id=t3.id where t3.id=20;
{noformat}
currently gives
{noformat}
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Alias - Map Operator Tree:
t1
TableScan
alias: t1
Reduce Output Operator
key expressions:
expr: id
type: int
sort order: +
Map-reduce partition columns:
expr: id
type: int
tag: 0
value expressions:
expr: id
type: int
expr: key
type: string
expr: value
type: string
t2
TableScan
alias: t2
Reduce Output Operator
key expressions:
expr: id
type: int
sort order: +
Map-reduce partition columns:
expr: id
type: int
tag: 1
value expressions:
expr: id
type: int
expr: key
type: string
expr: value
type: string
t3
TableScan
alias: t3
Reduce Output Operator
key expressions:
expr: id
type: int
sort order: +
Map-reduce partition columns:
expr: id
type: int
tag: 2
value expressions:
expr: id
type: int
expr: key
type: string
expr: value
type: string
Reduce Operator Tree:
Join Operator
condition map:
Outer Join 0 to 1
Inner Join 1 to 2
condition expressions:
0 {VALUE._col0} {VALUE._col1} {VALUE._col2}
1 {VALUE._col0} {VALUE._col1} {VALUE._col2}
2 {VALUE._col0} {VALUE._col1} {VALUE._col2}
handleSkewJoin: false
Re: Review Request: HIVE-1989: recognize transitivity of predicates on join keys
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-09-01 21:31:00.970058) Review request for hive. Changes --- Changed extractFinalCandidates() to decompose AND expression into parts so the parts can be replicated over transitively even if the whole thing can't. Added comments. Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/union.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf_when.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf_case.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_repeated_alias2.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_transitivity.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join8.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out 1163905
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[ https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095619#comment-13095619 ] jirapos...@reviews.apache.org commented on HIVE-1989: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-09-01 21:31:00.970058) Review request for hive. Changes --- Changed extractFinalCandidates() to decompose AND expression into parts so the parts can be replicated over transitively even if the whole thing can't. Added comments. Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/union.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf_when.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf_case.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_repeated_alias2.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_transitivity.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join8.q.out 1163905
[jira] [Updated] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charles Chen updated HIVE-1989:
---
Attachment: HIVE-1989v9.patch
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Fix For: 0.8.0
Attachments: HIVE-1989v1.patch, HIVE-1989v4.patch,
HIVE-1989v5-WITH-HIVE-2382v1.patch, HIVE-1989v6-WITH-HIVE-2383v1.patch,
HIVE-1989v8.patch, HIVE-1989v9.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2 via
transitivity. The current lack places a burden on the user to add a
redundant predicate, and makes impossible (at least in strict mode) join
views where both underlying tables are partitioned (the join select list has
to pick one of the tables arbitrarily).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charles Chen updated HIVE-1989:
---
Status: Patch Available (was: Open)
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Fix For: 0.8.0
Attachments: HIVE-1989v1.patch, HIVE-1989v4.patch,
HIVE-1989v5-WITH-HIVE-2382v1.patch, HIVE-1989v6-WITH-HIVE-2383v1.patch,
HIVE-1989v8.patch, HIVE-1989v9.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2 via
transitivity. The current lack places a burden on the user to add a
redundant predicate, and makes impossible (at least in strict mode) join
views where both underlying tables are partitioned (the join select list has
to pick one of the tables arbitrarily).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1545) Add a bunch of UDFs and UDAFs
[
https://issues.apache.org/jira/browse/HIVE-1545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095621#comment-13095621
]
John Sichi commented on HIVE-1545:
--
I'm way behind on the PDK (probably not gonna make it for 0.8), but I'm
planning to rework the UDFUtils into annotations as part of it.
Cyril, I think they are mostly used for validation purposes, in which case you
can just comment out the calls for now if you want to use the UDF without
validation.
Add a bunch of UDFs and UDAFs
-
Key: HIVE-1545
URL: https://issues.apache.org/jira/browse/HIVE-1545
Project: Hive
Issue Type: New Feature
Components: UDF
Reporter: Jonathan Chang
Assignee: Jonathan Chang
Priority: Minor
Attachments: core.tar.gz, ext.tar.gz, udfs.tar.gz, udfs.tar.gz
Here some UD(A)Fs which can be incorporated into the Hive distribution:
UDFArgMax - Find the 0-indexed index of the largest argument. e.g., ARGMAX(4,
5, 3) returns 1.
UDFBucket - Find the bucket in which the first argument belongs. e.g.,
BUCKET(x, b_1, b_2, b_3, ...), will return the smallest i such that x b_{i}
but = b_{i+1}. Returns 0 if x is smaller than all the buckets.
UDFFindInArray - Finds the 1-index of the first element in the array given as
the second argument. Returns 0 if not found. Returns NULL if either argument
is NULL. E.g., FIND_IN_ARRAY(5, array(1,2,5)) will return 3. FIND_IN_ARRAY(5,
array(1,2,3)) will return 0.
UDFGreatCircleDist - Finds the great circle distance (in km) between two
lat/long coordinates (in degrees).
UDFLDA - Performs LDA inference on a vector given fixed topics.
UDFNumberRows - Number successive rows starting from 1. Counter resets to 1
whenever any of its parameters changes.
UDFPmax - Finds the maximum of a set of columns. e.g., PMAX(4, 5, 3) returns
5.
UDFRegexpExtractAll - Like REGEXP_EXTRACT except that it returns all matches
in an array.
UDFUnescape - Returns the string unescaped (using C/Java style unescaping).
UDFWhich - Given a boolean array, return the indices which are TRUE.
UDFJaccard
UDAFCollect - Takes all the values associated with a row and converts it into
a list. Make sure to have: set hive.map.aggr = false;
UDAFCollectMap - Like collect except that it takes tuples and generates a map.
UDAFEntropy - Compute the entropy of a column.
UDAFPearson (BROKEN!!!) - Computes the pearson correlation between two
columns.
UDAFTop - TOP(KEY, VAL) - returns the KEY associated with the largest value
of VAL.
UDAFTopN (BROKEN!!!) - Like TOP except returns a list of the keys associated
with the N (passed as the third parameter) largest values of VAL.
UDAFHistogram
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2424) Don't bundle thrift, commons and json classes in the hive exec jar
Don't bundle thrift, commons and json classes in the hive exec jar --- Key: HIVE-2424 URL: https://issues.apache.org/jira/browse/HIVE-2424 Project: Hive Issue Type: Improvement Components: Build Infrastructure Reporter: Eli Collins The hive exec jar includes exploded thrift, json, and commons lang classes. These may conflict with the user's classpath. This could be fixed by jar jaring or using shade. A mechanism that allowed a user to substitute alternative versions w/o recompiling might be a useful intermediate step (though will require the user substitute alternative versions that work w/ Hive). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2424) Don't expose thrift, commons and json classes in the hive exec jar
[ https://issues.apache.org/jira/browse/HIVE-2424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HIVE-2424: -- Summary: Don't expose thrift, commons and json classes in the hive exec jar (was: Don't bundle thrift, commons and json classes in the hive exec jar ) Don't expose thrift, commons and json classes in the hive exec jar --- Key: HIVE-2424 URL: https://issues.apache.org/jira/browse/HIVE-2424 Project: Hive Issue Type: Improvement Components: Build Infrastructure Reporter: Eli Collins The hive exec jar includes exploded thrift, json, and commons lang classes. These may conflict with the user's classpath. This could be fixed by jar jaring or using shade. A mechanism that allowed a user to substitute alternative versions w/o recompiling might be a useful intermediate step (though will require the user substitute alternative versions that work w/ Hive). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2376) Upgrade Hive's Thrift dependency to version 0.6.1
[ https://issues.apache.org/jira/browse/HIVE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2376: - Attachment: HIVE-2376-complete.2.patch.txt Upgrade Hive's Thrift dependency to version 0.6.1 - Key: HIVE-2376 URL: https://issues.apache.org/jira/browse/HIVE-2376 Project: Hive Issue Type: Task Components: Build Infrastructure, Clients Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.8.0 Attachments: HIVE-2376-complete.1.patch.txt, HIVE-2376-complete.2.patch.txt, HIVE-2376-no-gen.1.patch.txt, libfb303-0.7.0.jar, libthrift-0.7.0.jar -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: HIVE-2376. Upgrade Hive's Thrift dependency to version 0.7.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1695/ --- Review request for hive and Amareshwari Sriramadasu. Summary --- * Upgrade Hive's Thrift dependency to version 0.7.0 * Resolve Thrift dependency using Ivy This addresses bug HIVE-2376. https://issues.apache.org/jira/browse/HIVE-2376 Diffs - service/src/java/org/apache/hadoop/hive/service/HiveServer.java c4b041f shims/ivy.xml 88fae44 metastore/ivy.xml 7647b37 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 285908e ql/build.xml 15a58ef ql/ivy.xml 99f3786 serde/ivy.xml 01c4d0c service/ivy.xml 7254a09 build.xml f20cbb6 cli/ivy.xml abe7232 eclipse-templates/.classpath 74e5542 ivy/libraries.properties 4236df1 jdbc/ivy.xml ec26e74 lib/README 6cc2fba lib/thrift-fb303.LICENSE 4eacb64 lib/thrift.LICENSE 9d189ef Diff: https://reviews.apache.org/r/1695/diff Testing --- Thanks, Carl
[jira] [Updated] (HIVE-2376) Upgrade Hive's Thrift dependency to version 0.6.1
[ https://issues.apache.org/jira/browse/HIVE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2376: - Attachment: HIVE-2376-no-gen.2.patch.txt Upgrade Hive's Thrift dependency to version 0.6.1 - Key: HIVE-2376 URL: https://issues.apache.org/jira/browse/HIVE-2376 Project: Hive Issue Type: Task Components: Build Infrastructure, Clients Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.8.0 Attachments: HIVE-2376-complete.1.patch.txt, HIVE-2376-complete.2.patch.txt, HIVE-2376-no-gen.1.patch.txt, HIVE-2376-no-gen.2.patch.txt, libfb303-0.7.0.jar, libthrift-0.7.0.jar -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2376) Upgrade Hive's Thrift dependency to version 0.6.1
[ https://issues.apache.org/jira/browse/HIVE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095639#comment-13095639 ] jirapos...@reviews.apache.org commented on HIVE-2376: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1695/ --- Review request for hive and Amareshwari Sriramadasu. Summary --- * Upgrade Hive's Thrift dependency to version 0.7.0 * Resolve Thrift dependency using Ivy This addresses bug HIVE-2376. https://issues.apache.org/jira/browse/HIVE-2376 Diffs - service/src/java/org/apache/hadoop/hive/service/HiveServer.java c4b041f shims/ivy.xml 88fae44 metastore/ivy.xml 7647b37 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 285908e ql/build.xml 15a58ef ql/ivy.xml 99f3786 serde/ivy.xml 01c4d0c service/ivy.xml 7254a09 build.xml f20cbb6 cli/ivy.xml abe7232 eclipse-templates/.classpath 74e5542 ivy/libraries.properties 4236df1 jdbc/ivy.xml ec26e74 lib/README 6cc2fba lib/thrift-fb303.LICENSE 4eacb64 lib/thrift.LICENSE 9d189ef Diff: https://reviews.apache.org/r/1695/diff Testing --- Thanks, Carl Upgrade Hive's Thrift dependency to version 0.6.1 - Key: HIVE-2376 URL: https://issues.apache.org/jira/browse/HIVE-2376 Project: Hive Issue Type: Task Components: Build Infrastructure, Clients Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.8.0 Attachments: HIVE-2376-complete.1.patch.txt, HIVE-2376-complete.2.patch.txt, HIVE-2376-no-gen.1.patch.txt, HIVE-2376-no-gen.2.patch.txt, libfb303-0.7.0.jar, libthrift-0.7.0.jar -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2376) Upgrade Hive's Thrift dependency to version 0.6.1
[ https://issues.apache.org/jira/browse/HIVE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095640#comment-13095640 ] Carl Steinbach commented on HIVE-2376: -- bq. I see that the old License is removed. And no new License is added. Any reason for doing so? I did this in anticipation of resolving the Thrift dependency with Ivy, which the new version of the patch does. Upgrade Hive's Thrift dependency to version 0.6.1 - Key: HIVE-2376 URL: https://issues.apache.org/jira/browse/HIVE-2376 Project: Hive Issue Type: Task Components: Build Infrastructure, Clients Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.8.0 Attachments: HIVE-2376-complete.1.patch.txt, HIVE-2376-complete.2.patch.txt, HIVE-2376-no-gen.1.patch.txt, HIVE-2376-no-gen.2.patch.txt, libfb303-0.7.0.jar, libthrift-0.7.0.jar -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2376) Upgrade Hive's Thrift dependency to version 0.6.1
[ https://issues.apache.org/jira/browse/HIVE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2376: - Status: Patch Available (was: Open) Upgrade Hive's Thrift dependency to version 0.6.1 - Key: HIVE-2376 URL: https://issues.apache.org/jira/browse/HIVE-2376 Project: Hive Issue Type: Task Components: Build Infrastructure, Clients Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.8.0 Attachments: HIVE-2376-complete.1.patch.txt, HIVE-2376-complete.2.patch.txt, HIVE-2376-no-gen.1.patch.txt, HIVE-2376-no-gen.2.patch.txt, libfb303-0.7.0.jar, libthrift-0.7.0.jar -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095643#comment-13095643
]
John Sichi commented on HIVE-1989:
--
Charles, can you add a test case for the original partitioned join view use
case? Separate JIRA is fine.
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Fix For: 0.8.0
Attachments: HIVE-1989v1.patch, HIVE-1989v4.patch,
HIVE-1989v5-WITH-HIVE-2382v1.patch, HIVE-1989v6-WITH-HIVE-2383v1.patch,
HIVE-1989v8.patch, HIVE-1989v9.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2 via
transitivity. The current lack places a burden on the user to add a
redundant predicate, and makes impossible (at least in strict mode) join
views where both underlying tables are partitioned (the join select list has
to pick one of the tables arbitrarily).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2376) Upgrade Hive's Thrift dependency to version 0.7.0
[ https://issues.apache.org/jira/browse/HIVE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HIVE-2376: -- Summary: Upgrade Hive's Thrift dependency to version 0.7.0 (was: Upgrade Hive's Thrift dependency to version 0.6.1) Upgrade Hive's Thrift dependency to version 0.7.0 - Key: HIVE-2376 URL: https://issues.apache.org/jira/browse/HIVE-2376 Project: Hive Issue Type: Task Components: Build Infrastructure, Clients Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.8.0 Attachments: HIVE-2376-complete.1.patch.txt, HIVE-2376-complete.2.patch.txt, HIVE-2376-no-gen.1.patch.txt, HIVE-2376-no-gen.2.patch.txt, libfb303-0.7.0.jar, libthrift-0.7.0.jar -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095669#comment-13095669 ] John Sichi commented on HIVE-2380: -- Ashutosh, maybe we can discuss this one at the contributor meetup next week (and then record the conclusions here). A few questions that I've heard so far: * Is there a design doc somewhere? * Since Hive already has an array type, but this feature is independent, we probably want a different type name than bytearray. * For conversions, is going through string for all types a good default behavior? An alternative would be to prevent implicit conversions altogether, and force users to pick the UDF with the desired behavior. E.g. for string/binary conversion, it's a good idea to be able to specify an encoding rather than always using the JVM default. * How does the new type work with TRANSFORM scripts, UDF's, saving to textfile, etc? * Don't we need more accessor functions (e.g. making the existing string functions such as LENGTH work)? Add ByteArray Datatype -- Key: HIVE-2380 URL: https://issues.apache.org/jira/browse/HIVE-2380 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: hive-2380.patch Add bytearray as a primitive data type. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2401) Show functions with regex not working
[ https://issues.apache.org/jira/browse/HIVE-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-2401: - Status: Open (was: Patch Available) The wiki already explains how to do this. I don't hink we need any behavior change here. hive show functions 'm.*'; OK map map_keys map_values max min minute month Show functions with regex not working - Key: HIVE-2401 URL: https://issues.apache.org/jira/browse/HIVE-2401 Project: Hive Issue Type: Improvement Components: CLI Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5) Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-2401.patch show functions a; If it gives all the function names starting with a is easy to search. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2402) Function like with empty string is throwing null pointer exception
[ https://issues.apache.org/jira/browse/HIVE-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-2402: - Status: Open (was: Patch Available) Function like with empty string is throwing null pointer exception -- Key: HIVE-2402 URL: https://issues.apache.org/jira/browse/HIVE-2402 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.8.0 Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5) Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-2402.patch select emp.ename from emp where ename like '' This query is throwing null pointer exception -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2369) Minor typo in error message in HiveConnection.java (JDBC)
[
https://issues.apache.org/jira/browse/HIVE-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sichi reassigned HIVE-2369:
Assignee: Clément Notin
Minor typo in error message in HiveConnection.java (JDBC)
-
Key: HIVE-2369
URL: https://issues.apache.org/jira/browse/HIVE-2369
Project: Hive
Issue Type: Bug
Components: JDBC
Affects Versions: 0.7.1, 0.8.0
Environment: Linux
Reporter: Clément Notin
Assignee: Clément Notin
Priority: Trivial
Attachments: HIVE-2369.patch
Original Estimate: 2m
Remaining Estimate: 2m
There is a minor typo issue in HiveConnection.java (jdbc) :
{code}throw new SQLException(Could not establish connecton to
+ uri + : + e.getMessage(), 08S01);{code}
It seems like there's a i missing.
I know it's a very minor typo but I report it anyway. I won't attach a patch
because it would be too long for me to SVN checkout just for 1 letter.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HIVE-2401) Show functions with regex not working
[ https://issues.apache.org/jira/browse/HIVE-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095671#comment-13095671 ] John Sichi edited comment on HIVE-2401 at 9/1/11 11:31 PM: --- The wiki already explains how to do this. I don't think we need any behavior change here. hive show functions 'm.*'; OK map map_keys map_values max min minute month was (Author: jvs): The wiki already explains how to do this. I don't hink we need any behavior change here. hive show functions 'm.*'; OK map map_keys map_values max min minute month Show functions with regex not working - Key: HIVE-2401 URL: https://issues.apache.org/jira/browse/HIVE-2401 Project: Hive Issue Type: Improvement Components: CLI Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5) Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-2401.patch show functions a; If it gives all the function names starting with a is easy to search. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2196) Ensure HiveConf includes all properties defined in hive-default.xml
[ https://issues.apache.org/jira/browse/HIVE-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095695#comment-13095695 ] Carl Steinbach commented on HIVE-2196: -- @Chinna: Please followup on the issue raised by Ashutosh. Configuration property values should match in HiveConf and hive-default.xml. If they don't match then the value that appears in hive-default.xml should be copied over to HiveConf. Thanks. Ensure HiveConf includes all properties defined in hive-default.xml --- Key: HIVE-2196 URL: https://issues.apache.org/jira/browse/HIVE-2196 Project: Hive Issue Type: Bug Components: Configuration Reporter: Carl Steinbach Assignee: Chinna Rao Lalam Attachments: HIVE-2196.1.patch, HIVE-2196.2.patch, HIVE-2196.build.log, HIVE-2196.patch There are a bunch of properties that are defined in hive-default.xml but not in HiveConf. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2020) Create a separate namespace for Hive variables
[
https://issues.apache.org/jira/browse/HIVE-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-2020:
-
Resolution: Fixed
Fix Version/s: 0.8.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed to branch-0.8 and trunk. Thanks Vaibhav!
Create a separate namespace for Hive variables
--
Key: HIVE-2020
URL: https://issues.apache.org/jira/browse/HIVE-2020
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Carl Steinbach
Assignee: Vaibhav Aggarwal
Fix For: 0.8.0
Attachments: HIVE-2020-2.patch, HIVE-2020-3.patch, HIVE-2020.patch
Support for variable substitution was added in HIVE-1096. However, variable
substitution was implemented by reusing the HiveConf namespace, so there is
no separation between Hive configuration properties and Hive variables.
This ticket encompasses the following enhancements:
* Create a separate namespace for managing Hive variables.
* Add support for setting variables on the command line via '-hivevar x=y'
* Add support for setting variables through the CLI via 'set hivevar:x=y'
* Add support for referencing variables in statements using either
'${hivevar:var_name}' or '${var_name}'
* Provide a means for differentiating between hiveconf, hivevar, system, and
environment properties in the output of 'set -v'
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2020) Create a separate namespace for Hive variables
[
https://issues.apache.org/jira/browse/HIVE-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095701#comment-13095701
]
Vaibhav Aggarwal commented on HIVE-2020:
Thanks for looking at this Carl!
Create a separate namespace for Hive variables
--
Key: HIVE-2020
URL: https://issues.apache.org/jira/browse/HIVE-2020
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Carl Steinbach
Assignee: Vaibhav Aggarwal
Fix For: 0.8.0
Attachments: HIVE-2020-2.patch, HIVE-2020-3.patch, HIVE-2020.patch
Support for variable substitution was added in HIVE-1096. However, variable
substitution was implemented by reusing the HiveConf namespace, so there is
no separation between Hive configuration properties and Hive variables.
This ticket encompasses the following enhancements:
* Create a separate namespace for managing Hive variables.
* Add support for setting variables on the command line via '-hivevar x=y'
* Add support for setting variables through the CLI via 'set hivevar:x=y'
* Add support for referencing variables in statements using either
'${hivevar:var_name}' or '${var_name}'
* Provide a means for differentiating between hiveconf, hivevar, system, and
environment properties in the output of 'set -v'
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2425) Print the counters SLOTS_MILLIS_MAPS and SLOTS_MILLIS_REDUCES to the Hive CLI output.
Print the counters SLOTS_MILLIS_MAPS and SLOTS_MILLIS_REDUCES to the Hive CLI output. - Key: HIVE-2425 URL: https://issues.apache.org/jira/browse/HIVE-2425 Project: Hive Issue Type: Improvement Reporter: Kevin Wilfong Assignee: Kevin Wilfong -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2425) Print the counters SLOTS_MILLIS_MAPS and SLOTS_MILLIS_REDUCES to the Hive CLI output.
[ https://issues.apache.org/jira/browse/HIVE-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2425: Attachment: HIVE-2425.1.patch.txt Print the counters SLOTS_MILLIS_MAPS and SLOTS_MILLIS_REDUCES to the Hive CLI output. - Key: HIVE-2425 URL: https://issues.apache.org/jira/browse/HIVE-2425 Project: Hive Issue Type: Improvement Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2425.1.patch.txt -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2425) Print the counters SLOTS_MILLIS_MAPS and SLOTS_MILLIS_REDUCES to the Hive CLI output.
[ https://issues.apache.org/jira/browse/HIVE-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095707#comment-13095707 ] jirapos...@reviews.apache.org commented on HIVE-2425: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1697/ --- Review request for hive and Siying Dong. Summary --- I print the new stats out where we are already printing out the other map reduce stats for each job. This addresses bug HIVE-2425. https://issues.apache.org/jira/browse/HIVE-2425 Diffs - trunk/ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java 1164320 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1164320 Diff: https://reviews.apache.org/r/1697/diff Testing --- I ran a query in local mode and verified the stats were not printed. I ran a map only query and verified only the map time was printed. I ran a query involving multiple jobs involving both map only and map and reduce, and verified that the appropriate stats were printed for each. In all cases, I verified the query completed successfully. Thanks, Kevin Print the counters SLOTS_MILLIS_MAPS and SLOTS_MILLIS_REDUCES to the Hive CLI output. - Key: HIVE-2425 URL: https://issues.apache.org/jira/browse/HIVE-2425 Project: Hive Issue Type: Improvement Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2425.1.patch.txt -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: HIVE-2304. Support PreparedStatement.setObject
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1700/ --- Review request for hive. Summary --- Review for HIVE-2304 (Ido Hadanny) This addresses bug HIVE-2304. https://issues.apache.org/jira/browse/HIVE-2304 Diffs - jdbc/src/java/org/apache/hadoop/hive/jdbc/HivePreparedStatement.java 679792c jdbc/src/test/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java e744d11 Diff: https://reviews.apache.org/r/1700/diff Testing --- Thanks, Carl
[jira] [Commented] (HIVE-2304) Support PreparedStatement.setObject
[ https://issues.apache.org/jira/browse/HIVE-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095709#comment-13095709 ] jirapos...@reviews.apache.org commented on HIVE-2304: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1700/ --- Review request for hive. Summary --- Review for HIVE-2304 (Ido Hadanny) This addresses bug HIVE-2304. https://issues.apache.org/jira/browse/HIVE-2304 Diffs - jdbc/src/java/org/apache/hadoop/hive/jdbc/HivePreparedStatement.java 679792c jdbc/src/test/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java e744d11 Diff: https://reviews.apache.org/r/1700/diff Testing --- Thanks, Carl Support PreparedStatement.setObject --- Key: HIVE-2304 URL: https://issues.apache.org/jira/browse/HIVE-2304 Project: Hive Issue Type: Sub-task Components: JDBC Affects Versions: 0.7.1 Reporter: Ido Hadanny Assignee: Ido Hadanny Priority: Minor Fix For: 0.8.0 Attachments: HIVE-0.8-SetObject.1.patch.txt Original Estimate: 1h Remaining Estimate: 1h PreparedStatement.setObject is important for spring's jdbcTemplate support -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-2304. Support PreparedStatement.setObject
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1700/#review1731 --- jdbc/src/java/org/apache/hadoop/hive/jdbc/HivePreparedStatement.java https://reviews.apache.org/r/1700/#comment3948 Please Add a negative testcase that exercises this exception. jdbc/src/test/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java https://reviews.apache.org/r/1700/#comment3947 Please add a new testcase or add to an existing testcase instead of modifying a pre-existing testcase. You have provided test coverage for PreparedStatement.setObject() at the expense of removing test coverage for PreparedStatement.setBoolean()/setInt(),etc. - Carl On 2011-09-02 01:35:33, Carl Steinbach wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1700/ --- (Updated 2011-09-02 01:35:33) Review request for hive. Summary --- Review for HIVE-2304 (Ido Hadanny) This addresses bug HIVE-2304. https://issues.apache.org/jira/browse/HIVE-2304 Diffs - jdbc/src/java/org/apache/hadoop/hive/jdbc/HivePreparedStatement.java 679792c jdbc/src/test/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java e744d11 Diff: https://reviews.apache.org/r/1700/diff Testing --- Thanks, Carl
[jira] [Commented] (HIVE-2304) Support PreparedStatement.setObject
[ https://issues.apache.org/jira/browse/HIVE-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095713#comment-13095713 ] jirapos...@reviews.apache.org commented on HIVE-2304: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1700/#review1731 --- jdbc/src/java/org/apache/hadoop/hive/jdbc/HivePreparedStatement.java https://reviews.apache.org/r/1700/#comment3948 Please Add a negative testcase that exercises this exception. jdbc/src/test/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java https://reviews.apache.org/r/1700/#comment3947 Please add a new testcase or add to an existing testcase instead of modifying a pre-existing testcase. You have provided test coverage for PreparedStatement.setObject() at the expense of removing test coverage for PreparedStatement.setBoolean()/setInt(),etc. - Carl On 2011-09-02 01:35:33, Carl Steinbach wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/1700/ bq. --- bq. bq. (Updated 2011-09-02 01:35:33) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Review for HIVE-2304 (Ido Hadanny) bq. bq. bq. This addresses bug HIVE-2304. bq. https://issues.apache.org/jira/browse/HIVE-2304 bq. bq. bq. Diffs bq. - bq. bq.jdbc/src/java/org/apache/hadoop/hive/jdbc/HivePreparedStatement.java 679792c bq.jdbc/src/test/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java e744d11 bq. bq. Diff: https://reviews.apache.org/r/1700/diff bq. bq. bq. Testing bq. --- bq. bq. bq. Thanks, bq. bq. Carl bq. bq. Support PreparedStatement.setObject --- Key: HIVE-2304 URL: https://issues.apache.org/jira/browse/HIVE-2304 Project: Hive Issue Type: Sub-task Components: JDBC Affects Versions: 0.7.1 Reporter: Ido Hadanny Assignee: Ido Hadanny Priority: Minor Attachments: HIVE-0.8-SetObject.1.patch.txt Original Estimate: 1h Remaining Estimate: 1h PreparedStatement.setObject is important for spring's jdbcTemplate support -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2304) Support PreparedStatement.setObject
[ https://issues.apache.org/jira/browse/HIVE-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2304: - Fix Version/s: (was: 0.8.0) Status: Open (was: Patch Available) Support PreparedStatement.setObject --- Key: HIVE-2304 URL: https://issues.apache.org/jira/browse/HIVE-2304 Project: Hive Issue Type: Sub-task Components: JDBC Affects Versions: 0.7.1 Reporter: Ido Hadanny Assignee: Ido Hadanny Priority: Minor Attachments: HIVE-0.8-SetObject.1.patch.txt Original Estimate: 1h Remaining Estimate: 1h PreparedStatement.setObject is important for spring's jdbcTemplate support -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2266) Fix compression parameters
[ https://issues.apache.org/jira/browse/HIVE-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2266: - Status: Open (was: Patch Available) bq. Besides the file size will differ on different platforms. Really? Which platforms are you talking about? Can you tell me how to reproduce this interesting behavior? bq. This test case executes the new code and makes sure that it works. How exactly does it verify that the new code works? How do you even know that your new code is getting exercised during the execution of this testcase? Fix compression parameters -- Key: HIVE-2266 URL: https://issues.apache.org/jira/browse/HIVE-2266 Project: Hive Issue Type: Bug Reporter: Vaibhav Aggarwal Assignee: Vaibhav Aggarwal Attachments: HIVE-2266-2.patch, HIVE-2266.patch There are a number of places where compression values are not set correctly in FileSinkOperator. This results in uncompressed files. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1545) Add a bunch of UDFs and UDAFs
[
https://issues.apache.org/jira/browse/HIVE-1545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095738#comment-13095738
]
cyril liao commented on HIVE-1545:
--
Neither in core.tar.gz nor ext.tar.gz,there is a class named
com.facebook.hive.udf.lib.UDFUtils,which is used by many UDFs.
In package com.facebook.hive.udf.lib ,only Counter and SetOps are included.
Add a bunch of UDFs and UDAFs
-
Key: HIVE-1545
URL: https://issues.apache.org/jira/browse/HIVE-1545
Project: Hive
Issue Type: New Feature
Components: UDF
Reporter: Jonathan Chang
Assignee: Jonathan Chang
Priority: Minor
Attachments: core.tar.gz, ext.tar.gz, udfs.tar.gz, udfs.tar.gz
Here some UD(A)Fs which can be incorporated into the Hive distribution:
UDFArgMax - Find the 0-indexed index of the largest argument. e.g., ARGMAX(4,
5, 3) returns 1.
UDFBucket - Find the bucket in which the first argument belongs. e.g.,
BUCKET(x, b_1, b_2, b_3, ...), will return the smallest i such that x b_{i}
but = b_{i+1}. Returns 0 if x is smaller than all the buckets.
UDFFindInArray - Finds the 1-index of the first element in the array given as
the second argument. Returns 0 if not found. Returns NULL if either argument
is NULL. E.g., FIND_IN_ARRAY(5, array(1,2,5)) will return 3. FIND_IN_ARRAY(5,
array(1,2,3)) will return 0.
UDFGreatCircleDist - Finds the great circle distance (in km) between two
lat/long coordinates (in degrees).
UDFLDA - Performs LDA inference on a vector given fixed topics.
UDFNumberRows - Number successive rows starting from 1. Counter resets to 1
whenever any of its parameters changes.
UDFPmax - Finds the maximum of a set of columns. e.g., PMAX(4, 5, 3) returns
5.
UDFRegexpExtractAll - Like REGEXP_EXTRACT except that it returns all matches
in an array.
UDFUnescape - Returns the string unescaped (using C/Java style unescaping).
UDFWhich - Given a boolean array, return the indices which are TRUE.
UDFJaccard
UDAFCollect - Takes all the values associated with a row and converts it into
a list. Make sure to have: set hive.map.aggr = false;
UDAFCollectMap - Like collect except that it takes tuples and generates a map.
UDAFEntropy - Compute the entropy of a column.
UDAFPearson (BROKEN!!!) - Computes the pearson correlation between two
columns.
UDAFTop - TOP(KEY, VAL) - returns the KEY associated with the largest value
of VAL.
UDAFTopN (BROKEN!!!) - Like TOP except returns a list of the keys associated
with the N (passed as the third parameter) largest values of VAL.
UDAFHistogram
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2337) Predicate pushdown erroneously conservative with outer joins
[
https://issues.apache.org/jira/browse/HIVE-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sichi updated HIVE-2337:
-
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Charles!
Predicate pushdown erroneously conservative with outer joins
Key: HIVE-2337
URL: https://issues.apache.org/jira/browse/HIVE-2337
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Charles Chen
Assignee: Charles Chen
Fix For: 0.9.0
Attachments: HIVE-2337v1.patch, HIVE-2337v2.patch, HIVE-2337v3.patch,
HIVE-2337v4.patch, HIVE-2337v5.patch, HIVE-2337v6.patch, HIVE-2337v7.patch
The predicate pushdown filter is not applying left associativity of joins
correctly in determining possible aliases for pushing predicates.
In hive.ql.ppd.OpProcFactory.JoinPPD.getQualifiedAliases, the criteria for
pushing aliases is specified as:
{noformat}
/**
* Figures out the aliases for whom it is safe to push predicates based on
* ANSI SQL semantics For inner join, all predicates for all aliases can
be
* pushed For full outer join, none of the predicates can be pushed as
that
* would limit the number of rows for join For left outer join, all the
* predicates on the left side aliases can be pushed up For right outer
* join, all the predicates on the right side aliases can be pushed up
Joins
* chain containing both left and right outer joins are treated as full
* outer join. [...]
*
* @param op
* Join Operator
* @param rr
* Row resolver
* @return set of qualified aliases
*/
{noformat}
Since hive joins are left associative, something like a RIGHT OUTER JOIN b
LEFT OUTER JOIN c INNER JOIN d should be interpreted as ((a RIGHT OUTER
JOIN b) LEFT OUTER JOIN c) INNER JOIN d, so there would be cases where joins
with both left and right outer joins can have aliases that can be pushed.
Here, aliases b and d are eligible to be pushed up while the current criteria
provide that none are eligible.
Using:
{noformat}
create table t1 (id int, key string, value string);
create table t2 (id int, key string, value string);
create table t3 (id int, key string, value string);
create table t4 (id int, key string, value string);
{noformat}
For example, the query
{noformat}
explain select * from t1 full outer join t2 on t1.id=t2.id join t3 on
t2.id=t3.id where t3.id=20;
{noformat}
currently gives
{noformat}
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Alias - Map Operator Tree:
t1
TableScan
alias: t1
Reduce Output Operator
key expressions:
expr: id
type: int
sort order: +
Map-reduce partition columns:
expr: id
type: int
tag: 0
value expressions:
expr: id
type: int
expr: key
type: string
expr: value
type: string
t2
TableScan
alias: t2
Reduce Output Operator
key expressions:
expr: id
type: int
sort order: +
Map-reduce partition columns:
expr: id
type: int
tag: 1
value expressions:
expr: id
type: int
expr: key
type: string
expr: value
type: string
t3
TableScan
alias: t3
Reduce Output Operator
key expressions:
expr: id
type: int
sort order: +
Map-reduce partition columns:
expr: id
type: int
tag: 2
value expressions:
expr: id
type: int
expr: key
type: string
expr: value
type: string
Reduce Operator Tree:
Join Operator
condition map:
Outer Join 0 to 1
Inner Join 1 to 2
condition expressions:
0 {VALUE._col0} {VALUE._col1} {VALUE._col2}
1 {VALUE._col0} {VALUE._col1} {VALUE._col2}
2 {VALUE._col0} {VALUE._col1} {VALUE._col2}
[jira] [Commented] (HIVE-2266) Fix compression parameters
[ https://issues.apache.org/jira/browse/HIVE-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095747#comment-13095747 ] Vaibhav Aggarwal commented on HIVE-2266: This patch attempts to fix a bug in the existing functionality in two ways: 1. In HiveFileFormatUtils.java, wrong jobconf is getting passed which is clear from the context. 2. In other cases the compression parameters are not getting set. The only difference this patch produces from the current behavior is smaller file sizes on file system. I am not sure how to write a hive query which can verify difference in file sizes. Do you have any ideas which can help me add some quick tests for this? The current test executes though the code checking that it does not result in any Exception or Error. It does not compare file size. Really? Which platforms are you talking about? Can you tell me how to reproduce this interesting behavior? Hadoop loads native compression libraries. I believe that they are platform dependent hence I do not assume that they always have same compression ratio. Please correct me if I am wrong here. In any case I think this is a broken existing functionality in Hive which we should fix. Fix compression parameters -- Key: HIVE-2266 URL: https://issues.apache.org/jira/browse/HIVE-2266 Project: Hive Issue Type: Bug Reporter: Vaibhav Aggarwal Assignee: Vaibhav Aggarwal Attachments: HIVE-2266-2.patch, HIVE-2266.patch There are a number of places where compression values are not set correctly in FileSinkOperator. This results in uncompressed files. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2426) Test that views with joins work properly
Test that views with joins work properly
Key: HIVE-2426
URL: https://issues.apache.org/jira/browse/HIVE-2426
Project: Hive
Issue Type: Test
Reporter: Charles Chen
With the testcase
{noformat}
drop table invites;
drop table invites2;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
set hive.mapred.mode=strict;
-- test join views: see HIVE-1989
create view v as select invites.bar, invites2.foo, invites2.ds from invites
join invites2 on invites.ds=invites2.ds;
explain select * from v where ds='2011-09-01';
drop view v;
drop table invites;
drop table invites2;
{noformat}
We should not have the partition pruner complain about invites.ds not having a
predicate because the predicate invites2.ds='2011-09-01' will be inferred with
the ppd transitivity optimization
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-1989: recognize transitivity of predicates on join keys
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-09-02 05:47:55.711939) Review request for hive. Changes --- Added unparseIdentifier for ASTNode handling Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_transitivity.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_repeated_alias2.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join8.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ppd_repeated_alias2.q PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ppd_transitivity.q PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join8.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1163905 http://svn.apache.org/repos/asf/hive/trunk/conf/hive-default.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 1163905
[jira] [Updated] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charles Chen updated HIVE-1989:
---
Attachment: HIVE-1989v10.patch
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Fix For: 0.8.0
Attachments: HIVE-1989v1.patch, HIVE-1989v10.patch,
HIVE-1989v4.patch, HIVE-1989v5-WITH-HIVE-2382v1.patch,
HIVE-1989v6-WITH-HIVE-2383v1.patch, HIVE-1989v8.patch, HIVE-1989v9.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2 via
transitivity. The current lack places a burden on the user to add a
redundant predicate, and makes impossible (at least in strict mode) join
views where both underlying tables are partitioned (the join select list has
to pick one of the tables arbitrarily).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[ https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095784#comment-13095784 ] jirapos...@reviews.apache.org commented on HIVE-1989: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-09-02 05:47:55.711939) Review request for hive. Changes --- Added unparseIdentifier for ASTNode handling Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_transitivity.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_repeated_alias2.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join8.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ppd_repeated_alias2.q PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ppd_transitivity.q PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join8.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1163905
[jira] [Assigned] (HIVE-2426) Test that views with joins work properly
[
https://issues.apache.org/jira/browse/HIVE-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charles Chen reassigned HIVE-2426:
--
Assignee: Charles Chen
Test that views with joins work properly
Key: HIVE-2426
URL: https://issues.apache.org/jira/browse/HIVE-2426
Project: Hive
Issue Type: Test
Reporter: Charles Chen
Assignee: Charles Chen
With the testcase
{noformat}
drop table invites;
drop table invites2;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
set hive.mapred.mode=strict;
-- test join views: see HIVE-1989
create view v as select invites.bar, invites2.foo, invites2.ds from invites
join invites2 on invites.ds=invites2.ds;
explain select * from v where ds='2011-09-01';
drop view v;
drop table invites;
drop table invites2;
{noformat}
We should not have the partition pruner complain about invites.ds not having
a predicate because the predicate invites2.ds='2011-09-01' will be inferred
with the ppd transitivity optimization
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: HIVE-2426: Test that views with joins work properly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1702/ --- Review request for hive. Summary --- https://issues.apache.org/jira/browse/HIVE-2426 This addresses bug HIVE-2426. https://issues.apache.org/jira/browse/HIVE-2426 Diffs - http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/UnparseTranslator.java 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/join_view.q PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_view.q.out PRE-CREATION Diff: https://reviews.apache.org/r/1702/diff Testing --- Thanks, Charles
[jira] [Updated] (HIVE-2426) Test that views with joins work properly
[
https://issues.apache.org/jira/browse/HIVE-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charles Chen updated HIVE-2426:
---
Attachment: HIVE-2426v2.patch
Test that views with joins work properly
Key: HIVE-2426
URL: https://issues.apache.org/jira/browse/HIVE-2426
Project: Hive
Issue Type: Test
Reporter: Charles Chen
Assignee: Charles Chen
Attachments: HIVE-2426v2.patch
With the testcase
{noformat}
drop table invites;
drop table invites2;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
set hive.mapred.mode=strict;
-- test join views: see HIVE-1989
create view v as select invites.bar, invites2.foo, invites2.ds from invites
join invites2 on invites.ds=invites2.ds;
explain select * from v where ds='2011-09-01';
drop view v;
drop table invites;
drop table invites2;
{noformat}
We should not have the partition pruner complain about invites.ds not having
a predicate because the predicate invites2.ds='2011-09-01' will be inferred
with the ppd transitivity optimization
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2426) Test that views with joins work properly
[
https://issues.apache.org/jira/browse/HIVE-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095791#comment-13095791
]
jirapos...@reviews.apache.org commented on HIVE-2426:
-
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1702/
---
Review request for hive.
Summary
---
https://issues.apache.org/jira/browse/HIVE-2426
This addresses bug HIVE-2426.
https://issues.apache.org/jira/browse/HIVE-2426
Diffs
-
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/UnparseTranslator.java
1163905
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/join_view.q
PRE-CREATION
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_view.q.out
PRE-CREATION
Diff: https://reviews.apache.org/r/1702/diff
Testing
---
Thanks,
Charles
Test that views with joins work properly
Key: HIVE-2426
URL: https://issues.apache.org/jira/browse/HIVE-2426
Project: Hive
Issue Type: Test
Reporter: Charles Chen
Assignee: Charles Chen
Attachments: HIVE-2426v2.patch
With the testcase
{noformat}
drop table invites;
drop table invites2;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
set hive.mapred.mode=strict;
-- test join views: see HIVE-1989
create view v as select invites.bar, invites2.foo, invites2.ds from invites
join invites2 on invites.ds=invites2.ds;
explain select * from v where ds='2011-09-01';
drop view v;
drop table invites;
drop table invites2;
{noformat}
We should not have the partition pruner complain about invites.ds not having
a predicate because the predicate invites2.ds='2011-09-01' will be inferred
with the ppd transitivity optimization
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
