[jira] [Commented] (HIVE-4018) MapJoin failing with Distributed Cache error
[ https://issues.apache.org/jira/browse/HIVE-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581110#comment-13581110 ] Amareshwari Sriramadasu commented on HIVE-4018: --- bq. this is a existing bug. Do you really think we should fix this now ? I mean, it is a pretty big and fundamental change. I would say this should be fixed. Because earlier we are able to run the same multi join query using 2 MR jobs with mapjoin hint passed in nested structure for each join as described in HIVE-3652. Now there is no way to do mapjoin for this multiway join, as the same query fails with this error, after changes to HIVE-3784. It becomes more trouble because there are no more mapjoin hints and will have to explicitly turn off autojoin for such queries. MapJoin failing with Distributed Cache error Key: HIVE-4018 URL: https://issues.apache.org/jira/browse/HIVE-4018 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.11.0 Reporter: Amareshwari Sriramadasu Assignee: Namit Jain Fix For: 0.11.0 Attachments: hive.4018.test.2.patch, HIVE-4018-test.patch When I'm a running a star join query after HIVE-3784, it is failing with following error: 2013-02-13 08:36:04,584 ERROR org.apache.hadoop.hive.ql.exec.MapJoinOperator: Load Distributed Cache Error 2013-02-13 08:36:04,585 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.EOFException at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:189) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.cleanUpInputFileChangedOp(MapJoinOperator.java:203) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1421) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:614) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:266) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:416) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278) at org.apache.hadoop.mapred.Child.main(Child.java:260) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4018) MapJoin failing with Distributed Cache error
[ https://issues.apache.org/jira/browse/HIVE-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581121#comment-13581121 ] Namit Jain commented on HIVE-4018: -- I agree from your point of view. Do you think you would be able to help on this ? Basically, MapJoinMetaData needs to be maintained per HashTableSink/Mapjoin, instead of the global. MapJoin failing with Distributed Cache error Key: HIVE-4018 URL: https://issues.apache.org/jira/browse/HIVE-4018 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.11.0 Reporter: Amareshwari Sriramadasu Assignee: Namit Jain Fix For: 0.11.0 Attachments: hive.4018.test.2.patch, HIVE-4018-test.patch When I'm a running a star join query after HIVE-3784, it is failing with following error: 2013-02-13 08:36:04,584 ERROR org.apache.hadoop.hive.ql.exec.MapJoinOperator: Load Distributed Cache Error 2013-02-13 08:36:04,585 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.EOFException at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:189) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.cleanUpInputFileChangedOp(MapJoinOperator.java:203) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1421) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:614) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:266) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:416) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278) at org.apache.hadoop.mapred.Child.main(Child.java:260) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4018) MapJoin failing with Distributed Cache error
[ https://issues.apache.org/jira/browse/HIVE-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581149#comment-13581149 ] Amareshwari Sriramadasu commented on HIVE-4018: --- bq. I agree from your point of view. Do you think you would be able to help on this ? Sure. Will give a try. MapJoin failing with Distributed Cache error Key: HIVE-4018 URL: https://issues.apache.org/jira/browse/HIVE-4018 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.11.0 Reporter: Amareshwari Sriramadasu Assignee: Namit Jain Fix For: 0.11.0 Attachments: hive.4018.test.2.patch, HIVE-4018-test.patch When I'm a running a star join query after HIVE-3784, it is failing with following error: 2013-02-13 08:36:04,584 ERROR org.apache.hadoop.hive.ql.exec.MapJoinOperator: Load Distributed Cache Error 2013-02-13 08:36:04,585 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.EOFException at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:189) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.cleanUpInputFileChangedOp(MapJoinOperator.java:203) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1421) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:614) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:266) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:416) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278) at org.apache.hadoop.mapred.Child.main(Child.java:260) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4024) Derby metastore update script will fail when upgrading from 0.9.0 to 0.10.0
[
https://issues.apache.org/jira/browse/HIVE-4024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581166#comment-13581166
]
Hudson commented on HIVE-4024:
--
Integrated in hive-trunk-hadoop1 #91 (See
[https://builds.apache.org/job/hive-trunk-hadoop1/91/])
HIVE-4024 Derby metastore update script will fail when upgrading from 0.9.0
to 0.10.0 (Jarek Jarcec Cecho via namit) (Revision 1447584)
Result = ABORTED
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1447584
Files :
* /hive/trunk/metastore/scripts/upgrade/derby/011-HIVE-3649.derby.sql
Derby metastore update script will fail when upgrading from 0.9.0 to 0.10.0
---
Key: HIVE-4024
URL: https://issues.apache.org/jira/browse/HIVE-4024
Project: Hive
Issue Type: Bug
Reporter: Jarek Jarcec Cecho
Assignee: Jarek Jarcec Cecho
Priority: Minor
Fix For: 0.11.0
Attachments: bugHIVE-4024.patch
The problem is in following file {{011-HIVE-3649.derby.sql}} that contains
following line:
{code}
ALTER TABLE SDS ADD IS_STOREDASSUBDIRECTORIES CHAR(1) NOT NULL;
{code}
This query will however fail if the table SDS have at least one row.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3403) user should not specify mapjoin to perform sort-merge bucketed join
[ https://issues.apache.org/jira/browse/HIVE-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581165#comment-13581165 ] Hudson commented on HIVE-3403: -- Integrated in hive-trunk-hadoop1 #91 (See [https://builds.apache.org/job/hive-trunk-hadoop1/91/]) HIVE-3403 user should not specify mapjoin to perform sort-merge bucketed join (Namit Jain via Ashutosh) (Revision 1447593) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1447593 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml.template * /hive/trunk/data/files/smallsrcsortbucket1outof4.txt * /hive/trunk/data/files/smallsrcsortbucket2outof4.txt * /hive/trunk/data/files/smallsrcsortbucket3outof4.txt * /hive/trunk/data/files/smallsrcsortbucket4outof4.txt * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SelectOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/AbstractBucketJoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/AbstractSMBJoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/AvgPartitionSizeBasedBigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketJoinProcCtx.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapjoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/LeftmostBigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SizeBasedBigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortBucketJoinProcCtx.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedMergeBucketMapJoinOptimizer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedMergeBucketMapjoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedMergeJoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TableSizeBasedBigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinResolver.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/SMBJoinDesc.java * /hive/trunk/ql/src/test/queries/clientnegative/smb_mapjoin_14.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_smb_mapjoin_14.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_1.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_10.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_2.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_3.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_4.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_5.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_6.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_7.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_8.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_9.q * /hive/trunk/ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_1.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_2.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_3.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_4.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_6.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_7.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_8.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out user should not specify mapjoin to perform sort-merge bucketed join --- Key: HIVE-3403 URL:
[jira] [Updated] (HIVE-3891) physical optimizer changes for auto sort-merge join
[ https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3891: - Attachment: hive.3891.5.patch physical optimizer changes for auto sort-merge join --- Key: HIVE-3891 URL: https://issues.apache.org/jira/browse/HIVE-3891 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, hive.3891.5.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-4018) MapJoin failing with Distributed Cache error
[ https://issues.apache.org/jira/browse/HIVE-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain reassigned HIVE-4018: Assignee: Amareshwari Sriramadasu (was: Namit Jain) Thanks MapJoin failing with Distributed Cache error Key: HIVE-4018 URL: https://issues.apache.org/jira/browse/HIVE-4018 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.11.0 Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.11.0 Attachments: hive.4018.test.2.patch, HIVE-4018-test.patch When I'm a running a star join query after HIVE-3784, it is failing with following error: 2013-02-13 08:36:04,584 ERROR org.apache.hadoop.hive.ql.exec.MapJoinOperator: Load Distributed Cache Error 2013-02-13 08:36:04,585 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.EOFException at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:189) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.cleanUpInputFileChangedOp(MapJoinOperator.java:203) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1421) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:614) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:266) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:416) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278) at org.apache.hadoop.mapred.Child.main(Child.java:260) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3971) testNegativeCliDriver_stats_aggregator_error_1 fails transiently
[ https://issues.apache.org/jira/browse/HIVE-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3971: - Status: Open (was: Patch Available) testNegativeCliDriver_stats_aggregator_error_1 fails transiently - Key: HIVE-3971 URL: https://issues.apache.org/jira/browse/HIVE-3971 Project: Hive Issue Type: Bug Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: error.rtf, HIVE-3971.patch.2 testNegativeCliDriver_stats_aggregator_error_1 fails transiently. In one case (not reproducible), it fails consistently. The root-cause is ClassNotFoundException for DummyStatsPublisher. please see attachment. In the environment, i did a test: move DummyStatsPublisher from src/test to src/java and the error is gone. this shows packaging is the issue. DummyStatsPublisher is in src/test package. ClassNotFoundException is thrown in MR task. This shows it's possible src/test package is not available in server side build path in some cases. Not sure it is the real and/or only root-cause since it's not always reproducible. If all tests pass, feel it's worthy to try. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3672) Support altering partition column type in Hive
[ https://issues.apache.org/jira/browse/HIVE-3672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581221#comment-13581221 ] Namit Jain commented on HIVE-3672: -- Can you refresh again ? The test is failing while applying the patch Support altering partition column type in Hive -- Key: HIVE-3672 URL: https://issues.apache.org/jira/browse/HIVE-3672 Project: Hive Issue Type: Improvement Components: CLI, SQL Reporter: Jingwei Lu Assignee: Jingwei Lu Labels: features Attachments: HIVE-3672.1.patch.txt, HIVE-3672.2.patch.txt, HIVE-3672.3.patch.txt, HIVE-3672.4.patch.txt, HIVE-3672.5.patch.txt, HIVE-3672.6.patch.txt, HIVE-3672.6.patch.txt Original Estimate: 72h Remaining Estimate: 72h Currently, Hive does not allow altering partition column types. As we've discouraged users from using non-string partition column types, this presents a problem for users who want to change there partition columns to be strings, they have to rename their table, create a new table, and copy all the data over. To support this via the CLI, adding a command like ALTER TABLE table_name PARTITION COLUMN (column_name new type); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join
[ https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581223#comment-13581223 ] Namit Jain commented on HIVE-3891: -- The tests passed physical optimizer changes for auto sort-merge join --- Key: HIVE-3891 URL: https://issues.apache.org/jira/browse/HIVE-3891 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, hive.3891.5.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3891) physical optimizer changes for auto sort-merge join
[ https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3891: - Attachment: hive.3891.6.patch physical optimizer changes for auto sort-merge join --- Key: HIVE-3891 URL: https://issues.apache.org/jira/browse/HIVE-3891 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, hive.3891.5.patch, hive.3891.6.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4025) Add reflect UDF for member method invocation of column
[ https://issues.apache.org/jira/browse/HIVE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581378#comment-13581378 ] Phabricator commented on HIVE-4025: --- brock has commented on the revision HIVE-4025 [jira] Add reflect UDF for member method invocation of column. Hi, Is there are a better name than reflect2? Perhaps reflect_obj, reflect_inst, or something else? Brock REVISION DETAIL https://reviews.facebook.net/D8601 To: JIRA, navis Cc: njain, brock Add reflect UDF for member method invocation of column -- Key: HIVE-4025 URL: https://issues.apache.org/jira/browse/HIVE-4025 Project: Hive Issue Type: Improvement Components: UDF Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-4025.D8601.1.patch There are many useful non-static methods on type of primitive types. But current reflect UDF cannot invoke those. For example, select reflect2(value, replace, val, VALUE) from src; which replaces 'val' part of value column with 'VALUE' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4007) Create abstract classes for serializer and deserializer
[
https://issues.apache.org/jira/browse/HIVE-4007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581395#comment-13581395
]
Jarek Jarcec Cecho commented on HIVE-4007:
--
Hi Namit,
thank you very much for working on this. I've took a look on the patch and I do
have one high level question. It seems to me that we're just adding
{{AbstractSerDe}} that all {{SerDe}} implementations will extend in the future
to allow the easy extension for future functionality. But it also seems that
we're keeping the old {{SerDe}} interface implementation. Since no {{SerDe}}
should implement that directly, it seems to me that it's unnecessary piece of
code. I can imagine that it would be great to keep it around for a release or
two to provide backward compatibility and allow Hive users to migrate they
codebase without immediately breaking the environment. I would suggest to mark
this interface as {{@Deprecated}} in case that this is the intent.
Jarcec
Create abstract classes for serializer and deserializer
---
Key: HIVE-4007
URL: https://issues.apache.org/jira/browse/HIVE-4007
Project: Hive
Issue Type: Improvement
Components: Serializers/Deserializers
Reporter: Namit Jain
Assignee: Namit Jain
Attachments: hive.4007.1.patch, hive.4007.2.patch
Currently, it is very difficult to change the Serializer/Deserializer
interface, since all the SerDes directly implement the interface.
Instead, we should have abstract classes for implementing these interfaces.
In case of a interface change, only the abstract class and the relevant
serde needs to change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2655) Ability to define functions in HQL
[ https://issues.apache.org/jira/browse/HIVE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-2655: --- Attachment: HIVE-2655-9.patch Ability to define functions in HQL -- Key: HIVE-2655 URL: https://issues.apache.org/jira/browse/HIVE-2655 Project: Hive Issue Type: New Feature Components: SQL Reporter: Jonathan Perlow Assignee: Brock Noland Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.2.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.3.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.4.patch, HIVE-2655-9.patch Ability to create functions in HQL as a substitute for creating them in Java. Jonathan Chang requested I create this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4022) Structs and struct fields cannot be NULL in INSERT statements
[
https://issues.apache.org/jira/browse/HIVE-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Malak updated HIVE-4022:
Description:
Originally thought to be Avro-specific, and first noted with respect to
HIVE-3528 Avro SerDe doesn't handle serializing Nullable types that require
access to a Schema, it turns out even native Hive tables cannot store NULL in
a STRUCT field or for the entire STRUCT itself, at least when the NULL is
specified directly in the INSERT statement.
Again, this affects both Avro-backed tables and native Hive tables.
***For native Hive tables:
The following:
echo 1,2 twovalues.csv
hive
CREATE TABLE tc (x INT, y INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
LOAD DATA LOCAL INPATH 'twovalues.csv' INTO TABLE tc;
CREATE TABLE oc (z STRUCTa: int, b: int);
INSERT INTO TABLE oc SELECT null FROM tc;
produces the error
FAILED: SemanticException [Error 10044]: Line 1:18 Cannot insert into target
table because column number/types are different 'oc': Cannot convert column 0
from void to structa:int,b:int.
The following:
INSERT INTO TABLE oc SELECT named_struct('a', null, 'b', null) FROM tc;
produces the error:
FAILED: SemanticException [Error 10044]: Line 1:18 Cannot insert into target
table because column number/types are different 'oc': Cannot convert column 0
from structa:void,b:void to structa:int,b:int.
***For Avro:
In HIVE-3528, there is in fact a null-struct test case in line 14 of
https://github.com/apache/hive/blob/15cc604bf10f4c2502cb88fb8bb3dcd45647cf2c/data/files/csv.txt
The test script at
https://github.com/apache/hive/blob/12d6f3e7d21f94e8b8490b7c6d291c9f4cac8a4f/ql/src/test/queries/clientpositive/avro_nullable_fields.q
does indeed work. But in that test, the query gets all of its data from a test
table verbatim:
INSERT OVERWRITE TABLE as_avro SELECT * FROM test_serializer;
If instead we stick in a hard-coded null for the struct directly into the
query, it fails:
INSERT OVERWRITE TABLE as_avro SELECT string1, int1, tinyint1, smallint1,
bigint1, boolean1, float1, double1, list1, map1, null, enum1, nullableint,
bytes1, fixed1 FROM test_serializer;
with the following error:
FAILED: SemanticException [Error 10044]: Line 1:23 Cannot insert into target
table because column number/types are different 'as_avro': Cannot convert
column 10 from void to structsint:int,sboolean:boolean,sstring:string.
Note, though, that substituting a hard-coded null for string1 (and restoring
struct1 into the query) does work:
INSERT OVERWRITE TABLE as_avro SELECT null, int1, tinyint1, smallint1, bigint1,
boolean1, float1, double1, list1, map1, struct1, enum1, nullableint, bytes1,
fixed1 FROM test_serializer;
was:
Related to HIVE-3528,
There is in fact a null-struct test case in line 14 of
https://github.com/apache/hive/blob/15cc604bf10f4c2502cb88fb8bb3dcd45647cf2c/data/files/csv.txt
The test script at
https://github.com/apache/hive/blob/12d6f3e7d21f94e8b8490b7c6d291c9f4cac8a4f/ql/src/test/queries/clientpositive/avro_nullable_fields.q
does indeed work. But in that test, the query gets all of its data from a test
table verbatim:
INSERT OVERWRITE TABLE as_avro SELECT * FROM test_serializer;
If instead we stick in a hard-coded null for the struct directly into the
query, it fails:
INSERT OVERWRITE TABLE as_avro SELECT string1, int1, tinyint1, smallint1,
bigint1, boolean1, float1, double1, list1, map1, null, enum1, nullableint,
bytes1, fixed1 FROM test_serializer;
with the following error:
FAILED: SemanticException [Error 10044]: Line 1:23 Cannot insert into target
table because column number/types are different 'as_avro': Cannot convert
column 10 from void to structsint:int,sboolean:boolean,sstring:string.
Note, though, that substituting a hard-coded null for string1 (and restoring
struct1 into the query) does work:
INSERT OVERWRITE TABLE as_avro SELECT null, int1, tinyint1, smallint1, bigint1,
boolean1, float1, double1, list1, map1, struct1, enum1, nullableint, bytes1,
fixed1 FROM test_serializer;
Summary: Structs and struct fields cannot be NULL in INSERT statements
(was: Avro SerDe queries don't handle hard-coded nulls for optional/nullable
structs)
Structs and struct fields cannot be NULL in INSERT statements
-
Key: HIVE-4022
URL: https://issues.apache.org/jira/browse/HIVE-4022
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Reporter: Michael Malak
Originally thought to be Avro-specific, and first noted with respect to
HIVE-3528 Avro SerDe doesn't handle serializing Nullable types that require
access to a Schema, it turns out even native Hive tables cannot store NULL
in a STRUCT field or for the entire STRUCT itself, at least when the NULL is
specified directly in the INSERT
[jira] [Updated] (HIVE-2655) Ability to define functions in HQL
[ https://issues.apache.org/jira/browse/HIVE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-2655: --- Status: Patch Available (was: Open) Marking as Patch Available since this is ready for a review. Ability to define functions in HQL -- Key: HIVE-2655 URL: https://issues.apache.org/jira/browse/HIVE-2655 Project: Hive Issue Type: New Feature Components: SQL Reporter: Jonathan Perlow Assignee: Brock Noland Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.2.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.3.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.4.patch, HIVE-2655-9.patch Ability to create functions in HQL as a substitute for creating them in Java. Jonathan Chang requested I create this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3980) Cleanup after HIVE-3403
[ https://issues.apache.org/jira/browse/HIVE-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3980: - Attachment: hive.3980.1.patch Cleanup after HIVE-3403 --- Key: HIVE-3980 URL: https://issues.apache.org/jira/browse/HIVE-3980 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3980.1.patch There have been a lot of comments on HIVE-3403, which involve changing variable names/function names/adding more comments/general cleanup etc. Since HIVE-3403 involves a lot of refactoring, it was fairly difficult to address the comments there, since refreshing becomes impossible. This jira is to track those cleanups. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4010) Failure finding iterate method with matching signature
[ https://issues.apache.org/jira/browse/HIVE-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581501#comment-13581501 ] Phabricator commented on HIVE-4010: --- mshang has commented on the revision HIVE-4010 [jira] Failure finding iterate method with matching signature. INLINE COMMENTS ql/src/test/org/apache/hadoop/hive/ql/udf/UDAFTestMethodOverloading.java:44 Is this currently implemented? How would I find the right terminate method if it is overloaded? By the return type? I don't see any way in GenericUDAFBridge to determine what the return type of the UDAF should be. REVISION DETAIL https://reviews.facebook.net/D8517 To: JIRA, jonchang, kevinwilfong, njain, mshang Failure finding iterate method with matching signature -- Key: HIVE-4010 URL: https://issues.apache.org/jira/browse/HIVE-4010 Project: Hive Issue Type: Bug Components: UDF Reporter: Miles Shang Assignee: Miles Shang Priority: Minor Attachments: HIVE-4010.D8517.1.patch, HIVE-4010.D8517.2.patch Original Estimate: 24h Remaining Estimate: 24h http://fburl.com/10467687 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21 #297
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/297/ -- [...truncated 5897 lines...] [ivy:resolve] downloading http://repo1.maven.org/maven2/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar ... [ivy:resolve] (98kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] javax.servlet.jsp#jsp-api;2.1!jsp-api.jar (14ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/commons-el/commons-el/1.0/commons-el-1.0.jar ... [ivy:resolve] ... (109kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] commons-el#commons-el;1.0!commons-el.jar (15ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar ... [ivy:resolve] (59kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] commons-logging#commons-logging;1.1.1!commons-logging.jar (14ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/commons-logging/commons-logging-api/1.1/commons-logging-api-1.1.jar ... [ivy:resolve] ... (43kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] commons-logging#commons-logging-api;1.1!commons-logging-api.jar (12ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/log4j/log4j/1.2.15/log4j-1.2.15.jar ... [ivy:resolve] ... (382kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] log4j#log4j;1.2.15!log4j.jar (19ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/net/java/dev/jets3t/jets3t/0.6.1/jets3t-0.6.1.jar ... [ivy:resolve] ... (314kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] net.java.dev.jets3t#jets3t;0.6.1!jets3t.jar (24ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/commons-lang/commons-lang/2.5/commons-lang-2.5.jar ... [ivy:resolve] .. (272kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] commons-lang#commons-lang;2.5!commons-lang.jar (19ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar ... [ivy:resolve] . (561kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] commons-collections#commons-collections;3.2.1!commons-collections.jar (25ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar ... [ivy:resolve] .. (291kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] commons-configuration#commons-configuration;1.6!commons-configuration.jar (21ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar ... [ivy:resolve] ... (24kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] org.slf4j#slf4j-api;1.6.1!slf4j-api.jar (10ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar ... [ivy:resolve] ... (9kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] org.slf4j#slf4j-log4j12;1.6.1!slf4j-log4j12.jar (26ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/org/eclipse/jdt/core/3.1.1/core-3.1.1.jar ... [ivy:resolve] .. (3483kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] org.eclipse.jdt#core;3.1.1!core.jar (84ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/oro/oro/2.0.8/oro-2.0.8.jar ... [ivy:resolve] ... (63kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] oro#oro;2.0.8!oro.jar (18ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/org/aspectj/aspectjrt/1.6.5/aspectjrt-1.6.5.jar ... [ivy:resolve] ... (113kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] org.aspectj#aspectjrt;1.6.5!aspectjrt.jar (63ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/org/apache/avro/avro/1.5.3/avro-1.5.3.jar ... [ivy:resolve] . (257kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] org.apache.avro#avro;1.5.3!avro.jar (47ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/org/apache/avro/avro-ipc/1.5.3/avro-ipc-1.5.3.jar ... [ivy:resolve] . (164kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] org.apache.avro#avro-ipc;1.5.3!avro-ipc.jar (57ms) [ivy:resolve] downloading
[jira] [Created] (HIVE-4039) Hive compiler sometimes fails in semantic analysis / optimisation stage when boolean variable appears in WHERE clause.
Jean Xu created HIVE-4039: - Summary: Hive compiler sometimes fails in semantic analysis / optimisation stage when boolean variable appears in WHERE clause. Key: HIVE-4039 URL: https://issues.apache.org/jira/browse/HIVE-4039 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Jean Xu Priority: Minor Hive compiler fails with a NullPointerException in semantic analysis / optimisation stage when a boolean variable appears in the WHERE clause in some cases. A minimal query to generate this error is here: SELECT 1 FROM ( SELECT TRUE AS flag FROM dim_one_row:measurementsystems ) a WHERE flag; On the other hand, the following query is perfectly fine: SELECT 1 FROM ( SELECT TRUE AS flag FROM dim_one_row:measurementsystems ) a WHERE flag=TRUE; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4039) Hive compiler sometimes fails in semantic analysis / optimisation stage when boolean variable appears in WHERE clause.
[ https://issues.apache.org/jira/browse/HIVE-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean Xu updated HIVE-4039: -- Assignee: Jean Xu Hive compiler sometimes fails in semantic analysis / optimisation stage when boolean variable appears in WHERE clause. -- Key: HIVE-4039 URL: https://issues.apache.org/jira/browse/HIVE-4039 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Jean Xu Assignee: Jean Xu Priority: Minor Hive compiler fails with a NullPointerException in semantic analysis / optimisation stage when a boolean variable appears in the WHERE clause in some cases. A minimal query to generate this error is here: SELECT 1 FROM ( SELECT TRUE AS flag FROM dim_one_row:measurementsystems ) a WHERE flag; On the other hand, the following query is perfectly fine: SELECT 1 FROM ( SELECT TRUE AS flag FROM dim_one_row:measurementsystems ) a WHERE flag=TRUE; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4039) Hive compiler sometimes fails in semantic analysis / optimisation stage when boolean variable appears in WHERE clause.
[ https://issues.apache.org/jira/browse/HIVE-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean Xu updated HIVE-4039: -- Attachment: HIVE_4039.1.patch.txt Hive compiler sometimes fails in semantic analysis / optimisation stage when boolean variable appears in WHERE clause. -- Key: HIVE-4039 URL: https://issues.apache.org/jira/browse/HIVE-4039 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Jean Xu Assignee: Jean Xu Priority: Minor Attachments: HIVE_4039.1.patch.txt Hive compiler fails with a NullPointerException in semantic analysis / optimisation stage when a boolean variable appears in the WHERE clause in some cases. A minimal query to generate this error is here: SELECT 1 FROM ( SELECT TRUE AS flag FROM dim_one_row:measurementsystems ) a WHERE flag; On the other hand, the following query is perfectly fine: SELECT 1 FROM ( SELECT TRUE AS flag FROM dim_one_row:measurementsystems ) a WHERE flag=TRUE; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4039) Hive compiler sometimes fails in semantic analysis / optimisation stage when boolean variable appears in WHERE clause.
[ https://issues.apache.org/jira/browse/HIVE-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean Xu updated HIVE-4039: -- Release Note: I have changed the code, and added in a test file Status: Patch Available (was: Open) Hive compiler sometimes fails in semantic analysis / optimisation stage when boolean variable appears in WHERE clause. -- Key: HIVE-4039 URL: https://issues.apache.org/jira/browse/HIVE-4039 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Jean Xu Assignee: Jean Xu Priority: Minor Attachments: HIVE_4039.1.patch.txt Hive compiler fails with a NullPointerException in semantic analysis / optimisation stage when a boolean variable appears in the WHERE clause in some cases. A minimal query to generate this error is here: SELECT 1 FROM ( SELECT TRUE AS flag FROM dim_one_row:measurementsystems ) a WHERE flag; On the other hand, the following query is perfectly fine: SELECT 1 FROM ( SELECT TRUE AS flag FROM dim_one_row:measurementsystems ) a WHERE flag=TRUE; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4010) Failure finding iterate method with matching signature
[ https://issues.apache.org/jira/browse/HIVE-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581548#comment-13581548 ] Phabricator commented on HIVE-4010: --- mshang has commented on the revision HIVE-4010 [jira] Failure finding iterate method with matching signature. INLINE COMMENTS ql/src/test/org/apache/hadoop/hive/ql/udf/UDAFTestMethodOverloading.java:44 cc @jonchang maybe you can enlighten me. REVISION DETAIL https://reviews.facebook.net/D8517 To: JIRA, jonchang, kevinwilfong, njain, mshang Failure finding iterate method with matching signature -- Key: HIVE-4010 URL: https://issues.apache.org/jira/browse/HIVE-4010 Project: Hive Issue Type: Bug Components: UDF Reporter: Miles Shang Assignee: Miles Shang Priority: Minor Attachments: HIVE-4010.D8517.1.patch, HIVE-4010.D8517.2.patch Original Estimate: 24h Remaining Estimate: 24h http://fburl.com/10467687 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3672) Support altering partition column type in Hive
[ https://issues.apache.org/jira/browse/HIVE-3672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingwei Lu updated HIVE-3672: - Attachment: HIVE-3672.7.patch.txt Refresh and updated the test result. Support altering partition column type in Hive -- Key: HIVE-3672 URL: https://issues.apache.org/jira/browse/HIVE-3672 Project: Hive Issue Type: Improvement Components: CLI, SQL Reporter: Jingwei Lu Assignee: Jingwei Lu Labels: features Attachments: HIVE-3672.1.patch.txt, HIVE-3672.2.patch.txt, HIVE-3672.3.patch.txt, HIVE-3672.4.patch.txt, HIVE-3672.5.patch.txt, HIVE-3672.6.patch.txt, HIVE-3672.6.patch.txt, HIVE-3672.7.patch.txt Original Estimate: 72h Remaining Estimate: 72h Currently, Hive does not allow altering partition column types. As we've discouraged users from using non-string partition column types, this presents a problem for users who want to change there partition columns to be strings, they have to rename their table, create a new table, and copy all the data over. To support this via the CLI, adding a command like ALTER TABLE table_name PARTITION COLUMN (column_name new type); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4027) Thrift alter_table api doesnt validate column type
[ https://issues.apache.org/jira/browse/HIVE-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-4027: --- Attachment: HIVE-4027.patch.2 Thrift alter_table api doesnt validate column type -- Key: HIVE-4027 URL: https://issues.apache.org/jira/browse/HIVE-4027 Project: Hive Issue Type: Bug Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-4027.patch.1, HIVE-4027.patch.2 Thrift alter_table api doesnt validate column type so that invalid column type can sneak it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4027) Thrift alter_table api doesnt validate column type
[ https://issues.apache.org/jira/browse/HIVE-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-4027: --- Status: Patch Available (was: Open) Thrift alter_table api doesnt validate column type -- Key: HIVE-4027 URL: https://issues.apache.org/jira/browse/HIVE-4027 Project: Hive Issue Type: Bug Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-4027.patch.1, HIVE-4027.patch.2 Thrift alter_table api doesnt validate column type so that invalid column type can sneak it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4027) Thrift alter_table api doesnt validate column type
[ https://issues.apache.org/jira/browse/HIVE-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581573#comment-13581573 ] Gang Tim Liu commented on HIVE-4027: patch is available. Thrift alter_table api doesnt validate column type -- Key: HIVE-4027 URL: https://issues.apache.org/jira/browse/HIVE-4027 Project: Hive Issue Type: Bug Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-4027.patch.1, HIVE-4027.patch.2 Thrift alter_table api doesnt validate column type so that invalid column type can sneak it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3403) user should not specify mapjoin to perform sort-merge bucketed join
[ https://issues.apache.org/jira/browse/HIVE-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581588#comment-13581588 ] Hudson commented on HIVE-3403: -- Integrated in Hive-trunk-hadoop2 #128 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/128/]) HIVE-3403 user should not specify mapjoin to perform sort-merge bucketed join (Namit Jain via Ashutosh) (Revision 1447593) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1447593 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml.template * /hive/trunk/data/files/smallsrcsortbucket1outof4.txt * /hive/trunk/data/files/smallsrcsortbucket2outof4.txt * /hive/trunk/data/files/smallsrcsortbucket3outof4.txt * /hive/trunk/data/files/smallsrcsortbucket4outof4.txt * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SelectOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/AbstractBucketJoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/AbstractSMBJoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/AvgPartitionSizeBasedBigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketJoinProcCtx.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapjoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/LeftmostBigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SizeBasedBigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortBucketJoinProcCtx.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedMergeBucketMapJoinOptimizer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedMergeBucketMapjoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedMergeJoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TableSizeBasedBigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinResolver.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/SMBJoinDesc.java * /hive/trunk/ql/src/test/queries/clientnegative/smb_mapjoin_14.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_smb_mapjoin_14.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_1.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_10.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_2.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_3.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_4.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_5.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_6.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_7.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_8.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_9.q * /hive/trunk/ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_1.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_2.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_3.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_4.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_6.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_7.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_8.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out user should not specify mapjoin to perform sort-merge bucketed join --- Key: HIVE-3403 URL:
[jira] [Commented] (HIVE-4024) Derby metastore update script will fail when upgrading from 0.9.0 to 0.10.0
[
https://issues.apache.org/jira/browse/HIVE-4024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581589#comment-13581589
]
Hudson commented on HIVE-4024:
--
Integrated in Hive-trunk-hadoop2 #128 (See
[https://builds.apache.org/job/Hive-trunk-hadoop2/128/])
HIVE-4024 Derby metastore update script will fail when upgrading from 0.9.0
to 0.10.0 (Jarek Jarcec Cecho via namit) (Revision 1447584)
Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1447584
Files :
* /hive/trunk/metastore/scripts/upgrade/derby/011-HIVE-3649.derby.sql
Derby metastore update script will fail when upgrading from 0.9.0 to 0.10.0
---
Key: HIVE-4024
URL: https://issues.apache.org/jira/browse/HIVE-4024
Project: Hive
Issue Type: Bug
Reporter: Jarek Jarcec Cecho
Assignee: Jarek Jarcec Cecho
Priority: Minor
Fix For: 0.11.0
Attachments: bugHIVE-4024.patch
The problem is in following file {{011-HIVE-3649.derby.sql}} that contains
following line:
{code}
ALTER TABLE SDS ADD IS_STOREDASSUBDIRECTORIES CHAR(1) NOT NULL;
{code}
This query will however fail if the table SDS have at least one row.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4023) Improve Error Logging in MetaStore
[ https://issues.apache.org/jira/browse/HIVE-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581590#comment-13581590 ] Hudson commented on HIVE-4023: -- Integrated in Hive-trunk-hadoop2 #128 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/128/]) HIVE-4023 Improve Error Logging in MetaStore (Bhushan Mandhani via namit) (Revision 1447601) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1447601 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RetryingHMSHandler.java Improve Error Logging in MetaStore -- Key: HIVE-4023 URL: https://issues.apache.org/jira/browse/HIVE-4023 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Bhushan Mandhani Assignee: Bhushan Mandhani Priority: Trivial Fix For: 0.11.0 Attachments: HIVE-4023.1.patch.txt The RetryingHMSHandler should log the entire stack trace before throwing an exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-hadoop2 - Build # 128 - Still Failing
Changes for Build #95 [namit] HIVE-3873 lot of tests failing for hadoop 23 (Gang Tim Liu via namit) Changes for Build #96 [hashutosh] Missed deleting empty file GenMRRedSink4.java while commiting 3784 [hashutosh] HIVE-de-emphasize mapjoin hint (Namit Jain via Ashutosh Chauhan) Changes for Build #97 [namit] HIVE-933 Infer bucketing/sorting properties (Kevin Wilfong via namit) [hashutosh] HIVE-3950 : Remove code for merging files via MR job (Ashutosh Chauhan, Reviewed by Namit Jain) Changes for Build #98 Changes for Build #99 [kevinwilfong] HIVE-3940. Track columns accessed in each table in a query. (Samuel Yuan via kevinwilfong) Changes for Build #100 [namit] HIVE-3778 Add MapJoinDesc.isBucketMapJoin() as part of explain plan (Gang Tim Liu via namit) Changes for Build #101 Changes for Build #102 Changes for Build #103 Changes for Build #104 [hashutosh] HIVE-3977 : Hive 0.10 postgres schema script is broken (Johnny Zhang via Ashutosh Chauhan) [hashutosh] HIVE-3932 : Hive release tarballs don't contain PostgreSQL metastore scripts (Mark Grover via Ashutosh Chauhan) Changes for Build #105 [hashutosh] HIVE-3918 : Normalize more CRLF line endings (Mark Grover via Ashutosh Chauhan) [namit] HIVE-3917 Support noscan operation for analyze command (Gang Tim Liu via namit) Changes for Build #106 [namit] HIVE-3937 Hive Profiler (Pamela Vagata via namit) [hashutosh] HIVE-3571 : add a way to run a small unit quickly (Navis via Ashutosh Chauhan) [hashutosh] HIVE-3956 : TestMetaStoreAuthorization always uses the same port (Navis via Ashutosh Chauhan) Changes for Build #107 Changes for Build #108 Changes for Build #109 Changes for Build #110 [namit] HIVE-2839 Filters on outer join with mapjoin hint is not applied correctly (Navis via namit) Changes for Build #111 Changes for Build #112 [namit] HIVE-3998 Oracle metastore update script will fail when upgrading from 0.9.0 to 0.10.0 (Jarek and Mark via namit) [namit] HIVE-3999 Mysql metastore upgrade script will end up with different schema than the full schema load (Jarek and Mark via namit) Changes for Build #113 Changes for Build #114 [namit] HIVE-3995 PostgreSQL upgrade scripts are not valid (Jarek and Mark via namit) Changes for Build #115 Changes for Build #116 [namit] HIVE-4001 Add o.a.h.h.serde.Constants for backward compatibility (Navis via namit) Changes for Build #117 Changes for Build #118 Changes for Build #119 Changes for Build #120 [kevinwilfong] HIVE-3252. Add environment context to metastore Thrift calls. (Samuel Yuan via kevinwilfong) Changes for Build #121 Changes for Build #122 Changes for Build #123 Changes for Build #124 Changes for Build #125 Changes for Build #126 [hashutosh] HIVE-4000 Hive client goes into infinite loop at 100% cpu (Owen Omalley via Ashutosh Chauhan) Changes for Build #127 [namit] HIVE-4021 PostgreSQL upgrade scripts are creating column with incorrect name (Jarek Jarcec Cecho via namit) [hashutosh] HIVE-4033 : NPE at runtime while selecting virtual column after joining three tables on different keys (Ashutosh Chauhan) [namit] HIVE-4029 Hive Profiler dies with NPE (Brock Noland via namit) Changes for Build #128 [namit] HIVE-4023 Improve Error Logging in MetaStore (Bhushan Mandhani via namit) [namit] HIVE-3403 user should not specify mapjoin to perform sort-merge bucketed join (Namit Jain via Ashutosh) [namit] HIVE-4024 Derby metastore update script will fail when upgrading from 0.9.0 to 0.10.0 (Jarek Jarcec Cecho via namit) 34 tests failed. FAILED: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_archive_excludeHadoop20 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:5850) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_archive_excludeHadoop20(TestCliDriver.java:1367) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at
[jira] [Commented] (HIVE-4010) Failure finding iterate method with matching signature
[ https://issues.apache.org/jira/browse/HIVE-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581616#comment-13581616 ] Phabricator commented on HIVE-4010: --- jonchang has commented on the revision HIVE-4010 [jira] Failure finding iterate method with matching signature. INLINE COMMENTS ql/src/test/org/apache/hadoop/hive/ql/udf/UDAFTestMethodOverloading.java:44 IIRC, you can't overload based on return type (in either Java or Hive). REVISION DETAIL https://reviews.facebook.net/D8517 To: JIRA, jonchang, kevinwilfong, njain, mshang Failure finding iterate method with matching signature -- Key: HIVE-4010 URL: https://issues.apache.org/jira/browse/HIVE-4010 Project: Hive Issue Type: Bug Components: UDF Reporter: Miles Shang Assignee: Miles Shang Priority: Minor Attachments: HIVE-4010.D8517.1.patch, HIVE-4010.D8517.2.patch Original Estimate: 24h Remaining Estimate: 24h http://fburl.com/10467687 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4010) Failure finding iterate method with matching signature
[ https://issues.apache.org/jira/browse/HIVE-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581644#comment-13581644 ] Phabricator commented on HIVE-4010: --- mshang has commented on the revision HIVE-4010 [jira] Failure finding iterate method with matching signature. INLINE COMMENTS ql/src/test/org/apache/hadoop/hive/ql/udf/UDAFTestMethodOverloading.java:44 @jonchang, right, totally forgot about that. @njain, can you specify what you mean? REVISION DETAIL https://reviews.facebook.net/D8517 To: JIRA, jonchang, kevinwilfong, njain, mshang Failure finding iterate method with matching signature -- Key: HIVE-4010 URL: https://issues.apache.org/jira/browse/HIVE-4010 Project: Hive Issue Type: Bug Components: UDF Reporter: Miles Shang Assignee: Miles Shang Priority: Minor Attachments: HIVE-4010.D8517.1.patch, HIVE-4010.D8517.2.patch Original Estimate: 24h Remaining Estimate: 24h http://fburl.com/10467687 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3403) user should not specify mapjoin to perform sort-merge bucketed join
[ https://issues.apache.org/jira/browse/HIVE-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581656#comment-13581656 ] Hudson commented on HIVE-3403: -- Integrated in Hive-trunk-h0.21 #1976 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1976/]) HIVE-3403 user should not specify mapjoin to perform sort-merge bucketed join (Namit Jain via Ashutosh) (Revision 1447593) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1447593 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml.template * /hive/trunk/data/files/smallsrcsortbucket1outof4.txt * /hive/trunk/data/files/smallsrcsortbucket2outof4.txt * /hive/trunk/data/files/smallsrcsortbucket3outof4.txt * /hive/trunk/data/files/smallsrcsortbucket4outof4.txt * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SelectOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/AbstractBucketJoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/AbstractSMBJoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/AvgPartitionSizeBasedBigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketJoinProcCtx.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapjoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/LeftmostBigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SizeBasedBigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortBucketJoinProcCtx.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedMergeBucketMapJoinOptimizer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedMergeBucketMapjoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedMergeJoinProc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TableSizeBasedBigTableSelectorForAutoSMJ.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinResolver.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/SMBJoinDesc.java * /hive/trunk/ql/src/test/queries/clientnegative/smb_mapjoin_14.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_smb_mapjoin_14.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_1.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_10.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_2.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_3.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_4.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_5.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_6.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_7.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_8.q * /hive/trunk/ql/src/test/queries/clientpositive/auto_sortmerge_join_9.q * /hive/trunk/ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_1.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_2.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_3.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_4.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_6.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_7.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_8.q.out * /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out user should not specify mapjoin to perform sort-merge bucketed join --- Key: HIVE-3403 URL:
[jira] [Commented] (HIVE-4024) Derby metastore update script will fail when upgrading from 0.9.0 to 0.10.0
[
https://issues.apache.org/jira/browse/HIVE-4024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581657#comment-13581657
]
Hudson commented on HIVE-4024:
--
Integrated in Hive-trunk-h0.21 #1976 (See
[https://builds.apache.org/job/Hive-trunk-h0.21/1976/])
HIVE-4024 Derby metastore update script will fail when upgrading from 0.9.0
to 0.10.0 (Jarek Jarcec Cecho via namit) (Revision 1447584)
Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1447584
Files :
* /hive/trunk/metastore/scripts/upgrade/derby/011-HIVE-3649.derby.sql
Derby metastore update script will fail when upgrading from 0.9.0 to 0.10.0
---
Key: HIVE-4024
URL: https://issues.apache.org/jira/browse/HIVE-4024
Project: Hive
Issue Type: Bug
Reporter: Jarek Jarcec Cecho
Assignee: Jarek Jarcec Cecho
Priority: Minor
Fix For: 0.11.0
Attachments: bugHIVE-4024.patch
The problem is in following file {{011-HIVE-3649.derby.sql}} that contains
following line:
{code}
ALTER TABLE SDS ADD IS_STOREDASSUBDIRECTORIES CHAR(1) NOT NULL;
{code}
This query will however fail if the table SDS have at least one row.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4023) Improve Error Logging in MetaStore
[ https://issues.apache.org/jira/browse/HIVE-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581658#comment-13581658 ] Hudson commented on HIVE-4023: -- Integrated in Hive-trunk-h0.21 #1976 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1976/]) HIVE-4023 Improve Error Logging in MetaStore (Bhushan Mandhani via namit) (Revision 1447601) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1447601 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RetryingHMSHandler.java Improve Error Logging in MetaStore -- Key: HIVE-4023 URL: https://issues.apache.org/jira/browse/HIVE-4023 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Bhushan Mandhani Assignee: Bhushan Mandhani Priority: Trivial Fix For: 0.11.0 Attachments: HIVE-4023.1.patch.txt The RetryingHMSHandler should log the entire stack trace before throwing an exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1976 - Still Failing
Changes for Build #1975 [namit] HIVE-4021 PostgreSQL upgrade scripts are creating column with incorrect name (Jarek Jarcec Cecho via namit) [hashutosh] HIVE-4033 : NPE at runtime while selecting virtual column after joining three tables on different keys (Ashutosh Chauhan) [namit] HIVE-4029 Hive Profiler dies with NPE (Brock Noland via namit) Changes for Build #1976 [namit] HIVE-4023 Improve Error Logging in MetaStore (Bhushan Mandhani via namit) [namit] HIVE-3403 user should not specify mapjoin to perform sort-merge bucketed join (Namit Jain via Ashutosh) [namit] HIVE-4024 Derby metastore update script will fail when upgrading from 0.9.0 to 0.10.0 (Jarek Jarcec Cecho via namit) 1 tests failed. FAILED: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_aggregator_error_1 Error Message: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit. Stack Trace: junit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit. at net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259) at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268) at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:299) at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244) The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1976) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1976/ to view the results.
[jira] [Updated] (HIVE-4027) Thrift alter_table api doesnt validate column type
[ https://issues.apache.org/jira/browse/HIVE-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-4027: --- Attachment: HIVE-4027.patch.3 Thrift alter_table api doesnt validate column type -- Key: HIVE-4027 URL: https://issues.apache.org/jira/browse/HIVE-4027 Project: Hive Issue Type: Bug Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-4027.patch.1, HIVE-4027.patch.2, HIVE-4027.patch.3 Thrift alter_table api doesnt validate column type so that invalid column type can sneak it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3984) Maintain a clear separation between Windowing PTF at the specification level.
[ https://issues.apache.org/jira/browse/HIVE-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-3984: Attachment: HIVE-3984.3.patch.txt Maintain a clear separation between Windowing PTF at the specification level. Key: HIVE-3984 URL: https://issues.apache.org/jira/browse/HIVE-3984 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-3984.1.patch.txt, HIVE-3984.2.patch.txt, HIVE-3984.3.patch.txt This has multiple pieces: - redefine the PTF Spec classes, as described in the Data Structs doc in Hive-896 - refactor PTFDesc classes based on this design - refactor translation: both Phase 1 GenPlan phases -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3984) Maintain a clear separation between Windowing PTF at the specification level.
[ https://issues.apache.org/jira/browse/HIVE-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581860#comment-13581860 ] Harish Butani commented on HIVE-3984: - patch attached. - added the comments - many of the lint issues - fixed the 2 issues: Input's alias must not be visible in output of Function; bug in creation of First Partition in PTFOp Maintain a clear separation between Windowing PTF at the specification level. Key: HIVE-3984 URL: https://issues.apache.org/jira/browse/HIVE-3984 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-3984.1.patch.txt, HIVE-3984.2.patch.txt, HIVE-3984.3.patch.txt This has multiple pieces: - redefine the PTF Spec classes, as described in the Data Structs doc in Hive-896 - refactor PTFDesc classes based on this design - refactor translation: both Phase 1 GenPlan phases -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3984) Maintain a clear separation between Windowing PTF at the specification level.
[ https://issues.apache.org/jira/browse/HIVE-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581879#comment-13581879 ] Ashutosh Chauhan commented on HIVE-3984: +1 Maintain a clear separation between Windowing PTF at the specification level. Key: HIVE-3984 URL: https://issues.apache.org/jira/browse/HIVE-3984 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-3984.1.patch.txt, HIVE-3984.2.patch.txt, HIVE-3984.3.patch.txt This has multiple pieces: - redefine the PTF Spec classes, as described in the Data Structs doc in Hive-896 - refactor PTFDesc classes based on this design - refactor translation: both Phase 1 GenPlan phases -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-3984) Maintain a clear separation between Windowing PTF at the specification level.
[ https://issues.apache.org/jira/browse/HIVE-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-3984. Resolution: Fixed Committed to branch. Thanks Harish and Prajkta for your hard work on this one! Maintain a clear separation between Windowing PTF at the specification level. Key: HIVE-3984 URL: https://issues.apache.org/jira/browse/HIVE-3984 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-3984.1.patch.txt, HIVE-3984.2.patch.txt, HIVE-3984.3.patch.txt This has multiple pieces: - redefine the PTF Spec classes, as described in the Data Structs doc in Hive-896 - refactor PTFDesc classes based on this design - refactor translation: both Phase 1 GenPlan phases -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-3986) Fix select expr processing in PTF Operator
[ https://issues.apache.org/jira/browse/HIVE-3986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-3986. Resolution: Fixed Assignee: Harish Butani Fixed via HIVE-3984 Fix select expr processing in PTF Operator -- Key: HIVE-3986 URL: https://issues.apache.org/jira/browse/HIVE-3986 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Harish Butani Select expressions that contain Lead/Lag functions are handled by the PTF Operator as a post processing step after output Partition is computed. The Select Exprs Node Descs for these are incorrectly created using the Input RR. These should be created, just like the having expression using the Output RR of the PTF. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-701) lots of reserved keywords in hive
[ https://issues.apache.org/jira/browse/HIVE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581896#comment-13581896 ] Kevin Wilfong commented on HIVE-701: +1 Looks good. lots of reserved keywords in hive - Key: HIVE-701 URL: https://issues.apache.org/jira/browse/HIVE-701 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Namit Jain Assignee: Samuel Yuan Attachments: HIVE-701.1.patch.txt, HIVE-701.2.patch.txt, HIVE-701.D8397.1.patch, HIVE-701.HIVE-701.D8397.2.patch, HIVE-701.HIVE-701.D8397.3.patch There is a problem if we want to use some reserved keywords: for example, creating a function of name left/right ? left/right is already a reserved keyword. The other way around should also be possible - if we want to add a 'show tables status' and some applications already use status as a column name, they should not break -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4040) fix ptf negative tests
Harish Butani created HIVE-4040: --- Summary: fix ptf negative tests Key: HIVE-4040 URL: https://issues.apache.org/jira/browse/HIVE-4040 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Prajakta Kalmegh Priority: Minor fix queries in -ve tests to match language changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4041) Support multiple partitionings in a single Query
Harish Butani created HIVE-4041: --- Summary: Support multiple partitionings in a single Query Key: HIVE-4041 URL: https://issues.apache.org/jira/browse/HIVE-4041 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Harish Butani Currently we disallow queries if the partition specifications of all Wdw fns are not the same. We can relax this by generating multiple PTFOps based on the unique partitionings in a Query. For partitionings that only differ in sort, we can introduce a sort step in between PTFOps, which can happen in the same Reduce task. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4007) Create abstract classes for serializer and deserializer
[ https://issues.apache.org/jira/browse/HIVE-4007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581928#comment-13581928 ] Namit Jain commented on HIVE-4007: -- [~jarcec], good point - will do Create abstract classes for serializer and deserializer --- Key: HIVE-4007 URL: https://issues.apache.org/jira/browse/HIVE-4007 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4007.1.patch, hive.4007.2.patch Currently, it is very difficult to change the Serializer/Deserializer interface, since all the SerDes directly implement the interface. Instead, we should have abstract classes for implementing these interfaces. In case of a interface change, only the abstract class and the relevant serde needs to change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3980) Cleanup after HIVE-3403
[ https://issues.apache.org/jira/browse/HIVE-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581931#comment-13581931 ] Namit Jain commented on HIVE-3980: -- https://reviews.facebook.net/D8703 Cleanup after HIVE-3403 --- Key: HIVE-3980 URL: https://issues.apache.org/jira/browse/HIVE-3980 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3980.1.patch There have been a lot of comments on HIVE-3403, which involve changing variable names/function names/adding more comments/general cleanup etc. Since HIVE-3403 involves a lot of refactoring, it was fairly difficult to address the comments there, since refreshing becomes impossible. This jira is to track those cleanups. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-684) add UDF make_set
[ https://issues.apache.org/jira/browse/HIVE-684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PRETTY SITHARA updated HIVE-684: Attachment: make_set.q.out make_set.q input.txt.txt HIVE-684.1.patch.txt Patch for HIVE-684 add UDF make_set Key: HIVE-684 URL: https://issues.apache.org/jira/browse/HIVE-684 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Namit Jain Attachments: HIVE-684.1.patch.txt, input.txt.txt, make_set.q, make_set.q.out add UDFmake_set look at http://dev.mysql.com/doc/refman/5.0/en/func-op-summary-ref.html for details -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3980) Cleanup after HIVE-3403
[ https://issues.apache.org/jira/browse/HIVE-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3980: - Attachment: hive.3980.2.patch Cleanup after HIVE-3403 --- Key: HIVE-3980 URL: https://issues.apache.org/jira/browse/HIVE-3980 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3980.1.patch, hive.3980.2.patch There have been a lot of comments on HIVE-3403, which involve changing variable names/function names/adding more comments/general cleanup etc. Since HIVE-3403 involves a lot of refactoring, it was fairly difficult to address the comments there, since refreshing becomes impossible. This jira is to track those cleanups. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4042) ignore mapjoin hint
Namit Jain created HIVE-4042: Summary: ignore mapjoin hint Key: HIVE-4042 URL: https://issues.apache.org/jira/browse/HIVE-4042 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain After HIVE-3784, in a production environment, it can become difficult to deploy since a lot of production queries can break. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3980) Cleanup after HIVE-3403
[ https://issues.apache.org/jira/browse/HIVE-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3980: - Status: Patch Available (was: Open) cleanup patch -- only involves comments/changing variable names/function names etc. Cleanup after HIVE-3403 --- Key: HIVE-3980 URL: https://issues.apache.org/jira/browse/HIVE-3980 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3980.1.patch, hive.3980.2.patch There have been a lot of comments on HIVE-3403, which involve changing variable names/function names/adding more comments/general cleanup etc. Since HIVE-3403 involves a lot of refactoring, it was fairly difficult to address the comments there, since refreshing becomes impossible. This jira is to track those cleanups. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4042) ignore mapjoin hint
[ https://issues.apache.org/jira/browse/HIVE-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581953#comment-13581953 ] Namit Jain commented on HIVE-4042: -- Mapjoin hint is only needed for bucketed/sorted tables. It might be a good idea to just ignore the hint, if the underlying table is not bucketed/sorted. ignore mapjoin hint --- Key: HIVE-4042 URL: https://issues.apache.org/jira/browse/HIVE-4042 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain After HIVE-3784, in a production environment, it can become difficult to deploy since a lot of production queries can break. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4007) Create abstract classes for serializer and deserializer
[ https://issues.apache.org/jira/browse/HIVE-4007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4007: - Attachment: hive.4007.3.patch Create abstract classes for serializer and deserializer --- Key: HIVE-4007 URL: https://issues.apache.org/jira/browse/HIVE-4007 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4007.1.patch, hive.4007.2.patch, hive.4007.3.patch Currently, it is very difficult to change the Serializer/Deserializer interface, since all the SerDes directly implement the interface. Instead, we should have abstract classes for implementing these interfaces. In case of a interface change, only the abstract class and the relevant serde needs to change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4040) fix ptf negative tests
[ https://issues.apache.org/jira/browse/HIVE-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prajakta Kalmegh updated HIVE-4040: --- Attachment: HIVE-4040.1.patch.txt Attached patch to fix negative tests. This change was triggered due to separation of PTF and Windowing in https://issues.apache.org/jira/browse/HIVE-3984 fix ptf negative tests -- Key: HIVE-4040 URL: https://issues.apache.org/jira/browse/HIVE-4040 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Prajakta Kalmegh Priority: Minor Attachments: HIVE-4040.1.patch.txt fix queries in -ve tests to match language changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4040) fix ptf negative tests
[ https://issues.apache.org/jira/browse/HIVE-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581985#comment-13581985 ] Harish Butani commented on HIVE-4040: - +1 looks good; just ran the tests and confirmed. fix ptf negative tests -- Key: HIVE-4040 URL: https://issues.apache.org/jira/browse/HIVE-4040 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Prajakta Kalmegh Priority: Minor Attachments: HIVE-4040.1.patch.txt fix queries in -ve tests to match language changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3874) Create a new Optimized Row Columnar file format for Hive
[ https://issues.apache.org/jira/browse/HIVE-3874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3874: -- Attachment: HIVE-3874.D8529.2.patch omalley updated the revision HIVE-3874 [jira] Create a new Optimized Row Columnar file format for Hive. Addressed Kevin's feedback. * Fixed 500+ warnings out of checkstyle - there are a few left that I couldn't avoid. * Fixed all of the cases that I could find where operators didn't have space around them. If we care about that, we should configure checkstyle to check for it. * Fixed the directory with the code to match the package name. * Removed the loop initializing the boolean array in OrcInputFormat. * Added code to include the dictionary size when estimating memory size. Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D8529 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D8529?vs=27621id=28101#toc AFFECTED FILES build.properties build.xml ivy/libraries.properties ql/build.xml ql/ivy.xml ql/src/gen/protobuf/gen-java/org/apache/hadoop/hive/ql/io/orc/OrcProto.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/BitFieldReader.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/BitFieldWriter.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/BooleanColumnStatistics.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/ColumnStatistics.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/ColumnStatisticsImpl.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/CompressionCodec.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/CompressionKind.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/DoubleColumnStatistics.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/DynamicByteArray.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/DynamicIntArray.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/FileDump.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/InStream.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/IntegerColumnStatistics.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFile.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcOutputFormat.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSerde.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcStruct.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcUnion.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/OutStream.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/PositionProvider.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/PositionRecorder.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/PositionedOutputStream.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/Reader.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/ReaderImpl.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReader.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/RedBlackTree.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthByteReader.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthByteWriter.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReader.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerWriter.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/SerializationUtils.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/SnappyCodec.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/StreamName.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/StringColumnStatistics.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/StringRedBlackTree.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/StripeInformation.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/Writer.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/WriterImpl.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/ZlibCodec.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/package-info.java ql/src/protobuf/org/apache/hadoop/hive/ql/io/orc/orc_proto.proto ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestBitFieldReader.java ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestDynamicArray.java ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestFileDump.java ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInStream.java ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcFile.java ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcStruct.java ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestRunLengthByteReader.java ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestRunLengthIntegerReader.java ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestSerializationUtils.java ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestStreamName.java ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestStringRedBlackTree.java ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestZlib.java
[jira] [Updated] (HIVE-4042) ignore mapjoin hint
[ https://issues.apache.org/jira/browse/HIVE-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4042: - Attachment: hive.4042.1.patch ignore mapjoin hint --- Key: HIVE-4042 URL: https://issues.apache.org/jira/browse/HIVE-4042 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4042.1.patch After HIVE-3784, in a production environment, it can become difficult to deploy since a lot of production queries can break. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4042) ignore mapjoin hint
[ https://issues.apache.org/jira/browse/HIVE-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13582019#comment-13582019 ] Namit Jain commented on HIVE-4042: -- https://reviews.facebook.net/D8709 ignore mapjoin hint --- Key: HIVE-4042 URL: https://issues.apache.org/jira/browse/HIVE-4042 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4042.1.patch After HIVE-3784, in a production environment, it can become difficult to deploy since a lot of production queries can break. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4007) Create abstract classes for serializer and deserializer
[ https://issues.apache.org/jira/browse/HIVE-4007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13582020#comment-13582020 ] Namit Jain commented on HIVE-4007: -- Refreshed, addressed comments, all tests passed Create abstract classes for serializer and deserializer --- Key: HIVE-4007 URL: https://issues.apache.org/jira/browse/HIVE-4007 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.4007.1.patch, hive.4007.2.patch, hive.4007.3.patch Currently, it is very difficult to change the Serializer/Deserializer interface, since all the SerDes directly implement the interface. Instead, we should have abstract classes for implementing these interfaces. In case of a interface change, only the abstract class and the relevant serde needs to change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3980) Cleanup after HIVE-3403
[ https://issues.apache.org/jira/browse/HIVE-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13582021#comment-13582021 ] Namit Jain commented on HIVE-3980: -- Tests passed Cleanup after HIVE-3403 --- Key: HIVE-3980 URL: https://issues.apache.org/jira/browse/HIVE-3980 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3980.1.patch, hive.3980.2.patch There have been a lot of comments on HIVE-3403, which involve changing variable names/function names/adding more comments/general cleanup etc. Since HIVE-3403 involves a lot of refactoring, it was fairly difficult to address the comments there, since refreshing becomes impossible. This jira is to track those cleanups. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
