[jira] [Updated] (HIVE-3552) HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a high number of grouping set keys
[ https://issues.apache.org/jira/browse/HIVE-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3552: - Attachment: hive.3552.3.patch HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a high number of grouping set keys - Key: HIVE-3552 URL: https://issues.apache.org/jira/browse/HIVE-3552 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3552.1.patch, hive.3552.2.patch, hive.3552.3.patch This is a follow up for HIVE-3433. Had a offline discussion with Sambavi - she pointed out a scenario where the implementation in HIVE-3433 will not scale. Assume that the user is performing a cube on many columns, say '8' columns. So, each row would generate 256 rows for the hash table, which may kill the current group by implementation. A better implementation would be to add an additional mr job - in the first mr job perform the group by assuming there was no cube. Add another mr job, where you would perform the cube. The assumption is that the group by would have decreased the output data significantly, and the rows would appear in the order of grouping keys which has a higher probability of hitting the hash table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3074) Create a new API which gets a new empty partition object.
[ https://issues.apache.org/jira/browse/HIVE-3074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3074: - Status: Open (was: Patch Available) Create a new API which gets a new empty partition object. - Key: HIVE-3074 URL: https://issues.apache.org/jira/browse/HIVE-3074 Project: Hive Issue Type: New Feature Components: Metastore Reporter: Nadeem Moidu Assignee: Namit Jain Attachments: hive.3074.1.patch, HIVE-3074.1.patch Gets a new partition object with the given parameters. Similar to append_partition, but does not add the partition to metastore. It only works for tables; does not work for other objects like views. The table is fetched from the metastore using the db name and the table name. However, the actual partition is not fetched from the metastore. It does not matter whether the partition exists or not.The partition values are used to construct a new partition. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3774) Sort merge join should work if join cols are a prefix of sort columns for each partition
[ https://issues.apache.org/jira/browse/HIVE-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13529808#comment-13529808 ] Namit Jain commented on HIVE-3774: -- comments addressed Sort merge join should work if join cols are a prefix of sort columns for each partition Key: HIVE-3774 URL: https://issues.apache.org/jira/browse/HIVE-3774 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3774.1.patch, hive.3774.2.patch Currently, a join is converted into a sort-merge join only if the join cols exactly matches the sort cols. This constraint can definitely be relaxed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3774) Sort merge join should work if join cols are a prefix of sort columns for each partition
[ https://issues.apache.org/jira/browse/HIVE-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3774: - Attachment: hive.3774.2.patch Sort merge join should work if join cols are a prefix of sort columns for each partition Key: HIVE-3774 URL: https://issues.apache.org/jira/browse/HIVE-3774 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3774.1.patch, hive.3774.2.patch Currently, a join is converted into a sort-merge join only if the join cols exactly matches the sort cols. This constraint can definitely be relaxed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3784) de-emphasize mapjoin hint
[ https://issues.apache.org/jira/browse/HIVE-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3784: - Attachment: hive.3784.1.patch de-emphasize mapjoin hint - Key: HIVE-3784 URL: https://issues.apache.org/jira/browse/HIVE-3784 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3784.1.patch hive.auto.convert.join has been around for a long time, and is pretty stable. When mapjoin hint was created, the above parameter did not exist. The only reason for the user to specify a mapjoin currently is if they want it to be converted to a bucketed-mapjoin or a sort-merge bucketed mapjoin. Eventually, that should also go away, but that may take some time to stabilize. There are many rules in SemanticAnalyzer to handle the following trees: ReduceSink - MapJoin Union - MapJoin MapJoin- MapJoin This should not be supported anymore. In any of the above scenarios, the user can get the mapjoin behavior by setting hive.auto.convert.join to true and not specifying the hint. This will simplify the code a lot. What does everyone think ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3784) de-emphasize mapjoin hint
[ https://issues.apache.org/jira/browse/HIVE-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3784: - Attachment: hive.3784.2.patch de-emphasize mapjoin hint - Key: HIVE-3784 URL: https://issues.apache.org/jira/browse/HIVE-3784 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3784.1.patch, hive.3784.2.patch hive.auto.convert.join has been around for a long time, and is pretty stable. When mapjoin hint was created, the above parameter did not exist. The only reason for the user to specify a mapjoin currently is if they want it to be converted to a bucketed-mapjoin or a sort-merge bucketed mapjoin. Eventually, that should also go away, but that may take some time to stabilize. There are many rules in SemanticAnalyzer to handle the following trees: ReduceSink - MapJoin Union - MapJoin MapJoin- MapJoin This should not be supported anymore. In any of the above scenarios, the user can get the mapjoin behavior by setting hive.auto.convert.join to true and not specifying the hint. This will simplify the code a lot. What does everyone think ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3793) Print number of fetched rows after query in CliDriver
[ https://issues.apache.org/jira/browse/HIVE-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13529876#comment-13529876 ] Phabricator commented on HIVE-3793: --- njain has accepted the revision HIVE-3793 [jira] Print number of fetched rows after query in CliDriver. REVISION DETAIL https://reviews.facebook.net/D7305 BRANCH DPAL-1942 To: JIRA, njain, navis Print number of fetched rows after query in CliDriver - Key: HIVE-3793 URL: https://issues.apache.org/jira/browse/HIVE-3793 Project: Hive Issue Type: Improvement Components: CLI Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-3793.D7305.1.patch Currently shows time taken only. But it would be useful showing number of rows fetched also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3552) HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a high number of grouping set keys
[ https://issues.apache.org/jira/browse/HIVE-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530021#comment-13530021 ] Namit Jain commented on HIVE-3552: -- comments addressed + tests passed HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a high number of grouping set keys - Key: HIVE-3552 URL: https://issues.apache.org/jira/browse/HIVE-3552 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3552.1.patch, hive.3552.2.patch, hive.3552.3.patch, hive.3552.4.patch This is a follow up for HIVE-3433. Had a offline discussion with Sambavi - she pointed out a scenario where the implementation in HIVE-3433 will not scale. Assume that the user is performing a cube on many columns, say '8' columns. So, each row would generate 256 rows for the hash table, which may kill the current group by implementation. A better implementation would be to add an additional mr job - in the first mr job perform the group by assuming there was no cube. Add another mr job, where you would perform the cube. The assumption is that the group by would have decreased the output data significantly, and the rows would appear in the order of grouping keys which has a higher probability of hitting the hash table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3552) HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a high number of grouping set keys
[ https://issues.apache.org/jira/browse/HIVE-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3552: - Attachment: hive.3552.4.patch HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a high number of grouping set keys - Key: HIVE-3552 URL: https://issues.apache.org/jira/browse/HIVE-3552 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3552.1.patch, hive.3552.2.patch, hive.3552.3.patch, hive.3552.4.patch This is a follow up for HIVE-3433. Had a offline discussion with Sambavi - she pointed out a scenario where the implementation in HIVE-3433 will not scale. Assume that the user is performing a cube on many columns, say '8' columns. So, each row would generate 256 rows for the hash table, which may kill the current group by implementation. A better implementation would be to add an additional mr job - in the first mr job perform the group by assuming there was no cube. Add another mr job, where you would perform the cube. The assumption is that the group by would have decreased the output data significantly, and the rows would appear in the order of grouping keys which has a higher probability of hitting the hash table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3783) stats19.q is failing on trunk
[ https://issues.apache.org/jira/browse/HIVE-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530075#comment-13530075 ] Ashutosh Chauhan commented on HIVE-3783: TestStatsPublisherEnhanced failures on apache jenkins machine. https://builds.apache.org/job/Hive-trunk-h0.21/1849/testReport/ stats19.q is failing on trunk - Key: HIVE-3783 URL: https://issues.apache.org/jira/browse/HIVE-3783 Project: Hive Issue Type: Bug Affects Versions: 0.11 Reporter: Ashutosh Chauhan Assignee: Kevin Wilfong Attachments: HIVE-3783.1.patch.txt This test-case was introduced in HIVE-3750 and is failing since as soon as it was introduced. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3774) Sort merge join should work if join cols are a prefix of sort columns for each partition
[ https://issues.apache.org/jira/browse/HIVE-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530082#comment-13530082 ] Namit Jain commented on HIVE-3774: -- tests passed Sort merge join should work if join cols are a prefix of sort columns for each partition Key: HIVE-3774 URL: https://issues.apache.org/jira/browse/HIVE-3774 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3774.1.patch, hive.3774.2.patch, hive.3774.3.patch Currently, a join is converted into a sort-merge join only if the join cols exactly matches the sort cols. This constraint can definitely be relaxed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3774) Sort merge join should work if join cols are a prefix of sort columns for each partition
[ https://issues.apache.org/jira/browse/HIVE-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3774: - Attachment: hive.3774.3.patch Sort merge join should work if join cols are a prefix of sort columns for each partition Key: HIVE-3774 URL: https://issues.apache.org/jira/browse/HIVE-3774 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3774.1.patch, hive.3774.2.patch, hive.3774.3.patch Currently, a join is converted into a sort-merge join only if the join cols exactly matches the sort cols. This constraint can definitely be relaxed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #227
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/227/ -- [...truncated 10010 lines...] compile-test: [echo] Project: serde [javac] Compiling 26 source files to https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/227/artifact/hive/build/serde/test/classes [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. create-dirs: [echo] Project: service [copy] Warning: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/service/src/test/resources does not exist. init: [echo] Project: service ivy-init-settings: [echo] Project: service ivy-resolve: [echo] Project: service [ivy:resolve] :: loading settings :: file = https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml [ivy:report] Processing https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/227/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-service-default.xml to https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/227/artifact/hive/build/ivy/report/org.apache.hive-hive-service-default.html ivy-retrieve: [echo] Project: service compile: [echo] Project: service ivy-resolve-test: [echo] Project: service ivy-retrieve-test: [echo] Project: service compile-test: [echo] Project: service [javac] Compiling 2 source files to https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/227/artifact/hive/build/service/test/classes test: [echo] Project: hive test-shims: [echo] Project: hive test-conditions: [echo] Project: shims gen-test: [echo] Project: shims create-dirs: [echo] Project: shims [copy] Warning: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/test/resources does not exist. init: [echo] Project: shims ivy-init-settings: [echo] Project: shims ivy-resolve: [echo] Project: shims [ivy:resolve] :: loading settings :: file = https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml [ivy:report] Processing https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/227/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-shims-default.xml to https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/227/artifact/hive/build/ivy/report/org.apache.hive-hive-shims-default.html ivy-retrieve: [echo] Project: shims compile: [echo] Project: shims [echo] Building shims 0.20 build_shims: [echo] Project: shims [echo] Compiling https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/common/java;/home/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20/java against hadoop 0.20.2 (https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/227/artifact/hive/build/hadoopcore/hadoop-0.20.2) ivy-init-settings: [echo] Project: shims ivy-resolve-hadoop-shim: [echo] Project: shims [ivy:resolve] :: loading settings :: file = https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml ivy-retrieve-hadoop-shim: [echo] Project: shims [echo] Building shims 0.20S build_shims: [echo] Project: shims [echo] Compiling https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/common/java;/home/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common-secure/java;/home/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20S/java against hadoop 1.0.0 (https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/227/artifact/hive/build/hadoopcore/hadoop-1.0.0) ivy-init-settings: [echo] Project: shims ivy-resolve-hadoop-shim: [echo] Project: shims [ivy:resolve] :: loading settings :: file = https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml ivy-retrieve-hadoop-shim: [echo] Project: shims [echo] Building shims 0.23 build_shims: [echo] Project: shims [echo] Compiling https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/common/java;/home/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common-secure/java;/home/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.23/java against hadoop 0.23.3 (https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/227/artifact/hive/build/hadoopcore/hadoop-0.23.3)
[jira] [Updated] (HIVE-3793) Print number of fetched rows after query in CliDriver
[ https://issues.apache.org/jira/browse/HIVE-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3793: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Navis Print number of fetched rows after query in CliDriver - Key: HIVE-3793 URL: https://issues.apache.org/jira/browse/HIVE-3793 Project: Hive Issue Type: Improvement Components: CLI Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-3793.D7305.1.patch Currently shows time taken only. But it would be useful showing number of rows fetched also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3794) Oracle upgrade script for Hive is broken
Deepesh Khandelwal created HIVE-3794:
Summary: Oracle upgrade script for Hive is broken
Key: HIVE-3794
URL: https://issues.apache.org/jira/browse/HIVE-3794
Project: Hive
Issue Type: Bug
Components: SQL
Affects Versions: 0.10.0
Environment: Oracle 11g r2
Reporter: Deepesh Khandelwal
Priority: Critical
As part of Hive configuration for Oracle I ran the schema creation script for
Oracle. Here is what I observed when ran the script:
% sqlplus hive/hive@xe
SQL*Plus: Release 11.2.0.2.0 Production on Mon Dec 10 18:47:11 2012
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
SQL @scripts/metastore/upgrade/oracle/hive-schema-0.10.0.oracle.sql;
.
ALTER TABLE SKEWED_STRING_LIST_VALUES ADD CONSTRAINT
SKEWED_STRING_LIST_VALUES_FK1 FOREIGN KEY (STRING_LIST_ID) REFERENCES
SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00904: STRING_LIST_ID: invalid identifier{color}
.
ALTER TABLE SKEWED_STRING_LIST_VALUES ADD CONSTRAINT
SKEWED_STRING_LIST_VALUES_FK1 FOREIGN KEY (STRING_LIST_ID) REFERENCES
SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00904: STRING_LIST_ID: invalid identifier{color}
Table created.
Table altered.
Table altered.
CREATE TABLE SKEWED_COL_VALUE_LOCATION_MAPPING
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}
Table created.
Table created.
ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT
SKEWED_COL_VALUE_LOCATION_MAPPING_PK PRIMARY KEY (SD_ID,STRING_LIST_ID_KID)
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}
ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT
SKEWED_COL_VALUE_LOCATION_MAPPING_FK1 FOREIGN KEY (STRING_LIST_ID_KID)
REFERENCES SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}
ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT
SKEWED_COL_VALUE_LOCATION_MAPPING_FK2 FOREIGN KEY (SD_ID) REFERENCES SDS
(SD_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}
Table created.
Table altered.
ALTER TABLE SKEWED_VALUES ADD CONSTRAINT SKEWED_VALUES_FK1 FOREIGN KEY
(STRING_LIST_ID_EID) REFERENCES SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY
DEFERRED
*
ERROR at line 1:
{color:red}ORA-00904: STRING_LIST_ID: invalid identifier{color}
Basically there are two issues here with the Oracle sql script:
(1) Table SKEWED_STRING_LIST is created with the column SD_ID. Later the
script tries to reference STRING_LIST_ID column in SKEWED_STRING_LIST which
is obviously not there. Comparing the sql with that for other flavors it seems
it should be STRING_LIST_ID.
(2) Table name SKEWED_COL_VALUE_LOCATION_MAPPING is too long for Oracle which
limits identifier names to 30 characters. Also impacted are identifiers
SKEWED_COL_VALUE_LOCATION_MAPPING_PK and
SKEWED_COL_VALUE_LOCATION_MAPPING_FK1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3794) Oracle upgrade script for Hive is broken
[
https://issues.apache.org/jira/browse/HIVE-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-3794:
-
Fix Version/s: 0.10.0
Status: Patch Available (was: Open)
I have a patch which I successfully tested on Oracle. It does impact other
flavors.
Oracle upgrade script for Hive is broken
Key: HIVE-3794
URL: https://issues.apache.org/jira/browse/HIVE-3794
Project: Hive
Issue Type: Bug
Components: SQL
Affects Versions: 0.10.0
Environment: Oracle 11g r2
Reporter: Deepesh Khandelwal
Priority: Critical
Fix For: 0.10.0
As part of Hive configuration for Oracle I ran the schema creation script for
Oracle. Here is what I observed when ran the script:
% sqlplus hive/hive@xe
SQL*Plus: Release 11.2.0.2.0 Production on Mon Dec 10 18:47:11 2012
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
SQL @scripts/metastore/upgrade/oracle/hive-schema-0.10.0.oracle.sql;
.
ALTER TABLE SKEWED_STRING_LIST_VALUES ADD CONSTRAINT
SKEWED_STRING_LIST_VALUES_FK1 FOREIGN KEY (STRING_LIST_ID) REFERENCES
SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00904: STRING_LIST_ID: invalid identifier{color}
.
ALTER TABLE SKEWED_STRING_LIST_VALUES ADD CONSTRAINT
SKEWED_STRING_LIST_VALUES_FK1 FOREIGN KEY (STRING_LIST_ID) REFERENCES
SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00904: STRING_LIST_ID: invalid identifier{color}
Table created.
Table altered.
Table altered.
CREATE TABLE SKEWED_COL_VALUE_LOCATION_MAPPING
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}
Table created.
Table created.
ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT
SKEWED_COL_VALUE_LOCATION_MAPPING_PK PRIMARY KEY (SD_ID,STRING_LIST_ID_KID)
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}
ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT
SKEWED_COL_VALUE_LOCATION_MAPPING_FK1 FOREIGN KEY (STRING_LIST_ID_KID)
REFERENCES SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}
ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT
SKEWED_COL_VALUE_LOCATION_MAPPING_FK2 FOREIGN KEY (SD_ID) REFERENCES SDS
(SD_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}
Table created.
Table altered.
ALTER TABLE SKEWED_VALUES ADD CONSTRAINT SKEWED_VALUES_FK1 FOREIGN KEY
(STRING_LIST_ID_EID) REFERENCES SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY
DEFERRED
*
ERROR at line 1:
{color:red}ORA-00904: STRING_LIST_ID: invalid identifier{color}
Basically there are two issues here with the Oracle sql script:
(1) Table SKEWED_STRING_LIST is created with the column SD_ID. Later the
script tries to reference STRING_LIST_ID column in SKEWED_STRING_LIST
which is obviously not there. Comparing the sql with that for other flavors
it seems it should be STRING_LIST_ID.
(2) Table name SKEWED_COL_VALUE_LOCATION_MAPPING is too long for Oracle
which limits identifier names to 30 characters. Also impacted are identifiers
SKEWED_COL_VALUE_LOCATION_MAPPING_PK and
SKEWED_COL_VALUE_LOCATION_MAPPING_FK1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3794) Oracle upgrade script for Hive is broken
[
https://issues.apache.org/jira/browse/HIVE-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-3794:
-
Attachment: HIVE-3794.patch
Oracle upgrade script for Hive is broken
Key: HIVE-3794
URL: https://issues.apache.org/jira/browse/HIVE-3794
Project: Hive
Issue Type: Bug
Components: SQL
Affects Versions: 0.10.0
Environment: Oracle 11g r2
Reporter: Deepesh Khandelwal
Priority: Critical
Fix For: 0.10.0
Attachments: HIVE-3794.patch
As part of Hive configuration for Oracle I ran the schema creation script for
Oracle. Here is what I observed when ran the script:
% sqlplus hive/hive@xe
SQL*Plus: Release 11.2.0.2.0 Production on Mon Dec 10 18:47:11 2012
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
SQL @scripts/metastore/upgrade/oracle/hive-schema-0.10.0.oracle.sql;
.
ALTER TABLE SKEWED_STRING_LIST_VALUES ADD CONSTRAINT
SKEWED_STRING_LIST_VALUES_FK1 FOREIGN KEY (STRING_LIST_ID) REFERENCES
SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00904: STRING_LIST_ID: invalid identifier{color}
.
ALTER TABLE SKEWED_STRING_LIST_VALUES ADD CONSTRAINT
SKEWED_STRING_LIST_VALUES_FK1 FOREIGN KEY (STRING_LIST_ID) REFERENCES
SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00904: STRING_LIST_ID: invalid identifier{color}
Table created.
Table altered.
Table altered.
CREATE TABLE SKEWED_COL_VALUE_LOCATION_MAPPING
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}
Table created.
Table created.
ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT
SKEWED_COL_VALUE_LOCATION_MAPPING_PK PRIMARY KEY (SD_ID,STRING_LIST_ID_KID)
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}
ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT
SKEWED_COL_VALUE_LOCATION_MAPPING_FK1 FOREIGN KEY (STRING_LIST_ID_KID)
REFERENCES SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}
ALTER TABLE SKEWED_COL_VALUE_LOCATION_MAPPING ADD CONSTRAINT
SKEWED_COL_VALUE_LOCATION_MAPPING_FK2 FOREIGN KEY (SD_ID) REFERENCES SDS
(SD_ID) INITIALLY DEFERRED
*
ERROR at line 1:
{color:red}ORA-00972: identifier is too long{color}
Table created.
Table altered.
ALTER TABLE SKEWED_VALUES ADD CONSTRAINT SKEWED_VALUES_FK1 FOREIGN KEY
(STRING_LIST_ID_EID) REFERENCES SKEWED_STRING_LIST (STRING_LIST_ID) INITIALLY
DEFERRED
*
ERROR at line 1:
{color:red}ORA-00904: STRING_LIST_ID: invalid identifier{color}
Basically there are two issues here with the Oracle sql script:
(1) Table SKEWED_STRING_LIST is created with the column SD_ID. Later the
script tries to reference STRING_LIST_ID column in SKEWED_STRING_LIST
which is obviously not there. Comparing the sql with that for other flavors
it seems it should be STRING_LIST_ID.
(2) Table name SKEWED_COL_VALUE_LOCATION_MAPPING is too long for Oracle
which limits identifier names to 30 characters. Also impacted are identifiers
SKEWED_COL_VALUE_LOCATION_MAPPING_PK and
SKEWED_COL_VALUE_LOCATION_MAPPING_FK1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3795) NPE in SELECT when WHERE-clause is an and/or/not operation involving null
Xiao Jiang created HIVE-3795: Summary: NPE in SELECT when WHERE-clause is an and/or/not operation involving null Key: HIVE-3795 URL: https://issues.apache.org/jira/browse/HIVE-3795 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Xiao Jiang Sometimes users forget to quote date constants in queries. For example, SELECT * FROM some_table WHERE ds = 2012-12-10 and ds = 2012-12-12; . In such cases, if the WHERE-clause contains and/or/not operation, it would throw NPE exception. That's because PcrExprProcFactory in ql/optimizer forgot to check null. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3795) NPE in SELECT when WHERE-clause is an and/or/not operation involving null
[ https://issues.apache.org/jira/browse/HIVE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Jiang updated HIVE-3795: - Priority: Trivial (was: Major) NPE in SELECT when WHERE-clause is an and/or/not operation involving null - Key: HIVE-3795 URL: https://issues.apache.org/jira/browse/HIVE-3795 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Xiao Jiang Priority: Trivial Sometimes users forget to quote date constants in queries. For example, SELECT * FROM some_table WHERE ds = 2012-12-10 and ds = 2012-12-12; . In such cases, if the WHERE-clause contains and/or/not operation, it would throw NPE exception. That's because PcrExprProcFactory in ql/optimizer forgot to check null. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3783) stats19.q is failing on trunk
[ https://issues.apache.org/jira/browse/HIVE-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530152#comment-13530152 ] Kevin Wilfong commented on HIVE-3783: - The failures in TestStatsPublisherEnhanced are coming from the init method of the stats Publisher. stats19.q is failing on trunk - Key: HIVE-3783 URL: https://issues.apache.org/jira/browse/HIVE-3783 Project: Hive Issue Type: Bug Affects Versions: 0.11 Reporter: Ashutosh Chauhan Assignee: Kevin Wilfong Attachments: HIVE-3783.1.patch.txt This test-case was introduced in HIVE-3750 and is failing since as soon as it was introduced. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3783) stats19.q is failing on trunk
[ https://issues.apache.org/jira/browse/HIVE-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530157#comment-13530157 ] Kevin Wilfong commented on HIVE-3783: - I suspect the two are unrelated. stats19.q is failing on trunk - Key: HIVE-3783 URL: https://issues.apache.org/jira/browse/HIVE-3783 Project: Hive Issue Type: Bug Affects Versions: 0.11 Reporter: Ashutosh Chauhan Assignee: Kevin Wilfong Attachments: HIVE-3783.1.patch.txt This test-case was introduced in HIVE-3750 and is failing since as soon as it was introduced. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3783) stats19.q is failing on trunk
[ https://issues.apache.org/jira/browse/HIVE-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530167#comment-13530167 ] Namit Jain commented on HIVE-3783: -- +1 stats19.q is failing on trunk - Key: HIVE-3783 URL: https://issues.apache.org/jira/browse/HIVE-3783 Project: Hive Issue Type: Bug Affects Versions: 0.11 Reporter: Ashutosh Chauhan Assignee: Kevin Wilfong Attachments: HIVE-3783.1.patch.txt This test-case was introduced in HIVE-3750 and is failing since as soon as it was introduced. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3537) release locks at the end of move tasks
[ https://issues.apache.org/jira/browse/HIVE-3537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530176#comment-13530176 ] Kevin Wilfong commented on HIVE-3537: - +1 release locks at the end of move tasks -- Key: HIVE-3537 URL: https://issues.apache.org/jira/browse/HIVE-3537 Project: Hive Issue Type: Bug Components: Locking, Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3537.1.patch, hive.3537.2.patch, hive.3537.3.patch Look at HIVE-3106 for details. In order to make sure that concurrency is not an issue for multi-table inserts, the current option is to introduce a dependency task, which thereby delays the creation of all partitions. It would be desirable to release the locks for the outputs as soon as the move task is completed. That way, for multi-table inserts, the concurrency can be enabled without delaying any table. Currently, the movetask contains a input/output, but they do not seem to be populated correctly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3795) NPE in SELECT when WHERE-clause is an and/or/not operation involving null
[ https://issues.apache.org/jira/browse/HIVE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Jiang reassigned HIVE-3795: Assignee: Xiao Jiang NPE in SELECT when WHERE-clause is an and/or/not operation involving null - Key: HIVE-3795 URL: https://issues.apache.org/jira/browse/HIVE-3795 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Xiao Jiang Assignee: Xiao Jiang Priority: Trivial Sometimes users forget to quote date constants in queries. For example, SELECT * FROM some_table WHERE ds = 2012-12-10 and ds = 2012-12-12; . In such cases, if the WHERE-clause contains and/or/not operation, it would throw NPE exception. That's because PcrExprProcFactory in ql/optimizer forgot to check null. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3796) Multi-insert involving bucketed/sorted table turns off merging on all outputs
Kevin Wilfong created HIVE-3796: --- Summary: Multi-insert involving bucketed/sorted table turns off merging on all outputs Key: HIVE-3796 URL: https://issues.apache.org/jira/browse/HIVE-3796 Project: Hive Issue Type: Improvement Components: Query Processor Affects Versions: 0.11 Reporter: Kevin Wilfong Assignee: Kevin Wilfong When a multi-insert query has at least one output that is bucketed, merging is turned off for all outputs, rather than just the bucketed ones. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3785) Core hive changes for HiveServer2 implementation
[
https://issues.apache.org/jira/browse/HIVE-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530287#comment-13530287
]
Mark Grover commented on HIVE-3785:
---
FWIW, this patch doesn't apply cleanly on trunk. There is a conflict with
conf/hive-default.xml.template
{code}
***
*** 1581,1590
/property
property
-namehive.decode.partition.name/name
-valuefalse/value
-descriptionWhether to show the unquoted partition names in query
results./description
/property
/configuration
--- 1580,1660
/property
property
+ namehive.server2.thrift.min.worker.threads/name
+ value5/value
+ descriptionMinimum number of Thrift worker threads/description
+ /property
+
+ property
+ namehive.server2.thrift.max.worker.threads/name
+ value100/value
+ descriptionMaximum number of Thrift worker threads/description
+ /property
+
{code}
I don't it's necessary to refresh the patch now (but it wouldn't hurt) since we
are still waiting for review but it would need to be refreshed before
submission.
Core hive changes for HiveServer2 implementation
Key: HIVE-3785
URL: https://issues.apache.org/jira/browse/HIVE-3785
Project: Hive
Issue Type: Sub-task
Components: Authentication, Build Infrastructure, Configuration,
Thrift API
Affects Versions: 0.10.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Attachments: HS2-changed-files-only.patch
The subtask to track changes in the core hive components for HiveServer2
implementation
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530288#comment-13530288 ] Phabricator commented on HIVE-3384: --- mbautin has abandoned the revision [jira] [HIVE-3384] HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification. Committed into Hive trunk and 0.9 branches. REVISION DETAIL https://reviews.facebook.net/D6873 To: cdrome, weidongbian, JIRA, ashutoshc, mbautin HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification -- Key: HIVE-3384 URL: https://issues.apache.org/jira/browse/HIVE-3384 Project: Hive Issue Type: Bug Components: JDBC Reporter: Weidong Bian Assignee: Mikhail Bautin Priority: Minor Fix For: 0.11 Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, D6873.7.patch, HIVE-3384-2012-12-02.patch, HIVE-3384-2012-12-04.patch, HIVE-3384.2.patch, HIVE-3384-branch-0.9.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch jdbc module couldn't be compiled with jdk7 as it adds some abstract method in the JDBC specification some error info: error: HiveCallableStatement is not abstract and does not override abstract method TgetObject(String,ClassT) in CallableStatement . . . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3796) Multi-insert involving bucketed/sorted table turns off merging on all outputs
[ https://issues.apache.org/jira/browse/HIVE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530320#comment-13530320 ] Kevin Wilfong commented on HIVE-3796: - https://reviews.facebook.net/D7335 Multi-insert involving bucketed/sorted table turns off merging on all outputs - Key: HIVE-3796 URL: https://issues.apache.org/jira/browse/HIVE-3796 Project: Hive Issue Type: Improvement Components: Query Processor Affects Versions: 0.11 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3796.1.patch.txt When a multi-insert query has at least one output that is bucketed, merging is turned off for all outputs, rather than just the bucketed ones. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3796) Multi-insert involving bucketed/sorted table turns off merging on all outputs
[ https://issues.apache.org/jira/browse/HIVE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3796: Attachment: HIVE-3796.1.patch.txt Multi-insert involving bucketed/sorted table turns off merging on all outputs - Key: HIVE-3796 URL: https://issues.apache.org/jira/browse/HIVE-3796 Project: Hive Issue Type: Improvement Components: Query Processor Affects Versions: 0.11 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3796.1.patch.txt When a multi-insert query has at least one output that is bucketed, merging is turned off for all outputs, rather than just the bucketed ones. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3796) Multi-insert involving bucketed/sorted table turns off merging on all outputs
[ https://issues.apache.org/jira/browse/HIVE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3796: Status: Patch Available (was: Open) Multi-insert involving bucketed/sorted table turns off merging on all outputs - Key: HIVE-3796 URL: https://issues.apache.org/jira/browse/HIVE-3796 Project: Hive Issue Type: Improvement Components: Query Processor Affects Versions: 0.11 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3796.1.patch.txt When a multi-insert query has at least one output that is bucketed, merging is turned off for all outputs, rather than just the bucketed ones. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1851 - Still Failing
Changes for Build #1844 [hashutosh] HIVE-3705 : Adding authorization capability to the metastore (Sushanth Sowmyan via Ashutosh Chauhan) Changes for Build #1845 [hashutosh] HIVE-3231 : msck repair should find partitions already containing data files (Keegan Mosley via Ashutosh Chauhan) [hashutosh] HIVE-2691 : Specify location of log4j configuration files via configuration properties (Zhenxiao Luo via Ashutosh Chauhan) [hashutosh] HIVE-2794 : Aggregations without grouping should return NULL when applied to partitioning column of a partitionless table (Zhenxiao Luo via Ashutosh Chauhan) [hashutosh] HIVE-3780 : RetryingMetaStoreClient Should Log the Caught Exception (Bhushan Mandhani via Ashutosh Chauhan) [hashutosh] HIVE-3084 : Hive CI failing due to script_broken_pipe1.q (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-3760 : TestNegativeMinimrCliDriver_mapreduce_stack_trace.q fails on hadoop-1 (Gunther Hagleitner via Ashutosh Chauhan) Changes for Build #1846 Changes for Build #1847 [hashutosh] HIVE-3714 : Patch: Hive's ivy internal resolvers need to use sourceforge for sqlline (Gopal V via Ashutosh Chauhan) Changes for Build #1848 [hashutosh] HIVE-3782 : testCliDriver_sample_islocalmode_hook fails on hadoop-1 (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-2288 : Adding the oracle nvl function to the UDF (Ed Capriolo, Guy Doulberg via Ashutosh Chauhan) [hashutosh] HIVE-2689 : ObjectInspectorConverters cannot convert Void types to Array/Map/Struct types. (Jonathan Chang via Ashutosh Chauhan) Changes for Build #1849 [hashutosh] HIVE-3622 : reflect udf cannot find method which has arguments of primitive types and String, Binary, Timestamp types mixed (Navis via Ashutosh Chauhan) [namit] HIVE-3401 Diversify grammar for split sampling (Navis via namit) Changes for Build #1850 Changes for Build #1851 7 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_aggregator_error_1 Error Message: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit. Stack Trace: junit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit. at net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259) at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268) at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:324) at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244) REGRESSION: org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherOneStat Error Message: null Stack Trace: junit.framework.AssertionFailedError: null at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at junit.framework.Assert.assertTrue(Assert.java:27) at org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherOneStat(TestStatsPublisherEnhanced.java:81) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:79) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785) REGRESSION: org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisher Error Message: null Stack Trace: junit.framework.AssertionFailedError: null at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at junit.framework.Assert.assertTrue(Assert.java:27) at org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisher(TestStatsPublisherEnhanced.java:129) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
[jira] [Commented] (HIVE-3401) Diversify grammar for split sampling
[ https://issues.apache.org/jira/browse/HIVE-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530505#comment-13530505 ] Lars Francke commented on HIVE-3401: I know Namit already asked about it but I think this'd be the relevant Wiki page to update: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Sampling Diversify grammar for split sampling Key: HIVE-3401 URL: https://issues.apache.org/jira/browse/HIVE-3401 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-3401.D4821.2.patch, HIVE-3401.D4821.3.patch, HIVE-3401.D4821.4.patch, HIVE-3401.D4821.5.patch, HIVE-3401.D4821.6.patch, HIVE-3401.D4821.7.patch Current split sampling only supports grammar like TABLESAMPLE(n PERCENT). But some users wants to specify just the size of input. It can be easily calculated with a few commands but it seemed good to support more grammars something like TABLESAMPLE(500M). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3401) Diversify grammar for split sampling
[ https://issues.apache.org/jira/browse/HIVE-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530508#comment-13530508 ] Navis commented on HIVE-3401: - How can I get the permission? Diversify grammar for split sampling Key: HIVE-3401 URL: https://issues.apache.org/jira/browse/HIVE-3401 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-3401.D4821.2.patch, HIVE-3401.D4821.3.patch, HIVE-3401.D4821.4.patch, HIVE-3401.D4821.5.patch, HIVE-3401.D4821.6.patch, HIVE-3401.D4821.7.patch Current split sampling only supports grammar like TABLESAMPLE(n PERCENT). But some users wants to specify just the size of input. It can be easily calculated with a few commands but it seemed good to support more grammars something like TABLESAMPLE(500M). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3789) Patch HIVE-3648 causing the majority of unit tests to fail on branch 0.9
[
https://issues.apache.org/jira/browse/HIVE-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530545#comment-13530545
]
Arup Malakar commented on HIVE-3789:
The exception gets eaten up and doesn't show up in console. Here is the exact
stacktrace which I caught and logged, before rethrowing.
{code}
[junit] Something wrong happened while moving to trash Wrong FS:
pfile:/Users/malakar/code/oss/hive_09/hive/build/metastore/test/data/warehouse/testtablefilter.db/table1,
expected: file:///[junit]
org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:581)
[junit] org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:680)
[junit]
org.apache.hadoop.fs.FilterFileSystem.resolvePath(FilterFileSystem.java:139)
[junit]
org.apache.hadoop.fs.FilterFileSystem.resolvePath(FilterFileSystem.java:139)
[junit]
org.apache.hadoop.fs.FilterFileSystem.resolvePath(FilterFileSystem.java:139)
[junit] org.apache.hadoop.fs.Trash.moveToAppropriateTrash(Trash.java:70)
[junit]
org.apache.hadoop.hive.shims.Hadoop23Shims.moveToAppropriateTrash(Hadoop23Shims.java:133)
[junit]
org.apache.hadoop.hive.metastore.HiveMetaStoreFsImpl.deleteDir(HiveMetaStoreFsImpl.java:45)
[junit]
org.apache.hadoop.hive.metastore.Warehouse.deleteDir(Warehouse.java:201)
[junit]
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:929)
[junit]
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table(HiveMetaStore.java:944)
[junit]
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table.getResult(ThriftHiveMetastore.java:4955)
[junit]
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table.getResult(ThriftHiveMetastore.java:4943)
[junit] org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
[junit] org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
[junit]
org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:48)
[junit]
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
[junit]
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[junit]
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[junit] java.lang.Thread.run(Thread.java:680)
{code}
Patch HIVE-3648 causing the majority of unit tests to fail on branch 0.9
Key: HIVE-3789
URL: https://issues.apache.org/jira/browse/HIVE-3789
Project: Hive
Issue Type: Bug
Components: Metastore, Tests
Affects Versions: 0.9.0
Environment: Hadooop 0.23.5, JDK 1.6.0_31
Reporter: Chris Drome
Rolling back to before this patch shows that the unit tests are passing,
after the patch, the majority of the unit tests are failing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2379) Hive/HBase integration could be improved
[ https://issues.apache.org/jira/browse/HIVE-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis reassigned HIVE-2379: --- Assignee: Navis Hive/HBase integration could be improved Key: HIVE-2379 URL: https://issues.apache.org/jira/browse/HIVE-2379 Project: Hive Issue Type: Improvement Components: CLI, Clients, HBase Handler Affects Versions: 0.7.1, 0.8.0, 0.9.0 Reporter: Roman Shaposhnik Assignee: Navis Priority: Minor For now any Hive/HBase queries would require the following jars to be explicitly added via hive's add jar command: add jar /usr/lib/hive/lib/hbase-0.90.1-cdh3u0.jar; add jar /usr/lib/hive/lib/hive-hbase-handler-0.7.0-cdh3u0.jar; add jar /usr/lib/hive/lib/zookeeper-3.3.1.jar; add jar /usr/lib/hive/lib/guava-r06.jar; the longer term solution, perhaps, should be to have the code at submit time call hbase's TableMapREduceUtil.addDependencyJar(job, HBaseStorageHandler.class) to ship it in distributedcache. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2379) Hive/HBase integration could be improved
[ https://issues.apache.org/jira/browse/HIVE-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2379: Status: Patch Available (was: Open) Hive/HBase integration could be improved Key: HIVE-2379 URL: https://issues.apache.org/jira/browse/HIVE-2379 Project: Hive Issue Type: Improvement Components: CLI, Clients, HBase Handler Affects Versions: 0.9.0, 0.8.0, 0.7.1 Reporter: Roman Shaposhnik Assignee: Navis Priority: Minor For now any Hive/HBase queries would require the following jars to be explicitly added via hive's add jar command: add jar /usr/lib/hive/lib/hbase-0.90.1-cdh3u0.jar; add jar /usr/lib/hive/lib/hive-hbase-handler-0.7.0-cdh3u0.jar; add jar /usr/lib/hive/lib/zookeeper-3.3.1.jar; add jar /usr/lib/hive/lib/guava-r06.jar; the longer term solution, perhaps, should be to have the code at submit time call hbase's TableMapREduceUtil.addDependencyJar(job, HBaseStorageHandler.class) to ship it in distributedcache. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2379) Hive/HBase integration could be improved
[ https://issues.apache.org/jira/browse/HIVE-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2379: -- Attachment: HIVE-2379.D7347.1.patch navis requested code review of HIVE-2379 [jira] Hive/HBase integration could be improved. Reviewers: JIRA DPAL-1936 Add libraries for hbase handler automatically For now any Hive/HBase queries would require the following jars to be explicitly added via hive's add jar command: add jar /usr/lib/hive/lib/hbase-0.90.1-cdh3u0.jar; add jar /usr/lib/hive/lib/hive-hbase-handler-0.7.0-cdh3u0.jar; add jar /usr/lib/hive/lib/zookeeper-3.3.1.jar; add jar /usr/lib/hive/lib/guava-r06.jar; the longer term solution, perhaps, should be to have the code at submit time call hbase's TableMapREduceUtil.addDependencyJar(job, HBaseStorageHandler.class) to ship it in distributedcache. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D7347 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java ql/src/java/org/apache/hadoop/hive/ql/metadata/DefaultStorageHandler.java ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/17559/ To: JIRA, navis Hive/HBase integration could be improved Key: HIVE-2379 URL: https://issues.apache.org/jira/browse/HIVE-2379 Project: Hive Issue Type: Improvement Components: CLI, Clients, HBase Handler Affects Versions: 0.7.1, 0.8.0, 0.9.0 Reporter: Roman Shaposhnik Assignee: Navis Priority: Minor Attachments: HIVE-2379.D7347.1.patch For now any Hive/HBase queries would require the following jars to be explicitly added via hive's add jar command: add jar /usr/lib/hive/lib/hbase-0.90.1-cdh3u0.jar; add jar /usr/lib/hive/lib/hive-hbase-handler-0.7.0-cdh3u0.jar; add jar /usr/lib/hive/lib/zookeeper-3.3.1.jar; add jar /usr/lib/hive/lib/guava-r06.jar; the longer term solution, perhaps, should be to have the code at submit time call hbase's TableMapREduceUtil.addDependencyJar(job, HBaseStorageHandler.class) to ship it in distributedcache. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3401) Diversify grammar for split sampling
[ https://issues.apache.org/jira/browse/HIVE-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530656#comment-13530656 ] Namit Jain commented on HIVE-3401: -- Navis, can you tell me your wiki id. ? I can add you. Diversify grammar for split sampling Key: HIVE-3401 URL: https://issues.apache.org/jira/browse/HIVE-3401 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-3401.D4821.2.patch, HIVE-3401.D4821.3.patch, HIVE-3401.D4821.4.patch, HIVE-3401.D4821.5.patch, HIVE-3401.D4821.6.patch, HIVE-3401.D4821.7.patch Current split sampling only supports grammar like TABLESAMPLE(n PERCENT). But some users wants to specify just the size of input. It can be easily calculated with a few commands but it seemed good to support more grammars something like TABLESAMPLE(500M). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3790) UDF to introduce an OFFSET(day,month or year) for a given date or timestamp
[
https://issues.apache.org/jira/browse/HIVE-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530658#comment-13530658
]
Jithin John commented on HIVE-3790:
---
hi Sun Rui,
currently we could acheive the scenario by nesting the fuction like
date_offset(date_offset(date,1,'year'),3,'month')
we could consider whether SQL inteval type is possible or not?
UDF to introduce an OFFSET(day,month or year) for a given date or timestamp
Key: HIVE-3790
URL: https://issues.apache.org/jira/browse/HIVE-3790
Project: Hive
Issue Type: New Feature
Components: UDF
Affects Versions: 0.9.0
Reporter: Jithin John
Current releases of Hive lacks a generic function which would find the date
offset to a date / timestamp. Current releases have date_add (date) and
date_sub(date) which allows user to add or substract days only.But we could
not use year or month as a unit.
The Function DATE_OFFSET(date,offset,unit) returns the date offset value from
start_date according to the unit. Here the unit can be year , month and day.
The function could be used for date range queries and is more flexible than
the existing functions.
Functionality :-
Function Name: DATE_OFFSET(date,offset,unit)
Add a offset value to the unit part of the date/timestamp.
Returns the date in the format of -MM-dd .
Example: hive select date_offset('2009-07-29', -1 ,'MONTH' ) FROM src LIMIT 1
- 2009-06-29
Usage :-
Case : To calculate the expiry date of a item from manufacturing date
Table :- ITEM_TAB
Manufacturing_date |item id|store id|value|unit|price
2012-12-01|110001|00003|0.99|1.00|0.99
2012-12-02|110001|00008|0.99|0.00|0.00
2012-12-03|110001|00009|0.99|0.00|0.00
2012-12-04|110001|001112002|0.99|0.00|0.00
2012-12-05|110001|001112003|0.99|0.00|0.00
2012-12-06|110001|001112006|0.99|1.00|0.99
2012-12-07|110001|001112007|0.99|0.00|0.00
2012-12-08|110001|001112008|0.99|0.00|0.00
2012-12-09|110001|001112009|0.99|0.00|0.00
2012-12-10|110001|001112010|0.99|0.00|0.00
2012-12-11|110001|001113003|0.99|0.00|0.00
2012-12-12|110001|001113006|0.99|0.00|0.00
2012-12-13|110001|001113008|0.99|0.00|0.00
2012-12-14|110001|001113010|0.99|0.00|0.00
2012-12-15|110001|001114002|0.99|0.00|0.00
2012-12-16|110001|001114004|0.99|1.00|0.99
2012-12-17|110001|001114005|0.99|0.00|0.00
2012-12-18|110001|001121004|0.99|0.00|0.00
QUERY:
select man_date , date_offset(man_date ,5 ,'year') as expiry_date from
item_tab;
RESULT:
2012-12-01 2017-12-01
2012-12-02 2017-12-02
2012-12-03 2017-12-03
2012-12-04 2017-12-04
2012-12-05 2017-12-05
2012-12-06 2017-12-06
2012-12-07 2017-12-07
2012-12-08 2017-12-08
2012-12-09 2017-12-09
2012-12-10 2017-12-10
2012-12-11 2017-12-11
2012-12-12 2017-12-12
2012-12-13 2017-12-13
2012-12-14 2017-12-14
2012-12-15 2017-12-15
2012-12-16 2017-12-16
2012-12-17 2017-12-17
2012-12-18 2017-12-18
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3401) Diversify grammar for split sampling
[ https://issues.apache.org/jira/browse/HIVE-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530659#comment-13530659 ] Navis commented on HIVE-3401: - It's 'navis' and.. as you know by reviewing my codes/comments, I'm really bad at English writing. Diversify grammar for split sampling Key: HIVE-3401 URL: https://issues.apache.org/jira/browse/HIVE-3401 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-3401.D4821.2.patch, HIVE-3401.D4821.3.patch, HIVE-3401.D4821.4.patch, HIVE-3401.D4821.5.patch, HIVE-3401.D4821.6.patch, HIVE-3401.D4821.7.patch Current split sampling only supports grammar like TABLESAMPLE(n PERCENT). But some users wants to specify just the size of input. It can be easily calculated with a few commands but it seemed good to support more grammars something like TABLESAMPLE(500M). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3790) UDF to introduce an OFFSET(day,month or year) for a given date or timestamp
[
https://issues.apache.org/jira/browse/HIVE-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530666#comment-13530666
]
Jithin John commented on HIVE-3790:
---
hi georgy,
since we are using java calendar , leap year scenario will work.
UDF to introduce an OFFSET(day,month or year) for a given date or timestamp
Key: HIVE-3790
URL: https://issues.apache.org/jira/browse/HIVE-3790
Project: Hive
Issue Type: New Feature
Components: UDF
Affects Versions: 0.9.0
Reporter: Jithin John
Current releases of Hive lacks a generic function which would find the date
offset to a date / timestamp. Current releases have date_add (date) and
date_sub(date) which allows user to add or substract days only.But we could
not use year or month as a unit.
The Function DATE_OFFSET(date,offset,unit) returns the date offset value from
start_date according to the unit. Here the unit can be year , month and day.
The function could be used for date range queries and is more flexible than
the existing functions.
Functionality :-
Function Name: DATE_OFFSET(date,offset,unit)
Add a offset value to the unit part of the date/timestamp.
Returns the date in the format of -MM-dd .
Example: hive select date_offset('2009-07-29', -1 ,'MONTH' ) FROM src LIMIT 1
- 2009-06-29
Usage :-
Case : To calculate the expiry date of a item from manufacturing date
Table :- ITEM_TAB
Manufacturing_date |item id|store id|value|unit|price
2012-12-01|110001|00003|0.99|1.00|0.99
2012-12-02|110001|00008|0.99|0.00|0.00
2012-12-03|110001|00009|0.99|0.00|0.00
2012-12-04|110001|001112002|0.99|0.00|0.00
2012-12-05|110001|001112003|0.99|0.00|0.00
2012-12-06|110001|001112006|0.99|1.00|0.99
2012-12-07|110001|001112007|0.99|0.00|0.00
2012-12-08|110001|001112008|0.99|0.00|0.00
2012-12-09|110001|001112009|0.99|0.00|0.00
2012-12-10|110001|001112010|0.99|0.00|0.00
2012-12-11|110001|001113003|0.99|0.00|0.00
2012-12-12|110001|001113006|0.99|0.00|0.00
2012-12-13|110001|001113008|0.99|0.00|0.00
2012-12-14|110001|001113010|0.99|0.00|0.00
2012-12-15|110001|001114002|0.99|0.00|0.00
2012-12-16|110001|001114004|0.99|1.00|0.99
2012-12-17|110001|001114005|0.99|0.00|0.00
2012-12-18|110001|001121004|0.99|0.00|0.00
QUERY:
select man_date , date_offset(man_date ,5 ,'year') as expiry_date from
item_tab;
RESULT:
2012-12-01 2017-12-01
2012-12-02 2017-12-02
2012-12-03 2017-12-03
2012-12-04 2017-12-04
2012-12-05 2017-12-05
2012-12-06 2017-12-06
2012-12-07 2017-12-07
2012-12-08 2017-12-08
2012-12-09 2017-12-09
2012-12-10 2017-12-10
2012-12-11 2017-12-11
2012-12-12 2017-12-12
2012-12-13 2017-12-13
2012-12-14 2017-12-14
2012-12-15 2017-12-15
2012-12-16 2017-12-16
2012-12-17 2017-12-17
2012-12-18 2017-12-18
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1955) Support non-constant expressions for array indexes.
[ https://issues.apache.org/jira/browse/HIVE-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530678#comment-13530678 ] Navis commented on HIVE-1955: - Is there any reason index value should be constant? Support non-constant expressions for array indexes. --- Key: HIVE-1955 URL: https://issues.apache.org/jira/browse/HIVE-1955 Project: Hive Issue Type: Improvement Reporter: Adam Kramer FAILED: Error in semantic analysis: line 4:8 Non Constant Expressions for Array Indexes not Supported dut ...just wrote my own UDF to do this, and it is trivial. We should support this natively. Let foo have these rows: arr i [1,2,3] 1 [3,4,5] 2 [5,4,3] 2 [0,0,1] 0 Then, SELECT arr[i] FROM foo should return: 2 5 3 1 Similarly, for the same table, SELECT 3 IN arr FROM foo should return: true true true false ...these use cases are needless limitations of functionality. We shouldn't need UDFs to accomplish these goals. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3383) MinimrCliDriver test hangs when run on 20S (aka 1.0) hadoop profile
[
https://issues.apache.org/jira/browse/HIVE-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530682#comment-13530682
]
Mark Grover commented on HIVE-3383:
---
I ran into the same problem today with simple {{ant test}} on trunk.
MinimrCliDriver test hangs when run on 20S (aka 1.0) hadoop profile
---
Key: HIVE-3383
URL: https://issues.apache.org/jira/browse/HIVE-3383
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Reporter: Ashutosh Chauhan
I ran
{code}
ant clean package test -Dtestcase=TestMinimrCliDriver -Dhadoop.mr.rev=20S
{code}
This hangs after printing following on console:
{code}
test:
[junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is
deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the
log4j.properties files.
[junit] Starting DataNode 0 with dfs.data.dir:
build/test/data/dfs/data/data1,build/test/data/dfs/data/data2
[junit] Starting DataNode 1 with dfs.data.dir:
build/test/data/dfs/data/data3,build/test/data/dfs/data/data4
[junit] Starting DataNode 2 with dfs.data.dir:
build/test/data/dfs/data/data5,build/test/data/dfs/data/data6
[junit] Starting DataNode 3 with dfs.data.dir:
build/test/data/dfs/data/data7,build/test/data/dfs/data/data8
[junit] Generating rack names for tasktrackers
[junit] Generating host names for tasktrackers
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3784) de-emphasize mapjoin hint
[ https://issues.apache.org/jira/browse/HIVE-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3784: - Attachment: hive.3784.3.patch de-emphasize mapjoin hint - Key: HIVE-3784 URL: https://issues.apache.org/jira/browse/HIVE-3784 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3784.1.patch, hive.3784.2.patch, hive.3784.3.patch hive.auto.convert.join has been around for a long time, and is pretty stable. When mapjoin hint was created, the above parameter did not exist. The only reason for the user to specify a mapjoin currently is if they want it to be converted to a bucketed-mapjoin or a sort-merge bucketed mapjoin. Eventually, that should also go away, but that may take some time to stabilize. There are many rules in SemanticAnalyzer to handle the following trees: ReduceSink - MapJoin Union - MapJoin MapJoin- MapJoin This should not be supported anymore. In any of the above scenarios, the user can get the mapjoin behavior by setting hive.auto.convert.join to true and not specifying the hint. This will simplify the code a lot. What does everyone think ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2991) Integrate Clover with Hive
[ https://issues.apache.org/jira/browse/HIVE-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Katsov updated HIVE-2991: -- Attachment: hive-trunk-clover-html-report.zip Latest report for trunk is attached (class-level info was removed to reduce the archive size). Integrate Clover with Hive -- Key: HIVE-2991 URL: https://issues.apache.org/jira/browse/HIVE-2991 Project: Hive Issue Type: Test Components: Testing Infrastructure Affects Versions: 0.9.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2991.D2985.1.patch, hive.2991.1.branch-0.10.patch, hive.2991.1.branch-0.9.patch, hive.2991.1.trunk.patch, hive-trunk-clover-html-report.zip Atlassian has donated license of their code coverage tool Clover to ASF. Lets make use of it to generate code coverage report to figure out which areas of Hive are well tested and which ones are not. More information about license can be found in Hadoop jira HADOOP-1718 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3793) Print number of fetched rows after query in CliDriver
[ https://issues.apache.org/jira/browse/HIVE-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13530747#comment-13530747 ] Hudson commented on HIVE-3793: -- Integrated in Hive-trunk-h0.21 #1852 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1852/]) HIVE-3793 Print number of fetched rows after query in CliDriver (Navis via namit) (Revision 1420817) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1420817 Files : * /hive/trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java Print number of fetched rows after query in CliDriver - Key: HIVE-3793 URL: https://issues.apache.org/jira/browse/HIVE-3793 Project: Hive Issue Type: Improvement Components: CLI Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-3793.D7305.1.patch Currently shows time taken only. But it would be useful showing number of rows fetched also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1852 - Still Failing
Changes for Build #1844 [hashutosh] HIVE-3705 : Adding authorization capability to the metastore (Sushanth Sowmyan via Ashutosh Chauhan) Changes for Build #1845 [hashutosh] HIVE-3231 : msck repair should find partitions already containing data files (Keegan Mosley via Ashutosh Chauhan) [hashutosh] HIVE-2691 : Specify location of log4j configuration files via configuration properties (Zhenxiao Luo via Ashutosh Chauhan) [hashutosh] HIVE-2794 : Aggregations without grouping should return NULL when applied to partitioning column of a partitionless table (Zhenxiao Luo via Ashutosh Chauhan) [hashutosh] HIVE-3780 : RetryingMetaStoreClient Should Log the Caught Exception (Bhushan Mandhani via Ashutosh Chauhan) [hashutosh] HIVE-3084 : Hive CI failing due to script_broken_pipe1.q (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-3760 : TestNegativeMinimrCliDriver_mapreduce_stack_trace.q fails on hadoop-1 (Gunther Hagleitner via Ashutosh Chauhan) Changes for Build #1846 Changes for Build #1847 [hashutosh] HIVE-3714 : Patch: Hive's ivy internal resolvers need to use sourceforge for sqlline (Gopal V via Ashutosh Chauhan) Changes for Build #1848 [hashutosh] HIVE-3782 : testCliDriver_sample_islocalmode_hook fails on hadoop-1 (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-2288 : Adding the oracle nvl function to the UDF (Ed Capriolo, Guy Doulberg via Ashutosh Chauhan) [hashutosh] HIVE-2689 : ObjectInspectorConverters cannot convert Void types to Array/Map/Struct types. (Jonathan Chang via Ashutosh Chauhan) Changes for Build #1849 [hashutosh] HIVE-3622 : reflect udf cannot find method which has arguments of primitive types and String, Binary, Timestamp types mixed (Navis via Ashutosh Chauhan) [namit] HIVE-3401 Diversify grammar for split sampling (Navis via namit) Changes for Build #1850 Changes for Build #1851 Changes for Build #1852 [namit] HIVE-3793 Print number of fetched rows after query in CliDriver (Navis via namit) 2 tests failed. FAILED: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_aggregator_error_1 Error Message: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit. Stack Trace: junit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit. at net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259) at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268) at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:324) at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244) FAILED: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats19 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:47) at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats19(TestCliDriver.java:41417) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785) The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1852) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1852/ to view the results.
