Meta data didn't rollback after accessing mysql fail
Hi all, I got a problem when using Hive 0.7.1. After I created a table with JDO failure, I can still find the table by using 'show tables;'. I grep the code and find there is a transaction mechanism when hive interacts with metastore. However, it seems that the transaction doesn't work. I have tested this for several times. I manually throw an exception before ObjectStore commit the 'CREATE TABLE' transaction. But I can find that a new record in TBLS in mysql. So I guess meta data didn't rollback correctly. I also notice this could also happen on Hive 0.8 and 0.9. However, I haven't tested it. I think this is a fatal error and can make hive metastore unavailable. Is there anyone can help me solve this problem? Thanks. Kai Zhang
[jira] [Updated] (HIVE-2954) The statement fails when a column part of an ORDER BY is not specified in the SELECT.
[ https://issues.apache.org/jira/browse/HIVE-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2954: -- Attachment: HIVE-2954.D2889.1.patch navis requested code review of HIVE-2954 [jira] The statement fails when a column part of an ORDER BY is not specified in the SELECT.. Reviewers: JIRA DPAL-1110 The statement fails when a column part of an ORDER BY is not specified in the SELECT Given the following table: CREATE TABLE `DBCSTB32` (`aaa` DOUBLE,`bbb` STRING,`ccc` STRING,`ddd` DOUBLE) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' STORED AS TEXTFILE; The following statement fails: select TXT_1`aaa`, TXT_1.`bbb` from `DBCSTB32` TXT_1 order by TXT_1.`bbb` asc, TXT_1.`aaa` asc, TXT_1.`ccc` asc ERROR: java.sql.SQLException: Query returned non-zero code: 10, cause: FAILED: Error in semantic analysis: Line 1:104 Invalid column reference '`ccc`' Adding `ccc` to the selected list of columns fixes the problem. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2889 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnInfo.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java ql/src/test/queries/clientpositive/orderby_not_selected.q ql/src/test/results/clientpositive/orderby_not_selected.q.out MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6597/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. The statement fails when a column part of an ORDER BY is not specified in the SELECT. - Key: HIVE-2954 URL: https://issues.apache.org/jira/browse/HIVE-2954 Project: Hive Issue Type: Improvement Components: SQL Affects Versions: 0.8.1 Reporter: Mauro Cazzari Assignee: Navis Attachments: HIVE-2954.D2889.1.patch Given the following table: CREATE TABLE `DBCSTB32` (`aaa` DOUBLE,`bbb` STRING,`ccc` STRING,`ddd` DOUBLE) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' STORED AS TEXTFILE; The following statement fails: select TXT_1`aaa`, TXT_1.`bbb` from `DBCSTB32` TXT_1 order by TXT_1.`bbb` asc, TXT_1.`aaa` asc, TXT_1.`ccc` asc ERROR: java.sql.SQLException: Query returned non-zero code: 10, cause: FAILED: Error in semantic analysis: Line 1:104 Invalid column reference '`ccc`' Adding `ccc` to the selected list of columns fixes the problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2961) Remove need for storage descriptors for view partitions
[ https://issues.apache.org/jira/browse/HIVE-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257292#comment-13257292 ] Carl Steinbach commented on HIVE-2961: -- Based on the discussion at today's contrib meeting it sounds like we can drop this patch and instead backout HIVE-2795 and HIVE-2612. Does that sound good? Remove need for storage descriptors for view partitions --- Key: HIVE-2961 URL: https://issues.apache.org/jira/browse/HIVE-2961 Project: Hive Issue Type: Improvement Affects Versions: 0.9.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2961.D2877.1.patch Storage descriptors were introduced for view partitions as part of HIVE-2795. This was to allow view partitions to have the concept of a region as well as to fix a NPE that resulted from calling describe formatted on them. Since regions are no longer necessary for view partitions and the NPE can be fixed by not displaying storage information for view partitions (or displaying the view's storage information if this is preferred, although, since a view partition is purely metadata, this does not seem necessary), these are no longer needed. This also means the Python script added which retroactively adds storage descriptors to existing view partitions can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2961) Remove need for storage descriptors for view partitions
[ https://issues.apache.org/jira/browse/HIVE-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257294#comment-13257294 ] Carl Steinbach commented on HIVE-2961: -- But if we end up taking that route, I think we should leave the upgrade-0.8.0-to-0.9.0.xxx.sql scripts for the sake of consistency. Remove need for storage descriptors for view partitions --- Key: HIVE-2961 URL: https://issues.apache.org/jira/browse/HIVE-2961 Project: Hive Issue Type: Improvement Affects Versions: 0.9.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2961.D2877.1.patch Storage descriptors were introduced for view partitions as part of HIVE-2795. This was to allow view partitions to have the concept of a region as well as to fix a NPE that resulted from calling describe formatted on them. Since regions are no longer necessary for view partitions and the NPE can be fixed by not displaying storage information for view partitions (or displaying the view's storage information if this is preferred, although, since a view partition is purely metadata, this does not seem necessary), these are no longer needed. This also means the Python script added which retroactively adds storage descriptors to existing view partitions can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2848) Add validation to HiveConf ConfVars
[ https://issues.apache.org/jira/browse/HIVE-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2848: Status: Patch Available (was: Open) Add validation to HiveConf ConfVars --- Key: HIVE-2848 URL: https://issues.apache.org/jira/browse/HIVE-2848 Project: Hive Issue Type: Improvement Affects Versions: 0.9.0 Reporter: Kevin Wilfong Assignee: Navis Attachments: HIVE-2848.D2313.1.patch, HIVE-2848.D2313.2.patch, HIVE-2848.D2313.3.patch, HIVE-2848.D2313.4.patch, HIVE-2848.D2313.5.patch It would be good if we could add some validation to ConfVars in the HiveConf when they are set. At least if a variable is supposed to be boolean, numeric, or strict/nonstrict, it shouldn't be too difficult to check for these. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2848) Add validation to HiveConf ConfVars
[ https://issues.apache.org/jira/browse/HIVE-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2848: -- Attachment: HIVE-2848.D2313.6.patch navis updated the revision HIVE-2848 [jira] Add validation to HiveConf ConfVars. Reviewers: JIRA, cwsteinbach 1. Changed to use CamelCase (VAR_TYPE to VarType) 2. Added test explanation 3. Fixed error message I've missed some messages from phabricator. Sorry for that. REVISION DETAIL https://reviews.facebook.net/D2313 AFFECTED FILES cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/java/org/apache/hadoop/hive/ql/processors/SetProcessor.java ql/src/test/queries/clientnegative/set_hiveconf_validation0.q ql/src/test/queries/clientnegative/set_hiveconf_validation1.q ql/src/test/queries/clientpositive/udaf_context_ngrams.q ql/src/test/queries/clientpositive/udaf_ngrams.q ql/src/test/queries/clientpositive/udaf_percentile_approx.q ql/src/test/results/clientnegative/set_hiveconf_validation0.q.out ql/src/test/results/clientnegative/set_hiveconf_validation1.q.out ql/src/test/results/clientpositive/udaf_context_ngrams.q.out ql/src/test/results/clientpositive/udaf_ngrams.q.out ql/src/test/results/clientpositive/udaf_percentile_approx.q.out Add validation to HiveConf ConfVars --- Key: HIVE-2848 URL: https://issues.apache.org/jira/browse/HIVE-2848 Project: Hive Issue Type: Improvement Affects Versions: 0.9.0 Reporter: Kevin Wilfong Assignee: Navis Attachments: HIVE-2848.D2313.1.patch, HIVE-2848.D2313.2.patch, HIVE-2848.D2313.3.patch, HIVE-2848.D2313.4.patch, HIVE-2848.D2313.5.patch, HIVE-2848.D2313.6.patch It would be good if we could add some validation to ConfVars in the HiveConf when they are set. At least if a variable is supposed to be boolean, numeric, or strict/nonstrict, it shouldn't be too difficult to check for these. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2961) Remove need for storage descriptors for view partitions
[ https://issues.apache.org/jira/browse/HIVE-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257330#comment-13257330 ] Kevin Wilfong commented on HIVE-2961: - We can revert those two patches instead. This patch does fix a null pointer exception that occurs for DESCRIBE FORMATTED on a view partition. But I can submit that in a separate patch. Remove need for storage descriptors for view partitions --- Key: HIVE-2961 URL: https://issues.apache.org/jira/browse/HIVE-2961 Project: Hive Issue Type: Improvement Affects Versions: 0.9.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2961.D2877.1.patch Storage descriptors were introduced for view partitions as part of HIVE-2795. This was to allow view partitions to have the concept of a region as well as to fix a NPE that resulted from calling describe formatted on them. Since regions are no longer necessary for view partitions and the NPE can be fixed by not displaying storage information for view partitions (or displaying the view's storage information if this is preferred, although, since a view partition is purely metadata, this does not seem necessary), these are no longer needed. This also means the Python script added which retroactively adds storage descriptors to existing view partitions can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2964) Provide object inspectors of all visible columns for TOK_FUNCTIONSTAR type UDAF
Provide object inspectors of all visible columns for TOK_FUNCTIONSTAR type UDAF --- Key: HIVE-2964 URL: https://issues.apache.org/jira/browse/HIVE-2964 Project: Hive Issue Type: Improvement Reporter: Navis Assignee: Navis Priority: Minor Currently, TOK_FUNCTIONSTAR type UDAFs are provided with empty OI at init time and is sufficient for sum, the only UDAF accepts START as a parameter. But for some UDAFs, it is needed to know all OIs that can be used to access values of a row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2964) Provide object inspectors of all visible columns for TOK_FUNCTIONSTAR type UDAF
[ https://issues.apache.org/jira/browse/HIVE-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2964: -- Attachment: HIVE-2964.D2895.1.patch navis requested code review of HIVE-2964 [jira] Provide object inspectors of all visible columns for TOK_FUNCTIONSTAR type UDAF. Reviewers: JIRA DPAL-1123 Provide object inspectors of all visible columns for TOK_FUNCTIONSTAR type UDAF Currently, TOK_FUNCTIONSTAR type UDAFs are provided with empty OI at init time and is sufficient for sum, the only UDAF accepts START as a parameter. But for some UDAFs, it is needed to know all OIs that can be used to access values of a row. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2895 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6609/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. Provide object inspectors of all visible columns for TOK_FUNCTIONSTAR type UDAF --- Key: HIVE-2964 URL: https://issues.apache.org/jira/browse/HIVE-2964 Project: Hive Issue Type: Improvement Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-2964.D2895.1.patch Currently, TOK_FUNCTIONSTAR type UDAFs are provided with empty OI at init time and is sufficient for sum, the only UDAF accepts START as a parameter. But for some UDAFs, it is needed to know all OIs that can be used to access values of a row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257474#comment-13257474 ] Thomas Weise commented on HIVE-2646: The patch needs to be applied through arc: arc patch D2883 chmod u+x testutils/hadoop svn propset svn:executable yes testutils/hadoop Al least for me, arc patch will choke on ivy/common-configurations.xml - download the file from phabricator directly. I had the problem with Andrew's patch also, so I'm not sure whether this is just me or an arc issue. Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs Key: HIVE-2646 URL: https://issues.apache.org/jira/browse/HIVE-2646 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.8.0 Reporter: Andrew Bayer Assignee: Andrew Bayer Priority: Critical Fix For: 0.9.0 Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated HIVE-2646: --- Attachment: (was: HIVE-2646-fixtests.patch) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs Key: HIVE-2646 URL: https://issues.apache.org/jira/browse/HIVE-2646 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.8.0 Reporter: Andrew Bayer Assignee: Andrew Bayer Priority: Critical Fix For: 0.9.0 Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated HIVE-2646: --- Attachment: HIVE-2646-fixtests.patch Patch file produced using arc export --unified --revision 2883 then replaced ivy/common-configurations section with output from svn diff ivy/common-configurations.xml Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs Key: HIVE-2646 URL: https://issues.apache.org/jira/browse/HIVE-2646 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.8.0 Reporter: Andrew Bayer Assignee: Andrew Bayer Priority: Critical Fix For: 0.9.0 Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Apache Hive 0.9.0 Release Candidate 0
This vote stands cancelled because of various problems people have found in RC0. Thanks to all who tried RC0. I will respin RC1 shortly. Thanks, Ashutosh On Fri, Apr 13, 2012 at 11:50, Ashutosh Chauhan hashut...@apache.orgwrote: Hey all, Apache Hive 0.9.0-rc0 is out and available at http://people.apache.org/~hashutosh/hive-0.9.0-rc0/ Release notes are available at: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310843version=12317742 Please give it a try, let us know. Hive PMC members: Please test and vote. Thanks, Ashutosh
[jira] [Created] (HIVE-2965) Revert HIVE-2612
Revert HIVE-2612 Key: HIVE-2965 URL: https://issues.apache.org/jira/browse/HIVE-2965 Project: Hive Issue Type: Task Reporter: Ashutosh Chauhan Priority: Blocker Fix For: 0.9.0 In 4/19 contrib meeting it was decided to revert HIVE-2612. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2966) Revert HIVE-2795
Revert HIVE-2795 Key: HIVE-2966 URL: https://issues.apache.org/jira/browse/HIVE-2966 Project: Hive Issue Type: Task Components: Metastore Affects Versions: 0.9.0 Reporter: Ashutosh Chauhan Priority: Blocker Fix For: 0.9.0 In 4/18/12 contrib meeting, it was decided to revert HIVE-2795 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2961) Remove need for storage descriptors for view partitions
[ https://issues.apache.org/jira/browse/HIVE-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2961: --- Status: Open (was: Patch Available) Canceling patch since current patch may not be valid after HIVE-2795 is reverted. Remove need for storage descriptors for view partitions --- Key: HIVE-2961 URL: https://issues.apache.org/jira/browse/HIVE-2961 Project: Hive Issue Type: Improvement Affects Versions: 0.9.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2961.D2877.1.patch Storage descriptors were introduced for view partitions as part of HIVE-2795. This was to allow view partitions to have the concept of a region as well as to fix a NPE that resulted from calling describe formatted on them. Since regions are no longer necessary for view partitions and the NPE can be fixed by not displaying storage information for view partitions (or displaying the view's storage information if this is preferred, although, since a view partition is purely metadata, this does not seem necessary), these are no longer needed. This also means the Python script added which retroactively adds storage descriptors to existing view partitions can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
hive 0.9.0 RC1
RC0 failed to pass because of variety of reasons. In the meantime, various folks have requested for inclusion of other fixes in 0.9. Following is the list. Following list is for committers to review these patches and to get them committed in 0.9 branch. These are in Patch Available status. HIVE-2958 HIVE-2777 HIVE-2646 HIVE-2883 HIVE-2585 HIVE-538 HIVE-2904 Following list is for contributors/committers to contribute patches. These are in Open status. HIVE-2961 HIVE-2965 HIVE-2966 Thanks, Ashutosh
[jira] [Assigned] (HIVE-2804) Task log retrieval fails on secure cluster
[ https://issues.apache.org/jira/browse/HIVE-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo reassigned HIVE-2804: -- Assignee: Zhenxiao Luo Task log retrieval fails on secure cluster -- Key: HIVE-2804 URL: https://issues.apache.org/jira/browse/HIVE-2804 Project: Hive Issue Type: Bug Components: Diagnosability, Query Processor, Security Reporter: Carl Steinbach Assignee: Zhenxiao Luo -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2804) Task log retrieval fails on secure cluster
[ https://issues.apache.org/jira/browse/HIVE-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257616#comment-13257616 ] Zhenxiao Luo commented on HIVE-2804: Also relates to the following: http://search-hadoop.com/m/LR0vP1Y5227subj=Hive+Hadoop+Log+Retrieval+Problem Task log retrieval fails on secure cluster -- Key: HIVE-2804 URL: https://issues.apache.org/jira/browse/HIVE-2804 Project: Hive Issue Type: Bug Components: Diagnosability, Query Processor, Security Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-0.8.1-SNAPSHOT-h0.21 - Build # 258 - Fixed
Changes for Build #257 Changes for Build #258 All tests passed The Apache Jenkins build system has built Hive-0.8.1-SNAPSHOT-h0.21 (build #258) Status: Fixed Check console output at https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/258/ to view the results.
[jira] [Commented] (HIVE-2961) Remove need for storage descriptors for view partitions
[ https://issues.apache.org/jira/browse/HIVE-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257772#comment-13257772 ] Carl Steinbach commented on HIVE-2961: -- bq. Do you mean delete the content of the file, thus effectively leaving empty files, since there is no upgrade needed when we revert those patches? Yup, that's basically what I meant, but now that I think more about it I agree that it doesn't make much sense. Remove need for storage descriptors for view partitions --- Key: HIVE-2961 URL: https://issues.apache.org/jira/browse/HIVE-2961 Project: Hive Issue Type: Improvement Affects Versions: 0.9.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2961.D2877.1.patch Storage descriptors were introduced for view partitions as part of HIVE-2795. This was to allow view partitions to have the concept of a region as well as to fix a NPE that resulted from calling describe formatted on them. Since regions are no longer necessary for view partitions and the NPE can be fixed by not displaying storage information for view partitions (or displaying the view's storage information if this is preferred, although, since a view partition is purely metadata, this does not seem necessary), these are no longer needed. This also means the Python script added which retroactively adds storage descriptors to existing view partitions can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257833#comment-13257833 ] Thomas Weise commented on HIVE-2646: All test pass in latest run: Tests FailuresErrors Success rateTime 16780 0 100.00% 19245.326 Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs Key: HIVE-2646 URL: https://issues.apache.org/jira/browse/HIVE-2646 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.8.0 Reporter: Andrew Bayer Assignee: Andrew Bayer Priority: Critical Fix For: 0.9.0 Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2967) Statement.updateQuery() returns Method Not Supported exception
Statement.updateQuery() returns Method Not Supported exception -- Key: HIVE-2967 URL: https://issues.apache.org/jira/browse/HIVE-2967 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.8.0 Environment: Remotely connecting from a Windows machine via JDBC to Hive installed on Linux Kernel-2.6.18-glibc-2.5 x86_64 64bit Reporter: Alex Piyevsky When our software sends a CREATE TABLE command to HIVE via JDBC, a Statement.updateQuery() statement is issued which results in a Method Not Supported exception. We did not get this error in the previous version of Hive (0.7.1). Please advise, did something change within the 0.8 JDBC driver, is a statement like this no longer supported? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Problems with Arc/Phabricator
Hit a new problem with arc today:
Fatal error: Uncaught exception 'Exception' with message 'Host returned
HTTP/200, but invalid JSON data in response to a Conduit method call:
br /
bWarning/b: Unknown: POST Content-Length of 9079953 bytes exceeds the
limit of 8388608 bytes in bUnknown/b on line b0/bbr /
for(;;);{result:null,error_code:ERR-INVALID-SESSION,error_info:Session
key is not present.}' in
/Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php:48
Stack trace:
#0
/Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(62):
ConduitFuture-didReceiveResult(Array)
#1
/Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(39):
FutureProxy-getResult()
#2
/Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitClient.php(52):
FutureProxy-resolve()
#3
/Users/ashutosh/work/hive/arcanist/src/workflow/diff/ArcanistDiffWorkflow.php(341):
ConduitClient-callMethodSynchronous('differential.cr...', Array)
#4 /Users/ashutosh/work/hive/arcanist/scripts/arcanist.php(266):
ArcanistDiffWo in
/Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php on
line 48
Any ideas how to solve this?
Thanks,
Ashutosh
On Wed, Apr 11, 2012 at 18:37, Edward Capriolo edlinuxg...@gmail.comwrote:
I think the most practical solution is try and use arc/phab and then
if there is a problem fall back to Jira and do it the old way.
Edward
On Wed, Apr 11, 2012 at 7:17 PM, Carl Steinbach c...@cloudera.com wrote:
+1 to switching over to Git.
As for the rest of the Phabricator/Gerrit/Reviewboard discussion, I think
we should pick this up again at the contributor meeting on Wednesday.
Thanks.
Carl
On Wed, Apr 11, 2012 at 12:19 PM, Ashutosh Chauhan hashut...@apache.org
wrote:
+1 on moving away from arc/phabricator. It works great when it works,
but
most of the time it doesnt work.
Ashutosh
On Wed, Apr 11, 2012 at 11:57, Owen O'Malley omal...@apache.org
wrote:
On Wed, Apr 11, 2012 at 11:48 AM, Edward Capriolo
edlinuxg...@gmail.com
wrote:
If we are going to switch from fabricator we just might as well go
back to not using anything. Review board was really clunky and
confusing.
I'm mostly +1 to that. If no one is supporting phabricator, then it
won't work for long. Personally, I'd love it if we could move Hive to
git completely. Has anyone used gerrit? The videos of it make it look
better than sliced bread.
-- Owen
[jira] [Updated] (HIVE-2965) Revert HIVE-2612
[ https://issues.apache.org/jira/browse/HIVE-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2965: --- Attachment: hive-2765.patch Arc is unwieldy again. Can't convince it to submit the patch. So, uploading directly, patch reverting HIVE-2612 generated using svn merge command. Revert HIVE-2612 Key: HIVE-2965 URL: https://issues.apache.org/jira/browse/HIVE-2965 Project: Hive Issue Type: Task Reporter: Ashutosh Chauhan Priority: Blocker Fix For: 0.9.0 Attachments: hive-2765.patch In 4/19 contrib meeting it was decided to revert HIVE-2612. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2965) Revert HIVE-2612
[ https://issues.apache.org/jira/browse/HIVE-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2965: --- Status: Patch Available (was: Open) Ready for review. Revert HIVE-2612 Key: HIVE-2965 URL: https://issues.apache.org/jira/browse/HIVE-2965 Project: Hive Issue Type: Task Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Priority: Blocker Fix For: 0.9.0 Attachments: hive-2765.patch In 4/19 contrib meeting it was decided to revert HIVE-2612. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2965) Revert HIVE-2612
[ https://issues.apache.org/jira/browse/HIVE-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reassigned HIVE-2965: -- Assignee: Ashutosh Chauhan Revert HIVE-2612 Key: HIVE-2965 URL: https://issues.apache.org/jira/browse/HIVE-2965 Project: Hive Issue Type: Task Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Priority: Blocker Fix For: 0.9.0 Attachments: hive-2765.patch In 4/19 contrib meeting it was decided to revert HIVE-2612. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Problems with Arc/Phabricator
I think as an Apache project, using code review tools outside apache should
never be a requirement. I agree that arc/phab are great tools, and they
help the patch/review process a lot. But I think what is essential for long
term sustainability as a project is to lower the entry barriers for
contributing. Not every developer who wants to contribute some code will be
willing to work with phab/rb.
What I would suggest is to always have the option to submit patches via
jira, and make rb / phab optional.
Just my 2 cents,
Enis
On Thu, Apr 19, 2012 at 4:35 PM, Ashutosh Chauhan hashut...@apache.orgwrote:
Hit a new problem with arc today:
Fatal error: Uncaught exception 'Exception' with message 'Host returned
HTTP/200, but invalid JSON data in response to a Conduit method call:
br /
bWarning/b: Unknown: POST Content-Length of 9079953 bytes exceeds the
limit of 8388608 bytes in bUnknown/b on line b0/bbr /
for(;;);{result:null,error_code:ERR-INVALID-SESSION,error_info:Session
key is not present.}' in
/Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php:48
Stack trace:
#0
/Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(62):
ConduitFuture-didReceiveResult(Array)
#1
/Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(39):
FutureProxy-getResult()
#2
/Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitClient.php(52):
FutureProxy-resolve()
#3
/Users/ashutosh/work/hive/arcanist/src/workflow/diff/ArcanistDiffWorkflow.php(341):
ConduitClient-callMethodSynchronous('differential.cr...', Array)
#4 /Users/ashutosh/work/hive/arcanist/scripts/arcanist.php(266):
ArcanistDiffWo in
/Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php on
line 48
Any ideas how to solve this?
Thanks,
Ashutosh
On Wed, Apr 11, 2012 at 18:37, Edward Capriolo edlinuxg...@gmail.com
wrote:
I think the most practical solution is try and use arc/phab and then
if there is a problem fall back to Jira and do it the old way.
Edward
On Wed, Apr 11, 2012 at 7:17 PM, Carl Steinbach c...@cloudera.com
wrote:
+1 to switching over to Git.
As for the rest of the Phabricator/Gerrit/Reviewboard discussion, I
think
we should pick this up again at the contributor meeting on Wednesday.
Thanks.
Carl
On Wed, Apr 11, 2012 at 12:19 PM, Ashutosh Chauhan
hashut...@apache.org
wrote:
+1 on moving away from arc/phabricator. It works great when it works,
but
most of the time it doesnt work.
Ashutosh
On Wed, Apr 11, 2012 at 11:57, Owen O'Malley omal...@apache.org
wrote:
On Wed, Apr 11, 2012 at 11:48 AM, Edward Capriolo
edlinuxg...@gmail.com
wrote:
If we are going to switch from fabricator we just might as well go
back to not using anything. Review board was really clunky and
confusing.
I'm mostly +1 to that. If no one is supporting phabricator, then it
won't work for long. Personally, I'd love it if we could move Hive
to
git completely. Has anyone used gerrit? The videos of it make it
look
better than sliced bread.
-- Owen
[jira] [Updated] (HIVE-2966) Revert HIVE-2795
[ https://issues.apache.org/jira/browse/HIVE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-2966: Attachment: HIVE-2966.1.patch HIVE-2966.1.patch - Attaching the patch directly, as arc command didn't do it for me. Here is the phabricator link - https://reviews.facebook.net/D2907 Revert HIVE-2795 Key: HIVE-2966 URL: https://issues.apache.org/jira/browse/HIVE-2966 Project: Hive Issue Type: Task Components: Metastore Affects Versions: 0.9.0 Reporter: Ashutosh Chauhan Priority: Blocker Fix For: 0.9.0 Attachments: HIVE-2966.1.patch In 4/18/12 contrib meeting, it was decided to revert HIVE-2795 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Problems with Arc/Phabricator
Just throwing this out there. The phabricator IRC has more people and
is usually more active then Hive IRC.
#JustSaying...
On Thu, Apr 19, 2012 at 7:35 PM, Ashutosh Chauhan hashut...@apache.org wrote:
Hit a new problem with arc today:
Fatal error: Uncaught exception 'Exception' with message 'Host returned
HTTP/200, but invalid JSON data in response to a Conduit method call:
br /
bWarning/b: Unknown: POST Content-Length of 9079953 bytes exceeds the
limit of 8388608 bytes in bUnknown/b on line b0/bbr /
for(;;);{result:null,error_code:ERR-INVALID-SESSION,error_info:Session
key is not present.}' in
/Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php:48
Stack trace:
#0
/Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(62):
ConduitFuture-didReceiveResult(Array)
#1
/Users/ashutosh/work/hive/libphutil/src/future/proxy/FutureProxy.php(39):
FutureProxy-getResult()
#2
/Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitClient.php(52):
FutureProxy-resolve()
#3
/Users/ashutosh/work/hive/arcanist/src/workflow/diff/ArcanistDiffWorkflow.php(341):
ConduitClient-callMethodSynchronous('differential.cr...', Array)
#4 /Users/ashutosh/work/hive/arcanist/scripts/arcanist.php(266):
ArcanistDiffWo in
/Users/ashutosh/work/hive/libphutil/src/conduit/client/ConduitFuture.php on
line 48
Any ideas how to solve this?
Thanks,
Ashutosh
On Wed, Apr 11, 2012 at 18:37, Edward Capriolo edlinuxg...@gmail.comwrote:
I think the most practical solution is try and use arc/phab and then
if there is a problem fall back to Jira and do it the old way.
Edward
On Wed, Apr 11, 2012 at 7:17 PM, Carl Steinbach c...@cloudera.com wrote:
+1 to switching over to Git.
As for the rest of the Phabricator/Gerrit/Reviewboard discussion, I think
we should pick this up again at the contributor meeting on Wednesday.
Thanks.
Carl
On Wed, Apr 11, 2012 at 12:19 PM, Ashutosh Chauhan hashut...@apache.org
wrote:
+1 on moving away from arc/phabricator. It works great when it works,
but
most of the time it doesnt work.
Ashutosh
On Wed, Apr 11, 2012 at 11:57, Owen O'Malley omal...@apache.org
wrote:
On Wed, Apr 11, 2012 at 11:48 AM, Edward Capriolo
edlinuxg...@gmail.com
wrote:
If we are going to switch from fabricator we just might as well go
back to not using anything. Review board was really clunky and
confusing.
I'm mostly +1 to that. If no one is supporting phabricator, then it
won't work for long. Personally, I'd love it if we could move Hive to
git completely. Has anyone used gerrit? The videos of it make it look
better than sliced bread.
-- Owen
[jira] [Commented] (HIVE-2958) GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]
[
https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257950#comment-13257950
]
Phabricator commented on HIVE-2958:
---
ashutoshc has accepted the revision HIVE-2958 [jira] GROUP BY causing
ClassCastException [LazyDioInteger cannot be cast LazyInteger].
+1
REVISION DETAIL
https://reviews.facebook.net/D2871
BRANCH
DPAL-
GROUP BY causing ClassCastException [LazyDioInteger cannot be cast
LazyInteger]
---
Key: HIVE-2958
URL: https://issues.apache.org/jira/browse/HIVE-2958
Project: Hive
Issue Type: Bug
Components: HBase Handler
Affects Versions: 0.9.0
Environment: HBase 0.90.4, Hive 0.90 snapshot (trunk) built today
Reporter: Tim Robertson
Assignee: Navis
Priority: Blocker
Attachments: HIVE-2958.D2871.1.patch
This relates to https://issues.apache.org/jira/browse/HIVE-1634.
The following work fine:
{code}
CREATE EXTERNAL TABLE tim_hbase_occurrence (
id int,
scientific_name string,
data_resource_id int
) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH
SERDEPROPERTIES (
hbase.columns.mapping = :key#b,v:scientific_name#s,v:data_resource_id#b
) TBLPROPERTIES(
hbase.table.name = mini_occurrences,
hbase.table.default.storage.type = binary
);
SELECT * FROM tim_hbase_occurrence LIMIT 3;
SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3;
{code}
However, the following fails:
{code}
SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY
data_resource_id;
{code}
The error given:
{code}
0 TS
2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Initialization Done 7 MAP
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Processing alias tim_hbase_occurrence for file
hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7
forwarding 1 rows
2012-04-17 16:58:45,714 INFO
org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1
forwarding 1 rows
2012-04-17 16:58:45,723 FATAL ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {id:1444,scientific_name:null,data_resource_id:1081}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
... 9 more
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
at
org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150)
at
[jira] [Updated] (HIVE-2958) GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]
[
https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-2958:
---
Resolution: Fixed
Fix Version/s: 0.9.0
Status: Resolved (was: Patch Available)
Committed to trunk and 0.9. Thanks, Navis!
GROUP BY causing ClassCastException [LazyDioInteger cannot be cast
LazyInteger]
---
Key: HIVE-2958
URL: https://issues.apache.org/jira/browse/HIVE-2958
Project: Hive
Issue Type: Bug
Components: HBase Handler
Affects Versions: 0.9.0
Environment: HBase 0.90.4, Hive 0.90 snapshot (trunk) built today
Reporter: Tim Robertson
Assignee: Navis
Priority: Blocker
Fix For: 0.9.0
Attachments: HIVE-2958.D2871.1.patch
This relates to https://issues.apache.org/jira/browse/HIVE-1634.
The following work fine:
{code}
CREATE EXTERNAL TABLE tim_hbase_occurrence (
id int,
scientific_name string,
data_resource_id int
) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH
SERDEPROPERTIES (
hbase.columns.mapping = :key#b,v:scientific_name#s,v:data_resource_id#b
) TBLPROPERTIES(
hbase.table.name = mini_occurrences,
hbase.table.default.storage.type = binary
);
SELECT * FROM tim_hbase_occurrence LIMIT 3;
SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3;
{code}
However, the following fails:
{code}
SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY
data_resource_id;
{code}
The error given:
{code}
0 TS
2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Initialization Done 7 MAP
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Processing alias tim_hbase_occurrence for file
hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7
forwarding 1 rows
2012-04-17 16:58:45,714 INFO
org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1
forwarding 1 rows
2012-04-17 16:58:45,723 FATAL ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {id:1444,scientific_name:null,data_resource_id:1081}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
... 9 more
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
at
org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150)
at
Re: hive 0.9.0 RC1
Release is not going to be blocked on it. If the patch gets committed on trunk by the time I roll RC1 I will merge it in. But, first step is to have it in trunk. Thanks, Ashutosh On Thu, Apr 19, 2012 at 17:32, Edward Capriolo edlinuxg...@gmail.comwrote: Am I missing something about? https://issues.apache.org/jira/browse/HIVE-2777 I see no way to access this feature from the hive QL language. Should we delay releases on features not usable? Edward On Thu, Apr 19, 2012 at 1:28 PM, Ashutosh Chauhan hashut...@apache.org wrote: RC0 failed to pass because of variety of reasons. In the meantime, various folks have requested for inclusion of other fixes in 0.9. Following is the list. Following list is for committers to review these patches and to get them committed in 0.9 branch. These are in Patch Available status. HIVE-2958 HIVE-2777 HIVE-2646 HIVE-2883 HIVE-2585 HIVE-538 HIVE-2904 Following list is for contributors/committers to contribute patches. These are in Open status. HIVE-2961 HIVE-2965 HIVE-2966 Thanks, Ashutosh
[jira] [Created] (HIVE-2968) Add char(n), character(n), character varying(n), and varchar(n) as datatype aliases for string.
Add char(n), character(n), character varying(n), and varchar(n) as datatype aliases for string. --- Key: HIVE-2968 URL: https://issues.apache.org/jira/browse/HIVE-2968 Project: Hive Issue Type: Improvement Reporter: Joey Echeverria MySQL and Postgres support char(n), varchar(n), character(n), and character varying(n) as data types. In Hive, those would map to strings. It would be useful to support these types as aliases when creating a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2968) Add char(n), character(n), character varying(n), and varchar(n) as datatype aliases for string.
[ https://issues.apache.org/jira/browse/HIVE-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joey Echeverria updated HIVE-2968: -- Description: MySQL and Postgres support char\(n), varchar\(n), character\(n), and character varying\(n) as data types. In Hive, those would map to strings. It would be useful to support these types as aliases when creating a table. (was: MySQL and Postgres support char(n), varchar(n), character(n), and character varying(n) as data types. In Hive, those would map to strings. It would be useful to support these types as aliases when creating a table.) Add char(n), character(n), character varying(n), and varchar(n) as datatype aliases for string. --- Key: HIVE-2968 URL: https://issues.apache.org/jira/browse/HIVE-2968 Project: Hive Issue Type: Improvement Reporter: Joey Echeverria MySQL and Postgres support char\(n), varchar\(n), character\(n), and character varying\(n) as data types. In Hive, those would map to strings. It would be useful to support these types as aliases when creating a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2804) Task log retrieval fails on secure cluster
[ https://issues.apache.org/jira/browse/HIVE-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257964#comment-13257964 ] Edward Capriolo commented on HIVE-2804: --- It seems like you will have to make a shim or variable that can be used to generate a version dependent URL. Task log retrieval fails on secure cluster -- Key: HIVE-2804 URL: https://issues.apache.org/jira/browse/HIVE-2804 Project: Hive Issue Type: Bug Components: Diagnosability, Query Processor, Security Reporter: Carl Steinbach Assignee: Zhenxiao Luo -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Meta data didn't rollback after accessing mysql fail
Hi Carl To reproduce it, I added one line throw new MetaException(test); at line 628 in ObjectStore.java and comment the following line //commited = commitTransaction(); I also added some codes for tracing the behaviors of open/commit/rollback transaction and executeWithRetry. Here are the results I saw in my console: hive create table test(str string); 12/04/20 11:01:18 ERROR metastore.HiveMetaStore: execute with retry: 0 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: opening 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter commit transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter commit transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter commit transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter rollback transaction 12/04/20 11:01:18 ERROR metastore.ObjectStore: rolling back 12/04/20 11:01:18 ERROR metastore.ObjectStore: enter rollback transaction FAILED: Error in metadata: MetaException(message:test) 12/04/20 11:01:18 ERROR exec.DDLTask: FAILED: Error in metadata: MetaException(message:test) org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:test) And then I use describe test;, I got this: hive describe test; 12/04/20 11:11:31 ERROR metastore.HiveMetaStore: execute with retry: 0 12/04/20 11:11:31 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:11:31 ERROR metastore.ObjectStore: opening 12/04/20 11:11:31 ERROR metastore.ObjectStore: enter open transaction 12/04/20 11:11:31 ERROR metastore.ObjectStore: enter commit transaction 12/04/20 11:11:31 ERROR metastore.ObjectStore: enter commit transaction 12/04/20 11:11:31 ERROR metastore.ObjectStore: commiting OK str string Time taken: 0.132 seconds It seems the table has been created successfully although the transaction has never been committed and has been rolled back. I think this could relate to JDO and MySQL. Because I cannot reproduce it when using Derby. My MySQL version is 5.1.18 Thanks. Kai Zhang 2012/4/20 Carl Steinbach c...@cloudera.com Hi Kai, I agree that this sounds like a bug. Can you describe the steps required to reproduce this behavior on 0.7.1? Thanks. Carl On Wed, Apr 18, 2012 at 11:03 PM, Zhang Kai zhangk1...@gmail.com wrote: Hi all, I got a problem when using Hive 0.7.1. After I created a table with JDO failure, I can still find the table by using 'show tables;'. I grep the code and find there is a transaction mechanism when hive interacts with metastore. However, it seems that the transaction doesn't work. I have tested this for several times. I manually throw an exception before ObjectStore commit the 'CREATE TABLE' transaction. But I can find that a new record in TBLS in mysql. So I guess meta data didn't rollback correctly. I also notice this could also happen on Hive 0.8 and 0.9. However, I haven't tested it. I think this is a fatal error and can make hive metastore unavailable. Is there anyone can help me solve this problem? Thanks. Kai Zhang
[jira] [Updated] (HIVE-2933) analyze command throw NPE when table doesn't exists
[ https://issues.apache.org/jira/browse/HIVE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ransom.hezhiqiang updated HIVE-2933: Attachment: (was: HIVE-2933-0.8.1-2.patch) analyze command throw NPE when table doesn't exists --- Key: HIVE-2933 URL: https://issues.apache.org/jira/browse/HIVE-2933 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.8.1 Reporter: alex gemini Priority: Minor analyze command throw NPE when table doesn't exists -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2933) analyze command throw NPE when table doesn't exists
[ https://issues.apache.org/jira/browse/HIVE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ransom.hezhiqiang updated HIVE-2933: Attachment: (was: HIVE-2933-0.8.1-1.patch) analyze command throw NPE when table doesn't exists --- Key: HIVE-2933 URL: https://issues.apache.org/jira/browse/HIVE-2933 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.8.1 Reporter: alex gemini Priority: Minor analyze command throw NPE when table doesn't exists -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
