[jira] [Updated] (HIVE-3891) physical optimizer changes for auto sort-merge join
[ https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3891: - Attachment: hive.3891.13.patch physical optimizer changes for auto sort-merge join --- Key: HIVE-3891 URL: https://issues.apache.org/jira/browse/HIVE-3891 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, hive.3891.10.patch, hive.3891.11.patch, hive.3891.12.patch, hive.3891.13.patch, hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch, HIVE-3891_8.patch, hive.3891.9.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HIVE-4106) SMB joins fail in multi-way joins
[ https://issues.apache.org/jira/browse/HIVE-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13633835#comment-13633835 ] Namit Jain edited comment on HIVE-4106 at 4/17/13 6:54 AM: --- [~ashutoshc], can you take a look ? was (Author: namit): [~ashutoshc], can you taje a look ? SMB joins fail in multi-way joins - Key: HIVE-4106 URL: https://issues.apache.org/jira/browse/HIVE-4106 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0 Reporter: Vikram Dixit K Assignee: Namit Jain Priority: Blocker Attachments: auto_sortmerge_join_12.q, hive.4106.1.patch, hive.4106.2.patch, HIVE-4106.patch I see array out of bounds exception in case of multi way smb joins. This is related to changes that went in as part of HIVE-3403. This issue has been discussed in HIVE-3891. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4106) SMB joins fail in multi-way joins
[ https://issues.apache.org/jira/browse/HIVE-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4106: - Assignee: Namit Jain (was: Vikram Dixit K) Status: Patch Available (was: Open) SMB joins fail in multi-way joins - Key: HIVE-4106 URL: https://issues.apache.org/jira/browse/HIVE-4106 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0 Reporter: Vikram Dixit K Assignee: Namit Jain Priority: Blocker Attachments: auto_sortmerge_join_12.q, hive.4106.1.patch, hive.4106.2.patch, HIVE-4106.patch I see array out of bounds exception in case of multi way smb joins. This is related to changes that went in as part of HIVE-3403. This issue has been discussed in HIVE-3891. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4106) SMB joins fail in multi-way joins
[ https://issues.apache.org/jira/browse/HIVE-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13633835#comment-13633835 ] Namit Jain commented on HIVE-4106: -- [~ashutoshc], can you taje a look ? SMB joins fail in multi-way joins - Key: HIVE-4106 URL: https://issues.apache.org/jira/browse/HIVE-4106 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0 Reporter: Vikram Dixit K Assignee: Namit Jain Priority: Blocker Attachments: auto_sortmerge_join_12.q, hive.4106.1.patch, hive.4106.2.patch, HIVE-4106.patch I see array out of bounds exception in case of multi way smb joins. This is related to changes that went in as part of HIVE-3403. This issue has been discussed in HIVE-3891. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2137) JDBC driver doesn't encode string properly.
[ https://issues.apache.org/jira/browse/HIVE-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13633856#comment-13633856 ] Kristof Vanbecelaere commented on HIVE-2137: Can some committer comment on this issue please? Will this issue be resolved and/or will a patch be applied? Which version could contain a fix for this? JDBC driver doesn't encode string properly. --- Key: HIVE-2137 URL: https://issues.apache.org/jira/browse/HIVE-2137 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.9.0 Reporter: Jin Adachi Attachments: HIVE-2137.patch JDBC driver decode string by client encoding. It ignore server encoding. For example, server = Linux (utf-8) client = Windows (shift-jis : it's japanese charset) It makes character corruption in the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
adding a new property for hive history file HIVE-1708
Hi Guys, I am trying to work on this JIRA HIVE-1708https://issues.apache.org/jira/browse/HIVE-1708 . I have added one property HIVE_CLI_ENABLE_LOGGING to enable or disable the history and tested it. I am stuck at a point what should be the default value for HIVE_CLI_HISTORY_FILE_PATH? Currently this is set to String historyDirectory = System.getProperty(user.home); String historyFile = historyDirectory + File.separator + HISTORYFILE; Any ideas on what will be the default path then ? -- Nitin Pawar
[jira] [Created] (HIVE-4372) When trying to populate an external HBase table using a hive query involving joins, the data is incorrectly getting mixed up inside the rows.
Yashaswy Andavilli created HIVE-4372: Summary: When trying to populate an external HBase table using a hive query involving joins, the data is incorrectly getting mixed up inside the rows. Key: HIVE-4372 URL: https://issues.apache.org/jira/browse/HIVE-4372 Project: Hive Issue Type: Bug Components: HBase Handler Affects Versions: 0.10.0 Reporter: Yashaswy Andavilli -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4236) JDBC2 HivePreparedStatement does not release resources
[ https://issues.apache.org/jira/browse/HIVE-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634059#comment-13634059 ] Chris Drome commented on HIVE-4236: --- [~navis]: I will start addressing your comments, etc when I'm back in the office next week. JDBC2 HivePreparedStatement does not release resources -- Key: HIVE-4236 URL: https://issues.apache.org/jira/browse/HIVE-4236 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4236.patch HivePreparedStatement does not close the associated server-side operation when close() is called. Nor does it call close() on the ResultSet. When execute() is called the current ResultSet is not closed first it is just set to null. Similarly, HiveStatement's close() does not call close() on the ResultSet, it just sets it to null. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4280) TestRetryingHMSHandler is failing on trunk.
[ https://issues.apache.org/jira/browse/HIVE-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-4280: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk and 0.11 Thanks, Teddy! TestRetryingHMSHandler is failing on trunk. --- Key: HIVE-4280 URL: https://issues.apache.org/jira/browse/HIVE-4280 Project: Hive Issue Type: Bug Affects Versions: 0.11.0 Reporter: Ashutosh Chauhan Assignee: Teddy Choi Fix For: 0.11.0 Attachments: HIVE-4280-1.patch.txt, HIVE-4280-2.patch.txt Newly added testcase TestRetryingHMSHandler fails on trunk. https://builds.apache.org/job/Hive-trunk-h0.21/2040/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4019) Ability to create and drop temporary partition function
[ https://issues.apache.org/jira/browse/HIVE-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634087#comment-13634087 ] Brock Noland commented on HIVE-4019: Hi, Thanks! Too be honest, I haven't looked at noop much. It was just used in the original patch. I'look at your updated patch and see what more work needs to be done. Ability to create and drop temporary partition function --- Key: HIVE-4019 URL: https://issues.apache.org/jira/browse/HIVE-4019 Project: Hive Issue Type: New Feature Components: PTF-Windowing Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-4019-1.patch, HIVE-4019.2.patch, hive-4019.q Just like udf/udaf/udtf functions, user should be able to add and drop custom partitioning functions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4278) HCat needs to get current Hive jars instead of pulling them from maven repo
[ https://issues.apache.org/jira/browse/HIVE-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-4278: --- Status: Open (was: Patch Available) After doing ant clean package, I got following error: BUILD FAILED /Users/ashutosh/workspace/hive/build.xml:517: /Users/ashutosh/workspace/hive/hcatalog/build/hcatalog-0.11.0-SNAPSHOT does not exist. HCat needs to get current Hive jars instead of pulling them from maven repo --- Key: HIVE-4278 URL: https://issues.apache.org/jira/browse/HIVE-4278 Project: Hive Issue Type: Sub-task Components: Build Infrastructure, HCatalog Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Sushanth Sowmyan Priority: Blocker Fix For: 0.11.0 Attachments: HIVE-4278.approach2.patch, HIVE-4278.approach2.patch.2.for.branch.11, HIVE-4278.approach2.patch.2.for.branch.12, HIVE-4278.D10257.1.patch, HIVE-4278.D9981.1.patch The HCatalog build is currently pulling Hive jars from the maven repo instead of using the ones built as part of the current build. Now that it is part of Hive it should use the jars being built instead of pulling them from maven. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4278) HCat needs to get current Hive jars instead of pulling them from maven repo
[ https://issues.apache.org/jira/browse/HIVE-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634090#comment-13634090 ] Ashutosh Chauhan commented on HIVE-4278: Above I found for trunk (.12) patch. 0.11 patch worked fine. HCat needs to get current Hive jars instead of pulling them from maven repo --- Key: HIVE-4278 URL: https://issues.apache.org/jira/browse/HIVE-4278 Project: Hive Issue Type: Sub-task Components: Build Infrastructure, HCatalog Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Sushanth Sowmyan Priority: Blocker Fix For: 0.11.0 Attachments: HIVE-4278.approach2.patch, HIVE-4278.approach2.patch.2.for.branch.11, HIVE-4278.approach2.patch.2.for.branch.12, HIVE-4278.D10257.1.patch, HIVE-4278.D9981.1.patch The HCatalog build is currently pulling Hive jars from the maven repo instead of using the ones built as part of the current build. Now that it is part of Hive it should use the jars being built instead of pulling them from maven. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4347) Hcatalog build fail on Windows because javadoc command exceed length limit
[ https://issues.apache.org/jira/browse/HIVE-4347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634098#comment-13634098 ] Hudson commented on HIVE-4347: -- Integrated in Hive-trunk-h0.21 #2068 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2068/]) HIVE-4347 Hcatalog build fail on Windows because javadoc command exceed length limit (Shuaishuai Nie via gates) (Revision 1468562) Result = FAILURE gates : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1468562 Files : * /hive/trunk/hcatalog/build.xml Hcatalog build fail on Windows because javadoc command exceed length limit -- Key: HIVE-4347 URL: https://issues.apache.org/jira/browse/HIVE-4347 Project: Hive Issue Type: Bug Components: Build Infrastructure, HCatalog, Windows Affects Versions: 0.11.0 Environment: Windows 8 Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Labels: build, patch Fix For: 0.12.0 Attachments: HIVE-4347.patch Original Estimate: 24h Remaining Estimate: 24h When building Hcatalog on Window 8, build fail because HIVE_DIR\hcatalog\build.xml:213: Javadoc failed: java.io.IOException: Cannot run program JAVA_HOME\bin\javadoc.exe: CreateProces s error=206, The filename or extension is too long -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4320) Consider extending max limit for precision to 38
[ https://issues.apache.org/jira/browse/HIVE-4320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634099#comment-13634099 ] Hudson commented on HIVE-4320: -- Integrated in Hive-trunk-h0.21 #2068 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2068/]) HIVE-4320 : Consider extending max limit for precision to 38 (Gunther Hagleitner via Ashutosh Chauhan) (Revision 1468588) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1468588 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/common/type/HiveDecimal.java * /hive/trunk/data/files/kv8.txt * /hive/trunk/ql/src/test/results/clientpositive/decimal_precision.q.out * /hive/trunk/ql/src/test/results/clientpositive/decimal_udf.q.out Consider extending max limit for precision to 38 Key: HIVE-4320 URL: https://issues.apache.org/jira/browse/HIVE-4320 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Fix For: 0.11.0 Attachments: HIVE-4320.1.patch, HIVE-4320.2.patch Max precision of 38 still fits in 128. It changes the way you do math on these numbers though. Need to see if there will be perf implications, but there's a strong case to support 38 (instead of 36) to comply with other DBs. (Oracle, SQL Server, Teradata). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4191) describe table output always prints as if formatted keyword is specified
[ https://issues.apache.org/jira/browse/HIVE-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634100#comment-13634100 ] Hudson commented on HIVE-4191: -- Integrated in Hive-trunk-h0.21 #2068 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2068/]) HIVE-4191 : describe table output always prints as if formatted keyword is specified (Thejas Nair via Ashutosh Chauhan) (Revision 1468551) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1468551 Files : * /hive/trunk/contrib/src/test/results/clientpositive/fileformat_base64.q.out * /hive/trunk/contrib/src/test/results/clientpositive/serde_s3.q.out * /hive/trunk/hbase-handler/src/test/results/positive/hbase_queries.q.out * /hive/trunk/hwi/src/test/org/apache/hadoop/hive/hwi/TestHWISessionManager.java * /hive/trunk/jdbc/src/test/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java * /hive/trunk/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java * /hive/trunk/ql/src/test/results/clientnegative/alter_partition_coltype_2columns.q.out * /hive/trunk/ql/src/test/results/clientnegative/alter_partition_coltype_invalidcolname.q.out * /hive/trunk/ql/src/test/results/clientnegative/alter_partition_coltype_invalidtype.q.out * /hive/trunk/ql/src/test/results/clientnegative/desc_failure2.q.out * /hive/trunk/ql/src/test/results/clientnegative/drop_partition_filter_failure2.q.out * /hive/trunk/ql/src/test/results/clientnegative/protectmode_part_no_drop.q.out * /hive/trunk/ql/src/test/results/clientnegative/protectmode_tbl2.q.out * /hive/trunk/ql/src/test/results/clientnegative/protectmode_tbl3.q.out * /hive/trunk/ql/src/test/results/clientnegative/protectmode_tbl4.q.out * /hive/trunk/ql/src/test/results/clientnegative/protectmode_tbl5.q.out * /hive/trunk/ql/src/test/results/clientnegative/protectmode_tbl_no_drop.q.out * /hive/trunk/ql/src/test/results/clientnegative/set_hiveconf_validation0.q.out * /hive/trunk/ql/src/test/results/clientnegative/set_hiveconf_validation1.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter1.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter2.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter3.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter4.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter5.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_index.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_merge_2.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_merge_stats.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_partition_coltype.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_partition_format_loc.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_table_serde.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_view_rename.q.out * /hive/trunk/ql/src/test/results/clientpositive/authorization_7.q.out * /hive/trunk/ql/src/test/results/clientpositive/autogen_colalias.q.out * /hive/trunk/ql/src/test/results/clientpositive/avro_change_schema.q.out * /hive/trunk/ql/src/test/results/clientpositive/avro_evolved_schemas.q.out * /hive/trunk/ql/src/test/results/clientpositive/avro_joins.q.out * /hive/trunk/ql/src/test/results/clientpositive/avro_sanity_test.q.out * /hive/trunk/ql/src/test/results/clientpositive/avro_schema_error_message.q.out * /hive/trunk/ql/src/test/results/clientpositive/avro_schema_literal.q.out * /hive/trunk/ql/src/test/results/clientpositive/ba_table1.q.out * /hive/trunk/ql/src/test/results/clientpositive/ba_table2.q.out * /hive/trunk/ql/src/test/results/clientpositive/ba_table_union.q.out * /hive/trunk/ql/src/test/results/clientpositive/binary_table_bincolserde.q.out * /hive/trunk/ql/src/test/results/clientpositive/binary_table_colserde.q.out * /hive/trunk/ql/src/test/results/clientpositive/bucket_groupby.q.out * /hive/trunk/ql/src/test/results/clientpositive/columnarserde_create_shortcut.q.out * /hive/trunk/ql/src/test/results/clientpositive/combine3.q.out * /hive/trunk/ql/src/test/results/clientpositive/convert_enum_to_string.q.out * /hive/trunk/ql/src/test/results/clientpositive/create_1.q.out * /hive/trunk/ql/src/test/results/clientpositive/create_default_prop.q.out * /hive/trunk/ql/src/test/results/clientpositive/create_escape.q.out * /hive/trunk/ql/src/test/results/clientpositive/create_insert_outputformat.q.out * /hive/trunk/ql/src/test/results/clientpositive/create_like_view.q.out * /hive/trunk/ql/src/test/results/clientpositive/create_nested_type.q.out * /hive/trunk/ql/src/test/results/clientpositive/create_view.q.out *
Hive-trunk-h0.21 - Build # 2068 - Still Failing
Changes for Build #2033 [gates] Removing old branches to limit size of Hive downloads. [gates] Removing tags directory as we no longer need them and they're in the history. [gates] Moving HCatalog into Hive. [gates] Test that perms work for hcatalog [hashutosh] HIVE-4007 : Create abstract classes for serializer and deserializer (Namit Jain via Ashutosh Chauhan) [hashutosh] HIVE-3381 : Result of outer join is not valid (Navis via Ashutosh Chauhan) [hashutosh] HIVE-3980 : Cleanup after 3403 (Namit Jain via Ashutosh Chauhan) [hashutosh] HIVE-4042 : ignore mapjoin hint (Namit Jain via Ashutosh Chauhan) [namit] HIVE-3348 semi-colon in comments in .q file does not work (Nick Collins via namit) [namit] HIVE-4212 sort merge join should work for outer joins for more than 8 inputs (Namit via Gang Tim Liu) Changes for Build #2034 [namit] HIVE-3958 support partial scan for analyze command - RCFile (Gang Tim Liu via namit) Changes for Build #2035 [kevinwilfong] HIVE-4235. CREATE TABLE IF NOT EXISTS uses inefficient way to check if table exists. (Gang Tim Liu via kevinwilfong) [gangtimliu] HIVE-4157: ORC runs out of heap when writing (Kevin Wilfong vi Gang Tim Liu) [gangtimliu] HIVE-4155: Expose ORC's FileDump as a service [gangtimliu] HIVE-4159:RetryingHMSHandler doesn't retry in enough cases (Kevin Wilfong vi Gang Tim Liu) [namit] HIVE-4149 wrong results big outer joins with array of ints (Navis via namit) Changes for Build #2036 [gates] HIVE-4264 Moved hcatalog trunk code up to hive/trunk/hcatalog [hashutosh] HIVE-4263 : Adjust build.xml package command to move all hcat jars and binaries into build (Alan Gates via Ashutosh Chauhan) [namit] HIVE-4258 Log logical plan tree for debugging (Navis via namit) [navis] HIVE-2264 Hive server is SHUTTING DOWN when invalid queries beeing executed Changes for Build #2037 Changes for Build #2038 [hashutosh] HIVE-4122 : Queries fail if timestamp data not in expected format (Prasad Mujumdar via Ashutosh Chauhan) [hashutosh] HIVE-4170 : [REGRESSION] FsShell.close closes filesystem, removing temporary directories (Navis via Ashutosh Chauhan) Changes for Build #2039 [hashutosh] HIVE-3850 : hour() function returns 12 hour clock value when using timestamp datatype (Anandha and Franklin via Ashutosh Chauhan) Changes for Build #2040 [hashutosh] HIVE-3951 : Allow Decimal type columns in Regex Serde (Mark Grover via Ashutosh Chauhan) [namit] HIVE-4270 bug in hive.map.groupby.sorted in the presence of multiple input partitions (Namit via Gang Tim Liu) Changes for Build #2041 Changes for Build #2042 Changes for Build #2043 [hashutosh] HIVE-4252 : hiveserver2 string representation of complex types are inconsistent with cli (Thejas Nair via Ashutosh Chauhan) [hashutosh] HIVE-4179 : NonBlockingOpDeDup does not merge SEL operators correctly (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-4269 : fix handling of binary type in hiveserver2, jdbc driver (Thejas Nair via Ashutosh Chauhan) [namit] HIVE-4174 Round UDF converts BigInts to double (Chen Chun via namit) [namit] HIVE-4240 optimize hive.enforce.bucketing and hive.enforce sorting insert (Gang Tim Liu via namit) [navis] HIVE-4288 Add IntelliJ project files files to .gitignore (Roshan Naik via Navis) Changes for Build #2044 [namit] HIVE-4289 HCatalog build fails when behind a firewall (Samuel Yuan via namit) [namit] HIVE-4281 add hive.map.groupby.sorted.testmode (Namit via Gang Tim Liu) [hashutosh] Moving hcatalog site outside of trunk [hashutosh] Moving hcatalog branches outside of trunk [hashutosh] HIVE-4259 : SEL operator created with missing columnExprMap for unions (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-4156 : need to add protobuf classes to hive-exec.jar (Owen Omalley via Ashutosh Chauhan) [hashutosh] HIVE-3464 : Merging join tree may reorder joins which could be invalid (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4138 : ORC's union object inspector returns a type name that isn't parseable by TypeInfoUtils (Owen Omalley via Ashutosh Chauhan) [cws] HIVE-4119. ANALYZE TABLE ... COMPUTE STATISTICS FOR COLUMNS fails with NPE if the table is empty (Shreepadma Venugopalan via cws) Changes for Build #2045 Changes for Build #2046 [hashutosh] HIVE-4067 : Followup to HIVE-701: reduce ambiguity in grammar (Samuel Yuan via Ashutosh Chauhan) Changes for Build #2047 Changes for Build #2048 [gangtimliu] HIVE-4298: add tests for distincts for hive.map.groutp.sorted. (Namit via Gang Tim Liu) [hashutosh] HIVE-4128 : Support avg(decimal) (Brock Noland via Ashutosh Chauhan) [kevinwilfong] HIVE-4151. HiveProfiler NPE with ScriptOperator. (Pamela Vagata via kevinwilfong) Changes for Build #2049 [hashutosh] HIVE-3985 : Update new UDAFs introduced for Windowing to work with new Decimal Type (Brock Noland via Ashutosh Chauhan) [hashutosh] HIVE-3840 : hive cli null representation in output is inconsistent (Thejas Nair via Ashutosh Chauhan)
[jira] [Commented] (HIVE-2019) Implement NOW() UDF
[ https://issues.apache.org/jira/browse/HIVE-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634112#comment-13634112 ] Zhuoluo (Clark) Yang commented on HIVE-2019: If we use now() for filters, the result will be uncertain. Because if a Map Task is scheduled first, the now() of this map task is earlier, if this task is scheduled later, the now() of this map task is later. In our production environment, a lot of hive tasks are scheduled at mid night, the now() of the Tasks may cross a day for the scheduling orders. I think it is necessary to add a kind of UDF called UDCF. (User Defined Client Functions). if we get the client side now() and make it a constant during compile time, it will be no such problems. Implement NOW() UDF --- Key: HIVE-2019 URL: https://issues.apache.org/jira/browse/HIVE-2019 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Priyadarshini Attachments: HIVE-2019.patch Reference: http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_now -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2019) Implement NOW() UDF
[ https://issues.apache.org/jira/browse/HIVE-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634121#comment-13634121 ] Zhuoluo (Clark) Yang commented on HIVE-2019: [~priyadarshini] I think the patch is a little bit simple, and should consider distributed situation. I think a better way is to fold the NOW() into a constant during compile time. Implement NOW() UDF --- Key: HIVE-2019 URL: https://issues.apache.org/jira/browse/HIVE-2019 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Priyadarshini Attachments: HIVE-2019.patch Reference: http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_now -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2019) Implement NOW() UDF
[ https://issues.apache.org/jira/browse/HIVE-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634148#comment-13634148 ] Edward Capriolo commented on HIVE-2019: --- UDFs have a deterministic annotation you should explore. Implement NOW() UDF --- Key: HIVE-2019 URL: https://issues.apache.org/jira/browse/HIVE-2019 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Priyadarshini Attachments: HIVE-2019.patch Reference: http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_now -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2019) Implement NOW() UDF
[ https://issues.apache.org/jira/browse/HIVE-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634176#comment-13634176 ] Zhuoluo (Clark) Yang commented on HIVE-2019: Actually, NOW() is not a non-deterministic UDF like rand(), for every time you call it, it returns different answers. Is HIVE-746 a related JIRA issue? Implement NOW() UDF --- Key: HIVE-2019 URL: https://issues.apache.org/jira/browse/HIVE-2019 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Priyadarshini Attachments: HIVE-2019.patch Reference: http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_now -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4019) Ability to create and drop temporary partition function
[ https://issues.apache.org/jira/browse/HIVE-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-4019: --- Attachment: HIVE-4019-3.patch Attached patch includes the drop and the test output. I tested it on a distributed cluster and it worked fine. I am running a full unit test run at present and will update when complete. Ability to create and drop temporary partition function --- Key: HIVE-4019 URL: https://issues.apache.org/jira/browse/HIVE-4019 Project: Hive Issue Type: New Feature Components: PTF-Windowing Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-4019-1.patch, HIVE-4019.2.patch, HIVE-4019-3.patch, hive-4019.q Just like udf/udaf/udtf functions, user should be able to add and drop custom partitioning functions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: is there set of queries, which can be used to benchmark the hive performance?
Thanks Swarnim for the quick response. On Tue, Apr 16, 2013 at 9:22 PM, kulkarni.swar...@gmail.com kulkarni.swar...@gmail.com wrote: Hi Rob, HiBench[1] is one I have seen most commonly used. [1] https://github.com/intel-hadoop/HiBench/tree/master/hivebench On Tue, Apr 16, 2013 at 6:42 PM, ur lops urlop...@gmail.com wrote: I am looking to benchmark my database with hive. but before I do that, I want to run a set of tests on hive to benchmark hive. Is there something exists in hive, similar to pig gridmix? Thanks in advance Rob. -- Swarnim
[jira] [Updated] (HIVE-4284) Implement class for vectorized row batch
[ https://issues.apache.org/jira/browse/HIVE-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Hanson updated HIVE-4284: -- Attachment: HIVE-4284.3.patch removed unwanted .rej file Implement class for vectorized row batch Key: HIVE-4284 URL: https://issues.apache.org/jira/browse/HIVE-4284 Project: Hive Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Eric Hanson Attachments: HIVE-4284.1.patch, HIVE-4284.2.patch, HIVE-4284.3.patch Vectorized row batch object will represent the row batch that vectorized operators will work on. Refer to design spec attached to HIVE-4160 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4284) Implement class for vectorized row batch
[ https://issues.apache.org/jira/browse/HIVE-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634300#comment-13634300 ] Eric Hanson commented on HIVE-4284: --- Code review link: https://reviews.facebook.net/differential/diff/32307/ Implement class for vectorized row batch Key: HIVE-4284 URL: https://issues.apache.org/jira/browse/HIVE-4284 Project: Hive Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Eric Hanson Attachments: HIVE-4284.1.patch, HIVE-4284.2.patch, HIVE-4284.3.patch Vectorized row batch object will represent the row batch that vectorized operators will work on. Refer to design spec attached to HIVE-4160 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4284) Implement class for vectorized row batch
[ https://issues.apache.org/jira/browse/HIVE-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Hanson updated HIVE-4284: -- Attachment: (was: HIVE-4284.1.patch) Implement class for vectorized row batch Key: HIVE-4284 URL: https://issues.apache.org/jira/browse/HIVE-4284 Project: Hive Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Eric Hanson Attachments: HIVE-4284.3.patch Vectorized row batch object will represent the row batch that vectorized operators will work on. Refer to design spec attached to HIVE-4160 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4284) Implement class for vectorized row batch
[ https://issues.apache.org/jira/browse/HIVE-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Hanson updated HIVE-4284: -- Attachment: (was: HIVE-4284.2.patch) Implement class for vectorized row batch Key: HIVE-4284 URL: https://issues.apache.org/jira/browse/HIVE-4284 Project: Hive Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Eric Hanson Attachments: HIVE-4284.3.patch Vectorized row batch object will represent the row batch that vectorized operators will work on. Refer to design spec attached to HIVE-4160 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4280) TestRetryingHMSHandler is failing on trunk.
[ https://issues.apache.org/jira/browse/HIVE-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634315#comment-13634315 ] Hudson commented on HIVE-4280: -- Integrated in Hive-trunk-h0.21 #2069 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2069/]) HIVE-4280 : TestRetryingHMSHandler is failing on trunk. (Teddy Choi via Ashutosh Chauhan) (Revision 1468917) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1468917 Files : * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreWithEnvironmentContext.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEndFunctionListener.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestRetryingHMSHandler.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/security/TestAuthorizationPreEventListener.java TestRetryingHMSHandler is failing on trunk. --- Key: HIVE-4280 URL: https://issues.apache.org/jira/browse/HIVE-4280 Project: Hive Issue Type: Bug Affects Versions: 0.11.0 Reporter: Ashutosh Chauhan Assignee: Teddy Choi Fix For: 0.11.0 Attachments: HIVE-4280-1.patch.txt, HIVE-4280-2.patch.txt Newly added testcase TestRetryingHMSHandler fails on trunk. https://builds.apache.org/job/Hive-trunk-h0.21/2040/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-hadoop2 - Build # 164 - Still Failing
Changes for Build #138 [namit] HIVE-4289 HCatalog build fails when behind a firewall (Samuel Yuan via namit) [namit] HIVE-4281 add hive.map.groupby.sorted.testmode (Namit via Gang Tim Liu) [hashutosh] Moving hcatalog site outside of trunk [hashutosh] Moving hcatalog branches outside of trunk [hashutosh] HIVE-4259 : SEL operator created with missing columnExprMap for unions (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-4156 : need to add protobuf classes to hive-exec.jar (Owen Omalley via Ashutosh Chauhan) [hashutosh] HIVE-3464 : Merging join tree may reorder joins which could be invalid (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4138 : ORC's union object inspector returns a type name that isn't parseable by TypeInfoUtils (Owen Omalley via Ashutosh Chauhan) [cws] HIVE-4119. ANALYZE TABLE ... COMPUTE STATISTICS FOR COLUMNS fails with NPE if the table is empty (Shreepadma Venugopalan via cws) [hashutosh] HIVE-4252 : hiveserver2 string representation of complex types are inconsistent with cli (Thejas Nair via Ashutosh Chauhan) [hashutosh] HIVE-4179 : NonBlockingOpDeDup does not merge SEL operators correctly (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-4269 : fix handling of binary type in hiveserver2, jdbc driver (Thejas Nair via Ashutosh Chauhan) [namit] HIVE-4174 Round UDF converts BigInts to double (Chen Chun via namit) [namit] HIVE-4240 optimize hive.enforce.bucketing and hive.enforce sorting insert (Gang Tim Liu via namit) [navis] HIVE-4288 Add IntelliJ project files files to .gitignore (Roshan Naik via Navis) [namit] HIVE-4272 partition wise metadata does not work for text files [hashutosh] HIVE-896 : Add LEAD/LAG/FIRST/LAST analytical windowing functions to Hive. (Harish Butani via Ashutosh Chauhan) [namit] HIVE-4260 union_remove_12, union_remove_13 are failing on hadoop2 (Gunther Hagleitner via namit) [hashutosh] HIVE-3951 : Allow Decimal type columns in Regex Serde (Mark Grover via Ashutosh Chauhan) [namit] HIVE-4270 bug in hive.map.groupby.sorted in the presence of multiple input partitions (Namit via Gang Tim Liu) [hashutosh] HIVE-3850 : hour() function returns 12 hour clock value when using timestamp datatype (Anandha and Franklin via Ashutosh Chauhan) [hashutosh] HIVE-4122 : Queries fail if timestamp data not in expected format (Prasad Mujumdar via Ashutosh Chauhan) [hashutosh] HIVE-4170 : [REGRESSION] FsShell.close closes filesystem, removing temporary directories (Navis via Ashutosh Chauhan) [gates] HIVE-4264 Moved hcatalog trunk code up to hive/trunk/hcatalog [hashutosh] HIVE-4263 : Adjust build.xml package command to move all hcat jars and binaries into build (Alan Gates via Ashutosh Chauhan) [namit] HIVE-4258 Log logical plan tree for debugging (Navis via namit) [navis] HIVE-2264 Hive server is SHUTTING DOWN when invalid queries beeing executed [kevinwilfong] HIVE-4235. CREATE TABLE IF NOT EXISTS uses inefficient way to check if table exists. (Gang Tim Liu via kevinwilfong) [gangtimliu] HIVE-4157: ORC runs out of heap when writing (Kevin Wilfong vi Gang Tim Liu) [gangtimliu] HIVE-4155: Expose ORC's FileDump as a service [gangtimliu] HIVE-4159:RetryingHMSHandler doesn't retry in enough cases (Kevin Wilfong vi Gang Tim Liu) [namit] HIVE-4149 wrong results big outer joins with array of ints (Navis via namit) [namit] HIVE-3958 support partial scan for analyze command - RCFile (Gang Tim Liu via namit) [gates] Removing old branches to limit size of Hive downloads. [gates] Removing tags directory as we no longer need them and they're in the history. [gates] Moving HCatalog into Hive. [gates] Test that perms work for hcatalog [hashutosh] HIVE-4007 : Create abstract classes for serializer and deserializer (Namit Jain via Ashutosh Chauhan) [hashutosh] HIVE-3381 : Result of outer join is not valid (Navis via Ashutosh Chauhan) [hashutosh] HIVE-3980 : Cleanup after 3403 (Namit Jain via Ashutosh Chauhan) [hashutosh] HIVE-4042 : ignore mapjoin hint (Namit Jain via Ashutosh Chauhan) [namit] HIVE-3348 semi-colon in comments in .q file does not work (Nick Collins via namit) [namit] HIVE-4212 sort merge join should work for outer joins for more than 8 inputs (Namit via Gang Tim Liu) [namit] HIVE-4219 explain dependency does not capture the input table (Namit via Gang Tim Liu) [kevinwilfong] HIVE-4092. Store complete names of tables in column access analyzer (Samuel Yuan via kevinwilfong) [namit] HIVE-4208 Clientpositive test parenthesis_star_by is non-deteministic (Mark Grover via namit) [cws] HIVE-4217. Fix show_create_table_*.q test failures (Carl Steinbach via cws) [namit] HIVE-4206 Sort merge join does not work for outer joins for 7 inputs (Namit via Gang Tim Liu) [kevinwilfong] HIVE-4188. TestJdbcDriver2.testDescribeTable failing consistently. (Prasad Mujumdar via kevinwilfong) [hashutosh] HIVE-3820 Consider creating a literal like D or BD for representing Decimal type constants (Gunther Hagleitner
Hive-trunk-h0.21 - Build # 2069 - Still Failing
Changes for Build #2034 [namit] HIVE-3958 support partial scan for analyze command - RCFile (Gang Tim Liu via namit) Changes for Build #2035 [kevinwilfong] HIVE-4235. CREATE TABLE IF NOT EXISTS uses inefficient way to check if table exists. (Gang Tim Liu via kevinwilfong) [gangtimliu] HIVE-4157: ORC runs out of heap when writing (Kevin Wilfong vi Gang Tim Liu) [gangtimliu] HIVE-4155: Expose ORC's FileDump as a service [gangtimliu] HIVE-4159:RetryingHMSHandler doesn't retry in enough cases (Kevin Wilfong vi Gang Tim Liu) [namit] HIVE-4149 wrong results big outer joins with array of ints (Navis via namit) Changes for Build #2036 [gates] HIVE-4264 Moved hcatalog trunk code up to hive/trunk/hcatalog [hashutosh] HIVE-4263 : Adjust build.xml package command to move all hcat jars and binaries into build (Alan Gates via Ashutosh Chauhan) [namit] HIVE-4258 Log logical plan tree for debugging (Navis via namit) [navis] HIVE-2264 Hive server is SHUTTING DOWN when invalid queries beeing executed Changes for Build #2037 Changes for Build #2038 [hashutosh] HIVE-4122 : Queries fail if timestamp data not in expected format (Prasad Mujumdar via Ashutosh Chauhan) [hashutosh] HIVE-4170 : [REGRESSION] FsShell.close closes filesystem, removing temporary directories (Navis via Ashutosh Chauhan) Changes for Build #2039 [hashutosh] HIVE-3850 : hour() function returns 12 hour clock value when using timestamp datatype (Anandha and Franklin via Ashutosh Chauhan) Changes for Build #2040 [hashutosh] HIVE-3951 : Allow Decimal type columns in Regex Serde (Mark Grover via Ashutosh Chauhan) [namit] HIVE-4270 bug in hive.map.groupby.sorted in the presence of multiple input partitions (Namit via Gang Tim Liu) Changes for Build #2041 Changes for Build #2042 Changes for Build #2043 [hashutosh] HIVE-4252 : hiveserver2 string representation of complex types are inconsistent with cli (Thejas Nair via Ashutosh Chauhan) [hashutosh] HIVE-4179 : NonBlockingOpDeDup does not merge SEL operators correctly (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-4269 : fix handling of binary type in hiveserver2, jdbc driver (Thejas Nair via Ashutosh Chauhan) [namit] HIVE-4174 Round UDF converts BigInts to double (Chen Chun via namit) [namit] HIVE-4240 optimize hive.enforce.bucketing and hive.enforce sorting insert (Gang Tim Liu via namit) [navis] HIVE-4288 Add IntelliJ project files files to .gitignore (Roshan Naik via Navis) Changes for Build #2044 [namit] HIVE-4289 HCatalog build fails when behind a firewall (Samuel Yuan via namit) [namit] HIVE-4281 add hive.map.groupby.sorted.testmode (Namit via Gang Tim Liu) [hashutosh] Moving hcatalog site outside of trunk [hashutosh] Moving hcatalog branches outside of trunk [hashutosh] HIVE-4259 : SEL operator created with missing columnExprMap for unions (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-4156 : need to add protobuf classes to hive-exec.jar (Owen Omalley via Ashutosh Chauhan) [hashutosh] HIVE-3464 : Merging join tree may reorder joins which could be invalid (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4138 : ORC's union object inspector returns a type name that isn't parseable by TypeInfoUtils (Owen Omalley via Ashutosh Chauhan) [cws] HIVE-4119. ANALYZE TABLE ... COMPUTE STATISTICS FOR COLUMNS fails with NPE if the table is empty (Shreepadma Venugopalan via cws) Changes for Build #2045 Changes for Build #2046 [hashutosh] HIVE-4067 : Followup to HIVE-701: reduce ambiguity in grammar (Samuel Yuan via Ashutosh Chauhan) Changes for Build #2047 Changes for Build #2048 [gangtimliu] HIVE-4298: add tests for distincts for hive.map.groutp.sorted. (Namit via Gang Tim Liu) [hashutosh] HIVE-4128 : Support avg(decimal) (Brock Noland via Ashutosh Chauhan) [kevinwilfong] HIVE-4151. HiveProfiler NPE with ScriptOperator. (Pamela Vagata via kevinwilfong) Changes for Build #2049 [hashutosh] HIVE-3985 : Update new UDAFs introduced for Windowing to work with new Decimal Type (Brock Noland via Ashutosh Chauhan) [hashutosh] HIVE-3840 : hive cli null representation in output is inconsistent (Thejas Nair via Ashutosh Chauhan) [hashutosh] HIVE-4262 : fix last_value UDAF behavior (Harish Butani via Ashutosh Chauhan) [hashutosh] HIVE-4292 : hiveserver2 should support -hiveconf commandline parameter (Thejas Nair via Ashutosh Chauhan) Changes for Build #2050 [hashutosh] HIVE-3908 : create view statement's outputs contains the view and a temporary dir. (Prasad Mujumdar via Ashutosh Chauhan) [hashutosh] HIVE-4302 : Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp (Harish Butani via Ashutosh Chauhan) Changes for Build #2051 [hashutosh] HIVE-3992 : Hive RCFile::sync(long) does a sub-sequence linear search for sync blocks (Gopal V via Ashutosh Chauhan) Changes for Build #2052 Changes for Build #2053 [navis] Missing test results from HIVE-1953 (Vikram Dixit K via Navis) [namit] HIVE-4314 Result of
Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21 #350
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/ -- [...truncated 5454 lines...] init: [echo] Project: jdbc create-dirs: [echo] Project: hwi [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/hwi [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/hwi/classes [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/hwi/test [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/hwi/test/src [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/hwi/test/classes [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/hwi/test/resources [copy] Warning: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/hwi/src/test/resources does not exist. init: [echo] Project: hwi create-dirs: [echo] Project: hbase-handler [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/hbase-handler [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/hbase-handler/classes [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/hbase-handler/test [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/hbase-handler/test/src [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/hbase-handler/test/classes [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/hbase-handler/test/resources [copy] Warning: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/hbase-handler/src/test/resources does not exist. init: [echo] Project: hbase-handler create-dirs: [echo] Project: pdk [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/pdk [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/pdk/classes [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/pdk/test [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/pdk/test/src [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/pdk/test/classes [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/pdk/test/resources [copy] Warning: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/pdk/src/test/resources does not exist. init: [echo] Project: pdk create-dirs: [echo] Project: builtins [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/builtins [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/builtins/classes [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/builtins/test [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/builtins/test/src [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/builtins/test/classes [mkdir] Created dir: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/builtins/test/resources [copy] Warning: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/builtins/src/test/resources does not exist. init: [echo] Project: builtins jar: [echo] Project: hive create-dirs: [echo] Project: shims [copy] Warning: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/shims/src/test/resources does not exist. init: [echo] Project: shims ivy-init-settings: [echo] Project: shims ivy-resolve: [echo] Project: shims [ivy:resolve] :: loading settings :: file = https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/ivy/ivysettings.xml [ivy:resolve] downloading http://repo1.maven.org/maven2/org/apache/thrift/libthrift/0.7.0/libthrift-0.7.0.jar ... [ivy:resolve] (294kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] org.apache.thrift#libthrift;0.7.0!libthrift.jar (25ms) [ivy:report] Processing https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/350/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-shims-default.xml to
[jira] [Commented] (HIVE-3938) Hive MetaStore should send a single AddPartitionEvent for atomically added partition-set.
[ https://issues.apache.org/jira/browse/HIVE-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634349#comment-13634349 ] Mithun Radhakrishnan commented on HIVE-3938: Sorry for the delay. I've only just gotten back to fixing this. I've incorporated your suggestions. I'm trying to get the tests going. Will update shortly. Hive MetaStore should send a single AddPartitionEvent for atomically added partition-set. - Key: HIVE-3938 URL: https://issues.apache.org/jira/browse/HIVE-3938 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Mithun Radhakrishnan Assignee: Mithun Radhakrishnan Attachments: HIVE-3938.trunk.patch HiveMetaStore::add_partitions() currently adds all partitions specified in one call using a single meta-store transaction. This acts correctly. However, there's one AddPartitionEvent created per partition specified. Ideally, the set of partitions added atomically can be communicated using a single AddPartitionEvent, such that they are consumed together. I'll post a patch that does this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4318) OperatorHooks hit performance even when not used
[
https://issues.apache.org/jira/browse/HIVE-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634350#comment-13634350
]
Gunther Hagleitner commented on HIVE-4318:
--
[~kevinwilfong]/[~pamelavagata]: Given the latest run, are you ok with the
proposal, or do you have another suggestion?
OperatorHooks hit performance even when not used
Key: HIVE-4318
URL: https://issues.apache.org/jira/browse/HIVE-4318
Project: Hive
Issue Type: Bug
Components: Query Processor
Environment: Ubuntu LXC (64 bit)
Reporter: Gopal V
Assignee: Gunther Hagleitner
Attachments: HIVE-4318.1.patch, HIVE-4318.2.patch,
HIVE-4318.patch.pam.txt
Operator Hooks inserted into Operator.java cause a performance hit even when
it is not being used.
For a count(1) query tested with without the operator hook calls.
{code:title=with}
2013-04-09 07:33:58,920 Stage-1 map = 100%, reduce = 100%, Cumulative CPU
84.07 sec
Total MapReduce CPU Time Spent: 1 minutes 24 seconds 70 msec
OK
28800991
Time taken: 40.407 seconds, Fetched: 1 row(s)
{code}
{code:title=without}
2013-04-09 07:33:02,355 Stage-1 map = 100%, reduce = 100%, Cumulative CPU
68.48 sec
...
Total MapReduce CPU Time Spent: 1 minutes 8 seconds 480 msec
OK
28800991
Time taken: 35.907 seconds, Fetched: 1 row(s)
{code}
The effect is multiplied by the number of operators in the pipeline that has
to forward the row - the more operators there are the, the slower the query.
The modification made to test this was
{code:title=Operator.java}
--- ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
+++ ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
@@ -526,16 +526,16 @@ public void process(Object row, int tag) throws
HiveException {
return;
}
OperatorHookContext opHookContext = new OperatorHookContext(this, row,
tag);
-preProcessCounter();
-enterOperatorHooks(opHookContext);
+//preProcessCounter();
+//enterOperatorHooks(opHookContext);
processOp(row, tag);
-exitOperatorHooks(opHookContext);
-postProcessCounter();
+//exitOperatorHooks(opHookContext);
+//postProcessCounter();
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4225) HiveServer2 does not support SASL QOP
[ https://issues.apache.org/jira/browse/HIVE-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634351#comment-13634351 ] Joey Echeverria commented on HIVE-4225: --- This also affects the thrift metastore server. HiveServer2 does not support SASL QOP - Key: HIVE-4225 URL: https://issues.apache.org/jira/browse/HIVE-4225 Project: Hive Issue Type: Bug Components: HiveServer2, Shims Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-4225.patch HiveServer2 implements Kerberos authentication through SASL framework, but does not support setting QOP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: New code for VectorizedRowBatch to form basis of vectorized query execution
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10592/ --- Review request for hive. Description --- New code for VectorizedRowBatch to form basis of vectorized query execution This addresses bug HIVE-4284. https://issues.apache.org/jira/browse/HIVE-4284 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizedRowBatch.java PRE-CREATION Diff: https://reviews.apache.org/r/10592/diff/ Testing --- Thanks, Eric Hanson
[jira] [Commented] (HIVE-4284) Implement class for vectorized row batch
[ https://issues.apache.org/jira/browse/HIVE-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634431#comment-13634431 ] Eric Hanson commented on HIVE-4284: --- Disregard above code review link. Instead please use this: https://reviews.apache.org/r/10592/ Implement class for vectorized row batch Key: HIVE-4284 URL: https://issues.apache.org/jira/browse/HIVE-4284 Project: Hive Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Eric Hanson Attachments: HIVE-4284.3.patch Vectorized row batch object will represent the row batch that vectorized operators will work on. Refer to design spec attached to HIVE-4160 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4278) HCat needs to get current Hive jars instead of pulling them from maven repo
[ https://issues.apache.org/jira/browse/HIVE-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-4278: --- Attachment: HIVE-4278.approach2.patch.3.for.branch.12 Attaching updated patch for trunk - there was one more hcatalog.version flag in build.properties that wasn't there earlier. HCat needs to get current Hive jars instead of pulling them from maven repo --- Key: HIVE-4278 URL: https://issues.apache.org/jira/browse/HIVE-4278 Project: Hive Issue Type: Sub-task Components: Build Infrastructure, HCatalog Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Sushanth Sowmyan Priority: Blocker Fix For: 0.11.0 Attachments: HIVE-4278.approach2.patch, HIVE-4278.approach2.patch.2.for.branch.11, HIVE-4278.approach2.patch.2.for.branch.12, HIVE-4278.approach2.patch.3.for.branch.12, HIVE-4278.D10257.1.patch, HIVE-4278.D9981.1.patch The HCatalog build is currently pulling Hive jars from the maven repo instead of using the ones built as part of the current build. Now that it is part of Hive it should use the jars being built instead of pulling them from maven. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4278) HCat needs to get current Hive jars instead of pulling them from maven repo
[ https://issues.apache.org/jira/browse/HIVE-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-4278: --- Status: Patch Available (was: Open) HCat needs to get current Hive jars instead of pulling them from maven repo --- Key: HIVE-4278 URL: https://issues.apache.org/jira/browse/HIVE-4278 Project: Hive Issue Type: Sub-task Components: Build Infrastructure, HCatalog Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Sushanth Sowmyan Priority: Blocker Fix For: 0.11.0 Attachments: HIVE-4278.approach2.patch, HIVE-4278.approach2.patch.2.for.branch.11, HIVE-4278.approach2.patch.2.for.branch.12, HIVE-4278.approach2.patch.3.for.branch.12, HIVE-4278.D10257.1.patch, HIVE-4278.D9981.1.patch The HCatalog build is currently pulling Hive jars from the maven repo instead of using the ones built as part of the current build. Now that it is part of Hive it should use the jars being built instead of pulling them from maven. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: New code for VectorizedRowBatch to form basis of vectorized query execution
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10592/#review19350 --- Note: I did not flag every formatting issue individually. ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java https://reviews.apache.org/r/10592/#comment40030 Formatting. ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java https://reviews.apache.org/r/10592/#comment40040 Should these methods be abstract? If not, then I think you want to throw an UnsupportedOperationException(Not Implemented) instead of doing an assert or returning a string. ql/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java https://reviews.apache.org/r/10592/#comment40031 Is there a reason these variables are public? ql/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java https://reviews.apache.org/r/10592/#comment40033 Public? ql/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java https://reviews.apache.org/r/10592/#comment40032 Formatting. ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java https://reviews.apache.org/r/10592/#comment40034 Public? ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java https://reviews.apache.org/r/10592/#comment40035 Formatting: http://www.oracle.com/technetwork/java/javase/documentation/codeconventions-141999.html#342 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java https://reviews.apache.org/r/10592/#comment40041 Refer to the naming conventions for constants in the Java coding conventions. ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java https://reviews.apache.org/r/10592/#comment40036 Use UnsupportedOperationException instead. ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java https://reviews.apache.org/r/10592/#comment40037 Ditto ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizedRowBatch.java https://reviews.apache.org/r/10592/#comment40038 Formatting ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizedRowBatch.java https://reviews.apache.org/r/10592/#comment40039 Formatting. - Carl Steinbach On April 17, 2013, 8:50 p.m., Eric Hanson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10592/ --- (Updated April 17, 2013, 8:50 p.m.) Review request for hive. Description --- New code for VectorizedRowBatch to form basis of vectorized query execution This addresses bug HIVE-4284. https://issues.apache.org/jira/browse/HIVE-4284 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizedRowBatch.java PRE-CREATION Diff: https://reviews.apache.org/r/10592/diff/ Testing --- Thanks, Eric Hanson
[jira] [Commented] (HIVE-4284) Implement class for vectorized row batch
[ https://issues.apache.org/jira/browse/HIVE-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634484#comment-13634484 ] Carl Steinbach commented on HIVE-4284: -- I left some comments on reviewboard. Implement class for vectorized row batch Key: HIVE-4284 URL: https://issues.apache.org/jira/browse/HIVE-4284 Project: Hive Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Eric Hanson Attachments: HIVE-4284.3.patch Vectorized row batch object will represent the row batch that vectorized operators will work on. Refer to design spec attached to HIVE-4160 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4373) Hive Version
Deepesh Khandelwal created HIVE-4373:
Summary: Hive Version
Key: HIVE-4373
URL: https://issues.apache.org/jira/browse/HIVE-4373
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.11.0
Reporter: Deepesh Khandelwal
Priority: Minor
Fix For: 0.11.0
When running beeline
{code}
% beeline -u 'jdbc:hive2://localhost:1' -n hive -p passwd -d
org.apache.hive.jdbc.HiveDriver
Connecting to jdbc:hive2://localhost:1
Connected to: Hive (version 0.10.0)
Driver: Hive (version 0.11.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
{code}
The Hive version in the Connected to: string says 0.10.0 instead of 0.11.0.
Looking at the code it seems that the version is hardcoded at two places:
line 250 in jdbc/src/java/org/apache/hive/jdbc/HiveDatabaseMetaData.java
line 833 in jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4373) Hive Version returned by HiveDatabaseMetaData.getDatabaseProductVersion is incorrect
[
https://issues.apache.org/jira/browse/HIVE-4373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-4373:
-
Summary: Hive Version returned by
HiveDatabaseMetaData.getDatabaseProductVersion is incorrect (was: Hive Version
)
Hive Version returned by HiveDatabaseMetaData.getDatabaseProductVersion is
incorrect
Key: HIVE-4373
URL: https://issues.apache.org/jira/browse/HIVE-4373
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.11.0
Reporter: Deepesh Khandelwal
Priority: Minor
Fix For: 0.11.0
When running beeline
{code}
% beeline -u 'jdbc:hive2://localhost:1' -n hive -p passwd -d
org.apache.hive.jdbc.HiveDriver
Connecting to jdbc:hive2://localhost:1
Connected to: Hive (version 0.10.0)
Driver: Hive (version 0.11.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
{code}
The Hive version in the Connected to: string says 0.10.0 instead of 0.11.0.
Looking at the code it seems that the version is hardcoded at two places:
line 250 in jdbc/src/java/org/apache/hive/jdbc/HiveDatabaseMetaData.java
line 833 in jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3938) Hive MetaStore should send a single AddPartitionEvent for atomically added partition-set.
[ https://issues.apache.org/jira/browse/HIVE-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634523#comment-13634523 ] Mithun Radhakrishnan commented on HIVE-3938: Ok, Dilip's comment about support for multi-table inserts poses a dilemma. Thus far, HiveMetaStore::add_partitions_core() seems to have assumed (at least in part) that all the parts belong to the same table (evidenced by the logInfo() line). If one might call add_partitions() with parts in different tables, should they all be considered to be part of a single atomic set? Or ought I to categorize them (per table) and send separate Events out? I'm inclined to do the former (because you retain the ability to break the partition-sets up downstream), but I'm open to suggestions. (+ [~ashutoshc], who wrote the initial version of this.) Hive MetaStore should send a single AddPartitionEvent for atomically added partition-set. - Key: HIVE-3938 URL: https://issues.apache.org/jira/browse/HIVE-3938 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Mithun Radhakrishnan Assignee: Mithun Radhakrishnan Attachments: HIVE-3938.trunk.patch HiveMetaStore::add_partitions() currently adds all partitions specified in one call using a single meta-store transaction. This acts correctly. However, there's one AddPartitionEvent created per partition specified. Ideally, the set of partitions added atomically can be communicated using a single AddPartitionEvent, such that they are consumed together. I'll post a patch that does this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4305) Use a single system for dependency resolution
[ https://issues.apache.org/jira/browse/HIVE-4305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634524#comment-13634524 ] Carl Steinbach commented on HIVE-4305: -- bq. Carl Steinbach, you had asked for a list of benefits, are you convinced by the arguments here? No, I remain unconvinced. So far the points that have been made in favor of switching to Maven are either factually inaccurate (e.g. Ivy doesn't support X when it actually does), or are personal opinions masquerading as statements of fact (e.g. Owen's five points). People need to give concrete reasons (backed with examples) that demonstrate the superiority of Maven over Ivy. I'm happy to reciprocate with counterexamples. Use a single system for dependency resolution - Key: HIVE-4305 URL: https://issues.apache.org/jira/browse/HIVE-4305 Project: Hive Issue Type: Improvement Components: Build Infrastructure, HCatalog Reporter: Travis Crawford Both Hive and HCatalog use ant as their build tool. However, Hive uses ivy for dependency resolution while HCatalog uses maven-ant-tasks. With the project merge we should converge on a single tool for dependency resolution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4200) Consolidate submodule dependencies using ivy inheritance
[ https://issues.apache.org/jira/browse/HIVE-4200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-4200: - Attachment: HIVE-4200.1.patch.txt Attaching a WIP patch that moves all third-party dependencies (i.e. non inter module deps) to the root ivy.xml file. @Gunther: If you'd like to push this forward I'm happy to review and commit it. Thanks. Consolidate submodule dependencies using ivy inheritance Key: HIVE-4200 URL: https://issues.apache.org/jira/browse/HIVE-4200 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-4200.1.patch.txt As discussed in 4187: For easier maintenance of ivy dependencies across submodules: Create parent ivy file with consolidated dependencies and include into submodules via inheritance. This way we're not relying on transitive dependencies, but also have the dependencies in a single place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4374) Hive does not periodically cleanup hive.exec.scratchdir
Jeff Lord created HIVE-4374: --- Summary: Hive does not periodically cleanup hive.exec.scratchdir Key: HIVE-4374 URL: https://issues.apache.org/jira/browse/HIVE-4374 Project: Hive Issue Type: Improvement Components: HiveServer2 Affects Versions: 0.10.0 Reporter: Jeff Lord Currently Hive does not cleanup hive.exec.scratchdir It does have a setting to cleanup this directory on start which works in that case. It would be nice if there was a setting such that if cleanup.scratch==true and days=14 than remove scratch files/dirs that are older than 14 days. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-4333) most windowing tests fail on hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Weaver reassigned HIVE-4333: Assignee: Matthew Weaver (was: Gunther Hagleitner) most windowing tests fail on hadoop 2 - Key: HIVE-4333 URL: https://issues.apache.org/jira/browse/HIVE-4333 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Matthew Weaver Problem is different order of results on hadoop 2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4333) most windowing tests fail on hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634570#comment-13634570 ] Matthew Weaver commented on HIVE-4333: -- The OVER clauses don't fully specify the ordering, causing different ordering of results and different values for FIRST and LAST. The fix is just to add enough fields to guarantee an unambiguous ordering in the window. This will fix many of the queries, maybe all. most windowing tests fail on hadoop 2 - Key: HIVE-4333 URL: https://issues.apache.org/jira/browse/HIVE-4333 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Matthew Weaver Problem is different order of results on hadoop 2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4189) ORC fails with String column that ends in lots of nulls
[ https://issues.apache.org/jira/browse/HIVE-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634583#comment-13634583 ] Owen O'Malley commented on HIVE-4189: - +1 looks good. ORC fails with String column that ends in lots of nulls --- Key: HIVE-4189 URL: https://issues.apache.org/jira/browse/HIVE-4189 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.11.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-4189.1.patch.txt, HIVE-4189.2.patch.txt When ORC attempts to write out a string column that ends in enough nulls to span an index stride, StringTreeWriter's writeStripe method will get an exception from TreeWriter's writeStripe method Column has wrong number of index entries found: x expected: y This is caused by rowIndexValueCount having multiple entries equal to the number of non-null rows in the column, combined with the fact that StringTreeWriter has special logic for constructing its index. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4305) Use a single system for dependency resolution
[ https://issues.apache.org/jira/browse/HIVE-4305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634593#comment-13634593 ] Brock Noland commented on HIVE-4305: Does ivy have a completely offline mode? That is what I am most interested in and haven't been able to find it. For example, ivy.cache.ttl.default=eternal doesn't stop the downloading. Use a single system for dependency resolution - Key: HIVE-4305 URL: https://issues.apache.org/jira/browse/HIVE-4305 Project: Hive Issue Type: Improvement Components: Build Infrastructure, HCatalog Reporter: Travis Crawford Both Hive and HCatalog use ant as their build tool. However, Hive uses ivy for dependency resolution while HCatalog uses maven-ant-tasks. With the project merge we should converge on a single tool for dependency resolution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4318) OperatorHooks hit performance even when not used
[
https://issues.apache.org/jira/browse/HIVE-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634621#comment-13634621
]
Pamela Vagata commented on HIVE-4318:
-
Gunther, thanks for taking the time to run the patch: given the results - I'm
ok with the proposal.
OperatorHooks hit performance even when not used
Key: HIVE-4318
URL: https://issues.apache.org/jira/browse/HIVE-4318
Project: Hive
Issue Type: Bug
Components: Query Processor
Environment: Ubuntu LXC (64 bit)
Reporter: Gopal V
Assignee: Gunther Hagleitner
Attachments: HIVE-4318.1.patch, HIVE-4318.2.patch,
HIVE-4318.patch.pam.txt
Operator Hooks inserted into Operator.java cause a performance hit even when
it is not being used.
For a count(1) query tested with without the operator hook calls.
{code:title=with}
2013-04-09 07:33:58,920 Stage-1 map = 100%, reduce = 100%, Cumulative CPU
84.07 sec
Total MapReduce CPU Time Spent: 1 minutes 24 seconds 70 msec
OK
28800991
Time taken: 40.407 seconds, Fetched: 1 row(s)
{code}
{code:title=without}
2013-04-09 07:33:02,355 Stage-1 map = 100%, reduce = 100%, Cumulative CPU
68.48 sec
...
Total MapReduce CPU Time Spent: 1 minutes 8 seconds 480 msec
OK
28800991
Time taken: 35.907 seconds, Fetched: 1 row(s)
{code}
The effect is multiplied by the number of operators in the pipeline that has
to forward the row - the more operators there are the, the slower the query.
The modification made to test this was
{code:title=Operator.java}
--- ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
+++ ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
@@ -526,16 +526,16 @@ public void process(Object row, int tag) throws
HiveException {
return;
}
OperatorHookContext opHookContext = new OperatorHookContext(this, row,
tag);
-preProcessCounter();
-enterOperatorHooks(opHookContext);
+//preProcessCounter();
+//enterOperatorHooks(opHookContext);
processOp(row, tag);
-exitOperatorHooks(opHookContext);
-postProcessCounter();
+//exitOperatorHooks(opHookContext);
+//postProcessCounter();
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4305) Use a single system for dependency resolution
[ https://issues.apache.org/jira/browse/HIVE-4305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634624#comment-13634624 ] Owen O'Malley commented on HIVE-4305: - Carl, I fully acknowledge that ant vs maven is a religious discussion. However, to back up my five points: * IDE support is much better From http://www.jetbrains.com/idea/features/ant_maven.html : Maven integration reads the files and builds the modules and dependencies between them. Ant integration executes ant targets. This is similar to eclipse too. For Maven projects, you don't need to maintain a set of helper files that set up the project in the IDE. They can build it automatically. Even with our eclipse helper scripts, users give up on building Hive in an IDE. * Offline support is much better Try turning off the internet and build Hive. It is relatively difficult. Maven will just work if you have the required jars in your cache. * You can download a Maven project and build it without reading the build file. This is obviously true from the fundamentals of each system. Ant provides a wide open playing field and you can build tar in one project and package in another. There are no rules. In Maven, I know what package will build. * Publishing to Maven central is much easier. Ivy can't publish to Maven central, so you end up use ant's maven tasks to publish. This requires that you have two different descriptions of the projects dependencies one for ivy and one for ant's maven tasks. Furthermore, based on my experience as the release manager for Hadoop, ant's maven tasks are much more error-prone. Futhermore, they don't support features like storing your password encrypted. * Profiles work much better in Maven. Ok, this one is debatable. In my opinion, Maven profiles are cleaner and better designed. Finally, I fully support Brock's point: * Maven is used by the other Hadoop ecosystem projects. Hadoop in particular was using ant, ivy, and maven ant tasks for a long time and traded them in for Maven. There is significant value in using similar tools. Use a single system for dependency resolution - Key: HIVE-4305 URL: https://issues.apache.org/jira/browse/HIVE-4305 Project: Hive Issue Type: Improvement Components: Build Infrastructure, HCatalog Reporter: Travis Crawford Both Hive and HCatalog use ant as their build tool. However, Hive uses ivy for dependency resolution while HCatalog uses maven-ant-tasks. With the project merge we should converge on a single tool for dependency resolution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4369) Many new failures on hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-4369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634680#comment-13634680 ] Vikram Dixit K commented on HIVE-4369: -- Review request: https://reviews.apache.org/r/10597/ Many new failures on hadoop 2 - Key: HIVE-4369 URL: https://issues.apache.org/jira/browse/HIVE-4369 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Vikram Dixit K Roughly half the tests are failing, this seems to be the exception: [junit] org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: Configuration and input path are inconsistent [junit] at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:522) [junit] at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:91) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103) [junit] at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72) [junit] at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) [junit] at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103) [junit] at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72) [junit] at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) [junit] at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395) [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:232) [junit] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) [junit] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [junit] at java.util.concurrent.FutureTask.run(FutureTask.java:138) [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [junit] at java.lang.Thread.run(Thread.java:680) [junit] Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Configuration and input path are inconsistent [junit] at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:516) [junit] ... 25 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4369) Many new failures on hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-4369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-4369: - Attachment: HIVE-4369.1.patch Bumping up the hadoop version fixes this issue. Running the full suite of tests but do not anticipate any more patch updates. Many new failures on hadoop 2 - Key: HIVE-4369 URL: https://issues.apache.org/jira/browse/HIVE-4369 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Vikram Dixit K Attachments: HIVE-4369.1.patch Roughly half the tests are failing, this seems to be the exception: [junit] org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: Configuration and input path are inconsistent [junit] at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:522) [junit] at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:91) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103) [junit] at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72) [junit] at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) [junit] at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103) [junit] at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72) [junit] at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) [junit] at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395) [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:232) [junit] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) [junit] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [junit] at java.util.concurrent.FutureTask.run(FutureTask.java:138) [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [junit] at java.lang.Thread.run(Thread.java:680) [junit] Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Configuration and input path are inconsistent [junit] at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:516) [junit] ... 25 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4369) Many new failures on hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-4369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-4369: - Status: Patch Available (was: Open) Many new failures on hadoop 2 - Key: HIVE-4369 URL: https://issues.apache.org/jira/browse/HIVE-4369 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Vikram Dixit K Attachments: HIVE-4369.1.patch Roughly half the tests are failing, this seems to be the exception: [junit] org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: Configuration and input path are inconsistent [junit] at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:522) [junit] at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:91) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103) [junit] at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72) [junit] at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) [junit] at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103) [junit] at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72) [junit] at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) [junit] at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395) [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:232) [junit] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) [junit] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [junit] at java.util.concurrent.FutureTask.run(FutureTask.java:138) [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [junit] at java.lang.Thread.run(Thread.java:680) [junit] Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Configuration and input path are inconsistent [junit] at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:516) [junit] ... 25 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4284) Implement class for vectorized row batch
[ https://issues.apache.org/jira/browse/HIVE-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Hanson updated HIVE-4284: -- Attachment: HIVE-4284.4.patch Updated based on code review comments. Implement class for vectorized row batch Key: HIVE-4284 URL: https://issues.apache.org/jira/browse/HIVE-4284 Project: Hive Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Eric Hanson Attachments: HIVE-4284.3.patch, HIVE-4284.4.patch Vectorized row batch object will represent the row batch that vectorized operators will work on. Refer to design spec attached to HIVE-4160 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: New code for VectorizedRowBatch to form basis of vectorized query execution
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10592/ --- (Updated April 18, 2013, 1:27 a.m.) Review request for hive. Changes --- Updated patch based on previous code review feedback (see detailed response below) Description --- New code for VectorizedRowBatch to form basis of vectorized query execution This addresses bug HIVE-4284. https://issues.apache.org/jira/browse/HIVE-4284 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizedRowBatch.java PRE-CREATION Diff: https://reviews.apache.org/r/10592/diff/ Testing --- Thanks, Eric Hanson
Re: Review Request: New code for VectorizedRowBatch to form basis of vectorized query execution
On April 17, 2013, 9:47 p.m., Carl Steinbach wrote: ql/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java, line 33 https://reviews.apache.org/r/10592/diff/1/?file=282042#file282042line33 Is there a reason these variables are public? The major fields of the column vector and vectorized row batch types are public for fast access in the inner loop of query execution. This is by design. This is described in code comments. On April 17, 2013, 9:47 p.m., Carl Steinbach wrote: ql/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java, line 33 https://reviews.apache.org/r/10592/diff/1/?file=282043#file282043line33 Public? see previous comment re: public fields On April 17, 2013, 9:47 p.m., Carl Steinbach wrote: ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java, line 36 https://reviews.apache.org/r/10592/diff/1/?file=282044#file282044line36 Public? see previous comment re: public fields On April 17, 2013, 9:47 p.m., Carl Steinbach wrote: ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java, line 30 https://reviews.apache.org/r/10592/diff/1/?file=282041#file282041line30 Should these methods be abstract? If not, then I think you want to throw an UnsupportedOperationException(Not Implemented) instead of doing an assert or returning a string. methods deleted - Eric --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10592/#review19350 --- On April 18, 2013, 1:27 a.m., Eric Hanson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10592/ --- (Updated April 18, 2013, 1:27 a.m.) Review request for hive. Description --- New code for VectorizedRowBatch to form basis of vectorized query execution This addresses bug HIVE-4284. https://issues.apache.org/jira/browse/HIVE-4284 Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizedRowBatch.java PRE-CREATION Diff: https://reviews.apache.org/r/10592/diff/ Testing --- Thanks, Eric Hanson
[jira] [Commented] (HIVE-4284) Implement class for vectorized row batch
[ https://issues.apache.org/jira/browse/HIVE-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634742#comment-13634742 ] Eric Hanson commented on HIVE-4284: --- Uploaded new diff to review tool at https://reviews.apache.org/r/10592/ Implement class for vectorized row batch Key: HIVE-4284 URL: https://issues.apache.org/jira/browse/HIVE-4284 Project: Hive Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Eric Hanson Attachments: HIVE-4284.3.patch, HIVE-4284.4.patch Vectorized row batch object will represent the row batch that vectorized operators will work on. Refer to design spec attached to HIVE-4160 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4371) some issue with merging join trees
[ https://issues.apache.org/jira/browse/HIVE-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-4371: -- Attachment: HIVE-4371.D10323.1.patch navis requested code review of HIVE-4371 [jira] some issue with merging join trees. Reviewers: JIRA HIVE-4371 some issue with merging join trees Navis, I would really appreciate if you can take a look. I am attaching a testcase, for which in the optimizer the join context left aliases and right aliases do not look correct. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D10323 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/optimizer/AvgPartitionSizeBasedBigTableSelectorForAutoSMJ.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/SizeBasedBigTableSelectorForAutoSMJ.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/TableSizeBasedBigTableSelectorForAutoSMJ.java MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/24753/ To: JIRA, navis some issue with merging join trees -- Key: HIVE-4371 URL: https://issues.apache.org/jira/browse/HIVE-4371 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0 Reporter: Namit Jain Assignee: Navis Attachments: HIVE-4371.D10323.1.patch [~navis], I would really appreciate if you can take a look. I am attaching a testcase, for which in the optimizer the join context left aliases and right aliases do not look correct. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4371) some issue with merging join trees
[ https://issues.apache.org/jira/browse/HIVE-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634800#comment-13634800 ] Navis commented on HIVE-4371: - [~namit] I've not been following SMB/BMJoins for months and afraid I could be on the road. I've attached a small patch excluding aliases belonging to other join, which makes the test case run. (but not verified the result) And in the test query, select count(*) FROM bucket_small a JOIN bucket_medium b ON a.key = b.key JOIN bucket_big c ON c.key = b.key JOIN bucket_medium d ON c.key = b.key; the last JOIN bucket_medium d ON c.key = b.key means cross join? some issue with merging join trees -- Key: HIVE-4371 URL: https://issues.apache.org/jira/browse/HIVE-4371 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0 Reporter: Namit Jain Assignee: Navis Attachments: HIVE-4371.D10323.1.patch [~navis], I would really appreciate if you can take a look. I am attaching a testcase, for which in the optimizer the join context left aliases and right aliases do not look correct. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4372) When trying to populate an external HBase table using a hive query involving joins, the data is incorrectly getting mixed up inside the rows.
[ https://issues.apache.org/jira/browse/HIVE-4372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634802#comment-13634802 ] Navis commented on HIVE-4372: - [~yashaswy] Could you describe the problem in detail? When trying to populate an external HBase table using a hive query involving joins, the data is incorrectly getting mixed up inside the rows. -- Key: HIVE-4372 URL: https://issues.apache.org/jira/browse/HIVE-4372 Project: Hive Issue Type: Bug Components: HBase Handler Affects Versions: 0.10.0 Reporter: Yashaswy Andavilli -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2019) Implement NOW() UDF
[ https://issues.apache.org/jira/browse/HIVE-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634805#comment-13634805 ] Navis commented on HIVE-2019: - Agreed. NOW() should be replaced with a constant right before executing the query. It's non-deterministic in compile time but can be regarded as deterministic in runtime. Implement NOW() UDF --- Key: HIVE-2019 URL: https://issues.apache.org/jira/browse/HIVE-2019 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Priyadarshini Attachments: HIVE-2019.patch Reference: http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_now -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4299) exported metadata by HIVE-3068 cannot be imported because of wrong file name
[
https://issues.apache.org/jira/browse/HIVE-4299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634811#comment-13634811
]
Sho Shimauchi commented on HIVE-4299:
-
Hi [~ashutoshc],
Sorry for late.
I'm trying to write a testcase, but it's a bit hard to add a case for this
feature.
When a table is dropped, this feature will dump a metadata in a directory which
contains current datetime like
/user/sho/test_table.2013-04-05-20-13-14/test_table.metadata .
I can't write IMPORT FROM query against the dynamic path.
Do you have any idea to solve this problem?
Thanks,
Sho
exported metadata by HIVE-3068 cannot be imported because of wrong file name
Key: HIVE-4299
URL: https://issues.apache.org/jira/browse/HIVE-4299
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.11.0
Reporter: Sho Shimauchi
Assignee: Sho Shimauchi
Attachments: HIVE-4299.patch
h2. Symptom
When DROP TABLE a table, metadata of the table is generated to be able to
import the dropped table again.
However, the exported metadata name is 'table name.metadata'.
Since ImportSemanticAnalyzer allows only '_metadata' as metadata filename,
user have to rename the metadata file to import the table.
h2. How to reproduce
Set the following setting to hive-site.xml:
{code}
property
namehive.metastore.pre.event.listeners/name
valueorg.apache.hadoop.hive.ql.parse.MetaDataExportListener/value
/property
{code}
Then run the following queries:
{code}
CREATE TABLE test_table (id INT, name STRING);
DROP TABLE test_table;
IMPORT TABLE test_table_imported FROM '/path/to/metadata/file';
FAILED: SemanticException [Error 10027]: Invalid path
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4280) TestRetryingHMSHandler is failing on trunk.
[ https://issues.apache.org/jira/browse/HIVE-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634838#comment-13634838 ] Teddy Choi commented on HIVE-4280: -- You're welcome, [~ashutoshc]. Thank you for committing. TestRetryingHMSHandler is failing on trunk. --- Key: HIVE-4280 URL: https://issues.apache.org/jira/browse/HIVE-4280 Project: Hive Issue Type: Bug Affects Versions: 0.11.0 Reporter: Ashutosh Chauhan Assignee: Teddy Choi Fix For: 0.11.0 Attachments: HIVE-4280-1.patch.txt, HIVE-4280-2.patch.txt Newly added testcase TestRetryingHMSHandler fails on trunk. https://builds.apache.org/job/Hive-trunk-h0.21/2040/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4375) Single sourced multi insert consists of native and non-native table mixed throws NPE
Navis created HIVE-4375:
---
Summary: Single sourced multi insert consists of native and
non-native table mixed throws NPE
Key: HIVE-4375
URL: https://issues.apache.org/jira/browse/HIVE-4375
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
CREATE TABLE src_x1(key string, value string);
CREATE TABLE src_x2(key string, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (hbase.columns.mapping = :key,cf:string);
explain
from src a
insert overwrite table src_x1
select key,value where a.key 0 AND a.key 50
insert overwrite table src_x2
select key,value where a.key 50 AND a.key 100;
throws,
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.optimizer.GenMRFileSink1.addStatsTask(GenMRFileSink1.java:236)
at
org.apache.hadoop.hive.ql.optimizer.GenMRFileSink1.process(GenMRFileSink1.java:126)
at
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:87)
at
org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:55)
at
org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:67)
at
org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:67)
at
org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:67)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:101)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genMapRedTasks(SemanticAnalyzer.java:8354)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8759)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:279)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:433)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4375) Single sourced multi insert consists of native and non-native table mixed throws NPE
[
https://issues.apache.org/jira/browse/HIVE-4375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-4375:
Status: Patch Available (was: Open)
Single sourced multi insert consists of native and non-native table mixed
throws NPE
Key: HIVE-4375
URL: https://issues.apache.org/jira/browse/HIVE-4375
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-4375.D10329.1.patch
CREATE TABLE src_x1(key string, value string);
CREATE TABLE src_x2(key string, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (hbase.columns.mapping = :key,cf:string);
explain
from src a
insert overwrite table src_x1
select key,value where a.key 0 AND a.key 50
insert overwrite table src_x2
select key,value where a.key 50 AND a.key 100;
throws,
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.optimizer.GenMRFileSink1.addStatsTask(GenMRFileSink1.java:236)
at
org.apache.hadoop.hive.ql.optimizer.GenMRFileSink1.process(GenMRFileSink1.java:126)
at
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:87)
at
org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:55)
at
org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:67)
at
org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:67)
at
org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:67)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:101)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genMapRedTasks(SemanticAnalyzer.java:8354)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8759)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:279)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:433)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4375) Single sourced multi insert consists of native and non-native table mixed throws NPE
[
https://issues.apache.org/jira/browse/HIVE-4375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-4375:
--
Attachment: HIVE-4375.D10329.1.patch
navis requested code review of HIVE-4375 [jira] Single sourced multi insert
consists of native and non-native table mixed throws NPE.
Reviewers: JIRA
HIVE-4375 Single sourced multi insert consists of native and non-native table
mixed throws NPE
CREATE TABLE src_x1(key string, value string);
CREATE TABLE src_x2(key string, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (hbase.columns.mapping = :key,cf:string);
explain
from src a
insert overwrite table src_x1
select key,value where a.key 0 AND a.key 50
insert overwrite table src_x2
select key,value where a.key 50 AND a.key 100;
throws,
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.optimizer.GenMRFileSink1.addStatsTask(GenMRFileSink1.java:236)
at
org.apache.hadoop.hive.ql.optimizer.GenMRFileSink1.process(GenMRFileSink1.java:126)
at
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:87)
at
org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:55)
at
org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:67)
at
org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:67)
at
org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:67)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:101)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genMapRedTasks(SemanticAnalyzer.java:8354)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8759)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:279)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:433)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D10329
AFFECTED FILES
hbase-handler/src/test/queries/positive/hbase_single_sorced_multi_insert.q
hbase-handler/src/test/results/positive/hbase_single_sorced_multi_insert.q.out
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java
MANAGE HERALD RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/24759/
To: JIRA, navis
Single sourced multi insert consists of native and non-native table mixed
throws NPE
Key: HIVE-4375
URL: https://issues.apache.org/jira/browse/HIVE-4375
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-4375.D10329.1.patch
CREATE TABLE src_x1(key string, value string);
CREATE TABLE src_x2(key string, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (hbase.columns.mapping = :key,cf:string);
explain
from src a
insert overwrite table src_x1
select key,value where a.key 0 AND a.key 50
insert overwrite table src_x2
select key,value where a.key 50 AND a.key 100;
throws,
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.optimizer.GenMRFileSink1.addStatsTask(GenMRFileSink1.java:236)
at
org.apache.hadoop.hive.ql.optimizer.GenMRFileSink1.process(GenMRFileSink1.java:126)
at
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:87)
at
