Build failed in Hudson: Hive-trunk-h0.20 #588
See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/588/changes Changes: [pauly] HIVE-1941 support explicit view partitioning (John Sichi via pauly) -- [...truncated 27264 lines...] [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select count(1) as cnt from testhivedrivertable [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-03-02_02-16-58_088_9062067489216058392/-mr-1 [junit] Total MapReduce jobs = 1 [junit] Launching Job 1 out of 1 [junit] Number of reduce tasks determined at compile time: 1 [junit] In order to change the average load for a reducer (in bytes): [junit] set hive.exec.reducers.bytes.per.reducer=number [junit] In order to limit the maximum number of reducers: [junit] set hive.exec.reducers.max=number [junit] In order to set a constant number of reducers: [junit] set mapred.reduce.tasks=number [junit] Job running in-process (local Hadoop) [junit] 2011-03-02 02:17:01,163 null map = 100%, reduce = 100% [junit] Ended Job = job_local_0001 [junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-03-02_02-16-58_088_9062067489216058392/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/service/tmp/hive_job_log_hudson_201103020217_1944860128.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select * from testhivedrivertable limit 10 [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-03-02_02-17-02_648_1750938048883260933/-mr-1 [junit] POSTHOOK: query: select * from testhivedrivertable limit 10 [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-03-02_02-17-02_648_1750938048883260933/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit]
[jira] Commented: (HIVE-2022) Making JDO thread-safe by default
[ https://issues.apache.org/jira/browse/HIVE-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001462#comment-13001462 ] Mac Yang commented on HIVE-2022: Will this be included in the 0.7 release? Making JDO thread-safe by default - Key: HIVE-2022 URL: https://issues.apache.org/jira/browse/HIVE-2022 Project: Hive Issue Type: Bug Reporter: Ning Zhang Assignee: Ning Zhang Attachments: HIVE-2022.patch If there are multiple thread accessing metastore concurrently, there are cases that JDO threw exceptions because of concurrent access of HashMap inside JDO. Setting javax.jdo.option.Multithreaded to true solves this issue. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.7.0-h0.20 #23
See https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/23/ -- Started by timer Building remotely on ubuntu1 Checking out http://svn.apache.org/repos/asf/hive/branches/branch-0.7 A eclipse-templates SCM check out aborted Recording test results
[jira] Commented: (HIVE-1644) use filter pushdown for automatically accessing indexes
[ https://issues.apache.org/jira/browse/HIVE-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001584#comment-13001584 ] He Yongqiang commented on HIVE-1644: Take SkewJoinResolver as an example, in its resolve method, it adds all root tasks to be iterated by the optimizer. (topNodes.addAll(pctx.rootTasks);) Adding all root task should be good now, but you can add all tasks, and in the second step, look at the table scan operator in the current task, if all table scan ops are not top table scan ops, then skip this task. And in the dispatcher, the dispatch is in process of current task. It creates a rule R1 ( the same optimizer coder you have now.) And adds the reducer operator tree to iterate (you may want to add the mapper operator tree.). Please let me know if you have any questions. use filter pushdown for automatically accessing indexes --- Key: HIVE-1644 URL: https://issues.apache.org/jira/browse/HIVE-1644 Project: Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: Russell Melick Attachments: HIVE-1644.1.patch, HIVE-1644.2.patch, HIVE-1644.3.patch, HIVE-1644.4.patch HIVE-1226 provides utilities for analyzing filters which have been pushed down to a table scan. The next step is to use these for selecting available indexes and generating access plans for those indexes. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (HIVE-1694) Accelerate GROUP BY execution using indexes
[ https://issues.apache.org/jira/browse/HIVE-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi reassigned HIVE-1694: Assignee: Prajakta Kalmegh (was: Nikhil Deshpande) Accelerate GROUP BY execution using indexes --- Key: HIVE-1694 URL: https://issues.apache.org/jira/browse/HIVE-1694 Project: Hive Issue Type: New Feature Components: Indexing, Query Processor Affects Versions: 0.7.0 Reporter: Nikhil Deshpande Assignee: Prajakta Kalmegh Attachments: HIVE-1694.1.patch.txt, HIVE-1694_2010-10-28.diff, demo_q1.hql, demo_q2.hql The index building patch (Hive-417) is checked into trunk, this JIRA issue tracks supporting indexes in Hive compiler execution engine for SELECT queries. This is in ref. to John's comment at https://issues.apache.org/jira/browse/HIVE-417?focusedCommentId=12884869page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12884869 on creating separate JIRA issue for tracking index usage in optimizer query execution. The aim of this effort is to use indexes to accelerate query execution (for certain class of queries). E.g. - Filters and range scans (already being worked on by He Yongqiang as part of HIVE-417?) - Joins (index based joins) - Group By, Order By and other misc cases The proposal is multi-step: 1. Building index based operators, compiler and execution engine changes 2. Optimizer enhancements (e.g. cost-based optimizer to compare and choose between index scans, full table scans etc.) This JIRA initially focuses on the first step. This JIRA is expected to hold the information about index based plans operator implementations for above mentioned cases. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1644) use filter pushdown for automatically accessing indexes
[ https://issues.apache.org/jira/browse/HIVE-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001618#comment-13001618 ] John Sichi commented on HIVE-1644: -- Yongqiang, thanks for the pointer. Guys, give that a try. My original thinking for doing it during logical optimization was that it's similar to the storage handler logic I had added previously, but if you can get it working here, physical optimization makes sense as the place for indexing. use filter pushdown for automatically accessing indexes --- Key: HIVE-1644 URL: https://issues.apache.org/jira/browse/HIVE-1644 Project: Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: Russell Melick Attachments: HIVE-1644.1.patch, HIVE-1644.2.patch, HIVE-1644.3.patch, HIVE-1644.4.patch HIVE-1226 provides utilities for analyzing filters which have been pushed down to a table scan. The next step is to use these for selecting available indexes and generating access plans for those indexes. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1644) use filter pushdown for automatically accessing indexes
[ https://issues.apache.org/jira/browse/HIVE-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001565#comment-13001565 ] He Yongqiang commented on HIVE-1644: take a look at one physical optimizer, it is pretty straightforward. I think the entire index optimization can be moves there (no big changes needed). use filter pushdown for automatically accessing indexes --- Key: HIVE-1644 URL: https://issues.apache.org/jira/browse/HIVE-1644 Project: Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: Russell Melick Attachments: HIVE-1644.1.patch, HIVE-1644.2.patch, HIVE-1644.3.patch, HIVE-1644.4.patch HIVE-1226 provides utilities for analyzing filters which have been pushed down to a table scan. The next step is to use these for selecting available indexes and generating access plans for those indexes. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2022) Making JDO thread-safe by default
[ https://issues.apache.org/jira/browse/HIVE-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-2022: Resolution: Fixed Fix Version/s: 0.8.0 Status: Resolved (was: Patch Available) Committed. Thanks Ning! Making JDO thread-safe by default - Key: HIVE-2022 URL: https://issues.apache.org/jira/browse/HIVE-2022 Project: Hive Issue Type: Bug Reporter: Ning Zhang Assignee: Ning Zhang Fix For: 0.8.0 Attachments: HIVE-2022.patch If there are multiple thread accessing metastore concurrently, there are cases that JDO threw exceptions because of concurrent access of HashMap inside JDO. Setting javax.jdo.option.Multithreaded to true solves this issue. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-2022) Making JDO thread-safe by default
[ https://issues.apache.org/jira/browse/HIVE-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001666#comment-13001666 ] Paul Yang commented on HIVE-2022: - @Mac - sounds like a good idea. I'll backport to 0.7. Making JDO thread-safe by default - Key: HIVE-2022 URL: https://issues.apache.org/jira/browse/HIVE-2022 Project: Hive Issue Type: Bug Reporter: Ning Zhang Assignee: Ning Zhang Fix For: 0.8.0 Attachments: HIVE-2022.patch If there are multiple thread accessing metastore concurrently, there are cases that JDO threw exceptions because of concurrent access of HashMap inside JDO. Setting javax.jdo.option.Multithreaded to true solves this issue. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-2022) Making JDO thread-safe by default
[ https://issues.apache.org/jira/browse/HIVE-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001686#comment-13001686 ] Mac Yang commented on HIVE-2022: Thanks Paul! Making JDO thread-safe by default - Key: HIVE-2022 URL: https://issues.apache.org/jira/browse/HIVE-2022 Project: Hive Issue Type: Bug Reporter: Ning Zhang Assignee: Ning Zhang Fix For: 0.8.0 Attachments: HIVE-2022.patch If there are multiple thread accessing metastore concurrently, there are cases that JDO threw exceptions because of concurrent access of HashMap inside JDO. Setting javax.jdo.option.Multithreaded to true solves this issue. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HIVE-2023) Add javax.jdo.option.Multithreaded configuration property to HiveConf
Add javax.jdo.option.Multithreaded configuration property to HiveConf - Key: HIVE-2023 URL: https://issues.apache.org/jira/browse/HIVE-2023 Project: Hive Issue Type: Bug Components: Configuration, Metastore Reporter: Carl Steinbach Assignee: Ning Zhang The configuration property javax.jdo.option.Multithreaded was added to hive-default.xml in HIVE-2022. This property also needs to be added to HiveConf.java. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2022) Making JDO thread-safe by default
[ https://issues.apache.org/jira/browse/HIVE-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2022: - Component/s: Metastore Configuration Making JDO thread-safe by default - Key: HIVE-2022 URL: https://issues.apache.org/jira/browse/HIVE-2022 Project: Hive Issue Type: Bug Components: Configuration, Metastore Reporter: Ning Zhang Assignee: Ning Zhang Fix For: 0.8.0 Attachments: HIVE-2022.patch If there are multiple thread accessing metastore concurrently, there are cases that JDO threw exceptions because of concurrent access of HashMap inside JDO. Setting javax.jdo.option.Multithreaded to true solves this issue. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HIVE-2024) In Driver.execute(), mapred.job.tracker is not restored if one of the task fails.
In Driver.execute(), mapred.job.tracker is not restored if one of the task fails. - Key: HIVE-2024 URL: https://issues.apache.org/jira/browse/HIVE-2024 Project: Hive Issue Type: Bug Reporter: Siying Dong If automatically one job is determined to run in local mode, and the task fails with error code not 0, mapred.job.tracker will remain to be local and might cause further problems. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-trunk-h0.20 #590
See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/590/changes Changes: [pauly] HIVE-2022 Making JDO thread-safe by default (Ning Zhang via pauly) -- [...truncated 27597 lines...] [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select count(1) as cnt from testhivedrivertable [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-03-02_17-50-22_680_8771498428565594347/-mr-1 [junit] Total MapReduce jobs = 1 [junit] Launching Job 1 out of 1 [junit] Number of reduce tasks determined at compile time: 1 [junit] In order to change the average load for a reducer (in bytes): [junit] set hive.exec.reducers.bytes.per.reducer=number [junit] In order to limit the maximum number of reducers: [junit] set hive.exec.reducers.max=number [junit] In order to set a constant number of reducers: [junit] set mapred.reduce.tasks=number [junit] Job running in-process (local Hadoop) [junit] 2011-03-02 17:50:25,726 null map = 100%, reduce = 100% [junit] Ended Job = job_local_0001 [junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-03-02_17-50-22_680_8771498428565594347/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/service/tmp/hive_job_log_hudson_201103021750_1018263334.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select * from testhivedrivertable limit 10 [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-03-02_17-50-27_212_5281354341203258090/-mr-1 [junit] POSTHOOK: query: select * from testhivedrivertable limit 10 [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-03-02_17-50-27_212_5281354341203258090/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit]
[jira] Commented: (HIVE-1902) create script for the metastore upgrade due to HIVE-78
[
https://issues.apache.org/jira/browse/HIVE-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001829#comment-13001829
]
Shrijeet Paliwal commented on HIVE-1902:
Stumbled upon one issue while running meta store upgrade script. Steps to
reproduce:
1. Have hive 0.6 metastore with no partitioned tables.
2. Run meta store script upgrade-0.7.0.mysql.sql
Get an error :
{noformat}
ERROR 1005 (HY000) at line 64: Can't create table
'hive_meta_data.PART_COL_PRIVS' (errno: 150)
{noformat}
I think the table 'PARTITIONS' is created when you have your first partition
registered. Hence following action while creating PART_COL_PRIVS fails:
{code}
CONSTRAINT `PART_COL_PRIVS_FK1` FOREIGN KEY (`PART_ID`) REFERENCES `PARTITIONS`
(`PART_ID`)
{code}
Another (minor) issue is with following entry in script :
{code}
ALTER TABLE `DBS` DROP COLUMN `PARAMETERS`;
{code}
The DBS table has not column PARAMETERS (at-least in 0.6).
create script for the metastore upgrade due to HIVE-78
--
Key: HIVE-1902
URL: https://issues.apache.org/jira/browse/HIVE-1902
Project: Hive
Issue Type: Bug
Components: Metastore
Reporter: Namit Jain
Assignee: Arvind Prabhakar
Priority: Blocker
Fix For: 0.7.0
Attachments: HIVE-1902.1.patch.txt
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-2022) Making JDO thread-safe by default
[ https://issues.apache.org/jira/browse/HIVE-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001857#comment-13001857 ] Carl Steinbach commented on HIVE-2022: -- This patch broke TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore: https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/590/ @Paul: Please don't backport this to 0.7.0 Making JDO thread-safe by default - Key: HIVE-2022 URL: https://issues.apache.org/jira/browse/HIVE-2022 Project: Hive Issue Type: Bug Components: Configuration, Metastore Reporter: Ning Zhang Assignee: Ning Zhang Fix For: 0.8.0 Attachments: HIVE-2022.patch If there are multiple thread accessing metastore concurrently, there are cases that JDO threw exceptions because of concurrent access of HashMap inside JDO. Setting javax.jdo.option.Multithreaded to true solves this issue. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HIVE-2025) Fix TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore broken by HIVE-2022
Fix TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore broken by HIVE-2022 - Key: HIVE-2025 URL: https://issues.apache.org/jira/browse/HIVE-2025 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Paul Yang Priority: Critical The patch for HIVE-2022 broke TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/590/ @Paul: Assigning this to you. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-2025) Fix TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore broken by HIVE-2022
[ https://issues.apache.org/jira/browse/HIVE-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001881#comment-13001881 ] Ning Zhang commented on HIVE-2025: -- This is due to a bug in ObjectStore. I'll upload a patch soon. Fix TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore broken by HIVE-2022 - Key: HIVE-2025 URL: https://issues.apache.org/jira/browse/HIVE-2025 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Paul Yang Priority: Critical The patch for HIVE-2022 broke TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/590/ @Paul: Assigning this to you. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (HIVE-2025) Fix TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore broken by HIVE-2022
[ https://issues.apache.org/jira/browse/HIVE-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang reassigned HIVE-2025: Assignee: Ning Zhang (was: Paul Yang) Fix TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore broken by HIVE-2022 - Key: HIVE-2025 URL: https://issues.apache.org/jira/browse/HIVE-2025 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Ning Zhang Priority: Critical The patch for HIVE-2022 broke TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/590/ @Paul: Assigning this to you. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2025) Fix TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore broken by HIVE-2022
[ https://issues.apache.org/jira/browse/HIVE-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-2025: - Status: Patch Available (was: Open) Fix TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore broken by HIVE-2022 - Key: HIVE-2025 URL: https://issues.apache.org/jira/browse/HIVE-2025 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Ning Zhang Priority: Critical Attachments: HIVE-2025.patch The patch for HIVE-2022 broke TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/590/ @Paul: Assigning this to you. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2025) Fix TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore broken by HIVE-2022
[ https://issues.apache.org/jira/browse/HIVE-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-2025: - Attachment: HIVE-2025.patch null-checking before delete persistent obj in ObjectStore.dropType(). Also added javax.jdo.option.Embedded to HiveConf.java (HIVE-2022). Fix TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore broken by HIVE-2022 - Key: HIVE-2025 URL: https://issues.apache.org/jira/browse/HIVE-2025 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Ning Zhang Priority: Critical Attachments: HIVE-2025.patch The patch for HIVE-2022 broke TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/590/ @Paul: Assigning this to you. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1490) More implicit type conversion: UNION ALL and COALESCE
[ https://issues.apache.org/jira/browse/HIVE-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001903#comment-13001903 ] Adam Kramer commented on HIVE-1490: --- Also present in CASE statements: FAILED: Error in semantic analysis: line 37:57 Argument Type Mismatch '$.foo': The expressions after THEN should have the same type: int is expected but string is found More implicit type conversion: UNION ALL and COALESCE - Key: HIVE-1490 URL: https://issues.apache.org/jira/browse/HIVE-1490 Project: Hive Issue Type: Bug Components: Query Processor, Server Infrastructure Reporter: Adam Kramer This is a usecase that frequently annoys me: SELECT TRANSFORM(stuff) USING 'script' AS thing1, thing2 FROM some_table UNION ALL SELECT a.thing1, a.thing2 FROM some_other_table a ...this fails when a.thing1 and a.thing2 are anything but STRING, because all output of TRANSFORM is STRING. In this case, a.thing1 and a.thing2 should be implicitly converted to string. COALESCE(a.thing1, a.thing2, a.thing3) should similarly do implicit type conversion among the arguments. If two are INT and one is BIGINT, upgrade the INTs, etc. At the very least, it would be nice to have syntax like SELECT TRANSFORM(stuff) USING 'script' AS thing1 INT, thing2 INT ...which would effectively cast the output column to the specified type. But really, type conversion should work. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HIVE-2026) Parallelize UpdateInputAccessTimeHook
Parallelize UpdateInputAccessTimeHook - Key: HIVE-2026 URL: https://issues.apache.org/jira/browse/HIVE-2026 Project: Hive Issue Type: Improvement Reporter: Ning Zhang Assignee: Ning Zhang UpdateInputAccessTimeHook is usually used as a pre-execution hook to update the metastore's lastAccessTime field of input partition/table. If a query touches a large number of partitions, this hooks takes a long time to execute. One approach is to make the hook itself to run in a separate thread. But it is hard to guarantee backward compatibility in semantics in case of exceptions encountered in the hook execution. This task takes another approach to parallelize the hook itself (update multiple partitions concurrently), but execute each pre-hook in sequential order. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
