[jira] Created: (HIVE-2029) MetaStore ConnectionURL updates need to trigger creation of Default DB if it doesn't exist
MetaStore ConnectionURL updates need to trigger creation of Default DB if it doesn't exist -- Key: HIVE-2029 URL: https://issues.apache.org/jira/browse/HIVE-2029 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.7.0 Reporter: Carl Steinbach HIVE-1219 defined the JDOConnectionURLHook plugin, and integrated this feature into HiveMetaStore. On MetaStore operation failures, this plugin is used to update the metastore ConnectionURL configuration property. Currently this update triggers the reinitialization of the underlying JDO PersistenceManager, but it does not trigger checks to see if the default database exists, nor will it create the default database if it does not exist. It needs to do both. This ticket also covers removing the 'hive.metastore.force.reload.conf' property from HiveConf and HiveMetaStore. This property should not have been added in the first place since its sole purpose is to facilitate testing of the JDOConnectionURLHook mechanism by unnaturally forcing reinitialization of the PersistenceManager. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: HIVE-1939: Fix test failure in TestContribCliDriver/url_hook.q
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/469/ --- Review request for hive. Summary --- Review request for HIVE-1939. This addresses bug HIVE-1939. https://issues.apache.org/jira/browse/HIVE-1939 Diffs - contrib/src/test/queries/clientnegative/url_hook.q PRE-CREATION contrib/src/test/queries/clientpositive/url_hook.q d9ffd28 contrib/src/test/results/clientnegative/url_hook.q.out PRE-CREATION contrib/src/test/results/clientpositive/url_hook.q.out bd5afb4 Diff: https://reviews.apache.org/r/469/diff Testing --- Thanks, Carl
[jira] Updated: (HIVE-1939) Fix test failure in TestContribCliDriver/url_hook.q
[ https://issues.apache.org/jira/browse/HIVE-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1939: - Attachment: HIVE-1939.1.patch.txt Fix test failure in TestContribCliDriver/url_hook.q --- Key: HIVE-1939 URL: https://issues.apache.org/jira/browse/HIVE-1939 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Carl Steinbach Priority: Blocker Fix For: 0.7.0 Attachments: HIVE-1939.1.patch.txt -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1939) Fix test failure in TestContribCliDriver/url_hook.q
[ https://issues.apache.org/jira/browse/HIVE-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13003320#comment-13003320 ] Carl Steinbach commented on HIVE-1939: -- Review request: https://reviews.apache.org/r/469/ This patch moves the url_hook.q testcase from the contrib/positive directory to the contrib/negative directory. As I noted in my previous comment, this testcase should always have thrown an error, but until recently this fact was masked by another bug in the implementation of SHOW TABLES. That other bug was fixed in HIVE-1636, which consequently caused the error to start popping up in url_hook. I spent a lot of time trying to resolve the underlying problem (described in HIVE-2029), only to convince myself that this will require substantial changes to the HiveMetaStore class, which I think run the risk of destabilizing the branch. Consequently, I propose backporting this patch to the 0.7 branch in order to resolve the test failure, and then investigate and fix the real problem on trunk in HIVE-2029. Fix test failure in TestContribCliDriver/url_hook.q --- Key: HIVE-1939 URL: https://issues.apache.org/jira/browse/HIVE-1939 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Carl Steinbach Priority: Blocker Fix For: 0.7.0 Attachments: HIVE-1939.1.patch.txt -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1939) Fix test failure in TestContribCliDriver/url_hook.q
[ https://issues.apache.org/jira/browse/HIVE-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1939: - Status: Patch Available (was: Open) Fix test failure in TestContribCliDriver/url_hook.q --- Key: HIVE-1939 URL: https://issues.apache.org/jira/browse/HIVE-1939 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Carl Steinbach Priority: Blocker Fix For: 0.7.0 Attachments: HIVE-1939.1.patch.txt -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: HIVE-2011: upgrade-0.6.0.mysql.sql script attempts to increase size of PK COLUMNS.TYPE_NAME to 4000
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/471/ --- Review request for hive. Summary --- Review request for HIVE-2011 and HIVE-2010. This addresses bugs HIVE-2010 and HIVE-2011. https://issues.apache.org/jira/browse/HIVE-2010 https://issues.apache.org/jira/browse/HIVE-2011 Diffs - metastore/scripts/upgrade/derby/001-HIVE-972.derby.sql PRE-CREATION metastore/scripts/upgrade/derby/002-HIVE-1068.derby.sql PRE-CREATION metastore/scripts/upgrade/derby/003-HIVE-675.derby.sql PRE-CREATION metastore/scripts/upgrade/derby/004-HIVE-1364.derby.sql PRE-CREATION metastore/scripts/upgrade/derby/005-HIVE-417.derby.sql PRE-CREATION metastore/scripts/upgrade/derby/006-HIVE-1823.derby.sql PRE-CREATION metastore/scripts/upgrade/derby/007-HIVE-78.derby.sql PRE-CREATION metastore/scripts/upgrade/derby/README 506eb54 metastore/scripts/upgrade/derby/upgrade-0.5.0-to-0.6.0.derby.sql PRE-CREATION metastore/scripts/upgrade/derby/upgrade-0.6.0-to-0.7.0.derby.sql PRE-CREATION metastore/scripts/upgrade/derby/upgrade-0.6.0.derby.sql 77a0f39 metastore/scripts/upgrade/derby/upgrade-0.7.0.derby.sql ccad4e7 metastore/scripts/upgrade/mysql/001-HIVE-972.mysql.sql PRE-CREATION metastore/scripts/upgrade/mysql/002-HIVE-1068.mysql.sql PRE-CREATION metastore/scripts/upgrade/mysql/003-HIVE-675.mysql.sql PRE-CREATION metastore/scripts/upgrade/mysql/004-HIVE-1364.mysql.sql PRE-CREATION metastore/scripts/upgrade/mysql/005-HIVE-417.mysql.sql PRE-CREATION metastore/scripts/upgrade/mysql/006-HIVE-1823.mysql.sql PRE-CREATION metastore/scripts/upgrade/mysql/007-HIVE-78.mysql.sql PRE-CREATION metastore/scripts/upgrade/mysql/README 3d4360f metastore/scripts/upgrade/mysql/upgrade-0.5.0-to-0.6.0.mysql.sql PRE-CREATION metastore/scripts/upgrade/mysql/upgrade-0.6.0-to-0.7.0.mysql.sql PRE-CREATION metastore/scripts/upgrade/mysql/upgrade-0.6.0.mysql.sql 8cbd501 metastore/scripts/upgrade/mysql/upgrade-0.7.0.mysql.sql aab664e Diff: https://reviews.apache.org/r/471/diff Testing --- Thanks, Carl
[jira] Updated: (HIVE-2011) upgrade-0.6.0.mysql.sql script attempts to increase size of PK COLUMNS.TYPE_NAME to 4000
[
https://issues.apache.org/jira/browse/HIVE-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-2011:
-
Attachment: HIVE-2011.1.patch.txt
upgrade-0.6.0.mysql.sql script attempts to increase size of PK
COLUMNS.TYPE_NAME to 4000
Key: HIVE-2011
URL: https://issues.apache.org/jira/browse/HIVE-2011
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.6.0
Reporter: Carl Steinbach
Assignee: Carl Steinbach
Priority: Blocker
Fix For: 0.7.0
Attachments: HIVE-2011.1.patch.txt
{code}
# mysql flumenewresearch upgrade-0.6.0.mysql.sql
ERROR 1071 (42000) at line 16: Specified key was too long; max key length is
767 bytes
{code}
Here's the cause of the problem from upgrade-0.6.0.mysql.sql:
{code}
...
ALTER TABLE `COLUMNS` MODIFY `TYPE_NAME` VARCHAR(4000);
...
ALTER TABLE `COLUMNS` DROP PRIMARY KEY;
ALTER TABLE `COLUMNS` ADD PRIMARY KEY (`SD_ID`, `COLUMN_NAME`);
...
{code}
We need to make sure that the PK on COLUMNS.TYPE_NAME is dropped before the
size of the column is bumped to 4000.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2011) upgrade-0.6.0.mysql.sql script attempts to increase size of PK COLUMNS.TYPE_NAME to 4000
[
https://issues.apache.org/jira/browse/HIVE-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-2011:
-
Status: Patch Available (was: Open)
This is a patch for HIVE-2011 and HIVE-2010:
* Change the name of the upgrade scripts from 'upgrade-x.y.z.dbname.sql' to
'upgrade-x.y.z-to-a.b.c.dbname.sql'
* Splits the contents of the upgrade-x.y.z.dbname.sql files into individual
SQL files corresponding to each Hive ticket.
* Fixes HIVE-2011 by rearranging the order of DDL statements in
004-HIVE-1364.dbname.sql
upgrade-0.6.0.mysql.sql script attempts to increase size of PK
COLUMNS.TYPE_NAME to 4000
Key: HIVE-2011
URL: https://issues.apache.org/jira/browse/HIVE-2011
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.6.0
Reporter: Carl Steinbach
Assignee: Carl Steinbach
Priority: Blocker
Fix For: 0.7.0
Attachments: HIVE-2011.1.patch.txt
{code}
# mysql flumenewresearch upgrade-0.6.0.mysql.sql
ERROR 1071 (42000) at line 16: Specified key was too long; max key length is
767 bytes
{code}
Here's the cause of the problem from upgrade-0.6.0.mysql.sql:
{code}
...
ALTER TABLE `COLUMNS` MODIFY `TYPE_NAME` VARCHAR(4000);
...
ALTER TABLE `COLUMNS` DROP PRIMARY KEY;
ALTER TABLE `COLUMNS` ADD PRIMARY KEY (`SD_ID`, `COLUMN_NAME`);
...
{code}
We need to make sure that the PK on COLUMNS.TYPE_NAME is dropped before the
size of the column is bumped to 4000.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-2023) Add javax.jdo.option.Multithreaded configuration property to HiveConf
[ https://issues.apache.org/jira/browse/HIVE-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-2023: -- Add javax.jdo.option.Multithreaded configuration property to HiveConf - Key: HIVE-2023 URL: https://issues.apache.org/jira/browse/HIVE-2023 Project: Hive Issue Type: Bug Components: Configuration, Metastore Reporter: Carl Steinbach Assignee: Ning Zhang The configuration property javax.jdo.option.Multithreaded was added to hive-default.xml in HIVE-2022. This property also needs to be added to HiveConf.java. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-2023) Add javax.jdo.option.Multithreaded configuration property to HiveConf
[ https://issues.apache.org/jira/browse/HIVE-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-2023. -- Resolution: Duplicate Add javax.jdo.option.Multithreaded configuration property to HiveConf - Key: HIVE-2023 URL: https://issues.apache.org/jira/browse/HIVE-2023 Project: Hive Issue Type: Bug Components: Configuration, Metastore Reporter: Carl Steinbach Assignee: Ning Zhang The configuration property javax.jdo.option.Multithreaded was added to hive-default.xml in HIVE-2022. This property also needs to be added to HiveConf.java. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2018) avoid loading Hive aux jars in CLI remote mode
[ https://issues.apache.org/jira/browse/HIVE-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2018: - Component/s: CLI Fix Version/s: 0.8.0 avoid loading Hive aux jars in CLI remote mode -- Key: HIVE-2018 URL: https://issues.apache.org/jira/browse/HIVE-2018 Project: Hive Issue Type: Improvement Components: CLI Reporter: Ning Zhang Assignee: Ning Zhang Fix For: 0.8.0 Attachments: HIVE-2018.patch CLI load a number of jars (aux jars) including serde, antlr, metastore etc. These jars could be large and takes time to load when they are deployed to heavy loaded NFS mount points. In CLI remote mode, all these jars are not needed by the client side. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1611) Add alternative search-provider to Hive site
[ https://issues.apache.org/jira/browse/HIVE-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1611: - Fix Version/s: 0.7.0 Add alternative search-provider to Hive site Key: HIVE-1611 URL: https://issues.apache.org/jira/browse/HIVE-1611 Project: Hive Issue Type: Improvement Components: Website Reporter: Alex Baranau Assignee: Alex Baranau Priority: Minor Fix For: 0.7.0 Attachments: HIVE-1611.patch, HIVE-1611.patch Use search-hadoop.com service to make available search in Hive sources, MLs, wiki, etc. This was initially proposed on user mailing list. The search service was already added in site's skin (common for all Hadoop related projects) before so this issue is about enabling it for Hive. The ultimate goal is to use it at all Hadoop's sub-projects' sites. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1870) TestRemoteHiveMetaStore.java accidentally deleted during commit of HIVE-1845
[ https://issues.apache.org/jira/browse/HIVE-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1870: - Fix Version/s: 0.7.0 TestRemoteHiveMetaStore.java accidentally deleted during commit of HIVE-1845 Key: HIVE-1870 URL: https://issues.apache.org/jira/browse/HIVE-1870 Project: Hive Issue Type: Bug Components: Testing Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.7.0 Attachments: HIVE-1870.1.patch.txt TestRemoteHiveMetaStore.java was removed by the commit of HIVE-1845. This change was not part of the patch for HIVE-1845. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1678) NPE in MapJoin
[ https://issues.apache.org/jira/browse/HIVE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1678: - Fix Version/s: 0.7.0 NPE in MapJoin --- Key: HIVE-1678 URL: https://issues.apache.org/jira/browse/HIVE-1678 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.7.0 Attachments: patch-1678.txt The query with two map joins and a group by fails with following NPE: Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:177) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697) at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:464) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-1723) The result of left semi join is not correct
[ https://issues.apache.org/jira/browse/HIVE-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-1723: -- The result of left semi join is not correct --- Key: HIVE-1723 URL: https://issues.apache.org/jira/browse/HIVE-1723 Project: Hive Issue Type: Bug Reporter: Liyin Tang Assignee: Liyin Tang In the test case semijoin.q, there is a query: select /*+ mapjoin(b) */ a.key from t3 a left semi join t1 b on a.key = b.key sort by a.key; I think this query will return a wrong result if table t1 is larger than 25000 different keys To be simple, I tried a very similar query: select /*+ mapjoin(b) */ a.key from test_semijoin a left semi join test_semijoin b on a.key = b.key sort by a.key; The table of test_semijoin is like 0 0 1 1 2 2 3 3 4 4 5 5 ...... ... 25000 25000 25001 25001 ... ... 25999 25999 26000 26000 So we can easily estimate the correct result of this query should be the same keys from table test_semijoin itsel. Actually, the result is only part of that: only from 0 to 24544. 0 1 2 .. .. 24543 24544 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-1723) The result of left semi join is not correct
[ https://issues.apache.org/jira/browse/HIVE-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-1723. -- Resolution: Duplicate The result of left semi join is not correct --- Key: HIVE-1723 URL: https://issues.apache.org/jira/browse/HIVE-1723 Project: Hive Issue Type: Bug Reporter: Liyin Tang Assignee: Liyin Tang In the test case semijoin.q, there is a query: select /*+ mapjoin(b) */ a.key from t3 a left semi join t1 b on a.key = b.key sort by a.key; I think this query will return a wrong result if table t1 is larger than 25000 different keys To be simple, I tried a very similar query: select /*+ mapjoin(b) */ a.key from test_semijoin a left semi join test_semijoin b on a.key = b.key sort by a.key; The table of test_semijoin is like 0 0 1 1 2 2 3 3 4 4 5 5 ...... ... 25000 25000 25001 25001 ... ... 25999 25999 26000 26000 So we can easily estimate the correct result of this query should be the same keys from table test_semijoin itsel. Actually, the result is only part of that: only from 0 to 24544. 0 1 2 .. .. 24543 24544 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-1722) The result of the test case mapjoin1.q is not correct
[ https://issues.apache.org/jira/browse/HIVE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-1722. -- Resolution: Duplicate The result of the test case mapjoin1.q is not correct -- Key: HIVE-1722 URL: https://issues.apache.org/jira/browse/HIVE-1722 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.6.0, 0.7.0 Reporter: Liyin Tang Assignee: Liyin Tang In the test case mapjoin1.q : SELECT /*+ MAPJOIN(b) */ sum(a.key) as sum_a FROM srcpart a JOIN src b ON a.key = b.key where a.ds is not null; The current result in mapjoin1.q.out shows the result is 76260.0 But actually, if user remove the map join hint, and run the query: SELECT sum(a.key) as sum_a FROM srcpart a JOIN src b ON a.key = b.key where a.ds is not null; The result is 1114788.0 And I import these input data into mysql to test, and test result is also 1114788.0. Obviously, the current result is not correct -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-1722) The result of the test case mapjoin1.q is not correct
[ https://issues.apache.org/jira/browse/HIVE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-1722: -- The result of the test case mapjoin1.q is not correct -- Key: HIVE-1722 URL: https://issues.apache.org/jira/browse/HIVE-1722 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.6.0, 0.7.0 Reporter: Liyin Tang Assignee: Liyin Tang In the test case mapjoin1.q : SELECT /*+ MAPJOIN(b) */ sum(a.key) as sum_a FROM srcpart a JOIN src b ON a.key = b.key where a.ds is not null; The current result in mapjoin1.q.out shows the result is 76260.0 But actually, if user remove the map join hint, and run the query: SELECT sum(a.key) as sum_a FROM srcpart a JOIN src b ON a.key = b.key where a.ds is not null; The result is 1114788.0 And I import these input data into mysql to test, and test result is also 1114788.0. Obviously, the current result is not correct -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1759) Many important broken links on Hive web page
[ https://issues.apache.org/jira/browse/HIVE-1759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1759: - Fix Version/s: 0.7.0 Many important broken links on Hive web page Key: HIVE-1759 URL: https://issues.apache.org/jira/browse/HIVE-1759 Project: Hive Issue Type: Bug Components: Documentation Reporter: Jeff Hammerbacher Assignee: Edward Capriolo Fix For: 0.7.0 The change log links are broken, perhaps because of the move to a TLP, and the Jira issue log links all point to the 0.5 issue log. Also, all of the documentation links are broken. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1798) Clear empty files in Hive
[ https://issues.apache.org/jira/browse/HIVE-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1798: - Fix Version/s: 0.7.0 Clear empty files in Hive -- Key: HIVE-1798 URL: https://issues.apache.org/jira/browse/HIVE-1798 Project: Hive Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Fix For: 0.7.0 There are 4 empty files in Hive right now. We should delete them from trunk. D ql/src/java/org/apache/hadoop/hive/ql/exec/JDBMDummyOperator.java D ql/src/java/org/apache/hadoop/hive/ql/exec/JDBMSinkOperator.java D ql/src/java/org/apache/hadoop/hive/ql/plan/JDBMSinkDesc.java D ql/src/java/org/apache/hadoop/hive/ql/plan/JDBMDummyDesc.java D ql/src/java/org/apache/hadoop/hive/ql/util/JoinUtil.java -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2000) adding comments to Hive Stats JDBC queries
[ https://issues.apache.org/jira/browse/HIVE-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2000: - Fix Version/s: 0.8.0 adding comments to Hive Stats JDBC queries --- Key: HIVE-2000 URL: https://issues.apache.org/jira/browse/HIVE-2000 Project: Hive Issue Type: Improvement Reporter: Ning Zhang Assignee: Ning Zhang Priority: Trivial Fix For: 0.8.0 Attachments: HIVE-2000.patch Stats gathering could issue a large number of JDBC queries to stats publisher and aggregator. In a shared DB environment, it's hard to tell which SQL statements are contributed by Hive stats gathering tasks. It would be easier to identify these workload by adding SQL comments that identifies the source of the query. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1973) Getting error when join on tables where name of table has uppercase letters
[
https://issues.apache.org/jira/browse/HIVE-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-1973:
-
Fix Version/s: 0.8.0
Getting error when join on tables where name of table has uppercase letters
---
Key: HIVE-1973
URL: https://issues.apache.org/jira/browse/HIVE-1973
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.5.0, 0.7.0
Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise
Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
Fix For: 0.8.0
Attachments: HIVE-1973.patch
When execute a join query on tables containing Uppercase letters in the table
names hit an exception
Ex:
{noformat}
create table a(b int);
create table tabForJoin(b int,c int);
select * from a join tabForJoin on(a.b=tabForJoin.b);
Got an exception like this
FAILED: Error in semantic analysis: Invalid Table Alias tabForJoin
{noformat}
But if i give without capital letters ,It is working
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1980) Merging using mapreduce rather than map-only job failed in case of dynamic partition inserts
[ https://issues.apache.org/jira/browse/HIVE-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1980: - Component/s: Query Processor Fix Version/s: 0.8.0 Merging using mapreduce rather than map-only job failed in case of dynamic partition inserts Key: HIVE-1980 URL: https://issues.apache.org/jira/browse/HIVE-1980 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Ning Zhang Assignee: Ning Zhang Fix For: 0.8.0 Attachments: HIVE-1980.patch In dynamic partition insert and if merge is set to true and hive.mergejob.maponly=false, the merge MapReduce job will fail. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1974) In error scenario some opened streams may not closed in ScriptOperator.java, Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1974: - Fix Version/s: 0.8.0 In error scenario some opened streams may not closed in ScriptOperator.java, Utilities.java Key: HIVE-1974 URL: https://issues.apache.org/jira/browse/HIVE-1974 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.5.0, 0.7.0 Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5). Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Fix For: 0.8.0 Attachments: HIVE-1974.patch 1)In error scenario StreamProcessor may not be closed in ScriptOperator.java 2)In error scenario XMLEncoder may not be closed in Utilities.java -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-1493) incorrect explanation when local mode not chosen automatically
[ https://issues.apache.org/jira/browse/HIVE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-1493. -- Resolution: Duplicate incorrect explanation when local mode not chosen automatically -- Key: HIVE-1493 URL: https://issues.apache.org/jira/browse/HIVE-1493 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Joydeep Sen Sarma Assignee: Joydeep Sen Sarma Priority: Minor slipped past in 1408: // check for max input size if (inputSummary.getLength() maxBytes) return Input Size (= + maxBytes + ) is larger than + HiveConf.ConfVars.LOCALMODEMAXBYTES.varname + (= + maxBytes + ); printing same value twice. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-1493) incorrect explanation when local mode not chosen automatically
[ https://issues.apache.org/jira/browse/HIVE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-1493: -- incorrect explanation when local mode not chosen automatically -- Key: HIVE-1493 URL: https://issues.apache.org/jira/browse/HIVE-1493 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Joydeep Sen Sarma Assignee: Joydeep Sen Sarma Priority: Minor slipped past in 1408: // check for max input size if (inputSummary.getLength() maxBytes) return Input Size (= + maxBytes + ) is larger than + HiveConf.ConfVars.LOCALMODEMAXBYTES.varname + (= + maxBytes + ); printing same value twice. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-1334) Add PERCENTILE for continuous (double) distributions
[ https://issues.apache.org/jira/browse/HIVE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-1334: -- Add PERCENTILE for continuous (double) distributions Key: HIVE-1334 URL: https://issues.apache.org/jira/browse/HIVE-1334 Project: Hive Issue Type: New Feature Reporter: Adam Kramer Priority: Minor As with the fresh-off-the-presses https://issues.apache.org/jira/browse/HIVE-259 ...but for double distributions. Oracle spec is at http://download.oracle.com/docs/cd/B19306_01/server.102/b14200/functions110.htm for this. I don't think it should be much more trouble than the first version with simple linear imputation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-1334) Add PERCENTILE for continuous (double) distributions
[ https://issues.apache.org/jira/browse/HIVE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-1334. -- Resolution: Duplicate Add PERCENTILE for continuous (double) distributions Key: HIVE-1334 URL: https://issues.apache.org/jira/browse/HIVE-1334 Project: Hive Issue Type: New Feature Reporter: Adam Kramer Priority: Minor As with the fresh-off-the-presses https://issues.apache.org/jira/browse/HIVE-259 ...but for double distributions. Oracle spec is at http://download.oracle.com/docs/cd/B19306_01/server.102/b14200/functions110.htm for this. I don't think it should be much more trouble than the first version with simple linear imputation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1113) LOAD DATA LOCAL INPATH does't work on windows
[ https://issues.apache.org/jira/browse/HIVE-1113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1113: - Fix Version/s: 0.6.0 LOAD DATA LOCAL INPATH does't work on windows - Key: HIVE-1113 URL: https://issues.apache.org/jira/browse/HIVE-1113 Project: Hive Issue Type: Bug Affects Versions: 0.6.0 Environment: cygwin1.7.1-1 WindowsXP Reporter: ruslan.usifov Priority: Minor Fix For: 0.6.0 Attachments: HIVE-1113.patch I had follow trace of this issue. 2010-01-05 01:49:04,109 ERROR ql.Driver (SessionState.java:printError(248)) - FAILED: Unknown exception: null java.lang.IllegalArgumentException at java.net.URI.create(URI.java:842) at org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.analyzeInternal(LoadSemanticAnalyzer.java:197) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:105) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:270) at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:315) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:307) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) Caused by: java.net.URISyntaxException: Illegal character in path at index 42: file:/F:/hadoop/tmp/hive-Ruslan/1564735857\1 at java.net.URI$Parser.fail(URI.java:2809) at java.net.URI$Parser.checkChars(URI.java:2982) at java.net.URI$Parser.parseHierarchical(URI.java:3066) at java.net.URI$Parser.parse(URI.java:3014) at java.net.URI.init(URI.java:578) at java.net.URI.create(URI.java:840) ... 13 more -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-trunk-h0.20 #597
See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/597/ -- [...truncated 26790 lines...] [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select count(1) as cnt from testhivedrivertable [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-03-07_11-11-09_482_8547975294348078583/-mr-1 [junit] Total MapReduce jobs = 1 [junit] Launching Job 1 out of 1 [junit] Number of reduce tasks determined at compile time: 1 [junit] In order to change the average load for a reducer (in bytes): [junit] set hive.exec.reducers.bytes.per.reducer=number [junit] In order to limit the maximum number of reducers: [junit] set hive.exec.reducers.max=number [junit] In order to set a constant number of reducers: [junit] set mapred.reduce.tasks=number [junit] Job running in-process (local Hadoop) [junit] 2011-03-07 11:11:12,552 null map = 100%, reduce = 100% [junit] Ended Job = job_local_0001 [junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-03-07_11-11-09_482_8547975294348078583/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/service/tmp/hive_job_log_hudson_20110307_1255500934.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select * from testhivedrivertable limit 10 [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-03-07_11-11-14_066_774551768663784855/-mr-1 [junit] POSTHOOK: query: select * from testhivedrivertable limit 10 [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-03-07_11-11-14_066_774551768663784855/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output:
[jira] Updated: (HIVE-139) subq1 union subq2 should work
[ https://issues.apache.org/jira/browse/HIVE-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-139: Fix Version/s: 0.3.0 subq1 union subq2 should work - Key: HIVE-139 URL: https://issues.apache.org/jira/browse/HIVE-139 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Priority: Critical Fix For: 0.3.0 subq1 union subq2 should work -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-138) Provide option to export a HEADER
[ https://issues.apache.org/jira/browse/HIVE-138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-138: Fix Version/s: 0.7.0 Provide option to export a HEADER - Key: HIVE-138 URL: https://issues.apache.org/jira/browse/HIVE-138 Project: Hive Issue Type: Improvement Components: Clients, Query Processor Reporter: Adam Kramer Assignee: Paul Butler Priority: Minor Fix For: 0.7.0 Attachments: HIVE-138.patch When writing data to directories or files for later analysis, or when exploring data in the hive CLI with raw SELECT statements, it'd be great if we could get a header or something so we know which columns our output comes from. Any chance this is easy to add? Just print the column names (or formula used to generate them) in the first row? SELECT foo.* WITH HEADER FROM some_table foo limit 3; col1col2col3 1 9 6 7 5 0 7 5 3 SELECT f.col1-f.col2, col3 WITH HEADER FROM some_table foo limit 3; f.col1-f.col2 col3 -8 6 2 0 2 3 ...etc -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-83) Set up a continuous build of Hive with Hudson
[ https://issues.apache.org/jira/browse/HIVE-83?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-83: --- Fix Version/s: 0.3.0 Set up a continuous build of Hive with Hudson - Key: HIVE-83 URL: https://issues.apache.org/jira/browse/HIVE-83 Project: Hive Issue Type: Task Components: Build Infrastructure Reporter: Jeff Hammerbacher Assignee: Johan Oskarsson Fix For: 0.3.0 Other projects like Zookeeper and HBase are leveraging Apache's hosted Hudson server (http://hudson.zones.apache.org/hudson/view/HBase). Perhaps Hive should as well? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-81) Make forrest docs for Hive web site along the lines of http://hadoop.apache.org/core/
[ https://issues.apache.org/jira/browse/HIVE-81?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-81: --- Fix Version/s: 0.3.0 Make forrest docs for Hive web site along the lines of http://hadoop.apache.org/core/ - Key: HIVE-81 URL: https://issues.apache.org/jira/browse/HIVE-81 Project: Hive Issue Type: New Feature Components: Documentation Reporter: Jeff Hammerbacher Assignee: Jeff Hammerbacher Fix For: 0.3.0 Attachments: HIVE-81.patch, favicon.ico, hadoop-logo.jpg, hive_small.jpg Hive should ship with documentation, like Hadoop, instead of using the wiki as the official documentation repository. To get there, we'll need a set of xml files to grind through forrest, if we want to reuse the same mechanisms as the other sites. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1099) Queries in input40.q should be deterministic
[ https://issues.apache.org/jira/browse/HIVE-1099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1099: - Fix Version/s: 0.4.1 Queries in input40.q should be deterministic Key: HIVE-1099 URL: https://issues.apache.org/jira/browse/HIVE-1099 Project: Hive Issue Type: Bug Affects Versions: 0.4.0 Reporter: Paul Yang Assignee: Paul Yang Fix For: 0.4.1 Attachments: HIVE-1099.1.patch, HIVE-1099.2.patch The SELECT * queries in input40.q are not deterministic because the rows could be output in any order. The fix is to add a sort by clause as it is done in branch-0.5. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-765) HiveServer doesn't start properly with auxlib directory
[
https://issues.apache.org/jira/browse/HIVE-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-765:
Fix Version/s: 0.5.0
HiveServer doesn't start properly with auxlib directory
---
Key: HIVE-765
URL: https://issues.apache.org/jira/browse/HIVE-765
Project: Hive
Issue Type: Bug
Reporter: Bill Graham
Assignee: Ning Zhang
Fix For: 0.5.0
See http://www.mail-archive.com/hive-user@hadoop.apache.org/msg01303.html
The org.apache.hadoop.hive.service.HiveServer.main(String[] args) method has
code that looks like this:
int port = 1;
if (args.length = 1) {
port = Integer.parseInt(args[0]);
}
But ./bin/hive --service hiveserver (which runs
./bin/hivebin/ext/hiveserver.sh) will run the following if the auxlib/
directory is present:
/path/to/hadoop jar -libjars file:///path/to/hive/auxlib/some_jar.jar
/path/to/hive/lib/hive_service.jar org.apache.hadoop.hive.service.HiveServer
-hiveconf hive.aux.jars.path=file:///path/to/hive/auxlib/some_jar.jar
This cause an error like so:
java.lang.NumberFormatException: For input string: -hiveconf
The workaround is to be explicit with the port:
HIVE_PORT=1 ./bin/hive --service hiveserver
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1939) Fix test failure in TestContribCliDriver/url_hook.q
[ https://issues.apache.org/jira/browse/HIVE-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13003526#comment-13003526 ] John Sichi commented on HIVE-1939: -- +1. Will commit when tests pass. Fix test failure in TestContribCliDriver/url_hook.q --- Key: HIVE-1939 URL: https://issues.apache.org/jira/browse/HIVE-1939 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Carl Steinbach Priority: Blocker Fix For: 0.7.0 Attachments: HIVE-1939.1.patch.txt -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-818) Create a Hive CLI that connects to hive ThriftServer
[ https://issues.apache.org/jira/browse/HIVE-818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-818: Fix Version/s: 0.8.0 Create a Hive CLI that connects to hive ThriftServer Key: HIVE-818 URL: https://issues.apache.org/jira/browse/HIVE-818 Project: Hive Issue Type: New Feature Components: Clients, Server Infrastructure Reporter: Edward Capriolo Assignee: Ning Zhang Fix For: 0.8.0 Attachments: HIVE-818.2.patch, HIVE-818.3.patch, HIVE-818.4.patch, HIVE-818.5.patch, HIVE-818.patch, Hive-881_2.patch We should have an alternate CLI that works by interacting with the HiveServer, in this way it will be ready when/if we deprecate the current CLI. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-513) UDFToString bug
[
https://issues.apache.org/jira/browse/HIVE-513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach resolved HIVE-513.
-
Resolution: Invalid
UDFToString bug
---
Key: HIVE-513
URL: https://issues.apache.org/jira/browse/HIVE-513
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Min Zhou
Assignee: Min Zhou
Attachments: HIVE-513.patch
execute a query like below,
{noformat}
select cast('my string' as string) from pokes;
{noformat}
you will get the result:
{noformat}
NULL
NULL
...
{noformat}
the reason is UDFToString hasn't an evaluate method acceptes Text or
BytesWritable argument.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-359) Short-circuiting expression evaluation
[ https://issues.apache.org/jira/browse/HIVE-359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-359: Component/s: Query Processor Fix Version/s: 0.5.0 Short-circuiting expression evaluation -- Key: HIVE-359 URL: https://issues.apache.org/jira/browse/HIVE-359 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Zheng Shao Assignee: Zheng Shao Fix For: 0.5.0 We don't need to evaluate some sub-expressions for AND, OR, CASE, and IF. We should support this kind of expression operators natively so we can change the evaluation order and do short-circuiting. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-311) Ouput of 'describe extended' commands are not getting compared correctly in unit tests
[ https://issues.apache.org/jira/browse/HIVE-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-311: - Ouput of 'describe extended' commands are not getting compared correctly in unit tests -- Key: HIVE-311 URL: https://issues.apache.org/jira/browse/HIVE-311 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.6.0 Reporter: Prasad Chakka Since the output contains the file locations and user names, the outputs are not being compared at all so we are not able to catch bugs in new patches. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-408) UDAFCount should count empty strings
[ https://issues.apache.org/jira/browse/HIVE-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-408. - Resolution: Invalid UDAFCount should count empty strings Key: HIVE-408 URL: https://issues.apache.org/jira/browse/HIVE-408 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Zheng Shao Assignee: Edward Capriolo Attachments: hive-408.diff UDAFCount is not counting empty strings right now. We should count that according to SQL standard. (Null is not counted.) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-621) Inconsistent reporting of null and delimiter across access types
[ https://issues.apache.org/jira/browse/HIVE-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-621: - Inconsistent reporting of null and delimiter across access types -- Key: HIVE-621 URL: https://issues.apache.org/jira/browse/HIVE-621 Project: Hive Issue Type: Wish Reporter: Adam Kramer If you INSERT OVERWRITE LOCAL DIRECTORY, you get a bunch of \N fields in your output...If you hive -e SELECT ... you get a bunch of null fields in your output. I also recall having seen NULL (in caps) before somewhere, too, but maybe I'm dreaming that. In any case, it would be nice if this were consistent! Preference: \N everywhere. Ditto, when you I O L D, your fields are ^A-delimited, but when you SELECT, they're ^V (tab)-delimited. It would be nice if this, too, were consistent. Preference: tabs everywhere. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-311) Ouput of 'describe extended' commands are not getting compared correctly in unit tests
[ https://issues.apache.org/jira/browse/HIVE-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-311. - Resolution: Invalid Ouput of 'describe extended' commands are not getting compared correctly in unit tests -- Key: HIVE-311 URL: https://issues.apache.org/jira/browse/HIVE-311 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.6.0 Reporter: Prasad Chakka Since the output contains the file locations and user names, the outputs are not being compared at all so we are not able to catch bugs in new patches. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-621) Inconsistent reporting of null and delimiter across access types
[ https://issues.apache.org/jira/browse/HIVE-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-621. - Resolution: Invalid Inconsistent reporting of null and delimiter across access types -- Key: HIVE-621 URL: https://issues.apache.org/jira/browse/HIVE-621 Project: Hive Issue Type: Wish Reporter: Adam Kramer If you INSERT OVERWRITE LOCAL DIRECTORY, you get a bunch of \N fields in your output...If you hive -e SELECT ... you get a bunch of null fields in your output. I also recall having seen NULL (in caps) before somewhere, too, but maybe I'm dreaming that. In any case, it would be nice if this were consistent! Preference: \N everywhere. Ditto, when you I O L D, your fields are ^A-delimited, but when you SELECT, they're ^V (tab)-delimited. It would be nice if this, too, were consistent. Preference: tabs everywhere. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-624) Fix bug in TypeConverter
[
https://issues.apache.org/jira/browse/HIVE-624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13003534#comment-13003534
]
Carl Steinbach commented on HIVE-624:
-
Committed as HIVE-616.
Fix bug in TypeConverter
Key: HIVE-624
URL: https://issues.apache.org/jira/browse/HIVE-624
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Zheng Shao
Assignee: Zheng Shao
Fix For: 0.4.0
Attachments: HIVE-624.1.patch, HIVE-624.2.patch
There is a bug in the Converter - we are converting all objects to Primitive
Java objects instead of Writable.
This has caused some queries to fail:
{code}
SELECT IF(false, 1, cast(2 as smallint)) + 3 FROM any_table;
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-624) Fix bug in TypeConverter
[
https://issues.apache.org/jira/browse/HIVE-624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-624:
Fix Version/s: 0.4.0
Fix bug in TypeConverter
Key: HIVE-624
URL: https://issues.apache.org/jira/browse/HIVE-624
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Zheng Shao
Assignee: Zheng Shao
Fix For: 0.4.0
Attachments: HIVE-624.1.patch, HIVE-624.2.patch
There is a bug in the Converter - we are converting all objects to Primitive
Java objects instead of Writable.
This has caused some queries to fail:
{code}
SELECT IF(false, 1, cast(2 as smallint)) + 3 FROM any_table;
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-932) enforce bucketing
[ https://issues.apache.org/jira/browse/HIVE-932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-932: - enforce bucketing - Key: HIVE-932 URL: https://issues.apache.org/jira/browse/HIVE-932 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: He Yongqiang If a table is bucketed and the number of buckets are also specified, it should be enforced. If a query tries to insert into the table which violates the bucketing, it should fail. Also, the number of reducers should be the same as the number of buckets, and be independent of the input size -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-971) allow more tables in union all
[ https://issues.apache.org/jira/browse/HIVE-971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-971: - allow more tables in union all --- Key: HIVE-971 URL: https://issues.apache.org/jira/browse/HIVE-971 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Hao Liu We used to union more than 128 tables in hive, which did not work. But it is a useful feature to support query aggregating multi fct tables. (not sue if it is fixed already, pls. close the task if it is not true any longer) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-786) Move ql/.../ql/util/typedbytes and ql/.../ql/exec/TypedBytesRecordReader.java to contrib
[ https://issues.apache.org/jira/browse/HIVE-786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-786: Fix Version/s: 0.5.0 Move ql/.../ql/util/typedbytes and ql/.../ql/exec/TypedBytesRecordReader.java to contrib Key: HIVE-786 URL: https://issues.apache.org/jira/browse/HIVE-786 Project: Hive Issue Type: Improvement Affects Versions: 0.5.0 Reporter: Zheng Shao Assignee: Namit Jain Fix For: 0.5.0 These files are introduced by HIVE-708. They should belong to contrib. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-971) allow more tables in union all
[ https://issues.apache.org/jira/browse/HIVE-971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-971. - Resolution: Invalid allow more tables in union all --- Key: HIVE-971 URL: https://issues.apache.org/jira/browse/HIVE-971 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Hao Liu We used to union more than 128 tables in hive, which did not work. But it is a useful feature to support query aggregating multi fct tables. (not sue if it is fixed already, pls. close the task if it is not true any longer) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-932) enforce bucketing
[ https://issues.apache.org/jira/browse/HIVE-932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-932. - Resolution: Invalid enforce bucketing - Key: HIVE-932 URL: https://issues.apache.org/jira/browse/HIVE-932 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: He Yongqiang If a table is bucketed and the number of buckets are also specified, it should be enforced. If a query tries to insert into the table which violates the bucketing, it should fail. Also, the number of reducers should be the same as the number of buckets, and be independent of the input size -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.7.0-h0.20 #29
See https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/29/ -- [...truncated 27300 lines...] [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/build/service/tmp/hive_job_log_hudson_201103071150_1807300119.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select count(1) as cnt from testhivedrivertable [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-03-07_11-50-26_921_6704366808916511340/-mr-1 [junit] Total MapReduce jobs = 1 [junit] Launching Job 1 out of 1 [junit] Number of reduce tasks determined at compile time: 1 [junit] In order to change the average load for a reducer (in bytes): [junit] set hive.exec.reducers.bytes.per.reducer=number [junit] In order to limit the maximum number of reducers: [junit] set hive.exec.reducers.max=number [junit] In order to set a constant number of reducers: [junit] set mapred.reduce.tasks=number [junit] Job running in-process (local Hadoop) [junit] 2011-03-07 11:50:29,944 null map = 100%, reduce = 100% [junit] Ended Job = job_local_0001 [junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-03-07_11-50-26_921_6704366808916511340/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/build/service/tmp/hive_job_log_hudson_201103071150_57173212.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select * from testhivedrivertable limit 10 [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-03-07_11-50-31_709_671116554725781142/-mr-1 [junit] POSTHOOK: query: select * from testhivedrivertable limit 10 [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-03-07_11-50-31_709_671116554725781142/-mr-1 [junit] OK [junit] PREHOOK: query: drop table
[jira] Updated: (HIVE-1168) Fix Hive build on Hudson
[
https://issues.apache.org/jira/browse/HIVE-1168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-1168:
-
Fix Version/s: 0.6.0
Fix Hive build on Hudson
Key: HIVE-1168
URL: https://issues.apache.org/jira/browse/HIVE-1168
Project: Hive
Issue Type: Bug
Components: Build Infrastructure
Reporter: Carl Steinbach
Assignee: John Sichi
Priority: Critical
Fix For: 0.6.0
{quote}
We need to delete the .ant directory containing the old ivy version in order
to fix it
(and if we're using the same environment for both trunk and branches, either
segregate them or script an rm to clean in between).
{quote}
It's worth noting that ant may have picked up the old version of Ivy from
somewhere else. In order Ant's classpath contains:
# Ant's startup JAR file, ant-launcher.jar
# Everything in the directory containing the version of ant-launcher.jar
that's
running, i.e. everything in ANT_HOME/lib
# All JAR files in ${user.home}/.ant/lib
# Directories and JAR files supplied via the -lib command line option.
# Everything in the CLASSPATH variable unless the -noclasspath option is used.
(2) implies that users on shared machines may have to install their own
version of ant in order to get around these problems, assuming that the
administrator has install the ivy.jar in $ANT_HOME/lib
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1148) Add Checkstyle documentation to developer guide
[ https://issues.apache.org/jira/browse/HIVE-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1148: - Fix Version/s: 0.6.0 Add Checkstyle documentation to developer guide --- Key: HIVE-1148 URL: https://issues.apache.org/jira/browse/HIVE-1148 Project: Hive Issue Type: Task Components: Documentation Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: checkstyle Fix For: 0.6.0 Add checkstyle documentation to the Hive developer manual. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-945) custom mappers/reducers should not be initialized at compile time
[
https://issues.apache.org/jira/browse/HIVE-945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-945:
Fix Version/s: 0.5.0
{code}
commit df1967f9a88fbb64351c3e97c540f36bc59f6c1a
Author: Yongqiang He heyongqi...@apache.org
Date: Tue Nov 24 07:12:55 2009 +
custom mappers/reducers should not be initialized at compile time
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hive/trunk@883608
13f79535-47bb-0310-9956-ffa450edef68
CHANGES.txt|2 +
.../apache/hadoop/hive/ql/exec/ScriptOperator.java | 206 +++-
2 files changed, 116 insertions(+), 92 deletions(-)
{code}
custom mappers/reducers should not be initialized at compile time
-
Key: HIVE-945
URL: https://issues.apache.org/jira/browse/HIVE-945
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.5.0
Reporter: Namit Jain
Assignee: Namit Jain
Fix For: 0.5.0
Attachments: hive.945.1.patch, hive.945.2.patch
custom mappers/reducers should not be initialized at compile time
Instead of being initialized at compile time, they should be initialized at
the first row, since the custom job may be doing some redundant work or
some task specific work
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-751) Rename serde/serdeFormat etc in Hive.g for readability
[ https://issues.apache.org/jira/browse/HIVE-751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-751: Fix Version/s: 0.5.0 Rename serde/serdeFormat etc in Hive.g for readability -- Key: HIVE-751 URL: https://issues.apache.org/jira/browse/HIVE-751 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.5.0 @hive.743.2.patch: Hive.g: Can we use rowFormat, rowFormatDelimited, rowFormatSerDe instead of serde, serdePropertiesFormat, and serdeFormat? I think that will be much more intuitive. We can also replace tableRowFormat with rowFormat. SemanticAnalyzer.genScriptPlan looks a bit ugly - can we add one level of nesting for inSerDe and outSerDe in Hive.g, and use a single loop over the children in genScriptPlan to get all corresponding children? We can also add a null placeholder in case inSerDe / outSerDe is missing, so we don't need to calculate the child positions. Otherwise it looks good. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-797) mappers should report life in ways other than emitting data
[ https://issues.apache.org/jira/browse/HIVE-797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-797. - Resolution: Invalid mappers should report life in ways other than emitting data --- Key: HIVE-797 URL: https://issues.apache.org/jira/browse/HIVE-797 Project: Hive Issue Type: Bug Reporter: S. Alex Smith Mappers which are performing a great deal of aggregation can be killed by time out even if they are running successfully. For example, in the following query the group by operator stops the mapper from returning any rows of data until the map is entirely finished. If the data processing takes longer than the time-out limit, the job will fail. The mapper should instead offer the tracker some indication that it is busy working. Alternatively, the tracker could ping the mapper with an appropriate question / warning before it sends a kill signal. FROM ( FROM my_table SELECT TRANSFORM(my_data) USING 'my_boolean_function' AS boolean_output) a SELECT boolean_output, COUNT(1) GROUP BY boolean_output -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-797) mappers should report life in ways other than emitting data
[ https://issues.apache.org/jira/browse/HIVE-797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-797: - mappers should report life in ways other than emitting data --- Key: HIVE-797 URL: https://issues.apache.org/jira/browse/HIVE-797 Project: Hive Issue Type: Bug Reporter: S. Alex Smith Mappers which are performing a great deal of aggregation can be killed by time out even if they are running successfully. For example, in the following query the group by operator stops the mapper from returning any rows of data until the map is entirely finished. If the data processing takes longer than the time-out limit, the job will fail. The mapper should instead offer the tracker some indication that it is busy working. Alternatively, the tracker could ping the mapper with an appropriate question / warning before it sends a kill signal. FROM ( FROM my_table SELECT TRANSFORM(my_data) USING 'my_boolean_function' AS boolean_output) a SELECT boolean_output, COUNT(1) GROUP BY boolean_output -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-703) change unit tests to add 'DROP FUNCTION' at the end which create a function
[ https://issues.apache.org/jira/browse/HIVE-703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-703: Fix Version/s: 0.5.0 change unit tests to add 'DROP FUNCTION' at the end which create a function --- Key: HIVE-703 URL: https://issues.apache.org/jira/browse/HIVE-703 Project: Hive Issue Type: Bug Components: Testing Infrastructure Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.5.0 change unit tests to add 'DROP FUNCTION' at the end which create a function -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-430) Ability to comment desired for hive query files
[ https://issues.apache.org/jira/browse/HIVE-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-430. - Resolution: Cannot Reproduce Ability to comment desired for hive query files --- Key: HIVE-430 URL: https://issues.apache.org/jira/browse/HIVE-430 Project: Hive Issue Type: Wish Components: Clients, Query Processor Reporter: Adam Kramer Priority: Minor It is, in many cases, quite useful to write a query, save it in a file, and run the file with hive \-f filename. However, as time passes and many such files are written, it becomes unclear why I did things the way I did--the solution for most code is to comment the code, and I would love to be able to comment hive code in .sql/.hql files or queries. Standard SQL comment structure is anything following a non-escaped non-quoted -- or # on a single line should cause the rest of the line to be ignored. I would love to have this in hive...both in the CLI (for when I paste a query in) and in the hive -f file reader option. C-style // and /* */ commenting would be great too, but only because I'm lazy. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-430) Ability to comment desired for hive query files
[ https://issues.apache.org/jira/browse/HIVE-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-430: - Ability to comment desired for hive query files --- Key: HIVE-430 URL: https://issues.apache.org/jira/browse/HIVE-430 Project: Hive Issue Type: Wish Components: Clients, Query Processor Reporter: Adam Kramer Priority: Minor It is, in many cases, quite useful to write a query, save it in a file, and run the file with hive \-f filename. However, as time passes and many such files are written, it becomes unclear why I did things the way I did--the solution for most code is to comment the code, and I would love to be able to comment hive code in .sql/.hql files or queries. Standard SQL comment structure is anything following a non-escaped non-quoted -- or # on a single line should cause the rest of the line to be ignored. I would love to have this in hive...both in the CLI (for when I paste a query in) and in the hive -f file reader option. C-style // and /* */ commenting would be great too, but only because I'm lazy. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-172) failed build with hadoop 0.17.2
[ https://issues.apache.org/jira/browse/HIVE-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-172. - Resolution: Cannot Reproduce failed build with hadoop 0.17.2 --- Key: HIVE-172 URL: https://issues.apache.org/jira/browse/HIVE-172 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Zheng Shao Command: ant -Dhadoop.version=0.17.2 package 0.17.0, 0.17.1 both works but not 0.17.2 The error message: http://archive.apache.org/dist/hadoop/core/hadoop-0.17.2/hadoop-0.17.2.tar.gz ... [ivy:retrieve] [FAILED ] hadoop#core;0.17.2!hadoop.tar.gz(source): invalid md5: expected=hadoop-0.17.2.tar.gz: computed=4819ec4aed328f699eeae29f42800f03 (5413ms) [ivy:retrieve] [FAILED ] hadoop#core;0.17.2!hadoop.tar.gz(source): invalid md5: expected=hadoop-0.17.2.tar.gz: computed=4819ec4aed328f699eeae29f42800f03 (5413ms) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-172) failed build with hadoop 0.17.2
[ https://issues.apache.org/jira/browse/HIVE-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-172: - failed build with hadoop 0.17.2 --- Key: HIVE-172 URL: https://issues.apache.org/jira/browse/HIVE-172 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Zheng Shao Command: ant -Dhadoop.version=0.17.2 package 0.17.0, 0.17.1 both works but not 0.17.2 The error message: http://archive.apache.org/dist/hadoop/core/hadoop-0.17.2/hadoop-0.17.2.tar.gz ... [ivy:retrieve] [FAILED ] hadoop#core;0.17.2!hadoop.tar.gz(source): invalid md5: expected=hadoop-0.17.2.tar.gz: computed=4819ec4aed328f699eeae29f42800f03 (5413ms) [ivy:retrieve] [FAILED ] hadoop#core;0.17.2!hadoop.tar.gz(source): invalid md5: expected=hadoop-0.17.2.tar.gz: computed=4819ec4aed328f699eeae29f42800f03 (5413ms) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-237) Hive does not allow ascii values 128 as a delimiter
[ https://issues.apache.org/jira/browse/HIVE-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-237: - Hive does not allow ascii values 128 as a delimiter - Key: HIVE-237 URL: https://issues.apache.org/jira/browse/HIVE-237 Project: Hive Issue Type: Bug Environment: All Reporter: Eric Gottschalk Hive does not allow ascii values 128 as a delimiter. Using delimiter '254' is currently not possible. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-237) Hive does not allow ascii values 128 as a delimiter
[ https://issues.apache.org/jira/browse/HIVE-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-237. - Resolution: Won't Fix Hive does not allow ascii values 128 as a delimiter - Key: HIVE-237 URL: https://issues.apache.org/jira/browse/HIVE-237 Project: Hive Issue Type: Bug Environment: All Reporter: Eric Gottschalk Hive does not allow ascii values 128 as a delimiter. Using delimiter '254' is currently not possible. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HIVE-226) SerDe should support null column values
[ https://issues.apache.org/jira/browse/HIVE-226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-226: - SerDe should support null column values - Key: HIVE-226 URL: https://issues.apache.org/jira/browse/HIVE-226 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Reporter: Josh Ferguson Currently if you attempt to load data that has null values it will work but selecting data back out of the table will fail due to a null pointer exception during deserialization. Suppose we have a generic users table with ^A separated fields. CREATE TABLE users (id STRING, properties MAPSTRING, STRING) ROW FORMAT DELIMITED COLLECTION ITEMS TERMINATED BY '44' MAP KEYS TERMINATED BY '58' STORED AS TEXTFILE; we might insert this data (where spaces are ^A characters) 1 key:value 2 3 key:value Then the following queries will fail SELECT id FROM users; SELECT id, properties FROM users; SELECT properties FROM users; But they should not fail. Anytime the field delimiter is encountered twice in a row or the field delimiter is encountered followed immediately by the line delimiter a non-existent value should be assumed for the appropriate column. To circumvent this in my application I have been substituting my own reserved world NULL and the key/value pair NULL:NULL to indicate to my application that particular fields currently have no value. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HIVE-226) SerDe should support null column values
[ https://issues.apache.org/jira/browse/HIVE-226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-226. - Resolution: Invalid SerDe should support null column values - Key: HIVE-226 URL: https://issues.apache.org/jira/browse/HIVE-226 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Reporter: Josh Ferguson Currently if you attempt to load data that has null values it will work but selecting data back out of the table will fail due to a null pointer exception during deserialization. Suppose we have a generic users table with ^A separated fields. CREATE TABLE users (id STRING, properties MAPSTRING, STRING) ROW FORMAT DELIMITED COLLECTION ITEMS TERMINATED BY '44' MAP KEYS TERMINATED BY '58' STORED AS TEXTFILE; we might insert this data (where spaces are ^A characters) 1 key:value 2 3 key:value Then the following queries will fail SELECT id FROM users; SELECT id, properties FROM users; SELECT properties FROM users; But they should not fail. Anytime the field delimiter is encountered twice in a row or the field delimiter is encountered followed immediately by the line delimiter a non-existent value should be assumed for the appropriate column. To circumvent this in my application I have been substituting my own reserved world NULL and the key/value pair NULL:NULL to indicate to my application that particular fields currently have no value. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-341) Specifying partition column without table alias causes unknown exception
[ https://issues.apache.org/jira/browse/HIVE-341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-341: Fix Version/s: 0.6.0 Specifying partition column without table alias causes unknown exception Key: HIVE-341 URL: https://issues.apache.org/jira/browse/HIVE-341 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.6.0 Reporter: Raghotham Murthy Fix For: 0.6.0 Created two tables - tmp_rsm_abc and tmp_rsm_abc1. The latter is partitioned on ds. Query on first table succeeds, but query on second fails. See the session below. hive describe tmp_rsm_abc; a string b int Time taken: 0.116 seconds hive select a, b from tmp_rsm_abc where b 5; - this query succeeds Unknown 19 Unknown 29 Unknown 29 Unknown 29 Unknown 30 Unknown 25 Unknown 15 Unknown 17 Unknown 28 Unknown 17 Time taken: 8.198 seconds hive create table tmp_rsm_abc1(a string, b int) partitioned by (ds string); OK Time taken: 0.118 seconds hive insert overwrite table tmp_rsm_abc1 partition (ds = '10') select a, b from tmp_rsm_abc where b 5; 10 Rows loaded to tmp_rsm_abc1 OK Time taken: 9.319 seconds hive select a, b from tmp_rsm_abc1 where ds = '10'; - this query fails FAILED: Unknown exception : null Time taken: 0.053 seconds hive -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-2011) upgrade-0.6.0.mysql.sql script attempts to increase size of PK COLUMNS.TYPE_NAME to 4000
[
https://issues.apache.org/jira/browse/HIVE-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13003613#comment-13003613
]
John Sichi commented on HIVE-2011:
--
I gave this one a try using MySQL. First, I started with an 0.6 build and got
it to autocreate a schema there. (I don't know of a way to force creation of
all tables, so the schema was incomplete.)
Then I applied this patch to 0.7, built there, and ran the upgrade script. It
failed because the PARTS table hadn't been created in 0.6 (I had only created
an unpartitioned table there). This is fine; the only part which could be
improved is that the script did not stop after the errors, so the tail looks
like this:
{noformat}
...
Query OK, 0 rows affected (0.00 sec)
ERROR 1005 (HY000): Can't create table
'./jsichi_hive_upgrade_test/PART_PRIVS.frm' (errno: 150)
ERROR 1005 (HY000): Can't create table
'./jsichi_hive_upgrade_test/PART_COL_PRIVS.frm' (errno: 150)
+-+
| |
+-+
| Finished upgrading MetaStore schema from 0.6.0 to 0.7.0 |
+-+
1 row in set, 1 warning (0.00 sec)
{noformat}
Should we change the README instructions to use something which will abort on
exception? Otherwise, for an error in the middle of a long upgrade sequence,
the administrator might miss it and think all was well.
upgrade-0.6.0.mysql.sql script attempts to increase size of PK
COLUMNS.TYPE_NAME to 4000
Key: HIVE-2011
URL: https://issues.apache.org/jira/browse/HIVE-2011
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.6.0
Reporter: Carl Steinbach
Assignee: Carl Steinbach
Priority: Blocker
Fix For: 0.7.0
Attachments: HIVE-2011.1.patch.txt
{code}
# mysql flumenewresearch upgrade-0.6.0.mysql.sql
ERROR 1071 (42000) at line 16: Specified key was too long; max key length is
767 bytes
{code}
Here's the cause of the problem from upgrade-0.6.0.mysql.sql:
{code}
...
ALTER TABLE `COLUMNS` MODIFY `TYPE_NAME` VARCHAR(4000);
...
ALTER TABLE `COLUMNS` DROP PRIMARY KEY;
ALTER TABLE `COLUMNS` ADD PRIMARY KEY (`SD_ID`, `COLUMN_NAME`);
...
{code}
We need to make sure that the PK on COLUMNS.TYPE_NAME is dropped before the
size of the column is bumped to 4000.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2011) upgrade-0.6.0.mysql.sql script attempts to increase size of PK COLUMNS.TYPE_NAME to 4000
[
https://issues.apache.org/jira/browse/HIVE-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sichi updated HIVE-2011:
-
Status: Open (was: Patch Available)
upgrade-0.6.0.mysql.sql script attempts to increase size of PK
COLUMNS.TYPE_NAME to 4000
Key: HIVE-2011
URL: https://issues.apache.org/jira/browse/HIVE-2011
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.6.0
Reporter: Carl Steinbach
Assignee: Carl Steinbach
Priority: Blocker
Fix For: 0.7.0
Attachments: HIVE-2011.1.patch.txt
{code}
# mysql flumenewresearch upgrade-0.6.0.mysql.sql
ERROR 1071 (42000) at line 16: Specified key was too long; max key length is
767 bytes
{code}
Here's the cause of the problem from upgrade-0.6.0.mysql.sql:
{code}
...
ALTER TABLE `COLUMNS` MODIFY `TYPE_NAME` VARCHAR(4000);
...
ALTER TABLE `COLUMNS` DROP PRIMARY KEY;
ALTER TABLE `COLUMNS` ADD PRIMARY KEY (`SD_ID`, `COLUMN_NAME`);
...
{code}
We need to make sure that the PK on COLUMNS.TYPE_NAME is dropped before the
size of the column is bumped to 4000.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1644) use filter pushdown for automatically accessing indexes
[ https://issues.apache.org/jira/browse/HIVE-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13003645#comment-13003645 ] John Sichi commented on HIVE-1644: -- I'm not sure about those task dependencies...the EXPLAIN output looks wonky. use filter pushdown for automatically accessing indexes --- Key: HIVE-1644 URL: https://issues.apache.org/jira/browse/HIVE-1644 Project: Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: Russell Melick Attachments: HIVE-1644.1.patch, HIVE-1644.2.patch, HIVE-1644.3.patch, HIVE-1644.4.patch, HIVE-1644.5.patch, HIVE-1644.6.patch HIVE-1226 provides utilities for analyzing filters which have been pushed down to a table scan. The next step is to use these for selecting available indexes and generating access plans for those indexes. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1434) Cassandra Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13003652#comment-13003652
]
John Sichi commented on HIVE-1434:
--
After applying patch:
{noformat}
ant clean package
... builds fine ...
ant test -Dtestcase=TestCassandraCliDriver
...
BUILD FAILED
/data/users/jsichi/open/hive-trunk/build-common.xml:317:
/data/users/jsichi/open/hive-trunk/cassandra-handler/lib does not exist.
{noformat}
Cassandra Storage Handler
-
Key: HIVE-1434
URL: https://issues.apache.org/jira/browse/HIVE-1434
Project: Hive
Issue Type: New Feature
Affects Versions: 0.7.0
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Attachments: cas-handle.tar.gz, cass_handler.diff, hive-1434-1.txt,
hive-1434-2-patch.txt, hive-1434-2011-02-26.patch.txt, hive-1434-3-patch.txt,
hive-1434-4-patch.txt, hive-1434-5.patch.txt, hive-1434.2011-02-27.diff.txt,
hive-cassandra.2011-02-25.txt, hive.diff
Add a cassandra storage handler.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1434) Cassandra Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13003654#comment-13003654 ] John Sichi commented on HIVE-1434: -- (Ignore deleted comment...I was in the wrong subdir.) Cassandra Storage Handler - Key: HIVE-1434 URL: https://issues.apache.org/jira/browse/HIVE-1434 Project: Hive Issue Type: New Feature Affects Versions: 0.7.0 Reporter: Edward Capriolo Assignee: Edward Capriolo Attachments: cas-handle.tar.gz, cass_handler.diff, hive-1434-1.txt, hive-1434-2-patch.txt, hive-1434-2011-02-26.patch.txt, hive-1434-3-patch.txt, hive-1434-4-patch.txt, hive-1434-5.patch.txt, hive-1434.2011-02-27.diff.txt, hive-cassandra.2011-02-25.txt, hive.diff Add a cassandra storage handler. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1434) Cassandra Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-1434: - Comment: was deleted (was: (Ignore deleted comment...I was in the wrong subdir.) ) Cassandra Storage Handler - Key: HIVE-1434 URL: https://issues.apache.org/jira/browse/HIVE-1434 Project: Hive Issue Type: New Feature Affects Versions: 0.7.0 Reporter: Edward Capriolo Assignee: Edward Capriolo Attachments: cas-handle.tar.gz, cass_handler.diff, hive-1434-1.txt, hive-1434-2-patch.txt, hive-1434-2011-02-26.patch.txt, hive-1434-3-patch.txt, hive-1434-4-patch.txt, hive-1434-5.patch.txt, hive-1434.2011-02-27.diff.txt, hive-cassandra.2011-02-25.txt, hive.diff Add a cassandra storage handler. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1434) Cassandra Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sichi updated HIVE-1434:
-
Comment: was deleted
(was: After applying patch:
{noformat}
ant clean package
... builds fine ...
ant test -Dtestcase=TestCassandraCliDriver
...
BUILD FAILED
/data/users/jsichi/open/hive-trunk/build-common.xml:317:
/data/users/jsichi/open/hive-trunk/cassandra-handler/lib does not exist.
{noformat}
)
Cassandra Storage Handler
-
Key: HIVE-1434
URL: https://issues.apache.org/jira/browse/HIVE-1434
Project: Hive
Issue Type: New Feature
Affects Versions: 0.7.0
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Attachments: cas-handle.tar.gz, cass_handler.diff, hive-1434-1.txt,
hive-1434-2-patch.txt, hive-1434-2011-02-26.patch.txt, hive-1434-3-patch.txt,
hive-1434-4-patch.txt, hive-1434-5.patch.txt, hive-1434.2011-02-27.diff.txt,
hive-cassandra.2011-02-25.txt, hive.diff
Add a cassandra storage handler.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1434) Cassandra Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13003655#comment-13003655
]
John Sichi commented on HIVE-1434:
--
After applying patch:
{noformat}
ant clean package
... builds fine ...
ant test -Dtestcase=TestCassandraCliDriver
...
BUILD FAILED
/data/users/jsichi/open/hive-trunk/build-common.xml:317:
/data/users/jsichi/open/hive-trunk/cassandra-handler/lib does not exist.
{noformat}
Cassandra Storage Handler
-
Key: HIVE-1434
URL: https://issues.apache.org/jira/browse/HIVE-1434
Project: Hive
Issue Type: New Feature
Affects Versions: 0.7.0
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Attachments: cas-handle.tar.gz, cass_handler.diff, hive-1434-1.txt,
hive-1434-2-patch.txt, hive-1434-2011-02-26.patch.txt, hive-1434-3-patch.txt,
hive-1434-4-patch.txt, hive-1434-5.patch.txt, hive-1434.2011-02-27.diff.txt,
hive-cassandra.2011-02-25.txt, hive.diff
Add a cassandra storage handler.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1434) Cassandra Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13003661#comment-13003661
]
John Sichi commented on HIVE-1434:
--
After manual mkdir cassandra-handler/lib, I tried again and got
{noformat}
...
test:
[junit] Running org.apache.hadoop.hive.cli.TestCassandraCliDriver
[junit] SLF4J: Class path contains multiple SLF4J bindings.
[junit] SLF4J: Found binding in
[jar:file:/data/users/jsichi/open/hive-trunk/build/ivy/lib/default/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
[junit] SLF4J: Found binding in
[jar:file:/data/users/jsichi/open/hive-trunk/build/hadoopcore/hadoop-0.20.1/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
[junit] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
[junit] org.apache.thrift.TApplicationException: Internal error processing
system_add_keyspace
[junit] at
org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
[junit] at
org.apache.cassandra.thrift.Cassandra$Client.recv_system_add_keyspace(Cassandra.java:1403)
[junit] at
org.apache.cassandra.thrift.Cassandra$Client.system_add_keyspace(Cassandra.java:1386)
[junit] at
org.apache.hadoop.hive.cassandra.CassandraTestSetup.preTest(CassandraTestSetup.java:56)
[junit] at
org.apache.hadoop.hive.cassandra.CassandraQTestUtil.init(CassandraQTestUtil.java:14)
[junit] at
org.apache.hadoop.hive.cli.TestCassandraCliDriver.setUp(TestCassandraCliDriver.java:41)
[junit] at junit.framework.TestCase.runBare(TestCase.java:125)
[junit] at junit.framework.TestResult$1.protect(TestResult.java:106)
[junit] at junit.framework.TestResult.runProtected(TestResult.java:124)
[junit] at junit.framework.TestResult.run(TestResult.java:109)
[junit] at junit.framework.TestCase.run(TestCase.java:118)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:208)
[junit] at junit.framework.TestSuite.run(TestSuite.java:203)
[junit] at
junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
[junit] at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
[junit] at junit.framework.TestResult.runProtected(TestResult.java:124)
[junit] at junit.extensions.TestSetup.run(TestSetup.java:23)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)
[junit] Exception: Internal error processing system_add_keyspace
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 1.329 sec
[junit] Test org.apache.hadoop.hive.cli.TestCassandraCliDriver FAILED
[for] /data/users/jsichi/open/hive-trunk/cassandra-handler/build.xml: The
following error occurred while executing this line:
[for] /data/users/jsichi/open/hive-trunk/build.xml:214: The following
error occurred while executing this line:
[for] /data/users/jsichi/open/hive-trunk/build-common.xml:455: Tests
failed!
...
{noformat}
Cassandra Storage Handler
-
Key: HIVE-1434
URL: https://issues.apache.org/jira/browse/HIVE-1434
Project: Hive
Issue Type: New Feature
Affects Versions: 0.7.0
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Attachments: cas-handle.tar.gz, cass_handler.diff, hive-1434-1.txt,
hive-1434-2-patch.txt, hive-1434-2011-02-26.patch.txt, hive-1434-3-patch.txt,
hive-1434-4-patch.txt, hive-1434-5.patch.txt, hive-1434.2011-02-27.diff.txt,
hive-cassandra.2011-02-25.txt, hive.diff
Add a cassandra storage handler.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Jenkins build is back to normal : Hive-trunk-h0.20 #598
See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/598/changes
Re: Review Request: HIVE-1803: Implement bitmap indexing in Hive
On None, John Sichi wrote: ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java, line 50 https://reviews.apache.org/r/466/diff/1/?file=13567#file13567line50 The point of factoring out the compact index handler base class was so that you could eliminate most of the code in this class, right? :) I don't know what happened with the refactoring here. I swear I did it right. Anyways, I'll fix it in the next patch. On None, John Sichi wrote: ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCollectBitmapSet.java, line 46 https://reviews.apache.org/r/466/diff/1/?file=13579#file13579line46 Couldn't you make this a single-parameter UDF which just tests whether a bitmap is empty or not? Then use the existing UDAF collect_set to collect the distinct block offsets. Duh. Should have thought of that. On None, John Sichi wrote: ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCollectBitmapSet.java, line 128 https://reviews.apache.org/r/466/diff/1/?file=13579#file13579line128 Hmmm...looking at the EWAH code, we could actually make our decision by reading just the header to avoid having to deserialize the whole thing. I don't think we can. The header shows the actual size in bits, but all of those bits can be zero. I think we still need to deserialize the entire bitmap in order to decide. On None, John Sichi wrote: ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBitmap.java, line 193 https://reviews.apache.org/r/466/diff/1/?file=13578#file13578line193 Couldn't you avoid this copying by having the BitmapObjectInput/Output already work in terms of LongWritable? When we convert the BitmapObjectInput/Output to work with LongWritables, we then need to worry about the bitmap_and, bitmap_or, (and the new bitmap_empty) udfs, since they take the hive datatype arraybigint as an argument, which is never represented as an ArrayLongWritable. For the next patch I'm preparing, I think I'm just going to keep the copying code in those UDFs except copy stuff into an ArrayLongWritable to pass to the BitmapObjectInput constructor. Do you think it would be better to pass the entire array object to BitmapObjectInput along with the ListObjectInspector and other classes required to read from the array? - Marquis --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/466/#review303 --- On 2011-03-04 14:34:35, John Sichi wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/466/ --- (Updated 2011-03-04 14:34:35) Review request for hive. Summary --- Review by JVS. This addresses bug HIVE-1803. https://issues.apache.org/jira/browse/HIVE-1803 Diffs - lib/README 1c2f0b1 lib/javaewah.jar PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java af2bacb ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java ff74f08 ql/src/java/org/apache/hadoop/hive/ql/index/AbstractIndexTableIndexHandler.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndex.java 308d985 ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexTableIndexInputFormat.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexTableIndexResult.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/index/IndexMetadataChangeTask.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/index/IndexMetadataChangeWork.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapObjectInput.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapObjectOutput.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 1f01446 ql/src/java/org/apache/hadoop/hive/ql/index/compact/HiveCompactIndexInputFormat.java 6c320c5 ql/src/java/org/apache/hadoop/hive/ql/index/compact/HiveCompactIndexResult.java 0c9ccea ql/src/java/org/apache/hadoop/hive/ql/index/compact/IndexMetadataChangeTask.java eac168f ql/src/java/org/apache/hadoop/hive/ql/index/compact/IndexMetadataChangeWork.java 26beb4e ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java 391e5de ql/src/java/org/apache/hadoop/hive/ql/io/IOContext.java 77220a1 ql/src/java/org/apache/hadoop/hive/ql/metadata/VirtualColumn.java 30714b8 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBitmap.java PRE-CREATION
clean continuous integration run
Yay, our first clean continuous integration test run in ages! There are still some known sporadic failures due to the System.exit race condition mentioned in HIVE-1872. From here on, when committing new patches, if you hit an exception during testing, please don't commit until you've made sure that it's a known sporadic failure. If it's anything else (e.g. a problem with your build/test environment), please clean that up first before proceeding. Once breakage gets checked in, it's very difficult to stop new ones from creeping in since everyone stops trusting the test results. Thanks, JVS Begin forwarded message: From: Apache Hudson Server hud...@hudson.apache.org Date: March 7, 2011 3:16:58 PM PST To: dev@hive.apache.org, cwsteinb...@gmail.com, jsi...@gmail.com, nzh...@facebook.com, jsensa...@gmail.com, py...@facebook.com, heyongqiang...@gmail.com, namit_j...@yahoo.com Subject: Jenkins build is back to normal : Hive-trunk-h0.20 #598 Reply-To: dev@hive.apache.org See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/598/changes
Build failed in Jenkins: Hive-0.7.0-h0.20 #30
See https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/30/changes Changes: [jvs] HIVE-1939. Fix test failure in TestContribCliDriver/url_hook.q (Carl Steinbach via jvs) -- [...truncated 26818 lines...] [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/build/service/tmp/hive_job_log_hudson_201103071544_2136713830.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select count(1) as cnt from testhivedrivertable [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-03-07_15-45-00_253_1050537854220498624/-mr-1 [junit] Total MapReduce jobs = 1 [junit] Launching Job 1 out of 1 [junit] Number of reduce tasks determined at compile time: 1 [junit] In order to change the average load for a reducer (in bytes): [junit] set hive.exec.reducers.bytes.per.reducer=number [junit] In order to limit the maximum number of reducers: [junit] set hive.exec.reducers.max=number [junit] In order to set a constant number of reducers: [junit] set mapred.reduce.tasks=number [junit] Job running in-process (local Hadoop) [junit] 2011-03-07 15:45:03,292 null map = 100%, reduce = 100% [junit] Ended Job = job_local_0001 [junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-03-07_15-45-00_253_1050537854220498624/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/build/service/tmp/hive_job_log_hudson_201103071545_811591670.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select * from testhivedrivertable limit 10 [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-03-07_15-45-05_671_8564071370878467063/-mr-1 [junit] POSTHOOK: query: select * from testhivedrivertable limit 10 [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output:
[jira] Updated: (HIVE-1803) Implement bitmap indexing in Hive
[ https://issues.apache.org/jira/browse/HIVE-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marquis Wang updated HIVE-1803: --- Status: Patch Available (was: Open) Implement bitmap indexing in Hive - Key: HIVE-1803 URL: https://issues.apache.org/jira/browse/HIVE-1803 Project: Hive Issue Type: New Feature Components: Indexing Reporter: Marquis Wang Assignee: Marquis Wang Attachments: HIVE-1803.1.patch, HIVE-1803.2.patch, HIVE-1803.3.patch, HIVE-1803.4.patch, HIVE-1803.5.patch, bitmap_index_1.png, bitmap_index_2.png, javaewah.jar, javaewah.jar Implement bitmap index handler to complement compact indexing. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1803) Implement bitmap indexing in Hive
[ https://issues.apache.org/jira/browse/HIVE-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13003735#comment-13003735 ] John Sichi commented on HIVE-1803: -- Table src has two columns (key and value). The value is equivalent to the key. For srcbucket, the value is one plus the key, so at least they're not exactly the same. Implement bitmap indexing in Hive - Key: HIVE-1803 URL: https://issues.apache.org/jira/browse/HIVE-1803 Project: Hive Issue Type: New Feature Components: Indexing Reporter: Marquis Wang Assignee: Marquis Wang Attachments: HIVE-1803.1.patch, HIVE-1803.2.patch, HIVE-1803.3.patch, HIVE-1803.4.patch, HIVE-1803.5.patch, bitmap_index_1.png, bitmap_index_2.png, javaewah.jar, javaewah.jar Implement bitmap index handler to complement compact indexing. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HIVE-2030) isEmptyPath() to use ContentSummary cache
isEmptyPath() to use ContentSummary cache - Key: HIVE-2030 URL: https://issues.apache.org/jira/browse/HIVE-2030 Project: Hive Issue Type: Improvement Reporter: Siying Dong Assignee: Siying Dong Priority: Minor addInputPaths() calls isEmptyPath() for every input path. Now every call is a DFS namenode call. Making isEmptyPath() to use cached ContentSummary, we should be able to avoid some namenode calls and reduce latency in the case of multiple partitions. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
