[jira] [Updated] (HIVE-2901) Hive union with NULL constant and string in same column returns all null
[ https://issues.apache.org/jira/browse/HIVE-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2901: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Navis Hive union with NULL constant and string in same column returns all null Key: HIVE-2901 URL: https://issues.apache.org/jira/browse/HIVE-2901 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Navis Priority: Critical Attachments: HIVE-2901.D2541.1.patch, HIVE-2901.D2541.2.patch, HIVE-2901.D2541.3.patch select x from (select value as x from src union all select NULL as x from src)a; This query produces all nulls, where value is a string column. Notably, select x from (select key as x from src union all select NULL as x from src)a; where key is a string, but can be cast to a double, the query returns correct results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2929) race condition in DAG execute tasks for hive
race condition in DAG execute tasks for hive Key: HIVE-2929 URL: https://issues.apache.org/jira/browse/HIVE-2929 Project: Hive Issue Type: Bug Reporter: Namit Jain select ... ( SubQuery involving MapReduce union all SubQuery involving MapReduce ); or select ... (SubQuery involving MapReduce) join (SubQuery involving MapReduce) ; If both the subQueries finish at nearly the same time, there is a race condition in which the results of the subQuery finishing last will be completely missed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2929) race condition in DAG execute tasks for hive
[ https://issues.apache.org/jira/browse/HIVE-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248114#comment-13248114 ] Namit Jain commented on HIVE-2929: -- https://reviews.facebook.net/differential/diff/8571/ race condition in DAG execute tasks for hive Key: HIVE-2929 URL: https://issues.apache.org/jira/browse/HIVE-2929 Project: Hive Issue Type: Bug Reporter: Namit Jain select ... ( SubQuery involving MapReduce union all SubQuery involving MapReduce ); or select ... (SubQuery involving MapReduce) join (SubQuery involving MapReduce) ; If both the subQueries finish at nearly the same time, there is a race condition in which the results of the subQuery finishing last will be completely missed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1721) use bloom filters to improve the performance of joins
[ https://issues.apache.org/jira/browse/HIVE-1721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248115#comment-13248115 ] alex gemini commented on HIVE-1721: --- The original thought is increase the map side small table size,this is very depend on how we chunk big table.if big table is chunk into buckets for 16 buckets,the small table must be automatic change to 16 buckets too(same logic bucket map join).if big table is partition by (region string),the small table also need to partition by (region) first and also make sure the smallest chunk size is not bigger than current small table size define by hive.The partition case is more often,we can avoid common join by always let small table chunk like big table's format. use bloom filters to improve the performance of joins - Key: HIVE-1721 URL: https://issues.apache.org/jira/browse/HIVE-1721 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Namit Jain Labels: gsoc, gsoc2012, optimization In case of map-joins, it is likely that the big table will not find many matching rows from the small table. Currently, we perform a hash-map lookup for every row in the big table, which can be pretty expensive. It might be useful to try out a bloom-filter containing all the elements in the small table. Each element from the big table is first searched in the bloom filter, and only in case of a positive match, the small table hash table is explored. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2863) Ambiguous table name or column reference message displays when table and column names are the same
[ https://issues.apache.org/jira/browse/HIVE-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248157#comment-13248157 ] Hudson commented on HIVE-2863: -- Integrated in Hive-trunk-h0.21 #1356 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1356/]) HIVE-2863 Ambiguous table name or column reference message displays when table and column names are the same (Navis via namit) (Revision 1310126) Result = SUCCESS namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1310126 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java * /hive/trunk/ql/src/test/queries/clientpositive/nomore_ambiguous_table_col.q * /hive/trunk/ql/src/test/queries/negative/ambiguous_table_col.q * /hive/trunk/ql/src/test/results/clientpositive/nomore_ambiguous_table_col.q.out * /hive/trunk/ql/src/test/results/compiler/errors/ambiguous_table_col.q.out Ambiguous table name or column reference message displays when table and column names are the same -- Key: HIVE-2863 URL: https://issues.apache.org/jira/browse/HIVE-2863 Project: Hive Issue Type: Bug Reporter: Mauro Cazzari Assignee: Navis Attachments: HIVE-2863.D2361.1.patch Given the following table: CREATE TABLE `Y` (`y` DOUBLE) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' STORED AS TEXTFILE; The following query fails: SELECT `Y`.`y` FROM `Y` WHERE ( `y` = 1 ) ERROR: java.sql.SQLException: Query returned non-zero code: 10, cause: FAILED: Error in semantic analysis: Line 1:36 Ambiguous table alias or column reference '`y`' ERROR: Unable to execute Hadoop query. ERROR: Prepare error. SQL statement: SELECT `Y`.`y` FROM `Y` WHERE ( `y` = 1 ). The problem goes away if the table and column names do not match. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2901) Hive union with NULL constant and string in same column returns all null
[ https://issues.apache.org/jira/browse/HIVE-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248260#comment-13248260 ] Hudson commented on HIVE-2901: -- Integrated in Hive-trunk-h0.21 #1357 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1357/]) HIVE-2901 Hive union with NULL constant and string in same column returns all null (Navis via namit) (Revision 1310179) Result = SUCCESS namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1310179 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java * /hive/trunk/ql/src/test/queries/clientpositive/union_null.q * /hive/trunk/ql/src/test/results/clientpositive/union_null.q.out Hive union with NULL constant and string in same column returns all null Key: HIVE-2901 URL: https://issues.apache.org/jira/browse/HIVE-2901 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Navis Priority: Critical Attachments: HIVE-2901.D2541.1.patch, HIVE-2901.D2541.2.patch, HIVE-2901.D2541.3.patch select x from (select value as x from src union all select NULL as x from src)a; This query produces all nulls, where value is a string column. Notably, select x from (select key as x from src union all select NULL as x from src)a; where key is a string, but can be cast to a double, the query returns correct results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2923) testAclPositive in TestZooKeeperTokenStore failing in clean checkout when run on Mac
[ https://issues.apache.org/jira/browse/HIVE-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2923: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Thomas! testAclPositive in TestZooKeeperTokenStore failing in clean checkout when run on Mac Key: HIVE-2923 URL: https://issues.apache.org/jira/browse/HIVE-2923 Project: Hive Issue Type: Bug Affects Versions: 0.9.0 Environment: Mac OSX Lion Reporter: Kevin Wilfong Assignee: Thomas Weise Priority: Blocker Fix For: 0.9.0 Attachments: HIVE-2923.patch When running testAclPositive in TestZooKeeperTokenStore in a clean checkout, it fails with the error: Failed to validate token path. org.apache.hadoop.hive.thrift.DelegationTokenStore$TokenStoreException: Failed to validate token path. at org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.init(ZooKeeperTokenStore.java:207) at org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.setConf(ZooKeeperTokenStore.java:225) at org.apache.hadoop.hive.thrift.TestZooKeeperTokenStore.testAclPositive(TestZooKeeperTokenStore.java:170) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /zktokenstore-testAcl at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:778) at org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.ensurePath(ZooKeeperTokenStore.java:119) at org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.init(ZooKeeperTokenStore.java:204) ... 17 more This message is also printed to standard out: Unable to load realm mapping info from SCDynamicStore The test seems to run fine in Linux, but more than one developer has reported this on a Mac. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2924) Clean up warnings in RCFile
[ https://issues.apache.org/jira/browse/HIVE-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley reassigned HIVE-2924: --- Assignee: Owen O'Malley Clean up warnings in RCFile --- Key: HIVE-2924 URL: https://issues.apache.org/jira/browse/HIVE-2924 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Currently we get a couple of warnings in compiling RCFile and I want to clean them up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2930) Add license to the
Add license to the --- Key: HIVE-2930 URL: https://issues.apache.org/jira/browse/HIVE-2930 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Priority: Blocker We need to clean up the RAT report to 0. Apache projects aren't supposed to release until they have shown that all of their files have proper headers. Note that although most of the files are just missing headers, some of them explicitly say copyright by facebook and released under the Thrift (not Apache) license. I'll generate a list of them, but I'd really appreciate it if someone from facebook could in verify that they intend to license them to Apache. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2930) Add license to the Hive files
[ https://issues.apache.org/jira/browse/HIVE-2930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-2930: Summary: Add license to the Hive files (was: Add license to the ) Add license to the Hive files - Key: HIVE-2930 URL: https://issues.apache.org/jira/browse/HIVE-2930 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley Priority: Blocker We need to clean up the RAT report to 0. Apache projects aren't supposed to release until they have shown that all of their files have proper headers. Note that although most of the files are just missing headers, some of them explicitly say copyright by facebook and released under the Thrift (not Apache) license. I'll generate a list of them, but I'd really appreciate it if someone from facebook could in verify that they intend to license them to Apache. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2930) Add license to the Hive files
[
https://issues.apache.org/jira/browse/HIVE-2930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248586#comment-13248586
]
Owen O'Malley commented on HIVE-2930:
-
The files that are copyright by Facebook are:
{code}
service/lib/php/ext/thrift_protocol/config.m4
service/lib/py/thrift/protocol/fastbinary.c
service/lib/py/thrift/reflection/__init__.py
{code}
Add license to the Hive files
-
Key: HIVE-2930
URL: https://issues.apache.org/jira/browse/HIVE-2930
Project: Hive
Issue Type: Improvement
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Priority: Blocker
We need to clean up the RAT report to 0. Apache projects aren't supposed to
release until they have shown that all of their files have proper headers.
Note that although most of the files are just missing headers, some of them
explicitly say copyright by facebook and released under the Thrift (not
Apache) license. I'll generate a list of them, but I'd really appreciate it
if someone from facebook could in verify that they intend to license them to
Apache.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2931) conf settings may be ignored
conf settings may be ignored Key: HIVE-2931 URL: https://issues.apache.org/jira/browse/HIVE-2931 Project: Hive Issue Type: Bug Reporter: Namit Jain This is a pretty serious problem. If a conf variable is changed, Hive may not pick up the variable unless the metastore variables are changed. When any session variables are changed, it might be simpler to update the corresponding Hive conf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Hive 0.9 release
Hi All, Seems like we have an agreement. So, unless someone has other ideas, I will cut the branch for 0.9 on 4/9. Below is the consolidated list which people have requested for 0.9, so as and when these gets checked-in trunk, we will merge it back in 0.9. 2084 2764 538 2646 2777 2585 2883 2926 Thanks, Ashutosh On Mon, Apr 2, 2012 at 19:02, Thomas Weise t...@yahoo-inc.com wrote: Would be great to get HIVE-2646 included. Thanks, Thomas On 4/2/12 6:59 PM, Ashutosh Chauhan hashut...@apache.org wrote: Here is a list of jiras which I plan to get in 0.9. HIVE-2084 HIVE-2822 HIVE-2764 HIVE-538 I will work with authors of these patches to see these can get in. Others, please feel free to add this list. Thanks, Ashutosh On Mon, Apr 2, 2012 at 18:39, Carl Steinbach c...@cloudera.com wrote: I'm +1 on doing an 0.9.0 release, but would also like to suggest that we put together a list of 0.9.0 blockers before cutting the release branch. In the past we have frequently underestimated the amount of work required to get trunk into a releasable state, with the consequence that we up wasting time doing a lot of backports from trunk to the release branch. It would be great if we could avoid all of that this time around. Thanks. Carl On Mon, Apr 2, 2012 at 6:33 PM, Ashutosh Chauhan hashut...@apache.org wrote: Hi all, Branch for 0.8-r2 was created on Dec 7, almost four months ago. Between then and now lots of cool stuff has landed in trunk waiting to be released and get to users. I think its a good time now to get the ball rolling for 0.9 release. If this sounds good, I would propose to cut a branch for 0.9 later this week. Then we can focus on stabilizing the branch and subsequent release from it. Thoughts? Thanks, Ashutosh
Re: Hive 0.9 release
Would it be possible/feasible to include 2910? Cheers, Hugo Verstuurd vanaf mijn iPhone Op 6 apr. 2012 om 21:18 heeft Ashutosh Chauhan hashut...@apache.org het volgende geschreven: Hi All, Seems like we have an agreement. So, unless someone has other ideas, I will cut the branch for 0.9 on 4/9. Below is the consolidated list which people have requested for 0.9, so as and when these gets checked-in trunk, we will merge it back in 0.9. 2084 2764 538 2646 2777 2585 2883 2926 Thanks, Ashutosh On Mon, Apr 2, 2012 at 19:02, Thomas Weise t...@yahoo-inc.com wrote: Would be great to get HIVE-2646 included. Thanks, Thomas On 4/2/12 6:59 PM, Ashutosh Chauhan hashut...@apache.org wrote: Here is a list of jiras which I plan to get in 0.9. HIVE-2084 HIVE-2822 HIVE-2764 HIVE-538 I will work with authors of these patches to see these can get in. Others, please feel free to add this list. Thanks, Ashutosh On Mon, Apr 2, 2012 at 18:39, Carl Steinbach c...@cloudera.com wrote: I'm +1 on doing an 0.9.0 release, but would also like to suggest that we put together a list of 0.9.0 blockers before cutting the release branch. In the past we have frequently underestimated the amount of work required to get trunk into a releasable state, with the consequence that we up wasting time doing a lot of backports from trunk to the release branch. It would be great if we could avoid all of that this time around. Thanks. Carl On Mon, Apr 2, 2012 at 6:33 PM, Ashutosh Chauhan hashut...@apache.org wrote: Hi all, Branch for 0.8-r2 was created on Dec 7, almost four months ago. Between then and now lots of cool stuff has landed in trunk waiting to be released and get to users. I think its a good time now to get the ball rolling for 0.9 release. If this sounds good, I would propose to cut a branch for 0.9 later this week. Then we can focus on stabilizing the branch and subsequent release from it. Thoughts? Thanks, Ashutosh
[jira] [Assigned] (HIVE-2931) conf settings may be ignored
[ https://issues.apache.org/jira/browse/HIVE-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain reassigned HIVE-2931: Assignee: Namit Jain conf settings may be ignored Key: HIVE-2931 URL: https://issues.apache.org/jira/browse/HIVE-2931 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain This is a pretty serious problem. If a conf variable is changed, Hive may not pick up the variable unless the metastore variables are changed. When any session variables are changed, it might be simpler to update the corresponding Hive conf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2931) conf settings may be ignored
[ https://issues.apache.org/jira/browse/HIVE-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248643#comment-13248643 ] Namit Jain commented on HIVE-2931: -- https://reviews.facebook.net/differential/diff/8583/ conf settings may be ignored Key: HIVE-2931 URL: https://issues.apache.org/jira/browse/HIVE-2931 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain This is a pretty serious problem. If a conf variable is changed, Hive may not pick up the variable unless the metastore variables are changed. When any session variables are changed, it might be simpler to update the corresponding Hive conf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2931) conf settings may be ignored
[ https://issues.apache.org/jira/browse/HIVE-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2931: - Attachment: hive.2931.1.patch conf settings may be ignored Key: HIVE-2931 URL: https://issues.apache.org/jira/browse/HIVE-2931 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.2931.1.patch This is a pretty serious problem. If a conf variable is changed, Hive may not pick up the variable unless the metastore variables are changed. When any session variables are changed, it might be simpler to update the corresponding Hive conf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
alter table set serde changes table schema?
Hey hive gurus - I'm creating some tables to represent thrift structs, and noticed when setting the serde the table schema changes. For example: hive describe foo_test; OK start_time_ms bigint bar structtransaction_id:string Set the serde: hive alter table foo_test set serde com.twitter.elephantbird.hive.serde.ThriftSerDe with serdeproperties (serialization.class=com.foo.Foo); Now the schema is different. Notice how the field type is a class name - it matches the thrift struct type stored in the field. hive describe foo_test; OK start_time_ms bigint from deserializer bar com.foo.Bar from deserializer I've spent some time watching how the object inspectors work in a debugger, and its correctly identifying the subfields inside field bar. However, the table column type gets set to the class name, instead of expanding to the schema of the thrift struct. Can anyone clue me into how this is intended to work? I'm using the standard thrift object inspector, and the serde I'm writing is: https://github.com/kevinweil/elephant-bird/blob/master/src/java/com/twitter/elephantbird/hive/serde/ThriftSerDe.java Any help in understanding how to correctly set the table schema would be greatly appreciated. Thanks! Travis
[jira] [Commented] (HIVE-2923) testAclPositive in TestZooKeeperTokenStore failing in clean checkout when run on Mac
[ https://issues.apache.org/jira/browse/HIVE-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248745#comment-13248745 ] Hudson commented on HIVE-2923: -- Integrated in Hive-trunk-h0.21 #1358 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1358/]) HIVE-2923 : testAclPositive in TestZooKeeperTokenStore failing in clean checkout when run on Mac (Thomas Weise via Ashutosh Chauhan) (Revision 1310406) Result = SUCCESS hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1310406 Files : * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java * /hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java testAclPositive in TestZooKeeperTokenStore failing in clean checkout when run on Mac Key: HIVE-2923 URL: https://issues.apache.org/jira/browse/HIVE-2923 Project: Hive Issue Type: Bug Affects Versions: 0.9.0 Environment: Mac OSX Lion Reporter: Kevin Wilfong Assignee: Thomas Weise Priority: Blocker Fix For: 0.9.0 Attachments: HIVE-2923.patch When running testAclPositive in TestZooKeeperTokenStore in a clean checkout, it fails with the error: Failed to validate token path. org.apache.hadoop.hive.thrift.DelegationTokenStore$TokenStoreException: Failed to validate token path. at org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.init(ZooKeeperTokenStore.java:207) at org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.setConf(ZooKeeperTokenStore.java:225) at org.apache.hadoop.hive.thrift.TestZooKeeperTokenStore.testAclPositive(TestZooKeeperTokenStore.java:170) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /zktokenstore-testAcl at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:778) at org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.ensurePath(ZooKeeperTokenStore.java:119) at org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.init(ZooKeeperTokenStore.java:204) ... 17 more This message is also printed to standard out: Unable to load realm mapping info from SCDynamicStore The test seems to run fine in Linux, but more than one developer has reported this on a Mac. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2926) Expose some information about the metastore through JMX
[ https://issues.apache.org/jira/browse/HIVE-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2926: -- Attachment: HIVE-2926.D2655.1.patch ddraj requested code review of HIVE-2926 [jira] Expose some information about the metastore through JMX. Reviewers: JIRA Patch that exposes some additional details in the metastore through jmx Expose some information about the metastore through JMX TEST PLAN Manually tested REVISION DETAIL https://reviews.facebook.net/D2655 AFFECTED FILES build-common.xml common/build.xml common/src/java/org/apache/hadoop/hive/common/HiveVersionAnnotation.java common/src/java/org/apache/hadoop/hive/common/VersionInfo.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java metastore/src/java/org/apache/hadoop/hive/metastore/MXBean.java metastore/src/java/org/apache/hadoop/hive/metastore/MXBeanImpl.java saveVersion.sh MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6105/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. Expose some information about the metastore through JMX --- Key: HIVE-2926 URL: https://issues.apache.org/jira/browse/HIVE-2926 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.9.0 Attachments: HIVE-2926.D2637.1.patch, HIVE-2926.D2655.1.patch, hive-2926.1.patch, metastore-jmx.patch Expose some information about the metastore through JMX -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2932) TestHBaseCliDriver breaking in trunk
TestHBaseCliDriver breaking in trunk Key: HIVE-2932 URL: https://issues.apache.org/jira/browse/HIVE-2932 Project: Hive Issue Type: Bug Reporter: Namit Jain I am getting 3 failures in clean trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2858) Cache remote map reduce job stack traces for additional logging
[ https://issues.apache.org/jira/browse/HIVE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2858: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Kevin Cache remote map reduce job stack traces for additional logging --- Key: HIVE-2858 URL: https://issues.apache.org/jira/browse/HIVE-2858 Project: Hive Issue Type: Improvement Components: Logging Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2858.D2223.1.patch, HIVE-2858.D2223.2.patch Currently we are parsing the task logs for failed jobs for information to display to the user in the CLI. In addition, we could parse those logs for stack traces and store e them in the SessionState. This way, when we log failed queries, these will give us a decent idea of why those queries failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2932) TestHBaseCliDriver breaking in trunk
[ https://issues.apache.org/jira/browse/HIVE-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248769#comment-13248769 ] Ashutosh Chauhan commented on HIVE-2932: I just ran the tests and all of them passed. I have run them on multiple machines on past few days, all of them always pass. Something peculiar to your environment ? TestHBaseCliDriver breaking in trunk Key: HIVE-2932 URL: https://issues.apache.org/jira/browse/HIVE-2932 Project: Hive Issue Type: Bug Reporter: Namit Jain I am getting 3 failures in clean trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2926) Expose some information about the metastore through JMX
[ https://issues.apache.org/jira/browse/HIVE-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248771#comment-13248771 ] Phabricator commented on HIVE-2926: --- ddraj has added reviewers to the revision HIVE-2926 [jira] Expose some information about the metastore through JMX. Added Reviewers: cwsteinbach Sorry Carl, I messed up my workspace and I had to generate a fresh review request instead of updating the previous patch with your comments incorporated.. I took into consideration most of your comments. I made some changes to fix issues found in my testing. REVISION DETAIL https://reviews.facebook.net/D2655 Expose some information about the metastore through JMX --- Key: HIVE-2926 URL: https://issues.apache.org/jira/browse/HIVE-2926 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.9.0 Attachments: HIVE-2926.D2637.1.patch, HIVE-2926.D2655.1.patch, hive-2926.1.patch, metastore-jmx.patch Expose some information about the metastore through JMX -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2932) TestHBaseCliDriver breaking in trunk
[ https://issues.apache.org/jira/browse/HIVE-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248770#comment-13248770 ] Ashutosh Chauhan commented on HIVE-2932: Also, apache build machines are reporting success too: https://builds.apache.org/job/Hive-trunk-h0.21/ TestHBaseCliDriver breaking in trunk Key: HIVE-2932 URL: https://issues.apache.org/jira/browse/HIVE-2932 Project: Hive Issue Type: Bug Reporter: Namit Jain I am getting 3 failures in clean trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248781#comment-13248781 ] Phabricator commented on HIVE-2585: --- ashutoshc has commented on the revision HIVE-2585 [jira] Collapse hive.metastore.uris and hive.metastore.local. INLINE COMMENTS conf/hive-default.xml.template:102 Valid point. Then, instead of throwing exception, we can log a message then. REVISION DETAIL https://reviews.facebook.net/D2559 BRANCH svn Collapse hive.metastore.uris and hive.metastore.local - Key: HIVE-2585 URL: https://issues.apache.org/jira/browse/HIVE-2585 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-2585.D2559.1.patch We should just have hive.metastore.uris. If it is empty, we shall assume local mode, if non-empty we shall use that string to connect to remote metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2929) race condition in DAG execute tasks for hive
[ https://issues.apache.org/jira/browse/HIVE-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2929: - Attachment: hive.2929.1.patch race condition in DAG execute tasks for hive Key: HIVE-2929 URL: https://issues.apache.org/jira/browse/HIVE-2929 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.2929.1.patch select ... ( SubQuery involving MapReduce union all SubQuery involving MapReduce ); or select ... (SubQuery involving MapReduce) join (SubQuery involving MapReduce) ; If both the subQueries finish at nearly the same time, there is a race condition in which the results of the subQuery finishing last will be completely missed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2929) race condition in DAG execute tasks for hive
[ https://issues.apache.org/jira/browse/HIVE-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain reassigned HIVE-2929: Assignee: Namit Jain race condition in DAG execute tasks for hive Key: HIVE-2929 URL: https://issues.apache.org/jira/browse/HIVE-2929 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.2929.1.patch select ... ( SubQuery involving MapReduce union all SubQuery involving MapReduce ); or select ... (SubQuery involving MapReduce) join (SubQuery involving MapReduce) ; If both the subQueries finish at nearly the same time, there is a race condition in which the results of the subQuery finishing last will be completely missed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2929) race condition in DAG execute tasks for hive
[ https://issues.apache.org/jira/browse/HIVE-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2929: - Status: Patch Available (was: Open) race condition in DAG execute tasks for hive Key: HIVE-2929 URL: https://issues.apache.org/jira/browse/HIVE-2929 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.2929.1.patch select ... ( SubQuery involving MapReduce union all SubQuery involving MapReduce ); or select ... (SubQuery involving MapReduce) join (SubQuery involving MapReduce) ; If both the subQueries finish at nearly the same time, there is a race condition in which the results of the subQuery finishing last will be completely missed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2930) Add license to the Hive files
[
https://issues.apache.org/jira/browse/HIVE-2930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248891#comment-13248891
]
Owen O'Malley commented on HIVE-2930:
-
I ran through the files without headers and found the commit that first added
them and converted it to a jira from the log message. My intent was to show
that all of the files that I was adding headers to had been properly
contributed to the project via the checkbox on jira or the contributor's ICLA.
Here's the list of Jiras that added the files that are currently without
headers:
{code}
HADOOP-3601
HADOOP-4230
HADOOP-4566
HIVE-65
HIVE-82
HIVE-166
HIVE-203
HIVE-251
HIVE-299
HIVE-343
HIVE-382
HIVE-675
HIVE-705
HIVE-713
HIVE-956
HIVE-1003
HIVE-1104
HIVE-1135
HIVE-1163
HIVE-1204
HIVE-1361
HIVE-1405
HIVE-1513
HIVE-1609
HIVE-1616
HIVE-1867
HIVE-1983
HIVE-2011
HIVE-2076
HIVE-2185
HIVE-2244
HIVE-2246 *** contributed by Sohan Jain
HIVE-2441
HIVE-2442
HIVE-2523 (covered by John Sichi's ICLA)
HIVE-2795 (covered by Kevin Wilfong's ICLA)
HIVE-2797
HIVE-2805 (covered by Kevin Wilfong's ICLA)
HIVE-2853 (covered by Kevin Wilfong's ICLA)
HIVE-2866 (covered by Kevin Wilfong's ICLA)
{code}
The only one that doesn't seem covered is HIVE-2246, which was contributed by
Sohan Jain. I'll start a discussion on the dev list rather than continue on in
a jira. *smile*
Add license to the Hive files
-
Key: HIVE-2930
URL: https://issues.apache.org/jira/browse/HIVE-2930
Project: Hive
Issue Type: Improvement
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Priority: Blocker
We need to clean up the RAT report to 0. Apache projects aren't supposed to
release until they have shown that all of their files have proper headers.
Note that although most of the files are just missing headers, some of them
explicitly say copyright by facebook and released under the Thrift (not
Apache) license. I'll generate a list of them, but I'd really appreciate it
if someone from facebook could in verify that they intend to license them to
Apache.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Hive 0.9 release
I think we also need to get the RAT report cleaned up. Apache projects aren't supposed to release while they have files without the Apache header. I've filed HIVE-2930 to fix all of the issues. While working on it, I found that one of the files was added by HIVE-2246. HIVE-2246 was contributed by Sohan Jain, who hasn't filed an ICLA, and doesn't have the jira box checked for contribution. Does someone know him and can ask him to state on the jira that he intended to contribute it? Failing that, I believe he was working at Facebook at the time, so someone else who is still there can upload the patch to the jira? All of this brings up a challenge in that Phabricator and the Apache review tool upload patches to jira without providing a way to check the contribute to Apache box. Without the checkbox we should only commit patches from people who have filed ICLAs. Is there a way to add an option the arc command that will check the box? Even having it *always* check the box is better than having it not check the box. (Although it should warn users that it is doing so.) Thoughts? -- Owen
[jira] [Commented] (HIVE-2929) race condition in DAG execute tasks for hive
[ https://issues.apache.org/jira/browse/HIVE-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248940#comment-13248940 ] Siying Dong commented on HIVE-2929: --- +1 committed. Thanks Namit! race condition in DAG execute tasks for hive Key: HIVE-2929 URL: https://issues.apache.org/jira/browse/HIVE-2929 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.2929.1.patch select ... ( SubQuery involving MapReduce union all SubQuery involving MapReduce ); or select ... (SubQuery involving MapReduce) join (SubQuery involving MapReduce) ; If both the subQueries finish at nearly the same time, there is a race condition in which the results of the subQuery finishing last will be completely missed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2790) Grouping on constant expressions is broken on 0.23
[ https://issues.apache.org/jira/browse/HIVE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13249015#comment-13249015 ] Zhenxiao Luo commented on HIVE-2790: This failure won't reproduce on the current branch, probably due to MAPREDUCE-3952. Grouping on constant expressions is broken on 0.23 -- Key: HIVE-2790 URL: https://issues.apache.org/jira/browse/HIVE-2790 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Zhenxiao Luo Assignee: Zhenxiao Luo Query result(-30.33) is missing in the running output: [junit] Begin query: groupby_neg_float.q [junit] 12/01/24 10:57:53 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir [junit] 12/01/24 10:57:53 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir [junit] 12/01/24 10:57:58 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir [junit] 12/01/24 10:57:58 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/groupby_neg_float.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/groupby_neg_float.q.out [junit] 14c14,15 [junit] POSTHOOK: Output: file:/tmp/cloudera/hive_2012-01-24_10-57-51_652_2049459353574360528/-mr-1 [junit] — [junit] POSTHOOK: Output: file:/data/users/njain/hive5/hive5/build/ql/tmp/815331462/1 [junit] -30.33 [junit] 28c29,30 [junit] POSTHOOK: Output: file:/tmp/cloudera/hive_2012-01-24_10-57-56_909_7738265803221872068/-mr-1 [junit] — [junit] POSTHOOK: Output: file:/data/users/njain/hive5/hive5/build/ql/tmp/2082702070/1 [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. [junit] at junit.framework.Assert.fail(Assert.java:50) [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_neg_float(TestCliDriver.java:129) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit] at java.lang.reflect.Method.invoke(Method.java:616) [junit] at junit.framework.TestCase.runTest(TestCase.java:168) [junit] at junit.framework.TestCase.runBare(TestCase.java:134) [junit] at junit.framework.TestResult$1.protect(TestResult.java:110) [junit] at junit.framework.TestResult.runProtected(TestResult.java:128) [junit] at junit.framework.TestResult.run(TestResult.java:113) [junit] at junit.framework.TestCase.run(TestCase.java:124) [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243) [junit] at junit.framework.TestSuite.run(TestSuite.java:238) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768) [junit] Exception: Client execution results failed with error code = 1 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. [junit] -30.33 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs.) When trying the following(GROUP BY on Floating Point type and string type): FROM src SELECT -30.33 GROUP BY -30.33 LIMIT 1; FROM src SELECT '-30.33' GROUP BY '-30.33' LIMIT 1; Get NULL in result -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2790) Grouping on constant expressions is broken on 0.23
[ https://issues.apache.org/jira/browse/HIVE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo resolved HIVE-2790. Resolution: Fixed This failure won't reproduce on the current branch, MAPREDUCE-3952 get it fixed. Grouping on constant expressions is broken on 0.23 -- Key: HIVE-2790 URL: https://issues.apache.org/jira/browse/HIVE-2790 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Zhenxiao Luo Assignee: Zhenxiao Luo Query result(-30.33) is missing in the running output: [junit] Begin query: groupby_neg_float.q [junit] 12/01/24 10:57:53 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir [junit] 12/01/24 10:57:53 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir [junit] 12/01/24 10:57:58 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir [junit] 12/01/24 10:57:58 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/groupby_neg_float.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/groupby_neg_float.q.out [junit] 14c14,15 [junit] POSTHOOK: Output: file:/tmp/cloudera/hive_2012-01-24_10-57-51_652_2049459353574360528/-mr-1 [junit] — [junit] POSTHOOK: Output: file:/data/users/njain/hive5/hive5/build/ql/tmp/815331462/1 [junit] -30.33 [junit] 28c29,30 [junit] POSTHOOK: Output: file:/tmp/cloudera/hive_2012-01-24_10-57-56_909_7738265803221872068/-mr-1 [junit] — [junit] POSTHOOK: Output: file:/data/users/njain/hive5/hive5/build/ql/tmp/2082702070/1 [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. [junit] at junit.framework.Assert.fail(Assert.java:50) [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_neg_float(TestCliDriver.java:129) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit] at java.lang.reflect.Method.invoke(Method.java:616) [junit] at junit.framework.TestCase.runTest(TestCase.java:168) [junit] at junit.framework.TestCase.runBare(TestCase.java:134) [junit] at junit.framework.TestResult$1.protect(TestResult.java:110) [junit] at junit.framework.TestResult.runProtected(TestResult.java:128) [junit] at junit.framework.TestResult.run(TestResult.java:113) [junit] at junit.framework.TestCase.run(TestCase.java:124) [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243) [junit] at junit.framework.TestSuite.run(TestSuite.java:238) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768) [junit] Exception: Client execution results failed with error code = 1 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. [junit] -30.33 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs.) When trying the following(GROUP BY on Floating Point type and string type): FROM src SELECT -30.33 GROUP BY -30.33 LIMIT 1; FROM src SELECT '-30.33' GROUP BY '-30.33' LIMIT 1; Get NULL in result -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2767) Optionally use framed transport with metastore
[
https://issues.apache.org/jira/browse/HIVE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2767:
--
Attachment: HIVE-2767.D2661.1.patch
travis requested code review of HIVE-2767 [jira] Optionally use framed
transport with metastore.
Reviewers: JIRA
Add support for optionally using the thrift framed transport, enabling
integration with environments where that is necessary.
Users may want/need to use thrift's framed transport when communicating with
the Hive MetaStore. This patch adds a new property
hive.metastore.thrift.framed.transport.enabled that enables the framed
transport (defaults to off, aka no change from before the patch). This property
must be set for both clients and the HMS server.
It wasn't immediately clear how to use the framed transport with SASL, so as
written an exception is thrown if you try starting the server with both
options. If SASL and the framed transport will indeed work together I can
update the patch (although I don't have a secured environment to test in).
TEST PLAN
Tested locally that client and server can connect, both with and without the
flag. Tests pass.
REVISION DETAIL
https://reviews.facebook.net/D2661
AFFECTED FILES
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/6111/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
Optionally use framed transport with metastore
--
Key: HIVE-2767
URL: https://issues.apache.org/jira/browse/HIVE-2767
Project: Hive
Issue Type: New Feature
Components: Metastore
Reporter: Travis Crawford
Assignee: Julien Le Dem
Attachments: HIVE-2767.D2661.1.patch, HIVE-2767.patch.txt,
HIVE-2767_a.patch.txt
Users may want/need to use thrift's framed transport when communicating with
the Hive MetaStore. This patch adds a new property
{{hive.metastore.thrift.framed.transport.enabled}} that enables the framed
transport (defaults to off, aka no change from before the patch). This
property must be set for both clients and the HMS server.
It wasn't immediately clear how to use the framed transport with SASL, so as
written an exception is thrown if you try starting the server with both
options. If SASL and the framed transport will indeed work together I can
update the patch (although I don't have a secured environment to test in).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2767) Optionally use framed transport with metastore
[
https://issues.apache.org/jira/browse/HIVE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Travis Crawford reassigned HIVE-2767:
-
Assignee: Travis Crawford (was: Julien Le Dem)
Optionally use framed transport with metastore
--
Key: HIVE-2767
URL: https://issues.apache.org/jira/browse/HIVE-2767
Project: Hive
Issue Type: New Feature
Components: Metastore
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-2767.D2661.1.patch, HIVE-2767.patch.txt,
HIVE-2767_a.patch.txt
Users may want/need to use thrift's framed transport when communicating with
the Hive MetaStore. This patch adds a new property
{{hive.metastore.thrift.framed.transport.enabled}} that enables the framed
transport (defaults to off, aka no change from before the patch). This
property must be set for both clients and the HMS server.
It wasn't immediately clear how to use the framed transport with SASL, so as
written an exception is thrown if you try starting the server with both
options. If SASL and the framed transport will indeed work together I can
update the patch (although I don't have a secured environment to test in).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2929) race condition in DAG execute tasks for hive
[ https://issues.apache.org/jira/browse/HIVE-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-2929: -- Resolution: Fixed Status: Resolved (was: Patch Available) race condition in DAG execute tasks for hive Key: HIVE-2929 URL: https://issues.apache.org/jira/browse/HIVE-2929 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.2929.1.patch select ... ( SubQuery involving MapReduce union all SubQuery involving MapReduce ); or select ... (SubQuery involving MapReduce) join (SubQuery involving MapReduce) ; If both the subQueries finish at nearly the same time, there is a race condition in which the results of the subQuery finishing last will be completely missed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2691) Specify location of log4j configuration files via configuration properties
[
https://issues.apache.org/jira/browse/HIVE-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2691:
--
Attachment: HIVE-2691.D2667.1.patch
zhenxiao requested code review of HIVE-2691 [jira] Specify location of log4j
configuration files via configuration properties.
Reviewers: JIRA
HIVE-2691. Specify location of log4j configuration files via configuration
properties
Oozie needs to be able to override the default location of the log4j
configuration
files from the Hive command line, e.g:
hive -hiveconf hive.log4j.file=/home/carl/hive-log4j.properties -hiveconf
hive.log4j.exec.file=/home/carl/hive-exec-log4j.properties
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D2667
AFFECTED FILES
common/src/java/org/apache/hadoop/hive/common/LogUtils.java
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java
common/src/test/org/apache/hadoop/hive/conf/TestHiveLogging.java
common/src/test/resources/hive-exec-log4j-test.properties
common/src/test/resources/hive-log4j-test.properties
conf/hive-default.xml.template
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java
service/src/java/org/apache/hadoop/hive/service/HiveServer.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/6117/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
Specify location of log4j configuration files via configuration properties
--
Key: HIVE-2691
URL: https://issues.apache.org/jira/browse/HIVE-2691
Project: Hive
Issue Type: New Feature
Components: Configuration, Logging
Reporter: Carl Steinbach
Assignee: Zhenxiao Luo
Attachments: HIVE-2691.D1131.1.patch, HIVE-2691.D1203.1.patch,
HIVE-2691.D1203.2.patch, HIVE-2691.D1203.3.patch, HIVE-2691.D1203.4.patch,
HIVE-2691.D1203.5.patch, HIVE-2691.D1203.6.patch, HIVE-2691.D2667.1.patch
Oozie needs to be able to override the default location of the log4j
configuration
files from the Hive command line, e.g:
{noformat}
hive -hiveconf hive.log4j.file=/home/carl/hive-log4j.properties -hiveconf
hive.log4j.exec.file=/home/carl/hive-exec-log4j.properties
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2691) Specify location of log4j configuration files via configuration properties
[
https://issues.apache.org/jira/browse/HIVE-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhenxiao Luo updated HIVE-2691:
---
Attachment: HIVE-2691.D2667.1.patch
Specify location of log4j configuration files via configuration properties
--
Key: HIVE-2691
URL: https://issues.apache.org/jira/browse/HIVE-2691
Project: Hive
Issue Type: New Feature
Components: Configuration, Logging
Reporter: Carl Steinbach
Assignee: Zhenxiao Luo
Attachments: HIVE-2691.D1131.1.patch, HIVE-2691.D1203.1.patch,
HIVE-2691.D1203.2.patch, HIVE-2691.D1203.3.patch, HIVE-2691.D1203.4.patch,
HIVE-2691.D1203.5.patch, HIVE-2691.D1203.6.patch, HIVE-2691.D2667.1.patch,
HIVE-2691.D2667.1.patch
Oozie needs to be able to override the default location of the log4j
configuration
files from the Hive command line, e.g:
{noformat}
hive -hiveconf hive.log4j.file=/home/carl/hive-log4j.properties -hiveconf
hive.log4j.exec.file=/home/carl/hive-exec-log4j.properties
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2694) Add FORMAT UDF
[ https://issues.apache.org/jira/browse/HIVE-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2694: -- Attachment: HIVE-2694.D2673.1.patch zhenxiao requested code review of HIVE-2694 [jira] Add FORMAT UDF. Reviewers: JIRA HIVE-2694: Add FORMAT UDF TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2673 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFormatNumber.java ql/src/test/queries/clientnegative/udf_format_number_wrong1.q ql/src/test/queries/clientnegative/udf_format_number_wrong2.q ql/src/test/queries/clientnegative/udf_format_number_wrong3.q ql/src/test/queries/clientnegative/udf_format_number_wrong4.q ql/src/test/queries/clientnegative/udf_format_number_wrong5.q ql/src/test/queries/clientnegative/udf_format_number_wrong6.q ql/src/test/queries/clientnegative/udf_format_number_wrong7.q ql/src/test/queries/clientpositive/udf_format_number.q ql/src/test/results/clientnegative/udf_format_number_wrong1.q.out ql/src/test/results/clientnegative/udf_format_number_wrong2.q.out ql/src/test/results/clientnegative/udf_format_number_wrong3.q.out ql/src/test/results/clientnegative/udf_format_number_wrong4.q.out ql/src/test/results/clientnegative/udf_format_number_wrong5.q.out ql/src/test/results/clientnegative/udf_format_number_wrong6.q.out ql/src/test/results/clientnegative/udf_format_number_wrong7.q.out ql/src/test/results/clientpositive/show_functions.q.out ql/src/test/results/clientpositive/udf_format_number.q.out MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6123/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. Add FORMAT UDF -- Key: HIVE-2694 URL: https://issues.apache.org/jira/browse/HIVE-2694 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Attachments: HIVE-2694.D1149.1.patch, HIVE-2694.D1149.2.patch, HIVE-2694.D1149.3.patch, HIVE-2694.D2673.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2694) Add FORMAT UDF
[ https://issues.apache.org/jira/browse/HIVE-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo updated HIVE-2694: --- Attachment: HIVE-2694.D2673.1.patch Add FORMAT UDF -- Key: HIVE-2694 URL: https://issues.apache.org/jira/browse/HIVE-2694 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Attachments: HIVE-2694.D1149.1.patch, HIVE-2694.D1149.2.patch, HIVE-2694.D1149.3.patch, HIVE-2694.D2673.1.patch, HIVE-2694.D2673.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2695) Add PRINTF() Udf
[ https://issues.apache.org/jira/browse/HIVE-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo updated HIVE-2695: --- Attachment: HIVE-2695.D1173.1.patch Add PRINTF() Udf Key: HIVE-2695 URL: https://issues.apache.org/jira/browse/HIVE-2695 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Fix For: 0.9.0 Attachments: HIVE-2695.D1155.1.patch, HIVE-2695.D1161.1.patch, HIVE-2695.D1173.1.patch, HIVE-2695.D1173.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2585: -- Attachment: HIVE-2585.D2559.2.patch ashutoshc updated the revision HIVE-2585 [jira] Collapse hive.metastore.uris and hive.metastore.local. Reviewers: JIRA, cwsteinbach Incorporated Thomas and Carl's comments. REVISION DETAIL https://reviews.facebook.net/D2559 AFFECTED FILES shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java conf/hive-default.xml.template metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartitionRemote.java metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreAuthorization.java metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStoreIpAddress.java metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java data/conf/hive-site.xml common/src/java/org/apache/hadoop/hive/conf/HiveConf.java Collapse hive.metastore.uris and hive.metastore.local - Key: HIVE-2585 URL: https://issues.apache.org/jira/browse/HIVE-2585 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-2585.D2559.1.patch, HIVE-2585.D2559.2.patch We should just have hive.metastore.uris. If it is empty, we shall assume local mode, if non-empty we shall use that string to connect to remote metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
