[jira] [Commented] (HIVE-4502) subquery smb joins fails
[ https://issues.apache.org/jira/browse/HIVE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649536#comment-13649536 ] Vikram Dixit K commented on HIVE-4502: -- java.lang.NullPointerException [junit] at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:82) [junit] at org.apache.hadoop.hive.ql.exec.JoinUtil.getObjectInspectorsFromEvaluators(JoinUtil.java:68) [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.initializeOp(CommonJoinOperator.java:224) [junit] at org.apache.hadoop.hive.ql.exec.JoinOperator.initializeOp(JoinOperator.java:61) [junit] at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:154) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) [junit] at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) [junit] at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:486) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:421) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:262) subquery smb joins fails Key: HIVE-4502 URL: https://issues.apache.org/jira/browse/HIVE-4502 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: smb_mapjoin_25.q Found this issue while running some SMB joins. Attaching test case that causes this error. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE
[ https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649542#comment-13649542 ] Thejas M Nair commented on HIVE-4501: - [~clarkyzl] Yes, HIVE-3098 and HIVE-3155 are related, as it is same kind of leak is seen, but with metastore and hive server1 instead. HS2 memory leak - FileSystem objects in FileSystem.CACHE Key: HIVE-4501 URL: https://issues.apache.org/jira/browse/HIVE-4501 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: Thejas M Nair org.apache.hadoop.fs.FileSystem objects are getting accumulated in FileSystem.CACHE, with HS2 in unsecure mode. As a workaround, it is possible to set fs.hdfs.impl.disable.cache and fs.file.impl.disable.cache to false. Users should not have to bother with this extra configuration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4501) HS2 memory leak - FileSystem objects in FileSystem.CACHE
[ https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649543#comment-13649543 ] Thejas M Nair commented on HIVE-4501: - I have updated hiveserver2 setup instructions to disable the fs caches - https://cwiki.apache.org/confluence/display/Hive/Setting+up+HiveServer2 HS2 memory leak - FileSystem objects in FileSystem.CACHE Key: HIVE-4501 URL: https://issues.apache.org/jira/browse/HIVE-4501 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: Thejas M Nair org.apache.hadoop.fs.FileSystem objects are getting accumulated in FileSystem.CACHE, with HS2 in unsecure mode. As a workaround, it is possible to set fs.hdfs.impl.disable.cache and fs.file.impl.disable.cache to false. Users should not have to bother with this extra configuration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4503) HiveServer have too many opened fd。
sutao bian created HIVE-4503: Summary: HiveServer have too many opened fd。 Key: HIVE-4503 URL: https://issues.apache.org/jira/browse/HIVE-4503 Project: Hive Issue Type: Bug Affects Versions: 0.8.1 Environment: Hive: hive-0.8.1 OS: Red Hat Enterprise Linux Server release 5.7 (Tikanga) Hadoop: 0.20.205 Reporter: sutao bian When i run hiveserver a while time it will occur error Caused by: java.io.FileNotFoundException: /opt/tmp/mapred/local/jobTracker/job_201301251143_76286.xml (Too many open files) more errors info : 013-05-06 02:54:47,426 WARN parse.SemanticAnalyzer (SemanticAnalyzer.java:genBodyPlan(5821)) - Common Gby keys:null 2013-05-06 02:54:50,386 WARN mapred.JobClient (JobClient.java:copyAndConfigureFiles(659)) - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 2013-05-06 02:54:52,565 ERROR exec.Task (SessionState.java:printError(380)) - Job Submission failed with exception 'org.apache.hadoop.ipc.RemoteException(java.io.IOException: java.io.FileNotFoundException: /opt/tmp/mapred/local/jobTracker/job_201301251143_76286.xml (Too many open files) at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943) at sun.reflect.GeneratedMethodAccessor1278.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382) Caused by: java.io.FileNotFoundException: /opt/tmp/mapred/local/jobTracker/job_201301251143_76286.xml (Too many open files) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:212) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.init(RawLocalFileSystem.java:188) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.init(RawLocalFileSystem.java:184) at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:242) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.init(ChecksumFileSystem.java:335) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:368) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:546) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:527) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:434) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:229) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:163) at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1164) at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1145) at org.apache.hadoop.mapred.JobInProgress.init(JobInProgress.java:415) at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941) ... 10 more )' when i restart the hiveserver it will be ok . Thanks -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4504) HiveServer have too many opened fd。
sutao bian created HIVE-4504: Summary: HiveServer have too many opened fd。 Key: HIVE-4504 URL: https://issues.apache.org/jira/browse/HIVE-4504 Project: Hive Issue Type: Bug Affects Versions: 0.8.1 Environment: Hive: hive-0.8.1 OS: Red Hat Enterprise Linux Server release 5.7 (Tikanga) Hadoop: 0.20.205 Reporter: sutao bian When i run hiveserver a while time it will occur error Caused by: java.io.FileNotFoundException: /opt/tmp/mapred/local/jobTracker/job_201301251143_76286.xml (Too many open files) more errors info : 013-05-06 02:54:47,426 WARN parse.SemanticAnalyzer (SemanticAnalyzer.java:genBodyPlan(5821)) - Common Gby keys:null 2013-05-06 02:54:50,386 WARN mapred.JobClient (JobClient.java:copyAndConfigureFiles(659)) - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 2013-05-06 02:54:52,565 ERROR exec.Task (SessionState.java:printError(380)) - Job Submission failed with exception 'org.apache.hadoop.ipc.RemoteException(java.io.IOException: java.io.FileNotFoundException: /opt/tmp/mapred/local/jobTracker/job_201301251143_76286.xml (Too many open files) at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943) at sun.reflect.GeneratedMethodAccessor1278.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382) Caused by: java.io.FileNotFoundException: /opt/tmp/mapred/local/jobTracker/job_201301251143_76286.xml (Too many open files) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:212) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.init(RawLocalFileSystem.java:188) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.init(RawLocalFileSystem.java:184) at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:242) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.init(ChecksumFileSystem.java:335) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:368) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:546) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:527) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:434) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:229) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:163) at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1164) at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1145) at org.apache.hadoop.mapred.JobInProgress.init(JobInProgress.java:415) at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941) ... 10 more )' when i restart the hiveserver it will be ok . Thanks -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4505) Hive can't load transforms added using 'ADD FILE'
Prasad Mujumdar created HIVE-4505:
-
Summary: Hive can't load transforms added using 'ADD FILE'
Key: HIVE-4505
URL: https://issues.apache.org/jira/browse/HIVE-4505
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.11.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Priority: Blocker
Fix For: 0.11.0, 0.12.0
ADD FILE mangles name of the resource when copying to resource download
directory. As a results following doesn't work:
{code:sql}
ADD FILE test.py;
SELECT TRANSFORM (id) USING 'python test.py' AS b FROM tab1;
{code}
The resource gets added with a different name every time which makes it
impossible to use transform in non-interactive mode.
This seems to be due to HIVE-3431
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: HIVE-4505: Hive can't load transforms added using 'ADD FILE'
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10945/
---
Review request for hive, Ashutosh Chauhan and Navis Ryu.
Description
---
This patch provides addional options for users to configure the resource dir in
order to avoid conflicts, instead of renaming the files.
- revert HIVE-3431
- Store the hiveserver2 session handle in the config so that it can be used for
the resource directory setting. eg
property
namehive.downloaded.resources.dir/name
value/tmp/resource_dir/${hive.server2.session}/value
/property
- support removing the resource directory at the end of the session.
One can configure the resource dir based on the session id or hiveserver2
session handle to avoid multiple users trying to use common resource directory.
This addresses bug HIVE-4505.
https://issues.apache.org/jira/browse/HIVE-4505
Diffs
-
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5c1b283
ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java d8c91bd
service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
8f0adb5
Diff: https://reviews.apache.org/r/10945/diff/
Testing
---
Thanks,
Prasad Mujumdar
[jira] [Updated] (HIVE-4505) Hive can't load transforms added using 'ADD FILE'
[
https://issues.apache.org/jira/browse/HIVE-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasad Mujumdar updated HIVE-4505:
--
Status: Patch Available (was: Open)
Review request on https://reviews.apache.org/r/10945/
Hive can't load transforms added using 'ADD FILE'
-
Key: HIVE-4505
URL: https://issues.apache.org/jira/browse/HIVE-4505
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.11.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Priority: Blocker
Fix For: 0.11.0, 0.12.0
Attachments: HIVE-4505-1.patch
ADD FILE mangles name of the resource when copying to resource download
directory. As a results following doesn't work:
{code:sql}
ADD FILE test.py;
SELECT TRANSFORM (id) USING 'python test.py' AS b FROM tab1;
{code}
The resource gets added with a different name every time which makes it
impossible to use transform in non-interactive mode.
This seems to be due to HIVE-3431
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4505) Hive can't load transforms added using 'ADD FILE'
[
https://issues.apache.org/jira/browse/HIVE-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasad Mujumdar updated HIVE-4505:
--
Attachment: HIVE-4505-1.patch
Patch for 0.11 branch
Hive can't load transforms added using 'ADD FILE'
-
Key: HIVE-4505
URL: https://issues.apache.org/jira/browse/HIVE-4505
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.11.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Priority: Blocker
Fix For: 0.11.0, 0.12.0
Attachments: HIVE-4505-1.patch
ADD FILE mangles name of the resource when copying to resource download
directory. As a results following doesn't work:
{code:sql}
ADD FILE test.py;
SELECT TRANSFORM (id) USING 'python test.py' AS b FROM tab1;
{code}
The resource gets added with a different name every time which makes it
impossible to use transform in non-interactive mode.
This seems to be due to HIVE-3431
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Apache Hive 0.11.0 Release Candidate 1
-1 (non-binding) My apologies, but HIVE-4505 is a regression that IMHO should be addressed. thanks Prasad On Tue, Apr 30, 2013 at 5:18 PM, Ashutosh Chauhan hashut...@apache.orgwrote: Hey all, Based on feedback from folks, I have respun release candidate, RC1. Please take a look. It basically fixes the size bloat of tarball. Source tag for RC1 is at: https://svn.apache.org/repos/asf/hive/tags/release-0.11.0-rc1 Source tar ball and convenience binary artifacts can be found at:http://people.apache.org/~hashutosh/hive-0.11.0-rc1/ Maven artifacts for hive are available at:https://repository.apache.org/content/repositories/orgapachehive-158/ Maven artifacts for hcatalog are available at: https://repository.apache.org/content/repositories/orgapachehcatalog-159/ This release has many goodies including HiveServer2, integrated hcatalog, windowing and analytical functions, decimal data type, better query planning, performance enhancements and various bug fixes. In total, we resolved more than 350 issues. Full list of fixed issues can be found at: http://s.apache.org/8Fr Voting will conclude in 72 hours. Hive PMC Members: Please test and vote. Thanks, Ashutosh (On behalf of Hive contributors who made 0.11 a possibility)
[jira] [Updated] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoluo (Clark) Yang updated HIVE-2616: --- Issue Type: New Feature (was: Bug) Passing user identity from metastore client to server in non-secure mode Key: HIVE-2616 URL: https://issues.apache.org/jira/browse/HIVE-2616 Project: Hive Issue Type: New Feature Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.8.1, 0.9.0 Attachments: hive-2616_1.patch, hive-2616_3.patch, hive-2616_4.patch, hive-2616_5.patch, hive-2616.patch Currently in unsecure mode client don't pass on user identity. As a result hdfs and other operations done by server gets executed by user running metastore process instead of being done in context of client. This results in problem as reported here: http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4506) join multi small tables
Fern created HIVE-4506: -- Summary: join multi small tables Key: HIVE-4506 URL: https://issues.apache.org/jira/browse/HIVE-4506 Project: Hive Issue Type: Wish Reporter: Fern -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4506) use one map reduce to join multiple small tables
[ https://issues.apache.org/jira/browse/HIVE-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fern updated HIVE-4506: --- Priority: Minor (was: Major) Description: I know we can use map side join for small table. by my test, if I use HQL like this -- select /*+mapjoin(b,c)*/... from a left join b on ... left join c on ... --- b and c are both small tables, I expect do the join in one map reduce using map side join. Actually, it would generate two map-reduce by sequence. Sorry, currently I am just a user of hive and not dig into the code, so I this is what I expect and have no idea about how to improve. Affects Version/s: 0.10.0 Summary: use one map reduce to join multiple small tables (was: join multi small tables ) use one map reduce to join multiple small tables - Key: HIVE-4506 URL: https://issues.apache.org/jira/browse/HIVE-4506 Project: Hive Issue Type: Wish Affects Versions: 0.10.0 Reporter: Fern Priority: Minor I know we can use map side join for small table. by my test, if I use HQL like this -- select /*+mapjoin(b,c)*/... from a left join b on ... left join c on ... --- b and c are both small tables, I expect do the join in one map reduce using map side join. Actually, it would generate two map-reduce by sequence. Sorry, currently I am just a user of hive and not dig into the code, so I this is what I expect and have no idea about how to improve. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4506) use one map reduce to join multiple small tables
[ https://issues.apache.org/jira/browse/HIVE-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fern updated HIVE-4506: --- Description: I know we can use map side join for small table. by my test, if I use HQL like this -- select /*+mapjoin(b,c)*/... from a left join b on ... left join c on ... --- b and c are both small tables, I expect do the join in one map reduce using map side join. Actually, it would generate two map-reduce jobs by sequence. Sorry, currently I am just a user of hive and not dig into the code, so I this is what I expect and have no idea about how to improve. was: I know we can use map side join for small table. by my test, if I use HQL like this -- select /*+mapjoin(b,c)*/... from a left join b on ... left join c on ... --- b and c are both small tables, I expect do the join in one map reduce using map side join. Actually, it would generate two map-reduce by sequence. Sorry, currently I am just a user of hive and not dig into the code, so I this is what I expect and have no idea about how to improve. use one map reduce to join multiple small tables - Key: HIVE-4506 URL: https://issues.apache.org/jira/browse/HIVE-4506 Project: Hive Issue Type: Wish Affects Versions: 0.10.0 Reporter: Fern Priority: Minor I know we can use map side join for small table. by my test, if I use HQL like this -- select /*+mapjoin(b,c)*/... from a left join b on ... left join c on ... --- b and c are both small tables, I expect do the join in one map reduce using map side join. Actually, it would generate two map-reduce jobs by sequence. Sorry, currently I am just a user of hive and not dig into the code, so I this is what I expect and have no idea about how to improve. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4506) use one map reduce to join multiple small tables
[ https://issues.apache.org/jira/browse/HIVE-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fern updated HIVE-4506: --- Description: I know we can use map side join for small table. by my test, if I use HQL like this -- select /*+mapjoin(b,c)*/... from a left join b on ... left join c on ... --- b and c are both small tables, I expect do the join in one map reduce using map side join. Actually, it would generate two map-reduce jobs by sequence. Sorry, currently I am just a user of hive and not dig into the code, so this is what I expect but I have no idea about how to improve now. was: I know we can use map side join for small table. by my test, if I use HQL like this -- select /*+mapjoin(b,c)*/... from a left join b on ... left join c on ... --- b and c are both small tables, I expect do the join in one map reduce using map side join. Actually, it would generate two map-reduce jobs by sequence. Sorry, currently I am just a user of hive and not dig into the code, so I this is what I expect and have no idea about how to improve. use one map reduce to join multiple small tables - Key: HIVE-4506 URL: https://issues.apache.org/jira/browse/HIVE-4506 Project: Hive Issue Type: Wish Affects Versions: 0.10.0 Reporter: Fern Priority: Minor I know we can use map side join for small table. by my test, if I use HQL like this -- select /*+mapjoin(b,c)*/... from a left join b on ... left join c on ... --- b and c are both small tables, I expect do the join in one map reduce using map side join. Actually, it would generate two map-reduce jobs by sequence. Sorry, currently I am just a user of hive and not dig into the code, so this is what I expect but I have no idea about how to improve now. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4507) Fix jdbc to compile under openjdk 7
Owen O'Malley created HIVE-4507: --- Summary: Fix jdbc to compile under openjdk 7 Key: HIVE-4507 URL: https://issues.apache.org/jira/browse/HIVE-4507 Project: Hive Issue Type: Bug Components: JDBC Reporter: Owen O'Malley Assignee: Owen O'Malley The newer Linux distros are shipping with just openjdk 7. Currently, the jdbc module doesn't compile because some new methods aren't implemented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4508) Fix various release issues in 0.11.0rc1
Owen O'Malley created HIVE-4508: --- Summary: Fix various release issues in 0.11.0rc1 Key: HIVE-4508 URL: https://issues.apache.org/jira/browse/HIVE-4508 Project: Hive Issue Type: Bug Reporter: Owen O'Malley Assignee: Owen O'Malley Fix For: 0.11.0 Carl described some non-code issues in the 0.11.0rc1 and I want to fix them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Apache Hive 0.11.0 Release Candidate 1
I've create a jira to track the non-code issues: https://issues.apache.org/jira/browse/HIVE-4508 I'll upload a patch in the next hour for comments. -- Owen On Sun, May 5, 2013 at 10:52 PM, Carl Steinbach cwsteinb...@gmail.comwrote: I took a quick look. Here's a list of things I noticed: * NOTICE ** Copyright is out of date. * README.txt ** Add one more '=' to the title underline. ** Requirements section only lists Hadoop 0.20.x ** Will it work with Java 1.7? * RELEASE_NOTES.txt ** The 0.11 section lists tickets that aren't marked as resolved/fixed on JIRA, and which are not included in the git commit log (e.g. HIVE-3979). This may also extend to the 0.10 release notes. Related question: How were these release notes generated? * docs/index.html ** Update email list links (e.g. hive-u...@hadoop.apache.org) * src/build.properties ** year=2012 * src/eclipse-files/.classpath ** The guava jar is not in the right location. This breaks the eclipse import. Thanks. Carl On Tue, Apr 30, 2013 at 5:18 PM, Ashutosh Chauhan hashut...@apache.org wrote: Hey all, Based on feedback from folks, I have respun release candidate, RC1. Please take a look. It basically fixes the size bloat of tarball. Source tag for RC1 is at: https://svn.apache.org/repos/asf/hive/tags/release-0.11.0-rc1 Source tar ball and convenience binary artifacts can be found at:http://people.apache.org/~hashutosh/hive-0.11.0-rc1/ Maven artifacts for hive are available at:https://repository.apache.org/content/repositories/orgapachehive-158/ Maven artifacts for hcatalog are available at: https://repository.apache.org/content/repositories/orgapachehcatalog-159/ This release has many goodies including HiveServer2, integrated hcatalog, windowing and analytical functions, decimal data type, better query planning, performance enhancements and various bug fixes. In total, we resolved more than 350 issues. Full list of fixed issues can be found at: http://s.apache.org/8Fr Voting will conclude in 72 hours. Hive PMC Members: Please test and vote. Thanks, Ashutosh (On behalf of Hive contributors who made 0.11 a possibility)
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649880#comment-13649880 ] Mark Grover commented on HIVE-3384: --- I verified that this was committed on trunk (Thanks!). Is there anything left to be done regarding this patch? If not, can we please update the JIRA to resolved status? HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification -- Key: HIVE-3384 URL: https://issues.apache.org/jira/browse/HIVE-3384 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.10.0 Reporter: Weidong Bian Assignee: Chris Drome Priority: Minor Fix For: 0.11.0 Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, D6873.7.patch, HIVE-3384-0.10.patch, HIVE-3384-2012-12-02.patch, HIVE-3384-2012-12-04.patch, HIVE-3384.2.patch, HIVE-3384-branch-0.9.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch jdbc module couldn't be compiled with jdk7 as it adds some abstract method in the JDBC specification some error info: error: HiveCallableStatement is not abstract and does not override abstract method TgetObject(String,ClassT) in CallableStatement . . . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: HIVE-4500 HS2 holding too many file handles of hive_job_log_hive_*.txt files
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10954/ --- Review request for hive and Carl Steinbach. Description --- HS2 holding too many file handles of hive_job_log_hive_*.txt files This addresses bug HIVE-4500. https://issues.apache.org/jira/browse/HIVE-4500 Diffs - trunk/ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistory.java 1478219 trunk/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 1478219 trunk/service/src/java/org/apache/hive/service/cli/operation/HiveCommandOperation.java 1478219 trunk/service/src/java/org/apache/hive/service/cli/operation/Operation.java 1478219 trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 1478219 Diff: https://reviews.apache.org/r/10954/diff/ Testing --- Thanks, Alan Gates
[jira] [Commented] (HIVE-4165) Union failures with mapjoin hints
[ https://issues.apache.org/jira/browse/HIVE-4165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649886#comment-13649886 ] Vikram Dixit K commented on HIVE-4165: -- [~namit] Could you please explain why union operator was explicitly not allowed before or after map join? This above change was introduced as part of the change converting joins to mapjoins automatically without need for hints. Thanks Vikram. Union failures with mapjoin hints - Key: HIVE-4165 URL: https://issues.apache.org/jira/browse/HIVE-4165 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0 Reporter: Vikram Dixit K While running the unit tests - union_remove_12.q which has mapjoin hints fails. Although this is a hadoop 23 only test, the same is reproducible on any union test with mapjoin hints. For e.g. union_22.q regardless of if we have the auto.convert.join=true/false, I see that we fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4500) HS2 holding too many file handles of hive_job_log_hive_*.txt files
[ https://issues.apache.org/jira/browse/HIVE-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649888#comment-13649888 ] Alan Gates commented on HIVE-4500: -- Review request at https://reviews.apache.org/r/10954/ HS2 holding too many file handles of hive_job_log_hive_*.txt files -- Key: HIVE-4500 URL: https://issues.apache.org/jira/browse/HIVE-4500 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Alan Gates Attachments: HIVE-4500.patch In the hiveserver2 setup used for testing, we see that it has 2444 files open and of them 2152 are /tmp/hive/hive_job_log_hive_*.txt files -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4039) Hive compiler sometimes fails in semantic analysis / optimisation stage when boolean variable appears in WHERE clause.
[ https://issues.apache.org/jira/browse/HIVE-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4039: Fix Version/s: 0.11.0 Hive compiler sometimes fails in semantic analysis / optimisation stage when boolean variable appears in WHERE clause. -- Key: HIVE-4039 URL: https://issues.apache.org/jira/browse/HIVE-4039 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Jean Xu Assignee: Jean Xu Priority: Minor Fix For: 0.11.0 Attachments: HIVE_4039.1.patch.txt Hive compiler fails with a NullPointerException in semantic analysis / optimisation stage when a boolean variable appears in the WHERE clause in some cases. A minimal query to generate this error is here: SELECT 1 FROM ( SELECT TRUE AS flag FROM dim_one_row:measurementsystems ) a WHERE flag; On the other hand, the following query is perfectly fine: SELECT 1 FROM ( SELECT TRUE AS flag FROM dim_one_row:measurementsystems ) a WHERE flag=TRUE; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4045) Modify PreDropPartitionEvent to pass Table parameter
[ https://issues.apache.org/jira/browse/HIVE-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4045: Fix Version/s: 0.11.0 Modify PreDropPartitionEvent to pass Table parameter Key: HIVE-4045 URL: https://issues.apache.org/jira/browse/HIVE-4045 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Li Yang Assignee: Li Yang Priority: Minor Fix For: 0.11.0 Attachments: HIVE-4045.1.patch.txt, HIVE-4045.2.patch.txt MetaStorePreEventListener which implements onEvent(PreEventContext context) sometimes needs to access Table properties when PreDropPartitionEvent is listened to. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4096) problem in hive.map.groupby.sorted with distincts
[ https://issues.apache.org/jira/browse/HIVE-4096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4096: Fix Version/s: 0.11.0 problem in hive.map.groupby.sorted with distincts - Key: HIVE-4096 URL: https://issues.apache.org/jira/browse/HIVE-4096 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.11.0 Attachments: hive.4096.1.patch set hive.enforce.bucketing = true; set hive.enforce.sorting = true; set hive.exec.reducers.max = 10; set hive.map.groupby.sorted=true; CREATE TABLE T1(key STRING, val STRING) PARTITIONED BY (ds string) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE; LOAD DATA LOCAL INPATH '../data/files/T1.txt' INTO TABLE T1 PARTITION (ds='1'); -- perform an insert to make sure there are 2 files INSERT OVERWRITE TABLE T1 PARTITION (ds='1') select key, val from T1 where ds = '1'; CREATE TABLE outputTbl1(cnt INT); -- The plan should be converted to a map-side group by, since the -- sorting columns and grouping columns match, and all the bucketing columns -- are part of sorting columns EXPLAIN select count(distinct key) from T1; select count(distinct key) from T1; explain INSERT OVERWRITE TABLE outputTbl1 select count(distinct key) from T1; INSERT OVERWRITE TABLE outputTbl1 select count(distinct key) from T1; SELECT * FROM outputTbl1; DROP TABLE T1; The above query gives wrong results -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4125) Expose metastore JMX metrics
[ https://issues.apache.org/jira/browse/HIVE-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4125: Fix Version/s: 0.11.0 Expose metastore JMX metrics Key: HIVE-4125 URL: https://issues.apache.org/jira/browse/HIVE-4125 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.11.0 Reporter: Samuel Yuan Assignee: Samuel Yuan Priority: Trivial Fix For: 0.11.0 Attachments: HIVE-4125.HIVE-4125.HIVE-4125.D9123.1.patch, HIVE-4125.HIVE-4125.HIVE-4125.D9123.2.patch Add a safe way to access the metrics stored for each MetricsScope, so that they can be used outside of JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4206) Sort merge join does not work for outer joins for 7 inputs
[ https://issues.apache.org/jira/browse/HIVE-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4206: Fix Version/s: 0.11.0 Sort merge join does not work for outer joins for 7 inputs -- Key: HIVE-4206 URL: https://issues.apache.org/jira/browse/HIVE-4206 Project: Hive Issue Type: Improvement Components: Query Processor Affects Versions: 0.11.0 Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.11.0 Attachments: hive.4206.1.patch, hive.4206.2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4219) explain dependency does not capture the input table
[
https://issues.apache.org/jira/browse/HIVE-4219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-4219:
Fix Version/s: 0.11.0
explain dependency does not capture the input table
---
Key: HIVE-4219
URL: https://issues.apache.org/jira/browse/HIVE-4219
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
Fix For: 0.11.0
Attachments: hive.4219.1.patch, hive.4219.2.patch, hive.4219.3.patch
hive explain dependency select * from srcpart where ds is not null;
OK
{input_partitions:[{partitionName:default@srcpart@ds=2008-04-08/hr=11},{partitionName:default@srcpart@ds=2008-04-08/hr=12},{partitionName:default@srcpart@ds=2008-04-09/hr=11},{partitionName:default@srcpart@ds=2008-04-09/hr=12}],input_tables:[]}
input_tables should contain srcpart
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4288) Add IntelliJ project files files to .gitignore
[ https://issues.apache.org/jira/browse/HIVE-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4288: Fix Version/s: 0.11.0 Add IntelliJ project files files to .gitignore -- Key: HIVE-4288 URL: https://issues.apache.org/jira/browse/HIVE-4288 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Roshan Naik Assignee: Roshan Naik Priority: Minor Fix For: 0.11.0 Attachments: 4288.patch Add *.iml files .idea dir to .gitignore -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4420) HCatalog unit tests stop after a failure
[ https://issues.apache.org/jira/browse/HIVE-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4420: Fix Version/s: 0.11.0 HCatalog unit tests stop after a failure Key: HIVE-4420 URL: https://issues.apache.org/jira/browse/HIVE-4420 Project: Hive Issue Type: Bug Components: Build Infrastructure, HCatalog Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.11.0, 0.12.0 Attachments: HIVE-4420.patch HCatalog splits it's different components (core, hcatalog-pig-adaptor, webhcat, etc.) into separate build.xml files, all controlled by the top build.xml. A side effect of this is that when the user does ant test and the top build file is iterating through the tests, if one of the component's test targets fail then unit tests are not run for subsequent components. This means a failure in one component can mask issues in others. test should run on all the components even when there is a failure in one of the components. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3140) Comment indenting is broken for describe in CLI
[ https://issues.apache.org/jira/browse/HIVE-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3140: Fix Version/s: 0.11.0 Comment indenting is broken for describe in CLI - Key: HIVE-3140 URL: https://issues.apache.org/jira/browse/HIVE-3140 Project: Hive Issue Type: Bug Components: CLI Reporter: Xiaoxiao Hou Assignee: Zhenxiao Luo Labels: patch Fix For: 0.10.0, 0.11.0 Attachments: HIVE-3140.1.patch.txt, HIVE-3140.2.patch.txt, hive.3140.3.patch, HIVE-3140.3.patch.txt Original Estimate: 72h Remaining Estimate: 72h Just go into the CLI and type describe [TABLE_NAME]. If a comment has multiple lines, it is completely unreadable due to poor comment indenting. For example: birthdayParam string 1 = comment1 2 = comment2 3 = comment3 But it supposed to display as: birthdayParam string 1 = comment1 2 = comment2 3 = comment3 Comments should be indented the same amount on each line, i.e., if the comment starts at row k for the first line of the comment, it should be indented by k on line 2. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3698) enhanceModel.notRequired is incorrectly determined
[ https://issues.apache.org/jira/browse/HIVE-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3698: Fix Version/s: 0.11.0 enhanceModel.notRequired is incorrectly determined -- Key: HIVE-3698 URL: https://issues.apache.org/jira/browse/HIVE-3698 Project: Hive Issue Type: Bug Components: Build Infrastructure, Metastore Affects Versions: 0.10.0, 0.9.1 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.9.1, 0.11.0 Attachments: HIVE-3698-0.9.patch, HIVE-3698-trunk.patch enhanceModel.notRequired in metastore/build.xml is not correctly determined which can result in datanucleus running twice. This in turn can cause other issues when building with JDK7. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3729) Error in groupSetExpression rule in Hive grammar
[ https://issues.apache.org/jira/browse/HIVE-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3729: Fix Version/s: 0.11.0 Error in groupSetExpression rule in Hive grammar Key: HIVE-3729 URL: https://issues.apache.org/jira/browse/HIVE-3729 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Environment: All Reporter: Harish Butani Assignee: Harish Butani Priority: Minor Fix For: 0.10.0, 0.11.0 Attachments: HIVE-3729.1.patch.txt Original Estimate: 5m Remaining Estimate: 5m Here is the error: Hive.g:1902:38: reference to rewrite element groupByExpression without reference on left of - -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3735) PTest doesn't work due to hive snapshot version upgrade to 11
[ https://issues.apache.org/jira/browse/HIVE-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3735: Fix Version/s: 0.11.0 PTest doesn't work due to hive snapshot version upgrade to 11 - Key: HIVE-3735 URL: https://issues.apache.org/jira/browse/HIVE-3735 Project: Hive Issue Type: Bug Components: Testing Infrastructure Affects Versions: 0.10.0 Reporter: Gang Tim Liu Assignee: Gang Tim Liu Priority: Critical Fix For: 0.10.0, 0.11.0 Attachments: HIVE-3735.patch.1 PTest fails. Error :: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :: [ivy:resolve] :: org.apache.hive#hive-builtins;0.11.0-SNAPSHOT: not found [ivy:resolve] :: -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4420) HCatalog unit tests stop after a failure
[ https://issues.apache.org/jira/browse/HIVE-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4420: Fix Version/s: (was: 0.12.0) HCatalog unit tests stop after a failure Key: HIVE-4420 URL: https://issues.apache.org/jira/browse/HIVE-4420 Project: Hive Issue Type: Bug Components: Build Infrastructure, HCatalog Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.11.0 Attachments: HIVE-4420.patch HCatalog splits it's different components (core, hcatalog-pig-adaptor, webhcat, etc.) into separate build.xml files, all controlled by the top build.xml. A side effect of this is that when the user does ant test and the top build file is iterating through the tests, if one of the component's test targets fail then unit tests are not run for subsequent components. This means a failure in one component can mask issues in others. test should run on all the components even when there is a failure in one of the components. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4508) Fix various release issues in 0.11.0rc1
[ https://issues.apache.org/jira/browse/HIVE-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649934#comment-13649934 ] Chris Drome commented on HIVE-4508: --- HIVE-4496 fixes the build issue with JDBC2 and JDK1.7. Fix various release issues in 0.11.0rc1 --- Key: HIVE-4508 URL: https://issues.apache.org/jira/browse/HIVE-4508 Project: Hive Issue Type: Bug Reporter: Owen O'Malley Assignee: Owen O'Malley Fix For: 0.11.0 Carl described some non-code issues in the 0.11.0rc1 and I want to fix them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-4507) Fix jdbc to compile under openjdk 7
[ https://issues.apache.org/jira/browse/HIVE-4507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HIVE-4507. - Resolution: Duplicate Fix jdbc to compile under openjdk 7 --- Key: HIVE-4507 URL: https://issues.apache.org/jira/browse/HIVE-4507 Project: Hive Issue Type: Bug Components: JDBC Reporter: Owen O'Malley Assignee: Owen O'Malley The newer Linux distros are shipping with just openjdk 7. Currently, the jdbc module doesn't compile because some new methods aren't implemented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4192) Use of LEAD in an OVER clauses causes the query to fail
[
https://issues.apache.org/jira/browse/HIVE-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-4192:
Fix Version/s: (was: 0.11.0)
Use of LEAD in an OVER clauses causes the query to fail
---
Key: HIVE-4192
URL: https://issues.apache.org/jira/browse/HIVE-4192
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.11.0
Reporter: Alan Gates
Running a query like:
{code}
select i, lead(s) over (partition by bin order by d desc
rows between current row and 1 following)
from over100k;
{code}
gives an error:
{code}
FAILED: SemanticException Function lead((TOK_TABLE_OR_COL s))
org.apache.hadoop.hive.ql.parse.WindowingSpec$WindowSpec@13e15f7 as _wcol0
doesn't support windowing
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4110) Aggregation functions must have aliases when multiple functions are used
[
https://issues.apache.org/jira/browse/HIVE-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-4110:
Fix Version/s: (was: 0.11.0)
Aggregation functions must have aliases when multiple functions are used
Key: HIVE-4110
URL: https://issues.apache.org/jira/browse/HIVE-4110
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Reporter: Brock Noland
Assignee: Brock Noland
Attachments: HIVE-4110-0.patch
The following query fails:
{noformat}
select p_mfgr, p_retailprice, p_size,
lead(p_retailprice) over(partition by p_mfgr order by p_size),
lag(p_retailprice) over(partition by p_mfgr order by p_size)
from part;
{noformat}
with the error below:
{noformat}
2013-03-02 16:10:47,126 ERROR ql.Driver (SessionState.java:printError(401)) -
FAILED: SemanticException [Error 10011]: Line 2:38 Invalid function 'p_mfgr'
org.apache.hadoop.hive.ql.parse.SemanticException: Line 2:38 Invalid function
'p_mfgr'
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:678)
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:908)
at
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:87)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:124)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:101)
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:166)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:8895)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:2634)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:2433)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:7234)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:7200)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:7978)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8651)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:259)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:898)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4249) current database is retained between sessions in hive server2
[
https://issues.apache.org/jira/browse/HIVE-4249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-4249:
Fix Version/s: (was: 0.11.0)
current database is retained between sessions in hive server2
---
Key: HIVE-4249
URL: https://issues.apache.org/jira/browse/HIVE-4249
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Thejas M Nair
Assignee: Thejas M Nair
current database is retained between sessions in hive server2.
To reproduce -
Run this serveral times -
bin/beeline -e '!connect jdbc:hive2://localhost:1 scott tiger
org.apache.hive.jdbc.HiveDriver' -e 'show tables;' -e ' use newdb;' -e ' show
tables;'
table ab is a table in default database, newtab is a table in newdb database.
Expected result is
{code}
+---+
| tab_name |
+---+
| ab|
+---+
1 row selected (0.457 seconds)
No rows affected (0.039 seconds)
+---+
| tab_name |
+---+
| newtab|
+---+
{code}
But after running it several, times you see threads having newdb as default
database, ie the output of above command becomes -
{code}
+---+
| tab_name |
+---+
| newtab|
+---+
1 row selected (0.518 seconds)
No rows affected (0.052 seconds)
+---+
| tab_name |
+---+
| newtab|
+---+
1 row selected (0.232 seconds)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4357) BeeLine tests are not getting executed
[ https://issues.apache.org/jira/browse/HIVE-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4357: Fix Version/s: (was: 0.11.0) BeeLine tests are not getting executed -- Key: HIVE-4357 URL: https://issues.apache.org/jira/browse/HIVE-4357 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.10.0 Reporter: Carl Steinbach Assignee: Rob Weltman Attachments: HIVE-4357.1.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4236) JDBC2 HivePreparedStatement does not release resources
[ https://issues.apache.org/jira/browse/HIVE-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4236: Fix Version/s: (was: 0.11.0) JDBC2 HivePreparedStatement does not release resources -- Key: HIVE-4236 URL: https://issues.apache.org/jira/browse/HIVE-4236 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4236.patch HivePreparedStatement does not close the associated server-side operation when close() is called. Nor does it call close() on the ResultSet. When execute() is called the current ResultSet is not closed first it is just set to null. Similarly, HiveStatement's close() does not call close() on the ResultSet, it just sets it to null. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3407) Update Hive CLI xdoc (sync with CLI wikidoc)
[ https://issues.apache.org/jira/browse/HIVE-3407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3407: Fix Version/s: (was: 0.11.0) Update Hive CLI xdoc (sync with CLI wikidoc) Key: HIVE-3407 URL: https://issues.apache.org/jira/browse/HIVE-3407 Project: Hive Issue Type: Bug Components: Documentation Affects Versions: 0.9.0 Reporter: Lefty Leverenz Assignee: Lefty Leverenz Labels: documentation Attachments: HIVE-3407.1.patch CLI documentation for Hive exists in two places (wikidocs and xdocs) and both of the versions are out of date, but the xdocs version was worse: * [http://hive.apache.org/docs/r0.9.0/language_manual/cli.html] * [https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Cli] A revised CLI wikidoc is available and will soon be exchanged for the old wikidoc. Although there's some resistance to moving more of the wikidocs into xdocs, for now let's have current information in both places instead of removing the xdocs version. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3384: Fix Version/s: (was: 0.11.0) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification -- Key: HIVE-3384 URL: https://issues.apache.org/jira/browse/HIVE-3384 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.10.0 Reporter: Weidong Bian Assignee: Chris Drome Priority: Minor Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, D6873.7.patch, HIVE-3384-0.10.patch, HIVE-3384-2012-12-02.patch, HIVE-3384-2012-12-04.patch, HIVE-3384.2.patch, HIVE-3384-branch-0.9.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch jdbc module couldn't be compiled with jdk7 as it adds some abstract method in the JDBC specification some error info: error: HiveCallableStatement is not abstract and does not override abstract method TgetObject(String,ClassT) in CallableStatement . . . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4193) OVER clauses with BETWEEN in the window definition produce wrong results
[
https://issues.apache.org/jira/browse/HIVE-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-4193:
Fix Version/s: (was: 0.11.0)
OVER clauses with BETWEEN in the window definition produce wrong results
Key: HIVE-4193
URL: https://issues.apache.org/jira/browse/HIVE-4193
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.11.0
Reporter: Alan Gates
Attachments: Range_5_diff
Window queries that define a windowing clause that has a termination row
often (though not all) return incorrect results. For example, from our test
queries all of the following return incorrect results:
{code}
select s, sum(f) over (partition by t order by b
rows between current row and unbounded following)
from over100k;
select s, avg(f) over (partition by b order by d
rows between 5 preceding and current row)
from over100k;
select s, avg(f) over (partition by bin order by s
rows between current row and 5 following)
from over100k;
select s, avg(d) over (partition by i order by f desc
rows between 5 preceding and 5 following)
from over100k;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4362) Allow Hive unit tests to run against fully-distributed cluster
[ https://issues.apache.org/jira/browse/HIVE-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4362: Fix Version/s: (was: 0.11.0) Allow Hive unit tests to run against fully-distributed cluster -- Key: HIVE-4362 URL: https://issues.apache.org/jira/browse/HIVE-4362 Project: Hive Issue Type: Improvement Components: Testing Infrastructure Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover It seems like Hive unit tests can run in (Hadoop) local mode or miniMR mode. It would be nice (especially for projects like Apache Bigtop) to be able to run Hive tests in fully distributed mode. This JIRA tracks the introduction of such functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4012) Unit test failures with Hadoop 23 due to HADOOP-8551
[ https://issues.apache.org/jira/browse/HIVE-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4012: Fix Version/s: (was: 0.10.1) (was: 0.11.0) Unit test failures with Hadoop 23 due to HADOOP-8551 Key: HIVE-4012 URL: https://issues.apache.org/jira/browse/HIVE-4012 Project: Hive Issue Type: Bug Affects Versions: 0.10.0, 0.11.0 Reporter: Thiruvel Thirumoolan Assignee: Thiruvel Thirumoolan Attachments: HIVE-4012_branch10.patch With HADOOP-8551 (=23.3 or =2.0.2-alpha), its not possible to do a dfs -mkdir of foo/bar when foo does not exist. One has to use '-p' option (not available in Hadoop 20.x). A bunch of our test cases rely on this feature and this was to make it interoperable with Windows too (HIVE-3204). However, all these unit tests fail when using Hadoop =23.3 or =2.0.2-alpha. Its also not possible to use the '-p' option in the tests as thats not supported in Hadoop 20.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4225) HiveServer2 does not support SASL QOP
[ https://issues.apache.org/jira/browse/HIVE-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4225: Fix Version/s: (was: 0.11.0) HiveServer2 does not support SASL QOP - Key: HIVE-4225 URL: https://issues.apache.org/jira/browse/HIVE-4225 Project: Hive Issue Type: Bug Components: HiveServer2, Shims Affects Versions: 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4225.patch HiveServer2 implements Kerberos authentication through SASL framework, but does not support setting QOP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3631) script_pipe.q fails when using JDK7
[ https://issues.apache.org/jira/browse/HIVE-3631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3631: Fix Version/s: (was: 0.11.0) script_pipe.q fails when using JDK7 --- Key: HIVE-3631 URL: https://issues.apache.org/jira/browse/HIVE-3631 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.10.0, 0.9.1, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-3631-0.10.patch, HIVE-3631-0.9.patch, HIVE-3631-trunk.patch Hive Runtime Error while closing operators: Hit error while closing .. The MR job fails on this test. Unfortunately, the exception is not all that helpful. I tracked this down to a class which attempts to close a stream that is already closed. Broken pipe exceptions are caught and not propagated further, but stream closed exception are not caught. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4257) java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator
[
https://issues.apache.org/jira/browse/HIVE-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-4257:
Fix Version/s: (was: 0.11.0)
java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator
--
Key: HIVE-4257
URL: https://issues.apache.org/jira/browse/HIVE-4257
Project: Hive
Issue Type: Bug
Components: Statistics
Affects Versions: 0.11.0
Reporter: Teddy Choi
Priority: Minor
Attachments: HIVE-4257.1.patch.txt
java.sql.SQLNonTransientConnectionException occurs on JDBCStatsAggregator
after executing dozens of Hive queries periodically, which inserts thousands
of rows. It may have a relation with DERBY-5098. To avoid this error, Hive
should use a more recent version of Derby(10.6.2.3, 10.7.1.4, 10.8.2.2,
10.9.1.0 or later). Hive 0.11.0-SNAPSHOT uses Derby 10.4.2.0.
{noformat}
2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator
(JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing
aggregation. java.sql.SQLNonTransientConnectionException: No current
connection.
2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator
(JDBCStatsAggregator.java:aggregateStats(168)) - Error during publishing
aggregation. java.sql.SQLNonTransientConnectionException: No current
connection.
2013-03-24 15:54:30,487 ERROR jdbc.JDBCStatsAggregator
(JDBCStatsAggregator.java:cleanUp(249)) - Error during publishing
aggregation. java.sql.SQLNonTransientConnectionException: No current
connection.
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3768) Document JDBC client configuration for secure clusters
[ https://issues.apache.org/jira/browse/HIVE-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3768: Fix Version/s: (was: 0.11.0) Document JDBC client configuration for secure clusters -- Key: HIVE-3768 URL: https://issues.apache.org/jira/browse/HIVE-3768 Project: Hive Issue Type: Bug Components: Documentation Affects Versions: 0.9.0 Reporter: Lefty Leverenz Assignee: Lefty Leverenz Labels: documentation Attachments: HIVE-3768.1.patch, HIVE-3768.2.patch Document the JDBC client configuration required for starting Hive on a secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2817) Drop any table even without privilege
[
https://issues.apache.org/jira/browse/HIVE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-2817:
Fix Version/s: (was: 0.11.0)
Drop any table even without privilege
-
Key: HIVE-2817
URL: https://issues.apache.org/jira/browse/HIVE-2817
Project: Hive
Issue Type: Bug
Affects Versions: 0.7.1, 0.8.0, 0.9.0, 0.10.0
Reporter: Benyi Wang
Assignee: Chen Chun
Attachments: HIVE-2817.D10371.1.patch, HIVE-2817.D10563.1.patch
You can drop any table if you use fully qualified name 'database.table' even
you don't have any previlige.
{code}
hive set hive.security.authorization.enabled=true;
hive revoke all on default from user test_user;
hive drop table abc;
hive drop table abc;
Authorization failed:No privilege 'Drop' found for outputs {
database:default, table:abc}. Use show grant to get more details.
hive drop table default.abc;
OK
Time taken: 0.13 seconds
{code}
The table and the file in {{/usr/hive/warehouse}} or external file will be
deleted. If you don't have hadoop access permission on
{{/usr/hive/warehouse}} or external files, you will see a hadoop access error
{code}
12/02/23 15:35:35 ERROR hive.log:
org.apache.hadoop.security.AccessControlException:
org.apache.hadoop.security.AccessControlException: Permission denied:
user=test_user, access=WRITE, inode=/user/myetl:myetl:etl:drwxr-xr-x
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3036) hive should support BigDecimal datatype
[ https://issues.apache.org/jira/browse/HIVE-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3036: Fix Version/s: (was: 0.11.0) hive should support BigDecimal datatype --- Key: HIVE-3036 URL: https://issues.apache.org/jira/browse/HIVE-3036 Project: Hive Issue Type: New Feature Components: Query Processor, Types Affects Versions: 0.7.1, 0.8.0, 0.8.1 Reporter: Anurag Tangri hive has support for big int but people have use cases where they need decimal precision to a big value. Values in question are like decimal(x,y). for eg. decimal of form (17,6) which cannot be represented by float/double. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3911) udaf_percentile_approx.q fails with Hadoop 0.23.5 when map-side aggr is disabled.
[ https://issues.apache.org/jira/browse/HIVE-3911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3911: Fix Version/s: (was: 0.10.1) (was: 0.11.0) udaf_percentile_approx.q fails with Hadoop 0.23.5 when map-side aggr is disabled. - Key: HIVE-3911 URL: https://issues.apache.org/jira/browse/HIVE-3911 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Thiruvel Thirumoolan Assignee: Thiruvel Thirumoolan Attachments: HIVE-3911_branch10.patch, HIVE-3911.patch I am running Hive10 unit tests against Hadoop 0.23.5 and udaf_percentile_approx.q fails with a different value when map-side aggr is disabled and only when 3rd argument to this UDAF is 100. Matches expected output when map-side aggr is enabled for the same arguments. This test passes when hadoop.version is 1.1.1 and fails when its 0.23.x or 2.0.0-alpha or 2.0.2-alpha. [junit] 20c20 [junit] 254.083331 [junit] --- [junit] 252.77 [junit] 47c47 [junit] 254.083331 [junit] --- [junit] 252.77 [junit] 74c74 [junit] [23.358,254.083331,477.0625,489.54667] [junit] --- [junit] [24.07,252.77,476.9,487.82] [junit] 101c101 [junit] [23.358,254.083331,477.0625,489.54667] [junit] --- [junit] [24.07,252.77,476.9,487.82] -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4161) create clean and small default set of tests for TestBeeLineDriver
[ https://issues.apache.org/jira/browse/HIVE-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4161: Fix Version/s: (was: 0.11.0) create clean and small default set of tests for TestBeeLineDriver - Key: HIVE-4161 URL: https://issues.apache.org/jira/browse/HIVE-4161 Project: Hive Issue Type: Bug Components: HiveServer2 Reporter: Thejas M Nair Assignee: Thejas M Nair Labels: HiveServer2 HiveServer2 (HIVE-2935) has added TestBeeLineDriver on the lines of TestCliDriver, which runs all the tests in TestCliDriver through the beeline commandline, which uses jdbc+hive server2. There are failures in many of the test cases after the rebase of the patch against latest hive code. The tests also almost double the time taken to run hive unit tests because TestCliDriver takes bulk of the hive unit test runtime. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3978) HIVE_AUX_JARS_PATH should have : instead of , as separator since it gets appended to HADOOP_CLASSPATH
[ https://issues.apache.org/jira/browse/HIVE-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3978: Fix Version/s: (was: 0.11.0) (was: 0.10.0) HIVE_AUX_JARS_PATH should have : instead of , as separator since it gets appended to HADOOP_CLASSPATH - Key: HIVE-3978 URL: https://issues.apache.org/jira/browse/HIVE-3978 Project: Hive Issue Type: Bug Environment: hive-0.10 hcatalog-0.5 hadoop 0.23 hbase 0.94 Reporter: Arup Malakar Assignee: Arup Malakar Attachments: HIVE-3978_branch_0.10_0.patch, HIVE-3978_trunk_0.patch The following code gets executed only in case of cygwin. HIVE_AUX_JARS_PATH=`echo $HIVE_AUX_JARS_PATH | sed 's/,/:/g'` But since HIVE_AUX_JARS_PATH gets added to HADOOP_CLASSPATH, the comma should get replaced by : for all cases. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4085) Incorrectly pruning columns for PTFOperator
[
https://issues.apache.org/jira/browse/HIVE-4085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-4085:
Fix Version/s: (was: 0.11.0)
Incorrectly pruning columns for PTFOperator
---
Key: HIVE-4085
URL: https://issues.apache.org/jira/browse/HIVE-4085
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Reporter: Ashutosh Chauhan
Following simple query used to work before HIVE-4035
{code}
select s, sum(b) over (distribute by i sort by si rows between unbounded
preceding and current row) from over100k;
{code}
but now it fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3957) Add pseudo-BNF grammar for RCFile to Javadoc
[ https://issues.apache.org/jira/browse/HIVE-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3957: Fix Version/s: (was: 0.11.0) Add pseudo-BNF grammar for RCFile to Javadoc Key: HIVE-3957 URL: https://issues.apache.org/jira/browse/HIVE-3957 Project: Hive Issue Type: Bug Components: Documentation Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Priority: Minor Attachments: HIVE-3957.1.patch, HIVE-3957.2.patch Carl Steinbach did wrote some nice pseudo-BNF grammar for RCFile. I think it would rad to put it in the Javadoc to benefit other developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4224) Upgrade to Thrift 1.0 when available
[ https://issues.apache.org/jira/browse/HIVE-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4224: Fix Version/s: (was: 0.11.0) Upgrade to Thrift 1.0 when available Key: HIVE-4224 URL: https://issues.apache.org/jira/browse/HIVE-4224 Project: Hive Issue Type: Sub-task Components: HiveServer2, Metastore, Server Infrastructure Affects Versions: 0.11.0 Reporter: Brock Noland Priority: Minor -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3979) Provide syntax for unescaped regex on rlike, and other regexp_* functions
[
https://issues.apache.org/jira/browse/HIVE-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-3979:
Fix Version/s: (was: 0.11.0)
Provide syntax for unescaped regex on rlike, and other regexp_* functions
-
Key: HIVE-3979
URL: https://issues.apache.org/jira/browse/HIVE-3979
Project: Hive
Issue Type: New Feature
Components: UDF
Affects Versions: 0.10.0
Environment: All Queries, JDBC and cli
Reporter: John Omernik
Labels: patch
To make hive more intuitive, can we provide a syntax for hive functions that
when encompassing a regex, that regex doesn't have to the standard hive
double escape? For example
column rlike '\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}'
could become
column rlike '/\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/'
In this case, just using PERL's regex brackets, but could be anything...
The point is to be able to take a standard JAVA/Perl type regex and cut and
paste it into a query. This would allow for faster query development, more
mainstream adoption by those with regex experience, and intuitive transition
to hive query writing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4003) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4003: Fix Version/s: (was: 0.11.0) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java - Key: HIVE-4003 URL: https://issues.apache.org/jira/browse/HIVE-4003 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Thomas Adam Assignee: Mark Grover Attachments: HIVE-4003.patch Utilities.java seems to be throwing a NPE. Change contributed by Thomas Adam. Reference: https://github.com/tecbot/hive/commit/1e29d88837e4101a76e870a716aadb729437355b#commitcomment-2588350 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4266) Refactor HCatalog code to org.apache.hive.hcatalog
[ https://issues.apache.org/jira/browse/HIVE-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4266: Fix Version/s: (was: 0.11.0) Refactor HCatalog code to org.apache.hive.hcatalog -- Key: HIVE-4266 URL: https://issues.apache.org/jira/browse/HIVE-4266 Project: Hive Issue Type: Sub-task Components: HCatalog Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Alan Gates Priority: Blocker Currently HCatalog code is in packages org.apache.hcatalog. It needs to now move to org.apache.hive.hcatalog. Shell classes/interface need to be created for public facing classes so that user's code does not break. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1016) Ability to access DistributedCache from UDFs
[ https://issues.apache.org/jira/browse/HIVE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-1016: Fix Version/s: (was: 0.11.0) Ability to access DistributedCache from UDFs Key: HIVE-1016 URL: https://issues.apache.org/jira/browse/HIVE-1016 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Carl Steinbach Assignee: Carl Steinbach Attachments: HIVE-1016.1.patch.txt, HIVE-1016.r1471197.patch.txt There have been several requests on the mailing list for information about how to access the DistributedCache from UDFs, e.g.: http://www.mail-archive.com/hive-user@hadoop.apache.org/msg01650.html http://www.mail-archive.com/hive-user@hadoop.apache.org/msg01926.html While responses to these emails suggested several workarounds, the only correct way of accessing the distributed cache is via the static methods of Hadoop's DistributedCache class, and all of these methods require that the JobConf be passed in as a parameter. Hence, giving UDFs access to the distributed cache reduces to giving UDFs access to the JobConf. I propose the following changes to GenericUDF/UDAF/UDTF: * Add an exec_init(Configuration conf) method that is called during Operator initialization at runtime. * Change the name of the initialize method to compile_init to make it clear that this method is called at compile-time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4101) Partition By field must be in select field list
[
https://issues.apache.org/jira/browse/HIVE-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-4101:
Fix Version/s: (was: 0.11.0)
Partition By field must be in select field list
---
Key: HIVE-4101
URL: https://issues.apache.org/jira/browse/HIVE-4101
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Reporter: Brock Noland
This following query:
{noformat}
SELECT year, quarter, sales,avg(sales) OVER (PARTITION BY department, year)
FROM quarterly_sales
WHERE department = 'Appliances';
{noformat}
fails as below. If department is moved to the select field list it passes.
{noformat}
Diagnostic Messages for this Task:java.lang.RuntimeException: Error in
configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:485)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 9 more
Caused by: java.lang.RuntimeException: Reduce operator initialization failed
at
org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:160)
... 14 more
Caused by: java.lang.RuntimeException: cannot find field _col0 from [0:_col1,
1:_col2, 2:_col3]
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:366)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:143)
at
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:57)
at
org.apache.hadoop.hive.ql.exec.PTFOperator.setupKeysWrapper(PTFOperator.java:193)
at
org.apache.hadoop.hive.ql.exec.PTFOperator.initializeOp(PTFOperator.java:100)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:377)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:481)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:409)
at
org.apache.hadoop.hive.ql.exec.ExtractOperator.initializeOp(ExtractOperator.java:40)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:377)
at
org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:152)
... 14 more
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4194) JDBC2: HiveDriver should not throw RuntimeException when passed an invalid URL
[ https://issues.apache.org/jira/browse/HIVE-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4194: Fix Version/s: (was: 0.11.0) JDBC2: HiveDriver should not throw RuntimeException when passed an invalid URL -- Key: HIVE-4194 URL: https://issues.apache.org/jira/browse/HIVE-4194 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Richard Ding Assignee: Richard Ding Attachments: HIVE-4194.patch As per JDBC 3.0 Spec (section 9.2) If the Driver implementation understands the URL, it will return a Connection object; otherwise it returns null Currently HiveConnection constructor will throw IllegalArgumentException if url string doesn't start with jdbc:hive2. This exception should be caught by HiveDriver.connect and return null. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4267) hiveserver2 jdbc driver - remove trailing .0 in resultset.getString() for timestamp field
[ https://issues.apache.org/jira/browse/HIVE-4267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4267: Fix Version/s: (was: 0.11.0) hiveserver2 jdbc driver - remove trailing .0 in resultset.getString() for timestamp field Key: HIVE-4267 URL: https://issues.apache.org/jira/browse/HIVE-4267 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-4267.1.patch In case of timestamp field, it is printed with a .0 at the end, which is inconsistent with hive cli. The format without .0 is compatible with java.sql.Timestamp.valueOf(String). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4421) Improve memory usage by ORC dictionaries
[ https://issues.apache.org/jira/browse/HIVE-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4421: Fix Version/s: (was: 0.11.0) Improve memory usage by ORC dictionaries Key: HIVE-4421 URL: https://issues.apache.org/jira/browse/HIVE-4421 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: HIVE-4421.D10545.1.patch, HIVE-4421.D10545.2.patch, HIVE-4421.D10545.3.patch, HIVE-4421.D10545.4.patch Currently, for tables with many string columns, it is possible to significantly underestimate the memory used by the ORC dictionaries and cause the query to run out of memory in the task. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3886) WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated
[ https://issues.apache.org/jira/browse/HIVE-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3886: Fix Version/s: (was: 0.11.0) WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated - Key: HIVE-3886 URL: https://issues.apache.org/jira/browse/HIVE-3886 Project: Hive Issue Type: Bug Components: Configuration Affects Versions: 0.9.0, 0.10.0, 0.11.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Minor Attachments: HIVE-3886.1.patch WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4508) Fix various release issues in 0.11.0rc1
[ https://issues.apache.org/jira/browse/HIVE-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4508: Fix Version/s: (was: 0.11.0) Fix various release issues in 0.11.0rc1 --- Key: HIVE-4508 URL: https://issues.apache.org/jira/browse/HIVE-4508 Project: Hive Issue Type: Bug Reporter: Owen O'Malley Assignee: Owen O'Malley Carl described some non-code issues in the 0.11.0rc1 and I want to fix them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4166) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data
[ https://issues.apache.org/jira/browse/HIVE-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4166: Fix Version/s: (was: 0.11.0) (was: 0.10.0) closeAllForUGI causes failure in hiveserver2 when fetching large amount of data --- Key: HIVE-4166 URL: https://issues.apache.org/jira/browse/HIVE-4166 Project: Hive Issue Type: Bug Components: HiveServer2, Security, Shims Affects Versions: 0.10.0, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4166-0.10.patch, HIVE-4166-trunk.patch HiveServer2 configured to use Kerberos authentication with doAs enabled throws an exception when fetching a large amount of data from a query. The exception is caused because FileSystem.closeAllForUGI is always called at the end of TUGIAssumingProcessor.process. This affects requests on the ResultSet for data from a SELECT query when the amount of data exceeds a certain size. At that point any subsequent calls to fetch more data throw an exception because the underlying DFSClient has been closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4467) HiveConnection does not handle failures correctly
[ https://issues.apache.org/jira/browse/HIVE-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4467: Fix Version/s: (was: 0.12.0) (was: 0.11.0) HiveConnection does not handle failures correctly - Key: HIVE-4467 URL: https://issues.apache.org/jira/browse/HIVE-4467 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Thiruvel Thirumoolan Assignee: Thiruvel Thirumoolan Attachments: HIVE-4467.patch HiveConnection uses Utils.verifySuccess* routines to check if there is any error from the server side. This is not handled well. In Utils.verifySuccess() when withInfo is 'false', the condition evaluates to 'false' and no SQLexception is thrown even though there could be a problem on the server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3953) Reading of partitioned Avro data fails because of missing properties
[ https://issues.apache.org/jira/browse/HIVE-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3953: Fix Version/s: (was: 0.11.0) Reading of partitioned Avro data fails because of missing properties Key: HIVE-3953 URL: https://issues.apache.org/jira/browse/HIVE-3953 Project: Hive Issue Type: Bug Affects Versions: 0.11.0 Reporter: Mark Wagner Priority: Blocker Attachments: avro_partition_test.q After HIVE-3833, reading partitioned Avro data fails due to missing properties. The avro.schema.(url|literal) properties are not making it all the way to the SerDe. Non-partitioned data can still be read. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4100) Improve regex_replace UDF to allow non-ascii characters
[ https://issues.apache.org/jira/browse/HIVE-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4100: Fix Version/s: (was: 0.11.0) Improve regex_replace UDF to allow non-ascii characters --- Key: HIVE-4100 URL: https://issues.apache.org/jira/browse/HIVE-4100 Project: Hive Issue Type: Improvement Components: UDF Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover There have a been a few email threads on the user mailing list regarding regex_replace UDF not supporting non-ASCII characters. We should validate that and improve the UDF to allow it. Translate UDF will be a good reference since it does that by using code points instead of characters -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4168) remove package-info.java from svn
[ https://issues.apache.org/jira/browse/HIVE-4168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4168: Fix Version/s: (was: 0.11.0) remove package-info.java from svn - Key: HIVE-4168 URL: https://issues.apache.org/jira/browse/HIVE-4168 Project: Hive Issue Type: Bug Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-4168.1.patch common/src/gen/org/apache/hive/common/package-info.java is autogenerated during compile (by saveVersion.sh). Looks like this was unintentionally checked-in. As the file includes timestamps and checksums, after a compile it shows up as a source code change. We should delete this file from svn repo. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1990) Logging fails due to moved EventCounter class in Hadoop 0.20.100
[
https://issues.apache.org/jira/browse/HIVE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-1990:
Fix Version/s: (was: 0.11.0)
Logging fails due to moved EventCounter class in Hadoop 0.20.100
Key: HIVE-1990
URL: https://issues.apache.org/jira/browse/HIVE-1990
Project: Hive
Issue Type: Bug
Components: Logging
Affects Versions: 0.6.0
Environment: Red Hat 2.6.18
Reporter: Joep Rottinghuis
Attachments: hive-1990.patch
When compiling Hive against Hadoop 0.20.100 logging on command line and in
unit tests fails due to the EventCounter class being moved from
o.a.h.metrics.jvm.EventCounter to o.a.h.log.EventCounter.
{code}
[junit] Running org.apache.hadoop.hive.serde2.TestTCTLSeparatedProtocol
[junit] log4j:ERROR Could not instantiate class
[org.apache.hadoop.metrics.jvm.EventCounter].
[junit] java.lang.ClassNotFoundException:
org.apache.hadoop.metrics.jvm.EventCounter
[junit] at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
[junit] at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
[junit] at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
[junit] at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
{code}
As a note: In order to re-produce I first applied patch as per HIVE-1264 to
0.6 branch in order to resolve jar naming issues in build.
Then I locally modified the build.properties to my locally built 0.20.100
Hadoop build:
{code}
hadoop.security.url=file:.../hadoop/core/hadoop-${hadoop.version}
hadoop.security.version=${hadoop.version}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3770) Test cases's broken in TestParse
[
https://issues.apache.org/jira/browse/HIVE-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-3770:
Fix Version/s: (was: 0.11.0)
Test cases's broken in TestParse
Key: HIVE-3770
URL: https://issues.apache.org/jira/browse/HIVE-3770
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Reporter: Gang Tim Liu
Assignee: Gang Tim Liu
20 TestParse test cases are broken. It's introduced from D7017 HIVE-3750:
20 Test cases
=
testParse_case_sensitivity
testParse_groupby1
testParse_input1
testParse_input2
testParse_input3
testParse_input4
testParse_input5
testParse_input6
testParse_input7
testParse_input9
testParse_input_testsequencefile
testParse_join1
testParse_join2
testParse_join3
testParse_sample2
testParse_sample3
testParse_sample4
testParse_sample5
testParse_sample6
testParse_sample7
sample error
{quote}
ant test -Dtestcase=TestParse -Dqfile=groupby1.q
[junit] diff -a ../build/ql/test/logs/positive/groupby1.q.out
../ql/src/test/results/compiler/parse/groupby1.q.out
[junit] diff -a -b ../build/ql/test/logs/positive/groupby1.q.xml
../ql/src/test/results/compiler/plan/groupby1.q.xml
[junit] 1224,1226d1223
[junit] void property=maxStatsKeyPrefixLength
[junit] int200/int
[junit] /void
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4505) Hive can't load transforms added using 'ADD FILE'
[
https://issues.apache.org/jira/browse/HIVE-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-4505:
Fix Version/s: (was: 0.12.0)
(was: 0.11.0)
Hive can't load transforms added using 'ADD FILE'
-
Key: HIVE-4505
URL: https://issues.apache.org/jira/browse/HIVE-4505
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.11.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Priority: Blocker
Attachments: HIVE-4505-1.patch
ADD FILE mangles name of the resource when copying to resource download
directory. As a results following doesn't work:
{code:sql}
ADD FILE test.py;
SELECT TRANSFORM (id) USING 'python test.py' AS b FROM tab1;
{code}
The resource gets added with a different name every time which makes it
impossible to use transform in non-interactive mode.
This seems to be due to HIVE-3431
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4198) Move HCatalog code into Hive
[ https://issues.apache.org/jira/browse/HIVE-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4198: Fix Version/s: (was: 0.11.0) Move HCatalog code into Hive Key: HIVE-4198 URL: https://issues.apache.org/jira/browse/HIVE-4198 Project: Hive Issue Type: Task Components: HCatalog Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Alan Gates The HCatalog code needs to be moved into Hive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3250) ArrayIndexOutOfBoundsException in ColumnPrunerProcFactory$ColumnPrunerSelectProc
[
https://issues.apache.org/jira/browse/HIVE-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-3250:
Fix Version/s: (was: 0.11.0)
ArrayIndexOutOfBoundsException in
ColumnPrunerProcFactory$ColumnPrunerSelectProc
Key: HIVE-3250
URL: https://issues.apache.org/jira/browse/HIVE-3250
Project: Hive
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Nicolas Lalevée
I have a query which was not selecting field and the optimizer fails to evict
them with the following stack trace:
{noformat}
FAILED: Hive Internal Error: java.lang.ArrayIndexOutOfBoundsException(-1)
java.lang.ArrayIndexOutOfBoundsException: -1
at java.util.ArrayList.get(ArrayList.java:324)
at
org.apache.hadoop.hive.ql.optimizer.ColumnPrunerProcFactory$ColumnPrunerSelectProc.process(ColumnPrunerProcFactory.java:397)
at
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:88)
at
org.apache.hadoop.hive.ql.optimizer.ColumnPruner$ColumnPrunerWalker.walk(ColumnPruner.java:143)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:102)
at
org.apache.hadoop.hive.ql.optimizer.ColumnPruner.transform(ColumnPruner.java:106)
at
org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:87)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7306)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:430)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:889)
at
org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:191)
at
org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:187)
{noformat}
The failing query reduced to the only failing part:
{noformat}
SELECT explodedUrls FROM
(
SELECT userid, array(named_struct('date', count(*))) AS urls, count(*) AS
user_lid FROM
(
SELECT * FROM NicoPageViewEvent WHERE day '20130801'
) pve
GROUP BY userid
) userViewData
LATERAL VIEW s_explode_pageflow(userViewData.urls) userViewDataLateralView AS
explodedUrls
{noformat}
Adding fields make it work:
{noformat}
SELECT userid, explodedUrls, user_lid FROM
(
SELECT userid, array(named_struct('date', count(*))) AS urls, count(*) AS
user_lid FROM
(
SELECT * FROM NicoPageViewEvent WHERE day '20130801'
) pve
GROUP BY userid
) userViewData
LATERAL VIEW s_explode_pageflow(userViewData.urls) userViewDataLateralView AS
explodedUrls
{noformat}
And s_explode_pageflow is a custom function which take an array of struct and
split them into arrays of struct
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4287) hive cli null representation is inconsistent
[
https://issues.apache.org/jira/browse/HIVE-4287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-4287:
Fix Version/s: (was: 0.11.0)
hive cli null representation is inconsistent
Key: HIVE-4287
URL: https://issues.apache.org/jira/browse/HIVE-4287
Project: Hive
Issue Type: Bug
Components: CLI
Reporter: Thejas M Nair
Assignee: Thejas M Nair
hive cli prints null value in upper case ('NULL') for primitives other than
binary, and in lower case ('null') for complex types and binary.
We should make it consistent for all data types.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4213) List bucketing error too restrictive
[
https://issues.apache.org/jira/browse/HIVE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-4213:
Fix Version/s: (was: 0.11.0)
List bucketing error too restrictive
Key: HIVE-4213
URL: https://issues.apache.org/jira/browse/HIVE-4213
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Mark Grover
Assignee: Gang Tim Liu
With the introduction of List bucketing, we introduced a config validation
step where we say:
{code}
SUPPORT_DIR_MUST_TRUE_FOR_LIST_BUCKETING(
10199,
hive.mapred.supports.subdirectories must be true
+ if any one of following is true:
hive.internal.ddl.list.bucketing.enable,
+ hive.optimize.listbucketing and mapred.input.dir.recursive),
{code}
This seems overly restrictive to because there are use cases where people may
want to use {{mapred.input.dir.recursive}} to {{true}} even when they don't
care about list bucketing.
Is that not true?
For example, here is the unit test code for {{clientpositive/recursive_dir.q}}
{code}
CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING)
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
SELECT key+11 FROM src WHERE key=484;
ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
ALTER TABLE fact_daily ADD PARTITION (ds='1')
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
SELECT * FROM fact_daily WHERE ds='1';
SELECT count(1) FROM fact_daily WHERE ds='1';
{code}
The unit test doesn't seem to be concerned about list bucketing but wants to
set {{mapred.input.dir.recursive}} to {{true}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4445) Fix the Hive unit test failures on Windows when Linux scripts or commands are used in test cases
[ https://issues.apache.org/jira/browse/HIVE-4445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4445: Fix Version/s: (was: 0.11.0) Fix the Hive unit test failures on Windows when Linux scripts or commands are used in test cases Key: HIVE-4445 URL: https://issues.apache.org/jira/browse/HIVE-4445 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.11.0 Environment: Windows Reporter: Xi Fang Attachments: HIVE-4445.1.patch The following unit tests fail on Windows because Linux scripts or commands are used in the test cases or .q files: 1. TestMinimrCliDriver: scriptfile1.q 2. TestNegativeMinimrCliDriver: mapreduce_stack_trace_hadoop20.q, minimr_broken_pipe.q 3. TestCliDriver: hiveprofiler_script0.q -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4232) JDBC2 HiveConnection has odd defaults
[ https://issues.apache.org/jira/browse/HIVE-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4232: Fix Version/s: (was: 0.12.0) (was: 0.11.0) JDBC2 HiveConnection has odd defaults - Key: HIVE-4232 URL: https://issues.apache.org/jira/browse/HIVE-4232 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-4232-1.patch, HIVE-4232-2.patch, HIVE-4232-3-0.11.patch, HIVE-4232-3-trunk.patch, HIVE-4232-4-0.11.patch, HIVE-4232-4-trunk.patch, HIVE-4232.patch HiveConnection defaults to using a plain SASL transport if auth is not set. To get a raw transport auth must be set to noSasl; furthermore noSasl is case sensitive. Code tries to infer Kerberos or plain authentication based on the presence of principal. There is no provision for specifying QOP level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4349) Fix the Hive unit test failures when the Hive enlistment root path is longer than ~12 characters
[ https://issues.apache.org/jira/browse/HIVE-4349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4349: Fix Version/s: (was: 0.11.0) Fix the Hive unit test failures when the Hive enlistment root path is longer than ~12 characters Key: HIVE-4349 URL: https://issues.apache.org/jira/browse/HIVE-4349 Project: Hive Issue Type: Bug Affects Versions: 0.11.0 Reporter: Xi Fang Attachments: HIVE-4349.1.patch If the Hive enlistment root path is longer than 12 chars then test classpath “hadoop.testcp” is exceeding the 8K chars so we are unable to run most of the Hive unit tests on Windows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3652) Join optimization for star schema
[ https://issues.apache.org/jira/browse/HIVE-3652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3652: Fix Version/s: (was: 0.11.0) Join optimization for star schema - Key: HIVE-3652 URL: https://issues.apache.org/jira/browse/HIVE-3652 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Amareshwari Sriramadasu Assignee: Vikram Dixit K Attachments: HIVE-3652-tests.patch, HIVE-3652-tests.patch Currently, if we join one fact table with multiple dimension tables, it results in multiple mapreduce jobs for each join with dimension table, because join would be on different keys for each dimension. Usually all the dimension tables will be small and can fit into memory and so map-side join can used to join with fact table. In this issue I want to look at optimizing such query to generate single mapreduce job sothat mapper loads dimension tables into memory and joins with fact table on different keys as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4500) HS2 holding too many file handles of hive_job_log_hive_*.txt files
[ https://issues.apache.org/jira/browse/HIVE-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649959#comment-13649959 ] Carl Steinbach commented on HIVE-4500: -- I left comments on rb. Thanks. HS2 holding too many file handles of hive_job_log_hive_*.txt files -- Key: HIVE-4500 URL: https://issues.apache.org/jira/browse/HIVE-4500 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Alan Gates Attachments: HIVE-4500.patch In the hiveserver2 setup used for testing, we see that it has 2444 files open and of them 2152 are /tmp/hive/hive_job_log_hive_*.txt files -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3176) implement returning values for SQLException getSQLState()
[ https://issues.apache.org/jira/browse/HIVE-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiu updated HIVE-3176: -- Attachment: HIVE-3176.patch.txt implement returning values for SQLException getSQLState() - Key: HIVE-3176 URL: https://issues.apache.org/jira/browse/HIVE-3176 Project: Hive Issue Type: Improvement Components: JDBC Affects Versions: 0.8.1 Reporter: N Campbell Attachments: HIVE-3176.patch.txt a dynamic SQL application should be able to check the values returned by getSQLState on a SQLException object. Currently the Hive driver is not doing this (throws exceptions etc). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Apache Hive 0.11.0 Release Candidate 1
All, Ashutosh is on vacation, so if it is ok with everyone, I'll roll a new RC in the next day. Issues that I think should be included: HIVE-4508 - update release notes HIVE-4505 - Hive can't load transforms added using 'ADD FILE' HIVE-4494 - ORC map columns get class cast exception in some context HIVE-4421 - Improve memory usage by ORC dictionaries Thanks, Owen On Mon, May 6, 2013 at 9:15 AM, Owen O'Malley omal...@apache.org wrote: I've create a jira to track the non-code issues: https://issues.apache.org/jira/browse/HIVE-4508 I'll upload a patch in the next hour for comments. -- Owen On Sun, May 5, 2013 at 10:52 PM, Carl Steinbach cwsteinb...@gmail.comwrote: I took a quick look. Here's a list of things I noticed: * NOTICE ** Copyright is out of date. * README.txt ** Add one more '=' to the title underline. ** Requirements section only lists Hadoop 0.20.x ** Will it work with Java 1.7? * RELEASE_NOTES.txt ** The 0.11 section lists tickets that aren't marked as resolved/fixed on JIRA, and which are not included in the git commit log (e.g. HIVE-3979). This may also extend to the 0.10 release notes. Related question: How were these release notes generated? * docs/index.html ** Update email list links (e.g. hive-u...@hadoop.apache.org) * src/build.properties ** year=2012 * src/eclipse-files/.classpath ** The guava jar is not in the right location. This breaks the eclipse import. Thanks. Carl On Tue, Apr 30, 2013 at 5:18 PM, Ashutosh Chauhan hashut...@apache.org wrote: Hey all, Based on feedback from folks, I have respun release candidate, RC1. Please take a look. It basically fixes the size bloat of tarball. Source tag for RC1 is at: https://svn.apache.org/repos/asf/hive/tags/release-0.11.0-rc1 Source tar ball and convenience binary artifacts can be found at:http://people.apache.org/~hashutosh/hive-0.11.0-rc1/ Maven artifacts for hive are available at: https://repository.apache.org/content/repositories/orgapachehive-158/ Maven artifacts for hcatalog are available at: https://repository.apache.org/content/repositories/orgapachehcatalog-159/ This release has many goodies including HiveServer2, integrated hcatalog, windowing and analytical functions, decimal data type, better query planning, performance enhancements and various bug fixes. In total, we resolved more than 350 issues. Full list of fixed issues can be found at: http://s.apache.org/8Fr Voting will conclude in 72 hours. Hive PMC Members: Please test and vote. Thanks, Ashutosh (On behalf of Hive contributors who made 0.11 a possibility)
Re: [VOTE] Apache Hive 0.11.0 Release Candidate 1
I think we add HIVE-4500 to the list, and I would also like to see us reach an agreement about when HIVE-4266 will be fixed. Thanks. Carl On Mon, May 6, 2013 at 11:35 AM, Owen O'Malley omal...@apache.org wrote: All, Ashutosh is on vacation, so if it is ok with everyone, I'll roll a new RC in the next day. Issues that I think should be included: HIVE-4508 - update release notes HIVE-4505 - Hive can't load transforms added using 'ADD FILE' HIVE-4494 - ORC map columns get class cast exception in some context HIVE-4421 - Improve memory usage by ORC dictionaries Thanks, Owen On Mon, May 6, 2013 at 9:15 AM, Owen O'Malley omal...@apache.org wrote: I've create a jira to track the non-code issues: https://issues.apache.org/jira/browse/HIVE-4508 I'll upload a patch in the next hour for comments. -- Owen On Sun, May 5, 2013 at 10:52 PM, Carl Steinbach cwsteinb...@gmail.com wrote: I took a quick look. Here's a list of things I noticed: * NOTICE ** Copyright is out of date. * README.txt ** Add one more '=' to the title underline. ** Requirements section only lists Hadoop 0.20.x ** Will it work with Java 1.7? * RELEASE_NOTES.txt ** The 0.11 section lists tickets that aren't marked as resolved/fixed on JIRA, and which are not included in the git commit log (e.g. HIVE-3979). This may also extend to the 0.10 release notes. Related question: How were these release notes generated? * docs/index.html ** Update email list links (e.g. hive-u...@hadoop.apache.org) * src/build.properties ** year=2012 * src/eclipse-files/.classpath ** The guava jar is not in the right location. This breaks the eclipse import. Thanks. Carl On Tue, Apr 30, 2013 at 5:18 PM, Ashutosh Chauhan hashut...@apache.org wrote: Hey all, Based on feedback from folks, I have respun release candidate, RC1. Please take a look. It basically fixes the size bloat of tarball. Source tag for RC1 is at: https://svn.apache.org/repos/asf/hive/tags/release-0.11.0-rc1 Source tar ball and convenience binary artifacts can be found at:http://people.apache.org/~hashutosh/hive-0.11.0-rc1/ Maven artifacts for hive are available at: https://repository.apache.org/content/repositories/orgapachehive-158/ Maven artifacts for hcatalog are available at: https://repository.apache.org/content/repositories/orgapachehcatalog-159/ This release has many goodies including HiveServer2, integrated hcatalog, windowing and analytical functions, decimal data type, better query planning, performance enhancements and various bug fixes. In total, we resolved more than 350 issues. Full list of fixed issues can be found at: http://s.apache.org/8Fr Voting will conclude in 72 hours. Hive PMC Members: Please test and vote. Thanks, Ashutosh (On behalf of Hive contributors who made 0.11 a possibility)
[jira] [Updated] (HIVE-4266) Refactor HCatalog code to org.apache.hive.hcatalog
[ https://issues.apache.org/jira/browse/HIVE-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-4266: - Fix Version/s: 0.12.0 Refactor HCatalog code to org.apache.hive.hcatalog -- Key: HIVE-4266 URL: https://issues.apache.org/jira/browse/HIVE-4266 Project: Hive Issue Type: Sub-task Components: HCatalog Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Alan Gates Priority: Blocker Fix For: 0.12.0 Currently HCatalog code is in packages org.apache.hcatalog. It needs to now move to org.apache.hive.hcatalog. Shell classes/interface need to be created for public facing classes so that user's code does not break. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-4497) beeline module tests don't get run by default
[ https://issues.apache.org/jira/browse/HIVE-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-4497. -- Resolution: Fixed Fix Version/s: 0.12.0 Committed to trunk. Thanks Thejas. beeline module tests don't get run by default - Key: HIVE-4497 URL: https://issues.apache.org/jira/browse/HIVE-4497 Project: Hive Issue Type: Bug Components: CLI, HiveServer2 Reporter: Thejas M Nair Assignee: Thejas M Nair Fix For: 0.12.0 Attachments: HIVE-4497.1.patch beeline tests are not getting run by default . See https://builds.apache.org/job/Hive-trunk-h0.21/lastCompletedBuild/testReport/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4357) BeeLine tests are not getting executed
[ https://issues.apache.org/jira/browse/HIVE-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-4357: - Resolution: Duplicate Status: Resolved (was: Patch Available) @Rob: I apologize for not committing this in a timely manner. I committed HIVE-4497 instead since beeline depends on JDBC and should follow it in the test order. BeeLine tests are not getting executed -- Key: HIVE-4357 URL: https://issues.apache.org/jira/browse/HIVE-4357 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.10.0 Reporter: Carl Steinbach Assignee: Rob Weltman Attachments: HIVE-4357.1.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3957) Add pseudo-BNF grammar for RCFile to Javadoc
[ https://issues.apache.org/jira/browse/HIVE-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3957: - Resolution: Fixed Fix Version/s: 0.12.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) +1. Committed to trunk. Thanks Mark! Add pseudo-BNF grammar for RCFile to Javadoc Key: HIVE-3957 URL: https://issues.apache.org/jira/browse/HIVE-3957 Project: Hive Issue Type: Bug Components: Documentation Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Priority: Minor Fix For: 0.12.0 Attachments: HIVE-3957.1.patch, HIVE-3957.2.patch Carl Steinbach did wrote some nice pseudo-BNF grammar for RCFile. I think it would rad to put it in the Javadoc to benefit other developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Apache Hive 0.11.0 Release Candidate 1
* README.txt ** The Upgrade section says that you need to replace old copies of hive-default.xml. This is no longer true. On Mon, May 6, 2013 at 11:44 AM, Carl Steinbach cwsteinb...@gmail.comwrote: I think we add HIVE-4500 to the list, and I would also like to see us reach an agreement about when HIVE-4266 will be fixed. Thanks. Carl On Mon, May 6, 2013 at 11:35 AM, Owen O'Malley omal...@apache.org wrote: All, Ashutosh is on vacation, so if it is ok with everyone, I'll roll a new RC in the next day. Issues that I think should be included: HIVE-4508 - update release notes HIVE-4505 - Hive can't load transforms added using 'ADD FILE' HIVE-4494 - ORC map columns get class cast exception in some context HIVE-4421 - Improve memory usage by ORC dictionaries Thanks, Owen On Mon, May 6, 2013 at 9:15 AM, Owen O'Malley omal...@apache.org wrote: I've create a jira to track the non-code issues: https://issues.apache.org/jira/browse/HIVE-4508 I'll upload a patch in the next hour for comments. -- Owen On Sun, May 5, 2013 at 10:52 PM, Carl Steinbach cwsteinb...@gmail.com wrote: I took a quick look. Here's a list of things I noticed: * NOTICE ** Copyright is out of date. * README.txt ** Add one more '=' to the title underline. ** Requirements section only lists Hadoop 0.20.x ** Will it work with Java 1.7? * RELEASE_NOTES.txt ** The 0.11 section lists tickets that aren't marked as resolved/fixed on JIRA, and which are not included in the git commit log (e.g. HIVE-3979). This may also extend to the 0.10 release notes. Related question: How were these release notes generated? * docs/index.html ** Update email list links (e.g. hive-u...@hadoop.apache.org) * src/build.properties ** year=2012 * src/eclipse-files/.classpath ** The guava jar is not in the right location. This breaks the eclipse import. Thanks. Carl On Tue, Apr 30, 2013 at 5:18 PM, Ashutosh Chauhan hashut...@apache.org wrote: Hey all, Based on feedback from folks, I have respun release candidate, RC1. Please take a look. It basically fixes the size bloat of tarball. Source tag for RC1 is at: https://svn.apache.org/repos/asf/hive/tags/release-0.11.0-rc1 Source tar ball and convenience binary artifacts can be found at:http://people.apache.org/~hashutosh/hive-0.11.0-rc1/ Maven artifacts for hive are available at: https://repository.apache.org/content/repositories/orgapachehive-158/ Maven artifacts for hcatalog are available at: https://repository.apache.org/content/repositories/orgapachehcatalog-159/ This release has many goodies including HiveServer2, integrated hcatalog, windowing and analytical functions, decimal data type, better query planning, performance enhancements and various bug fixes. In total, we resolved more than 350 issues. Full list of fixed issues can be found at: http://s.apache.org/8Fr Voting will conclude in 72 hours. Hive PMC Members: Please test and vote. Thanks, Ashutosh (On behalf of Hive contributors who made 0.11 a possibility)
[jira] [Updated] (HIVE-4508) Fix various release issues in 0.11.0rc1
[ https://issues.apache.org/jira/browse/HIVE-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4508: Attachment: h-4508.patch Removed paragraph about replacing hive-default.xml. Fix various release issues in 0.11.0rc1 --- Key: HIVE-4508 URL: https://issues.apache.org/jira/browse/HIVE-4508 Project: Hive Issue Type: Bug Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: h-4508.patch, h-4508.patch Carl described some non-code issues in the 0.11.0rc1 and I want to fix them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3384: Resolution: Fixed Fix Version/s: 0.11.0 Status: Resolved (was: Patch Available) This was committed to 0.11 HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification -- Key: HIVE-3384 URL: https://issues.apache.org/jira/browse/HIVE-3384 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.10.0 Reporter: Weidong Bian Assignee: Chris Drome Priority: Minor Fix For: 0.11.0 Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, D6873.7.patch, HIVE-3384-0.10.patch, HIVE-3384-2012-12-02.patch, HIVE-3384-2012-12-04.patch, HIVE-3384.2.patch, HIVE-3384-branch-0.9.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch jdbc module couldn't be compiled with jdk7 as it adds some abstract method in the JDBC specification some error info: error: HiveCallableStatement is not abstract and does not override abstract method TgetObject(String,ClassT) in CallableStatement . . . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3631) script_pipe.q fails when using JDK7
[ https://issues.apache.org/jira/browse/HIVE-3631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-3631: Resolution: Fixed Fix Version/s: 0.11.0 Status: Resolved (was: Patch Available) Probably won't make 0.9 or 0.10 releases. script_pipe.q fails when using JDK7 --- Key: HIVE-3631 URL: https://issues.apache.org/jira/browse/HIVE-3631 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.10.0, 0.9.1, 0.11.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.11.0 Attachments: HIVE-3631-0.10.patch, HIVE-3631-0.9.patch, HIVE-3631-trunk.patch Hive Runtime Error while closing operators: Hit error while closing .. The MR job fails on this test. Unfortunately, the exception is not all that helpful. I tracked this down to a class which attempts to close a stream that is already closed. Broken pipe exceptions are caught and not propagated further, but stream closed exception are not caught. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4509) Integer division should be cast to double.
Jitendra Nath Pandey created HIVE-4509: -- Summary: Integer division should be cast to double. Key: HIVE-4509 URL: https://issues.apache.org/jira/browse/HIVE-4509 Project: Hive Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey In current hive, the division always returns a double. Also, division by zero returns infinity following java semantics. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4503) HiveServer have too many opened fd。
[ https://issues.apache.org/jira/browse/HIVE-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13650106#comment-13650106 ] Alan Gates commented on HIVE-4503: -- This looks like a duplicate of HIVE-4500 HiveServer have too many opened fd。 --- Key: HIVE-4503 URL: https://issues.apache.org/jira/browse/HIVE-4503 Project: Hive Issue Type: Bug Affects Versions: 0.8.1 Environment: Hive: hive-0.8.1 OS: Red Hat Enterprise Linux Server release 5.7 (Tikanga) Hadoop: 0.20.205 Reporter: sutao bian When i run hiveserver a while time it will occur error Caused by: java.io.FileNotFoundException: /opt/tmp/mapred/local/jobTracker/job_201301251143_76286.xml (Too many open files) more errors info : 013-05-06 02:54:47,426 WARN parse.SemanticAnalyzer (SemanticAnalyzer.java:genBodyPlan(5821)) - Common Gby keys:null 2013-05-06 02:54:50,386 WARN mapred.JobClient (JobClient.java:copyAndConfigureFiles(659)) - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 2013-05-06 02:54:52,565 ERROR exec.Task (SessionState.java:printError(380)) - Job Submission failed with exception 'org.apache.hadoop.ipc.RemoteException(java.io.IOException: java.io.FileNotFoundException: /opt/tmp/mapred/local/jobTracker/job_201301251143_76286.xml (Too many open files) at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943) at sun.reflect.GeneratedMethodAccessor1278.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382) Caused by: java.io.FileNotFoundException: /opt/tmp/mapred/local/jobTracker/job_201301251143_76286.xml (Too many open files) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:212) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.init(RawLocalFileSystem.java:188) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.init(RawLocalFileSystem.java:184) at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:242) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.init(ChecksumFileSystem.java:335) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:368) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:546) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:527) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:434) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:229) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:163) at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1164) at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1145) at org.apache.hadoop.mapred.JobInProgress.init(JobInProgress.java:415) at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941) ... 10 more )' when i restart the hiveserver it will be ok . Thanks -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-4504) HiveServer have too many opened fd。
[ https://issues.apache.org/jira/browse/HIVE-4504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates resolved HIVE-4504. -- Resolution: Duplicate Duplicate of HIVE-4503 HiveServer have too many opened fd。 --- Key: HIVE-4504 URL: https://issues.apache.org/jira/browse/HIVE-4504 Project: Hive Issue Type: Bug Affects Versions: 0.8.1 Environment: Hive: hive-0.8.1 OS: Red Hat Enterprise Linux Server release 5.7 (Tikanga) Hadoop: 0.20.205 Reporter: sutao bian When i run hiveserver a while time it will occur error Caused by: java.io.FileNotFoundException: /opt/tmp/mapred/local/jobTracker/job_201301251143_76286.xml (Too many open files) more errors info : 013-05-06 02:54:47,426 WARN parse.SemanticAnalyzer (SemanticAnalyzer.java:genBodyPlan(5821)) - Common Gby keys:null 2013-05-06 02:54:50,386 WARN mapred.JobClient (JobClient.java:copyAndConfigureFiles(659)) - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 2013-05-06 02:54:52,565 ERROR exec.Task (SessionState.java:printError(380)) - Job Submission failed with exception 'org.apache.hadoop.ipc.RemoteException(java.io.IOException: java.io.FileNotFoundException: /opt/tmp/mapred/local/jobTracker/job_201301251143_76286.xml (Too many open files) at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943) at sun.reflect.GeneratedMethodAccessor1278.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382) Caused by: java.io.FileNotFoundException: /opt/tmp/mapred/local/jobTracker/job_201301251143_76286.xml (Too many open files) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:212) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.init(RawLocalFileSystem.java:188) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.init(RawLocalFileSystem.java:184) at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:242) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.init(ChecksumFileSystem.java:335) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:368) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:546) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:527) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:434) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:229) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:163) at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1164) at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1145) at org.apache.hadoop.mapred.JobInProgress.init(JobInProgress.java:415) at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941) ... 10 more )' when i restart the hiveserver it will be ok . Thanks -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
