[jira] [Created] (HIVE-6902) schematool should return exit codes according to current status.
Srinath Mantripragada created HIVE-6902: --- Summary: schematool should return exit codes according to current status. Key: HIVE-6902 URL: https://issues.apache.org/jira/browse/HIVE-6902 Project: Hive Issue Type: Improvement Components: Database/Schema Reporter: Srinath Mantripragada schematool should return exit codes according to current status when using -info flag, for example: Return code 0: Connection was successful and DB is up to date. Return code 1: Connection was successful and DB needs upgrade. Return code 2: Connection was not successful. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6901:
--
Attachment: HIVE-6901.1.patch
Patch #1 regenerated some test outputs.
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6901.1.patch, HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6432) Remove deprecated methods in HCatalog
[ https://issues.apache.org/jira/browse/HIVE-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6432: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Sushanth! Remove deprecated methods in HCatalog - Key: HIVE-6432 URL: https://issues.apache.org/jira/browse/HIVE-6432 Project: Hive Issue Type: Task Components: HCatalog Affects Versions: 0.14.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Fix For: 0.14.0 Attachments: 6432-addendum.patch, 6432-full.patch, HIVE-6432.2.patch, HIVE-6432.3.patch, HIVE-6432.4.patch, HIVE-6432.5.patch, HIVE-6432.patch, HIVE-6432.wip.1.patch, HIVE-6432.wip.2.patch, hcat.6432.test.out There are a lot of methods in HCatalog that have been deprecated in HCatalog 0.5, and some that were recently deprecated in Hive 0.11 (joint release with HCatalog). The goal for HCatalog deprecation is that in general, after something has been deprecated, it is expected to stay around for 2 releases, which means hive-0.13 will be the last release to ship with all the methods that were deprecated in hive-0.11 (the org.apache.hcatalog.* files should all be removed afterwards), and it is also good for us to clean out and nuke all other older deprecated methods. We should take this on early in a dev/release cycle to allow us time to resolve all fallout, so I propose that we remove all HCatalog deprecated methods after we branch out 0.13 and 0.14 becomes trunk. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968505#comment-13968505
]
Ashutosh Chauhan commented on HIVE-6901:
Dupe of HIVE-3925 ?
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6901.1.patch, HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-6903) Change default value of hive.metastore.execute.setugi to true
Ashutosh Chauhan created HIVE-6903: -- Summary: Change default value of hive.metastore.execute.setugi to true Key: HIVE-6903 URL: https://issues.apache.org/jira/browse/HIVE-6903 Project: Hive Issue Type: Task Components: Metastore Affects Versions: 0.12.0, 0.11.0, 0.10.0, 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Since its introduction in HIVE-2616 I havent seen any bug reported for it, only grief from users who expect system to work as if this is true by default. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968516#comment-13968516
]
Xuefu Zhang commented on HIVE-6901:
---
Actually HIVE-3925, while closely related to this, is concerned with stage
dependency. This JIRA is concerned with the plan manifesting for the fetch
stage. Thus, they seem orthogonal to each other.
It's nice to have both though.
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6901.1.patch, HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6903) Change default value of hive.metastore.execute.setugi to true
[ https://issues.apache.org/jira/browse/HIVE-6903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6903: --- Status: Patch Available (was: Open) Change default value of hive.metastore.execute.setugi to true - Key: HIVE-6903 URL: https://issues.apache.org/jira/browse/HIVE-6903 Project: Hive Issue Type: Task Components: Metastore Affects Versions: 0.12.0, 0.11.0, 0.10.0, 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6903.patch Since its introduction in HIVE-2616 I havent seen any bug reported for it, only grief from users who expect system to work as if this is true by default. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6903) Change default value of hive.metastore.execute.setugi to true
[ https://issues.apache.org/jira/browse/HIVE-6903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6903: --- Attachment: HIVE-6903.patch Change default value of hive.metastore.execute.setugi to true - Key: HIVE-6903 URL: https://issues.apache.org/jira/browse/HIVE-6903 Project: Hive Issue Type: Task Components: Metastore Affects Versions: 0.10.0, 0.11.0, 0.12.0, 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6903.patch Since its introduction in HIVE-2616 I havent seen any bug reported for it, only grief from users who expect system to work as if this is true by default. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6745) HCat MultiOutputFormat hardcodes DistributedCache keynames
[ https://issues.apache.org/jira/browse/HIVE-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6745: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Sushanth! HCat MultiOutputFormat hardcodes DistributedCache keynames -- Key: HIVE-6745 URL: https://issues.apache.org/jira/browse/HIVE-6745 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.13.0, 0.14.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Fix For: 0.14.0 Attachments: HIVE-6745.patch There's a bug in how MultiOutputFormat deals with DistributedCache, in that it hardcodes the parameter name to merge for distributed cache entries in the jobconf. This parameter name has changed with recent builds of 2.x, thus causing a test failure. These parameters need to be properly shimmed out. -- This message was sent by Atlassian JIRA (v6.2#6252)
[ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan
The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth Sowmyan committers on the Apache Hive Project. Please join me in congratulating Alan, Daniel, and Sushanth! - Carl
Re: [ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan
Congrats Alan, Daniel and Sushanth! On Mon, Apr 14, 2014 at 7:51 PM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth Sowmyan committers on the Apache Hive Project. Please join me in congratulating Alan, Daniel, and Sushanth! - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan
Congratulations everyone!! Thanks Prasanth Jayachandran On Apr 14, 2014, at 10:51 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth Sowmyan committers on the Apache Hive Project. Please join me in congratulating Alan, Daniel, and Sushanth! - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Updated] (HIVE-6890) Bug in HiveStreaming API causes problems if hive-site.xml is missing on streaming client side
[ https://issues.apache.org/jira/browse/HIVE-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-6890: - Resolution: Fixed Status: Resolved (was: Patch Available) Patch committed. Thanks Roshan. Bug in HiveStreaming API causes problems if hive-site.xml is missing on streaming client side - Key: HIVE-6890 URL: https://issues.apache.org/jira/browse/HIVE-6890 Project: Hive Issue Type: Bug Reporter: Roshan Naik Assignee: Roshan Naik Attachments: HIVE-6890.patch Incorrect conf object being passed to MetaStore client in AbstractRecordWriter is causing the issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan
Congratulations to all! Thanks, --Vaibhav On Mon, Apr 14, 2014 at 10:55 AM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congratulations everyone!! Thanks Prasanth Jayachandran On Apr 14, 2014, at 10:51 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth Sowmyan committers on the Apache Hive Project. Please join me in congratulating Alan, Daniel, and Sushanth! - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan
Congrats! On Mon, Apr 14, 2014 at 10:55 AM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congratulations everyone!! Thanks Prasanth Jayachandran On Apr 14, 2014, at 10:51 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth Sowmyan committers on the Apache Hive Project. Please join me in congratulating Alan, Daniel, and Sushanth! - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Updated] (HIVE-6890) Bug in HiveStreaming API causes problems if hive-site.xml is missing on streaming client side
[ https://issues.apache.org/jira/browse/HIVE-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-6890: - Fix Version/s: 0.14.0 Bug in HiveStreaming API causes problems if hive-site.xml is missing on streaming client side - Key: HIVE-6890 URL: https://issues.apache.org/jira/browse/HIVE-6890 Project: Hive Issue Type: Bug Reporter: Roshan Naik Assignee: Roshan Naik Fix For: 0.14.0 Attachments: HIVE-6890.patch Incorrect conf object being passed to MetaStore client in AbstractRecordWriter is causing the issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5072) [WebHCat]Enable directly invoke Sqoop job through Templeton
[ https://issues.apache.org/jira/browse/HIVE-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968598#comment-13968598 ] Eugene Koifman commented on HIVE-5072: -- [~shuainie] Here are a few more thoughts: 0. I was thinking that the options file would originate at the client making the REST call. It would be uploaded to WebHCat via the same REST call and then WebHCat can copy to Dist Cache or handle as appropriate, but your use case also makes sense and as currently implemented, it's on par with how pig/hive jobs are handled. 5. Your implementation supports automatically shipping of Sqoop tar file to the target node where the job will be executed. The tar file doesn't contain JDBC drivers for all DB types so you can't expect it to be in the lib folder on target node. 7. Since this API would mostly be used by Sqoop users, I think they'll find it confusing that items with the same name in Sqoop have different meaning in WebHCat/Sqoop. I would try to either change the names of parameters or make the definition in the 2 systems match. 7.5 Please file Documentation Jira to make sure this API gets documented on the wiki 9. Does your implementation work with Sqoop 1 and Sqoop 2? If not, it should probably assert the right version is installed/tar file is available and produce a meaningful error to client. 10. There is now (in Server.java) version/hadoop, version/hive, version/pig methods. I think it would be useful to add the same for Sqoop. [WebHCat]Enable directly invoke Sqoop job through Templeton --- Key: HIVE-5072 URL: https://issues.apache.org/jira/browse/HIVE-5072 Project: Hive Issue Type: Improvement Components: WebHCat Affects Versions: 0.12.0 Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5072.1.patch, HIVE-5072.2.patch, HIVE-5072.3.patch, Templeton-Sqoop-Action.pdf Now it is hard to invoke a Sqoop job through templeton. The only way is to use the classpath jar generated by a sqoop job and use the jar delegator in Templeton. We should implement Sqoop Delegator to enable directly invoke Sqoop job through Templeton. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan
Congrats! On Mon, Apr 14, 2014 at 11:09 AM, Tao Li litao.bupt...@gmail.com wrote: Congrats！ On Apr 15, 2014 1:55 AM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congratulations everyone!! Thanks Prasanth Jayachandran On Apr 14, 2014, at 10:51 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth Sowmyan committers on the Apache Hive Project. Please join me in congratulating Alan, Daniel, and Sushanth! - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan
Congrats！ On Apr 15, 2014 1:55 AM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congratulations everyone!! Thanks Prasanth Jayachandran On Apr 14, 2014, at 10:51 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth Sowmyan committers on the Apache Hive Project. Please join me in congratulating Alan, Daniel, and Sushanth! - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Apache Hive 0.13.0 Release Candidate 0
I've validated checksums and signatures. They check out. Unfortunately I am not able to build the src tarball. The problem is that it is missing the top level pom. Thanks, Gunther. On Fri, Apr 11, 2014 at 9:19 PM, Harish Butani rhbut...@apache.org wrote: Apache Hive 0.13.0 Release Candidate 0 is available here: http://people.apache.org/~rhbutani/hive-0.13.0-candidate-0 Maven artifacts are available here: https://repository.apache.org/content/repositories/orgapachehive-1008 Source tag for RCN is at: https://svn.apache.org/repos/asf/hive/tags/release-0.13.0-rc0/ Voting will conclude in 72 hours. Hive PMC Members: Please test and vote. Thanks. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan
Congratulations, guys! On Mon, Apr 14, 2014 at 11:14 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats! On Mon, Apr 14, 2014 at 11:09 AM, Tao Li litao.bupt...@gmail.com wrote: Congrats！ On Apr 15, 2014 1:55 AM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congratulations everyone!! Thanks Prasanth Jayachandran On Apr 14, 2014, at 10:51 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth Sowmyan committers on the Apache Hive Project. Please join me in congratulating Alan, Daniel, and Sushanth! - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HIVE-6904) out of sequence error in HS2 when connecting to the metastore server
sukhendu chakraborty created HIVE-6904: -- Summary: out of sequence error in HS2 when connecting to the metastore server Key: HIVE-6904 URL: https://issues.apache.org/jira/browse/HIVE-6904 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Environment: CDH5.0.0 Reporter: sukhendu chakraborty 6:45:14.965 PM ERROR hive.log Got exception: org.apache.thrift.TApplicationException get_tables failed: out of sequence response org.apache.thrift.TApplicationException: get_tables failed: out of sequence response at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:76) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_tables(ThriftHiveMetastore.java:887) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_tables(ThriftHiveMetastore.java:873) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:927) at org.apache.hive.service.cli.operation.GetTablesOperation.run(GetTablesOperation.java:92) at org.apache.hive.service.cli.session.HiveSessionImpl.runOperationWithLogCapture(HiveSessionImpl.java:514) at org.apache.hive.service.cli.session.HiveSessionImpl.getTables(HiveSessionImpl.java:305) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:62) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:554) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:57) at com.sun.proxy.$Proxy12.getTables(Unknown Source) at org.apache.hive.service.cli.CLIService.getTables(CLIService.java:236) at org.apache.hive.service.cli.thrift.ThriftCLIService.GetTables(ThriftCLIService.java:375) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1453) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1438) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:57) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) 6:45:15.074 PM ERROR hive.log Converting exception to MetaException -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6904) out of sequence error in HS2 when connecting to the metastore server
[ https://issues.apache.org/jira/browse/HIVE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968639#comment-13968639 ] sukhendu chakraborty commented on HIVE-6904: Only suggested workaround is to restart HiveServer2 and metastore but it does not fix the issue as the problems starts happening after a few tries. out of sequence error in HS2 when connecting to the metastore server Key: HIVE-6904 URL: https://issues.apache.org/jira/browse/HIVE-6904 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Environment: CDH5.0.0 Reporter: sukhendu chakraborty 6:45:14.965 PMERROR hive.log Got exception: org.apache.thrift.TApplicationException get_tables failed: out of sequence response org.apache.thrift.TApplicationException: get_tables failed: out of sequence response at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:76) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_tables(ThriftHiveMetastore.java:887) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_tables(ThriftHiveMetastore.java:873) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:927) at org.apache.hive.service.cli.operation.GetTablesOperation.run(GetTablesOperation.java:92) at org.apache.hive.service.cli.session.HiveSessionImpl.runOperationWithLogCapture(HiveSessionImpl.java:514) at org.apache.hive.service.cli.session.HiveSessionImpl.getTables(HiveSessionImpl.java:305) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:62) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:554) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:57) at com.sun.proxy.$Proxy12.getTables(Unknown Source) at org.apache.hive.service.cli.CLIService.getTables(CLIService.java:236) at org.apache.hive.service.cli.thrift.ThriftCLIService.GetTables(ThriftCLIService.java:375) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1453) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1438) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:57) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) 6:45:15.074 PMERROR hive.log Converting exception to MetaException -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6904) out of sequence error in HiveMetastore server when connecting from HS2
[ https://issues.apache.org/jira/browse/HIVE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sukhendu chakraborty updated HIVE-6904: --- Summary: out of sequence error in HiveMetastore server when connecting from HS2 (was: out of sequence error in HiveMetastore server when connecting to the metastore server) out of sequence error in HiveMetastore server when connecting from HS2 -- Key: HIVE-6904 URL: https://issues.apache.org/jira/browse/HIVE-6904 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Environment: CDH5.0.0 Reporter: sukhendu chakraborty 6:45:14.965 PMERROR hive.log Got exception: org.apache.thrift.TApplicationException get_tables failed: out of sequence response org.apache.thrift.TApplicationException: get_tables failed: out of sequence response at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:76) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_tables(ThriftHiveMetastore.java:887) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_tables(ThriftHiveMetastore.java:873) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:927) at org.apache.hive.service.cli.operation.GetTablesOperation.run(GetTablesOperation.java:92) at org.apache.hive.service.cli.session.HiveSessionImpl.runOperationWithLogCapture(HiveSessionImpl.java:514) at org.apache.hive.service.cli.session.HiveSessionImpl.getTables(HiveSessionImpl.java:305) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:62) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:554) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:57) at com.sun.proxy.$Proxy12.getTables(Unknown Source) at org.apache.hive.service.cli.CLIService.getTables(CLIService.java:236) at org.apache.hive.service.cli.thrift.ThriftCLIService.GetTables(ThriftCLIService.java:375) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1453) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1438) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:57) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) 6:45:15.074 PMERROR hive.log Converting exception to MetaException -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6904) out of sequence error in HiveMetastore server when connecting to the metastore server
[ https://issues.apache.org/jira/browse/HIVE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sukhendu chakraborty updated HIVE-6904: --- Summary: out of sequence error in HiveMetastore server when connecting to the metastore server (was: out of sequence error in HS2 when connecting to the metastore server) out of sequence error in HiveMetastore server when connecting to the metastore server - Key: HIVE-6904 URL: https://issues.apache.org/jira/browse/HIVE-6904 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Environment: CDH5.0.0 Reporter: sukhendu chakraborty 6:45:14.965 PMERROR hive.log Got exception: org.apache.thrift.TApplicationException get_tables failed: out of sequence response org.apache.thrift.TApplicationException: get_tables failed: out of sequence response at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:76) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_tables(ThriftHiveMetastore.java:887) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_tables(ThriftHiveMetastore.java:873) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:927) at org.apache.hive.service.cli.operation.GetTablesOperation.run(GetTablesOperation.java:92) at org.apache.hive.service.cli.session.HiveSessionImpl.runOperationWithLogCapture(HiveSessionImpl.java:514) at org.apache.hive.service.cli.session.HiveSessionImpl.getTables(HiveSessionImpl.java:305) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:62) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:554) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:57) at com.sun.proxy.$Proxy12.getTables(Unknown Source) at org.apache.hive.service.cli.CLIService.getTables(CLIService.java:236) at org.apache.hive.service.cli.thrift.ThriftCLIService.GetTables(ThriftCLIService.java:375) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1453) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1438) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:57) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) 6:45:15.074 PMERROR hive.log Converting exception to MetaException -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6904) out of sequence error in HiveMetastore server when connecting from HS2
[ https://issues.apache.org/jira/browse/HIVE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968646#comment-13968646 ] sukhendu chakraborty commented on HIVE-6904: Detailed discussion can be found at: https://groups.google.com/a/cloudera.org/forum/#!topic/hue-user/MCfGRfG3lIg out of sequence error in HiveMetastore server when connecting from HS2 -- Key: HIVE-6904 URL: https://issues.apache.org/jira/browse/HIVE-6904 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Environment: CDH5.0.0 Reporter: sukhendu chakraborty 6:45:14.965 PMERROR hive.log Got exception: org.apache.thrift.TApplicationException get_tables failed: out of sequence response org.apache.thrift.TApplicationException: get_tables failed: out of sequence response at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:76) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_tables(ThriftHiveMetastore.java:887) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_tables(ThriftHiveMetastore.java:873) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:927) at org.apache.hive.service.cli.operation.GetTablesOperation.run(GetTablesOperation.java:92) at org.apache.hive.service.cli.session.HiveSessionImpl.runOperationWithLogCapture(HiveSessionImpl.java:514) at org.apache.hive.service.cli.session.HiveSessionImpl.getTables(HiveSessionImpl.java:305) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:62) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:554) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:57) at com.sun.proxy.$Proxy12.getTables(Unknown Source) at org.apache.hive.service.cli.CLIService.getTables(CLIService.java:236) at org.apache.hive.service.cli.thrift.ThriftCLIService.GetTables(ThriftCLIService.java:375) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1453) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1438) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:57) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) 6:45:15.074 PMERROR hive.log Converting exception to MetaException -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6891) Alter rename partition Perm inheritance and general partition/table group inheritance
[ https://issues.apache.org/jira/browse/HIVE-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968656#comment-13968656 ] Szehon Ho commented on HIVE-6891: - [~brocknoland] Thanks for the review, these test failures dont look like related to this patch. I ran and they passed locally. TestDBTokenStore looks like a transient derby error, and the other looks like a transient error with the test itself. Alter rename partition Perm inheritance and general partition/table group inheritance - Key: HIVE-6891 URL: https://issues.apache.org/jira/browse/HIVE-6891 Project: Hive Issue Type: Bug Reporter: Szehon Ho Assignee: Szehon Ho Attachments: HIVE-6891.2.patch, HIVE-6891.3.patch, HIVE-6891.4.patch, HIVE-6891.patch Found this issue while looking at the method mentioned by HIVE-6648. 'alter table .. partition .. rename to ..' and other commands calling Warehouse.mkdirs() doesn't inherit permission on the partition directories and consequently the data, when hive.warehouse.subdir.inherit.perms is set. Also, in these scenarios of directory creation, group is not being inherited. Data files are already inheriting group by HIVE-3756. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6549) remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh
[ https://issues.apache.org/jira/browse/HIVE-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-6549: - Status: Open (was: Patch Available) remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh - Key: HIVE-6549 URL: https://issues.apache.org/jira/browse/HIVE-6549 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Priority: Minor Attachments: HIVE-6549.patch this property is no longer used also removed corresponding AppConfig.TEMPLETON_JAR_NAME hcatalog/bin/hive-config.sh is not used should also remove hive.metastore.local=false from templeton.hive.properties in webhcat-default.xml. hive.metastore.local has not been used since Hive .010. NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6549) remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh
[ https://issues.apache.org/jira/browse/HIVE-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-6549: - Description: this property is no longer used also removed corresponding AppConfig.TEMPLETON_JAR_NAME hcatalog/bin/hive-config.sh is not used should also remove hive.metastore.local=false from templeton.hive.properties in webhcat-default.xml. hive.metastore.local has not been used since Hive .010. was: this property is no longer used also removed corresponding AppConfig.TEMPLETON_JAR_NAME hcatalog/bin/hive-config.sh is not used remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh - Key: HIVE-6549 URL: https://issues.apache.org/jira/browse/HIVE-6549 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Priority: Minor Attachments: HIVE-6549.patch this property is no longer used also removed corresponding AppConfig.TEMPLETON_JAR_NAME hcatalog/bin/hive-config.sh is not used should also remove hive.metastore.local=false from templeton.hive.properties in webhcat-default.xml. hive.metastore.local has not been used since Hive .010. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6549) remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh
[ https://issues.apache.org/jira/browse/HIVE-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-6549: - Description: this property is no longer used also removed corresponding AppConfig.TEMPLETON_JAR_NAME hcatalog/bin/hive-config.sh is not used should also remove hive.metastore.local=false from templeton.hive.properties in webhcat-default.xml. hive.metastore.local has not been used since Hive .010. NO PRECOMMIT TESTS was: this property is no longer used also removed corresponding AppConfig.TEMPLETON_JAR_NAME hcatalog/bin/hive-config.sh is not used should also remove hive.metastore.local=false from templeton.hive.properties in webhcat-default.xml. hive.metastore.local has not been used since Hive .010. remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh - Key: HIVE-6549 URL: https://issues.apache.org/jira/browse/HIVE-6549 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Priority: Minor Attachments: HIVE-6549.patch this property is no longer used also removed corresponding AppConfig.TEMPLETON_JAR_NAME hcatalog/bin/hive-config.sh is not used should also remove hive.metastore.local=false from templeton.hive.properties in webhcat-default.xml. hive.metastore.local has not been used since Hive .010. NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6904) out of sequence error in HiveMetastore server when connecting from HS2
[ https://issues.apache.org/jira/browse/HIVE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968657#comment-13968657 ] Szehon Ho commented on HIVE-6904: - This looks like a duplicate of HIVE-6893, can you confirm? out of sequence error in HiveMetastore server when connecting from HS2 -- Key: HIVE-6904 URL: https://issues.apache.org/jira/browse/HIVE-6904 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Environment: CDH5.0.0 Reporter: sukhendu chakraborty 6:45:14.965 PMERROR hive.log Got exception: org.apache.thrift.TApplicationException get_tables failed: out of sequence response org.apache.thrift.TApplicationException: get_tables failed: out of sequence response at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:76) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_tables(ThriftHiveMetastore.java:887) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_tables(ThriftHiveMetastore.java:873) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:927) at org.apache.hive.service.cli.operation.GetTablesOperation.run(GetTablesOperation.java:92) at org.apache.hive.service.cli.session.HiveSessionImpl.runOperationWithLogCapture(HiveSessionImpl.java:514) at org.apache.hive.service.cli.session.HiveSessionImpl.getTables(HiveSessionImpl.java:305) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:62) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:554) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:57) at com.sun.proxy.$Proxy12.getTables(Unknown Source) at org.apache.hive.service.cli.CLIService.getTables(CLIService.java:236) at org.apache.hive.service.cli.thrift.ThriftCLIService.GetTables(ThriftCLIService.java:375) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1453) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1438) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:57) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) 6:45:15.074 PMERROR hive.log Converting exception to MetaException -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 20096: HIVE-6835: Reading of partitioned Avro data fails if partition schema does not match table schema
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20096/ --- (Updated April 14, 2014, 6:49 p.m.) Review request for hive. Changes --- Addressed Carl's comments. Changes: - Reverted whitespace changes. - Moved the TABLE_PROP_PREFIX (table.) to serdeConstants. - Removed code that mutated the Properties passed to the AvroSerDe - Added/improved comments - Synced with latest Repository: hive-git Description --- The problem occurs when you store the avro.schema.(literal|url) in the SERDEPROPERTIES instead of the TBLPROPERTIES, add a partition, change the table's schema, and then try reading from the old partition. I fixed this problem by passing the table properties to the partition with a table. prefix, and changing the Avro SerDe to always use the table properties when available. Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java 43cef5c ql/src/test/queries/clientpositive/avro_partitioned.q 6fe5117 ql/src/test/results/clientpositive/avro_partitioned.q.out 644716d serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java 515cf25 serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 9d58d13 serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerdeUtils.java 67d5570 Diff: https://reviews.apache.org/r/20096/diff/ Testing --- Added test cases Thanks, Anthony Hsu
Re: [ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan
Congrats Alan, Daniel, Sushanth! On Mon, Apr 14, 2014 at 11:29 AM, Szehon Ho sze...@cloudera.com wrote: Congratulations, guys! On Mon, Apr 14, 2014 at 11:14 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats! On Mon, Apr 14, 2014 at 11:09 AM, Tao Li litao.bupt...@gmail.com wrote: Congrats！ On Apr 15, 2014 1:55 AM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congratulations everyone!! Thanks Prasanth Jayachandran On Apr 14, 2014, at 10:51 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth Sowmyan committers on the Apache Hive Project. Please join me in congratulating Alan, Daniel, and Sushanth! - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan
Thanks, all. :) And congrats to Alan and Daniel too! On Apr 14, 2014 11:52 AM, Thejas Nair the...@hortonworks.com wrote: Congrats Alan, Daniel, Sushanth! On Mon, Apr 14, 2014 at 11:29 AM, Szehon Ho sze...@cloudera.com wrote: Congratulations, guys! On Mon, Apr 14, 2014 at 11:14 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats! On Mon, Apr 14, 2014 at 11:09 AM, Tao Li litao.bupt...@gmail.com wrote: Congrats！ On Apr 15, 2014 1:55 AM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congratulations everyone!! Thanks Prasanth Jayachandran On Apr 14, 2014, at 10:51 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth Sowmyan committers on the Apache Hive Project. Please join me in congratulating Alan, Daniel, and Sushanth! - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-6549) remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh
[ https://issues.apache.org/jira/browse/HIVE-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968682#comment-13968682 ] Eugene Koifman commented on HIVE-6549: -- [~leftylev] If [WebHCat Configuration: Configuration Variables|https://cwiki.apache.org/confluence/display/Hive/WebHCat+Configure#WebHCatConfigure-ConfigurationVariables] is maintained then, hive.metastore.local=false in templeton.hive.properties should be removed. It's be deprecated/removed since Hive 0.10 remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh - Key: HIVE-6549 URL: https://issues.apache.org/jira/browse/HIVE-6549 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Priority: Minor Attachments: HIVE-6549.patch this property is no longer used also removed corresponding AppConfig.TEMPLETON_JAR_NAME hcatalog/bin/hive-config.sh is not used NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6893) out of sequence error in HiveMetastore server
[
https://issues.apache.org/jira/browse/HIVE-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sukhendu chakraborty updated HIVE-6893:
---
Summary: out of sequence error in HiveMetastore server (was: Out of
sequence response)
out of sequence error in HiveMetastore server
-
Key: HIVE-6893
URL: https://issues.apache.org/jira/browse/HIVE-6893
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.12.0
Reporter: Romain Rigaux
Calls listing databases or tables fail. It seems to be a concurrency problem.
{code}
014-03-06 05:34:00,785 ERROR hive.log:
org.apache.thrift.TApplicationException: get_databases failed: out of
sequence response
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:76)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:472)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:459)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:648)
at
org.apache.hive.service.cli.operation.GetSchemasOperation.run(GetSchemasOperation.java:66)
at
org.apache.hive.service.cli.session.HiveSessionImpl.getSchemas(HiveSessionImpl.java:278)
at sun.reflect.GeneratedMethodAccessor323.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:62)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:582)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:57)
at com.sun.proxy.$Proxy9.getSchemas(Unknown Source)
at
org.apache.hive.service.cli.CLIService.getSchemas(CLIService.java:192)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.GetSchemas(ThriftCLIService.java:263)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$GetSchemas.getResult(TCLIService.java:1433)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$GetSchemas.getResult(TCLIService.java:1418)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.cli.thrift.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:38)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6549) remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh
[ https://issues.apache.org/jira/browse/HIVE-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-6549: - Description: this property is no longer used also removed corresponding AppConfig.TEMPLETON_JAR_NAME hcatalog/bin/hive-config.sh is not used NO PRECOMMIT TESTS was: this property is no longer used also removed corresponding AppConfig.TEMPLETON_JAR_NAME hcatalog/bin/hive-config.sh is not used should also remove hive.metastore.local=false from templeton.hive.properties in webhcat-default.xml. hive.metastore.local has not been used since Hive .010. NO PRECOMMIT TESTS remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh - Key: HIVE-6549 URL: https://issues.apache.org/jira/browse/HIVE-6549 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Priority: Minor Attachments: HIVE-6549.patch this property is no longer used also removed corresponding AppConfig.TEMPLETON_JAR_NAME hcatalog/bin/hive-config.sh is not used NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HIVE-6904) out of sequence error in HiveMetastore server when connecting from HS2
[ https://issues.apache.org/jira/browse/HIVE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sukhendu chakraborty resolved HIVE-6904. Resolution: Duplicate Duplicate of HIVE-6893 out of sequence error in HiveMetastore server when connecting from HS2 -- Key: HIVE-6904 URL: https://issues.apache.org/jira/browse/HIVE-6904 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Environment: CDH5.0.0 Reporter: sukhendu chakraborty 6:45:14.965 PMERROR hive.log Got exception: org.apache.thrift.TApplicationException get_tables failed: out of sequence response org.apache.thrift.TApplicationException: get_tables failed: out of sequence response at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:76) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_tables(ThriftHiveMetastore.java:887) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_tables(ThriftHiveMetastore.java:873) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:927) at org.apache.hive.service.cli.operation.GetTablesOperation.run(GetTablesOperation.java:92) at org.apache.hive.service.cli.session.HiveSessionImpl.runOperationWithLogCapture(HiveSessionImpl.java:514) at org.apache.hive.service.cli.session.HiveSessionImpl.getTables(HiveSessionImpl.java:305) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:62) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:554) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:57) at com.sun.proxy.$Proxy12.getTables(Unknown Source) at org.apache.hive.service.cli.CLIService.getTables(CLIService.java:236) at org.apache.hive.service.cli.thrift.ThriftCLIService.GetTables(ThriftCLIService.java:375) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1453) at org.apache.hive.service.cli.thrift.TCLIService$Processor$GetTables.getResult(TCLIService.java:1438) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:57) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) 6:45:15.074 PMERROR hive.log Converting exception to MetaException -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan
Congrats Alan, Daniel and Sushanth! --Xuefu On Mon, Apr 14, 2014 at 12:01 PM, Sushanth Sowmyan khorg...@gmail.comwrote: Thanks, all. :) And congrats to Alan and Daniel too! On Apr 14, 2014 11:52 AM, Thejas Nair the...@hortonworks.com wrote: Congrats Alan, Daniel, Sushanth! On Mon, Apr 14, 2014 at 11:29 AM, Szehon Ho sze...@cloudera.com wrote: Congratulations, guys! On Mon, Apr 14, 2014 at 11:14 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats! On Mon, Apr 14, 2014 at 11:09 AM, Tao Li litao.bupt...@gmail.com wrote: Congrats！ On Apr 15, 2014 1:55 AM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congratulations everyone!! Thanks Prasanth Jayachandran On Apr 14, 2014, at 10:51 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth Sowmyan committers on the Apache Hive Project. Please join me in congratulating Alan, Daniel, and Sushanth! - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-5317) Implement insert, update, and delete in Hive with full ACID support
[ https://issues.apache.org/jira/browse/HIVE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968696#comment-13968696 ] Pardeep Kumar commented on HIVE-5317: - Vinod.. it is very much obvious.. ACID, updates and Deletes are one of the most awaited features of Hive and many people like me are waiting for the same.. Implement insert, update, and delete in Hive with full ACID support --- Key: HIVE-5317 URL: https://issues.apache.org/jira/browse/HIVE-5317 Project: Hive Issue Type: New Feature Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: InsertUpdatesinHive.pdf Many customers want to be able to insert, update and delete rows from Hive tables with full ACID support. The use cases are varied, but the form of the queries that should be supported are: * INSERT INTO tbl SELECT … * INSERT INTO tbl VALUES ... * UPDATE tbl SET … WHERE … * DELETE FROM tbl WHERE … * MERGE INTO tbl USING src ON … WHEN MATCHED THEN ... WHEN NOT MATCHED THEN ... * SET TRANSACTION LEVEL … * BEGIN/END TRANSACTION Use Cases * Once an hour, a set of inserts and updates (up to 500k rows) for various dimension tables (eg. customer, inventory, stores) needs to be processed. The dimension tables have primary keys and are typically bucketed and sorted on those keys. * Once a day a small set (up to 100k rows) of records need to be deleted for regulatory compliance. * Once an hour a log of transactions is exported from a RDBS and the fact tables need to be updated (up to 1m rows) to reflect the new data. The transactions are a combination of inserts, updates, and deletes. The table is partitioned and bucketed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5317) Implement insert, update, and delete in Hive with full ACID support
[ https://issues.apache.org/jira/browse/HIVE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968698#comment-13968698 ] Pardeep Kumar commented on HIVE-5317: - Will these features be supported on all Hive file formats i.e. Sequencefile, Text, ORC, RC etc. Implement insert, update, and delete in Hive with full ACID support --- Key: HIVE-5317 URL: https://issues.apache.org/jira/browse/HIVE-5317 Project: Hive Issue Type: New Feature Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: InsertUpdatesinHive.pdf Many customers want to be able to insert, update and delete rows from Hive tables with full ACID support. The use cases are varied, but the form of the queries that should be supported are: * INSERT INTO tbl SELECT … * INSERT INTO tbl VALUES ... * UPDATE tbl SET … WHERE … * DELETE FROM tbl WHERE … * MERGE INTO tbl USING src ON … WHEN MATCHED THEN ... WHEN NOT MATCHED THEN ... * SET TRANSACTION LEVEL … * BEGIN/END TRANSACTION Use Cases * Once an hour, a set of inserts and updates (up to 500k rows) for various dimension tables (eg. customer, inventory, stores) needs to be processed. The dimension tables have primary keys and are typically bucketed and sorted on those keys. * Once a day a small set (up to 100k rows) of records need to be deleted for regulatory compliance. * Once an hour a log of transactions is exported from a RDBS and the fact tables need to be updated (up to 1m rows) to reflect the new data. The transactions are a combination of inserts, updates, and deletes. The table is partitioned and bucketed. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [VOTE] Apache Hive 0.13.0 Release Candidate 0
Thanks Gunther for finding this. I run: mvn install -Pdist -DskipTests -Phadoop-1 -Dmaven.javadoc.skip=true -DcreateChecksum=true to generate the src and bin tars. Looking into the issue. Any mvn experts, please provide suggestions. regards, Harish. On Apr 14, 2014, at 11:20 AM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: I've validated checksums and signatures. They check out. Unfortunately I am not able to build the src tarball. The problem is that it is missing the top level pom. Thanks, Gunther. On Fri, Apr 11, 2014 at 9:19 PM, Harish Butani rhbut...@apache.org wrote: Apache Hive 0.13.0 Release Candidate 0 is available here: http://people.apache.org/~rhbutani/hive-0.13.0-candidate-0 Maven artifacts are available here: https://repository.apache.org/content/repositories/orgapachehive-1008 Source tag for RCN is at: https://svn.apache.org/repos/asf/hive/tags/release-0.13.0-rc0/ Voting will conclude in 72 hours. Hive PMC Members: Please test and vote. Thanks. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HIVE-6905) Implement Auto increment, primary-foreign Key, not null constraints and default value in Hive Table columns
Pardeep Kumar created HIVE-6905: --- Summary: Implement Auto increment, primary-foreign Key, not null constraints and default value in Hive Table columns Key: HIVE-6905 URL: https://issues.apache.org/jira/browse/HIVE-6905 Project: Hive Issue Type: New Feature Reporter: Pardeep Kumar -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6549) remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh
[ https://issues.apache.org/jira/browse/HIVE-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-6549: - Status: Patch Available (was: Open) remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh - Key: HIVE-6549 URL: https://issues.apache.org/jira/browse/HIVE-6549 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Priority: Minor Attachments: HIVE-6549.patch this property is no longer used also removed corresponding AppConfig.TEMPLETON_JAR_NAME hcatalog/bin/hive-config.sh is not used NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6835) Reading of partitioned Avro data fails if partition schema does not match table schema
[
https://issues.apache.org/jira/browse/HIVE-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anthony Hsu updated HIVE-6835:
--
Attachment: HIVE-6835.2.patch
Reading of partitioned Avro data fails if partition schema does not match
table schema
--
Key: HIVE-6835
URL: https://issues.apache.org/jira/browse/HIVE-6835
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Anthony Hsu
Assignee: Anthony Hsu
Attachments: HIVE-6835.1.patch, HIVE-6835.2.patch
To reproduce:
{code}
create table testarray (a arraystring);
load data local inpath '/home/ahsu/test/array.txt' into table testarray;
# create partitioned Avro table with one array column
create table avroarray partitioned by (y string) row format serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with serdeproperties
('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ { name:a, type:{type:array,items:string}
} ] }') STORED as INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat';
insert into table avroarray partition(y=1) select * from testarray;
# add an int column with a default value of 0
alter table avroarray set serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with
serdeproperties('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ {name:intfield,type:int,default:0},{
name:a, type:{type:array,items:string} } ] }');
# fails with ClassCastException
select * from avroarray;
{code}
The select * fails with:
{code}
Failed with exception java.io.IOException:java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: [VOTE] Apache Hive 0.13.0 Release Candidate 0
I think it's in: packaging/src/main/assembly/src.xml Thanks, Gunther. On Mon, Apr 14, 2014 at 9:16 PM, Harish Butani hbut...@hortonworks.comwrote: Thanks Gunther for finding this. I run: mvn install -Pdist -DskipTests -Phadoop-1 -Dmaven.javadoc.skip=true -DcreateChecksum=true to generate the src and bin tars. Looking into the issue. Any mvn experts, please provide suggestions. regards, Harish. On Apr 14, 2014, at 11:20 AM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: I've validated checksums and signatures. They check out. Unfortunately I am not able to build the src tarball. The problem is that it is missing the top level pom. Thanks, Gunther. On Fri, Apr 11, 2014 at 9:19 PM, Harish Butani rhbut...@apache.org wrote: Apache Hive 0.13.0 Release Candidate 0 is available here: http://people.apache.org/~rhbutani/hive-0.13.0-candidate-0 Maven artifacts are available here: https://repository.apache.org/content/repositories/orgapachehive-1008 Source tag for RCN is at: https://svn.apache.org/repos/asf/hive/tags/release-0.13.0-rc0/ Voting will conclude in 72 hours. Hive PMC Members: Please test and vote. Thanks. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Updated] (HIVE-6835) Reading of partitioned Avro data fails if partition schema does not match table schema
[
https://issues.apache.org/jira/browse/HIVE-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anthony Hsu updated HIVE-6835:
--
Status: Patch Available (was: Open)
Thanks for the very thorough code review, [~cwsteinbach]. I've uploaded a new
patch that addresses your comments and also updated the Review Board request.
Reading of partitioned Avro data fails if partition schema does not match
table schema
--
Key: HIVE-6835
URL: https://issues.apache.org/jira/browse/HIVE-6835
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Anthony Hsu
Assignee: Anthony Hsu
Attachments: HIVE-6835.1.patch, HIVE-6835.2.patch
To reproduce:
{code}
create table testarray (a arraystring);
load data local inpath '/home/ahsu/test/array.txt' into table testarray;
# create partitioned Avro table with one array column
create table avroarray partitioned by (y string) row format serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with serdeproperties
('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ { name:a, type:{type:array,items:string}
} ] }') STORED as INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat';
insert into table avroarray partition(y=1) select * from testarray;
# add an int column with a default value of 0
alter table avroarray set serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with
serdeproperties('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ {name:intfield,type:int,default:0},{
name:a, type:{type:array,items:string} } ] }');
# fails with ClassCastException
select * from avroarray;
{code}
The select * fails with:
{code}
Failed with exception java.io.IOException:java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6905) Implement Auto increment, primary-foreign Key, not null constraints and default value in Hive Table columns
[ https://issues.apache.org/jira/browse/HIVE-6905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pardeep Kumar updated HIVE-6905: Description: For Hive to replace a modern datawarehouse based on RDBMS, it must have support for keys, constraints, auto-increment values, surrogate keys and not null features etc. Many customers do not move their EDW to Hive due to these reasons as these have been challenging to maintain in Hive. This must be implemented once https://issues.apache.org/jira/browse/HIVE-5317 for Updates, Deletes and Inserts are done in Hive. This should be next stop for Hive enhancement to take it closer to a very wide mainstream adoption.. Implement Auto increment, primary-foreign Key, not null constraints and default value in Hive Table columns Key: HIVE-6905 URL: https://issues.apache.org/jira/browse/HIVE-6905 Project: Hive Issue Type: New Feature Reporter: Pardeep Kumar For Hive to replace a modern datawarehouse based on RDBMS, it must have support for keys, constraints, auto-increment values, surrogate keys and not null features etc. Many customers do not move their EDW to Hive due to these reasons as these have been challenging to maintain in Hive. This must be implemented once https://issues.apache.org/jira/browse/HIVE-5317 for Updates, Deletes and Inserts are done in Hive. This should be next stop for Hive enhancement to take it closer to a very wide mainstream adoption.. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [VOTE] Apache Hive 0.13.0 Release Candidate 0
Yes, thanks again. Looking into changing it so that the tar ball contains all the files. regards, Harish. On Apr 14, 2014, at 12:25 PM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: I think it's in: packaging/src/main/assembly/src.xml Thanks, Gunther. On Mon, Apr 14, 2014 at 9:16 PM, Harish Butani hbut...@hortonworks.comwrote: Thanks Gunther for finding this. I run: mvn install -Pdist -DskipTests -Phadoop-1 -Dmaven.javadoc.skip=true -DcreateChecksum=true to generate the src and bin tars. Looking into the issue. Any mvn experts, please provide suggestions. regards, Harish. On Apr 14, 2014, at 11:20 AM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: I've validated checksums and signatures. They check out. Unfortunately I am not able to build the src tarball. The problem is that it is missing the top level pom. Thanks, Gunther. On Fri, Apr 11, 2014 at 9:19 PM, Harish Butani rhbut...@apache.org wrote: Apache Hive 0.13.0 Release Candidate 0 is available here: http://people.apache.org/~rhbutani/hive-0.13.0-candidate-0 Maven artifacts are available here: https://repository.apache.org/content/repositories/orgapachehive-1008 Source tag for RCN is at: https://svn.apache.org/repos/asf/hive/tags/release-0.13.0-rc0/ Voting will conclude in 72 hours. Hive PMC Members: Please test and vote. Thanks. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HIVE-6906) Fix assembly/src.xml so that sr tar ball contains top level pom.xml
Harish Butani created HIVE-6906: --- Summary: Fix assembly/src.xml so that sr tar ball contains top level pom.xml Key: HIVE-6906 URL: https://issues.apache.org/jira/browse/HIVE-6906 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani This affects the tarballs we package with Hive 0.13. Current src tarballs being generated are not buildable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6906) Fix assembly/src.xml so that sr tar ball contains top level pom.xml
[ https://issues.apache.org/jira/browse/HIVE-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-6906: Fix Version/s: 0.13.0 Fix assembly/src.xml so that sr tar ball contains top level pom.xml --- Key: HIVE-6906 URL: https://issues.apache.org/jira/browse/HIVE-6906 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Fix For: 0.13.0 This affects the tarballs we package with Hive 0.13. Current src tarballs being generated are not buildable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6906) Fix assembly/src.xml so that sr tar ball contains top level pom.xml
[ https://issues.apache.org/jira/browse/HIVE-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-6906: Attachment: HIVE-6906.1.patch Fix assembly/src.xml so that sr tar ball contains top level pom.xml --- Key: HIVE-6906 URL: https://issues.apache.org/jira/browse/HIVE-6906 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Fix For: 0.13.0 Attachments: HIVE-6906.1.patch This affects the tarballs we package with Hive 0.13. Current src tarballs being generated are not buildable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6868) Create table in HCatalog sets different SerDe defaults than what is set through the CLI
[ https://issues.apache.org/jira/browse/HIVE-6868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6868: --- Assignee: Harish Butani Create table in HCatalog sets different SerDe defaults than what is set through the CLI --- Key: HIVE-6868 URL: https://issues.apache.org/jira/browse/HIVE-6868 Project: Hive Issue Type: Bug Components: HCatalog Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-6868.1.patch, HIVE-6868.2.patch, HIVE-6868.3.patch HCatCreateTableDesc doesn't invoke the getEmptyTable function on org.apache.hadoop.hive.ql.metadata.Table -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5317) Implement insert, update, and delete in Hive with full ACID support
[ https://issues.apache.org/jira/browse/HIVE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968789#comment-13968789 ] Alan Gates commented on HIVE-5317: -- Currently they are being supported in ORC. It is done in such a way that it could be extended to any file format that can support a row id, though there is some code to write to make it happen. It could be extended to support text or sequence file by using offset in the base file as the surrogate for rowid. I'm not sure if this would work for RC file or not. Implement insert, update, and delete in Hive with full ACID support --- Key: HIVE-5317 URL: https://issues.apache.org/jira/browse/HIVE-5317 Project: Hive Issue Type: New Feature Reporter: Owen O'Malley Assignee: Owen O'Malley Attachments: InsertUpdatesinHive.pdf Many customers want to be able to insert, update and delete rows from Hive tables with full ACID support. The use cases are varied, but the form of the queries that should be supported are: * INSERT INTO tbl SELECT … * INSERT INTO tbl VALUES ... * UPDATE tbl SET … WHERE … * DELETE FROM tbl WHERE … * MERGE INTO tbl USING src ON … WHEN MATCHED THEN ... WHEN NOT MATCHED THEN ... * SET TRANSACTION LEVEL … * BEGIN/END TRANSACTION Use Cases * Once an hour, a set of inserts and updates (up to 500k rows) for various dimension tables (eg. customer, inventory, stores) needs to be processed. The dimension tables have primary keys and are typically bucketed and sorted on those keys. * Once a day a small set (up to 100k rows) of records need to be deleted for regulatory compliance. * Once an hour a log of transactions is exported from a RDBS and the fact tables need to be updated (up to 1m rows) to reflect the new data. The transactions are a combination of inserts, updates, and deletes. The table is partitioned and bucketed. -- This message was sent by Atlassian JIRA (v6.2#6252)
bigtop01 not taking new pre-commit jobs
Hi, New precommit builds haven't been submitted successfully on bigtop01 since yesterday morning. http://bigtop01.cloudera.org:8080/view/Hive/job/PreCommit-HIVE-Build/ The machine might be out of space again, or other issue. I mailed the Bigtop dev list, hopefully they can respond soon. Until then, new patches submitted to the Hive JIRA wont get picked up for testing. I'll notify if there are any updates. Thanks, Szehon
[jira] [Updated] (HIVE-4801) output deprecation warning for hive.mapred.map.tasks.speculative.execution
[ https://issues.apache.org/jira/browse/HIVE-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-4801: --- Status: Open (was: Patch Available) This one slipped through cracks. [~stayhf] Would you like to rebase it, so that it runs through Hive QA. Also, I think instead of iterating all keys of conf in handleDeprecation() method (which could be 100s), its better just to look for couple of those which we are deprecated in conf and see if they are set. output deprecation warning for hive.mapred.map.tasks.speculative.execution -- Key: HIVE-4801 URL: https://issues.apache.org/jira/browse/HIVE-4801 Project: Hive Issue Type: Bug Components: Configuration Affects Versions: 0.10.0 Reporter: Chu Tong Assignee: Chu Tong Attachments: HIVE-4801.patch, HIVE-4801.patch Output deprecation warning for hive.mapred.map.tasks.speculative.execution, use are encouraged to use mapred.reduce.tasks.speculative.execution -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6907) HS2 - wrong user gets used for metastore operation with embedded metastore
Thejas M Nair created HIVE-6907:
---
Summary: HS2 - wrong user gets used for metastore operation with
embedded metastore
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6907) HS2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-6907:
Attachment: HIVE-6907.1.patch
HIVE-6907.1.patch - Doing an explicit doAs() asynchronous thread with the user
in the thrift thread solves the problem.
HS2 - wrong user gets used for metastore operation with embedded metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6907) HS2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-6907:
Attachment: HIVE-6907.2.patch
HIVE-6907.2.patch - minor error message improvement
HS2 - wrong user gets used for metastore operation with embedded metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: [ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan
Congratulations [ Sushanth | Daniel | Alan ] -- well deserved! -- Lefty On Mon, Apr 14, 2014 at 3:06 PM, Xuefu Zhang xzh...@cloudera.com wrote: Congrats Alan, Daniel and Sushanth! --Xuefu On Mon, Apr 14, 2014 at 12:01 PM, Sushanth Sowmyan khorg...@gmail.com wrote: Thanks, all. :) And congrats to Alan and Daniel too! On Apr 14, 2014 11:52 AM, Thejas Nair the...@hortonworks.com wrote: Congrats Alan, Daniel, Sushanth! On Mon, Apr 14, 2014 at 11:29 AM, Szehon Ho sze...@cloudera.com wrote: Congratulations, guys! On Mon, Apr 14, 2014 at 11:14 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats! On Mon, Apr 14, 2014 at 11:09 AM, Tao Li litao.bupt...@gmail.com wrote: Congrats！ On Apr 15, 2014 1:55 AM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congratulations everyone!! Thanks Prasanth Jayachandran On Apr 14, 2014, at 10:51 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth Sowmyan committers on the Apache Hive Project. Please join me in congratulating Alan, Daniel, and Sushanth! - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Updated] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-6907:
Summary: HiveServer2 - wrong user gets used for metastore operation with
embedded metastore (was: HS2 - wrong user gets used for metastore operation
with embedded metastore)
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968890#comment-13968890
]
Ashutosh Chauhan commented on HIVE-6907:
Dupe of HIVE-6245 ?
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Review Request 20340: HIVE-6907 - HiveServer2 - wrong user gets used for metastore operation with embedded metastore
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20340/
---
Review request for hive.
Bugs: HIVE-6907
https://issues.apache.org/jira/browse/HIVE-6907
Repository: hive-git
Description
---
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
Diffs
-
service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
a09590a
Diff: https://reviews.apache.org/r/20340/diff/
Testing
---
Testing done manually with several concurrent queries that also perform
metastore operations, each thread uses different users.
Thanks,
Thejas Nair
[jira] [Resolved] (HIVE-2063) jdbc return only 1 collumn
[ https://issues.apache.org/jira/browse/HIVE-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-2063. Resolution: Cannot Reproduce Feel free to reopen if you can still repro this one. jdbc return only 1 collumn -- Key: HIVE-2063 URL: https://issues.apache.org/jira/browse/HIVE-2063 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.7.0 Reporter: Alexey Diomin Assignee: Alexey Diomin Priority: Critical Attachments: HIVE-2063.patch, HIVE-2063.patch we not set separator for data and all data return in first columns and all other fields set NULL addition we get WARNING: Missing fields! Expected 27 fields but only got 1! Ignoring similar problems. it's regresion after HIVE-1378 bug: use delimiter '\t' for fields on server side use default delimiter with code '1' on client side -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6245) HS2 creates DBs/Tables with wrong ownership when HMS setugi is true
[ https://issues.apache.org/jira/browse/HIVE-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968907#comment-13968907 ] Thejas M Nair commented on HIVE-6245: - This issue should be fixed through the changes in HIVE-6312 (which changed non-kerberos codepath to be same as kerberos code path in this aspect), and HIVE-6864. HS2 creates DBs/Tables with wrong ownership when HMS setugi is true --- Key: HIVE-6245 URL: https://issues.apache.org/jira/browse/HIVE-6245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Reporter: Chaoyu Tang Assignee: Chaoyu Tang Attachments: HIVE-6245.patch The case with following settings is valid but does not work correctly in current HS2: == hive.server2.authentication=NONE (or LDAP) hive.server2.enable.doAs= true hive.metastore.sasl.enabled=false hive.metastore.execute.setugi=true == Ideally, HS2 is able to impersonate the logged in user (from Beeline, or JDBC application) and create DBs/Tables with user's ownership. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968911#comment-13968911
]
Thejas M Nair commented on HIVE-6907:
-
bq. Dupe of HIVE-6245 ?
HIVE-6245 was talking about issue when remote metastore is used. There have
been changes that should fix the issue for remote metastore as I commented
there just now.
This fixes the case when embedded metastore is used in hive server2.
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968918#comment-13968918
]
Thejas M Nair commented on HIVE-6907:
-
cc [~vaibhavgumashta] [~prasadm] [~jnp] [~ashutoshc] [~xuefuz] [~rhbutani]
Since we are going to need another 0.13 rc because pom.xml was not included, I
would like to get this in into the new rc. This is a serious concurrency issue,
I think we should include in 0.13 .
Also, I am hoping we can avoid the 24 hr wait before commit for this one,
otherwise we will end up delaying the RC further.
Can one or more people review it ? (Or at least approve including it without 24
hr commit wait period).
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6245) HS2 creates DBs/Tables with wrong ownership when HMS setugi is true
[ https://issues.apache.org/jira/browse/HIVE-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968923#comment-13968923 ] Thejas M Nair commented on HIVE-6245: - [~ctang.ma] Apologies for not responding to your comments and patch sooner! HS2 creates DBs/Tables with wrong ownership when HMS setugi is true --- Key: HIVE-6245 URL: https://issues.apache.org/jira/browse/HIVE-6245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Reporter: Chaoyu Tang Assignee: Chaoyu Tang Attachments: HIVE-6245.patch The case with following settings is valid but does not work correctly in current HS2: == hive.server2.authentication=NONE (or LDAP) hive.server2.enable.doAs= true hive.metastore.sasl.enabled=false hive.metastore.execute.setugi=true == Ideally, HS2 is able to impersonate the logged in user (from Beeline, or JDBC application) and create DBs/Tables with user's ownership. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-2817) Drop any table even without privilege
[
https://issues.apache.org/jira/browse/HIVE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-2817:
---
Resolution: Fixed
Fix Version/s: 0.13.0
Status: Resolved (was: Patch Available)
Fixed via HIVE-2818 Feel free to reopen if you can still repro.
Drop any table even without privilege
-
Key: HIVE-2817
URL: https://issues.apache.org/jira/browse/HIVE-2817
Project: Hive
Issue Type: Bug
Affects Versions: 0.7.1, 0.8.0, 0.9.0, 0.10.0
Reporter: Benyi Wang
Assignee: Chun Chen
Fix For: 0.13.0
Attachments: HIVE-2817.D10371.1.patch, HIVE-2817.D10563.1.patch
You can drop any table if you use fully qualified name 'database.table' even
you don't have any previlige.
{code}
hive set hive.security.authorization.enabled=true;
hive revoke all on default from user test_user;
hive drop table abc;
hive drop table abc;
Authorization failed:No privilege 'Drop' found for outputs {
database:default, table:abc}. Use show grant to get more details.
hive drop table default.abc;
OK
Time taken: 0.13 seconds
{code}
The table and the file in {{/usr/hive/warehouse}} or external file will be
deleted. If you don't have hadoop access permission on
{{/usr/hive/warehouse}} or external files, you will see a hadoop access error
{code}
12/02/23 15:35:35 ERROR hive.log:
org.apache.hadoop.security.AccessControlException:
org.apache.hadoop.security.AccessControlException: Permission denied:
user=test_user, access=WRITE, inode=/user/myetl:myetl:etl:drwxr-xr-x
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6862) add DB schema DDL and upgrade 12to13 scripts for MS SQL Server
[ https://issues.apache.org/jira/browse/HIVE-6862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-6862: - Status: Open (was: Patch Available) add DB schema DDL and upgrade 12to13 scripts for MS SQL Server -- Key: HIVE-6862 URL: https://issues.apache.org/jira/browse/HIVE-6862 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.13.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-6862.patch need to add a unifed 0.13 script and a separate script for ACID support NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968934#comment-13968934
]
Vaibhav Gumashta commented on HIVE-6907:
+1 (non-binding).
I've tested the fix with following 7 combinations:
*Binary mode*
1. Secure MR: 291 passed in 2061.69 seconds
2. Unsecure MR: 334 passed in 2011.29 seconds
3. Secure Tez: 2 failed, 378 passed in 2016.36 seconds (All create index
failures - known issue)
4. Unsecure Tez: 4 failed, 435 passed in 1997.65 seconds (All create index
failures - known issue)
*Http mode*
5. Secure MR: 287 passed in 2161.14 seconds
6. Secure Tez: 1 failed, 369 passed in 2142.57 seconds(All create index
failures - known issue)
7. Unsecure Tez: 3 failed, 429 passed in 2137.11 seconds (All create index
failures - known issue)
My setup: Ran tests for 30 mins, with 20 test threads (firing different queries
in parallel; [~deepesh], [~taksaito] thanks for the test code) and HS2 running
only 10 async threads.
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968936#comment-13968936
]
Vaibhav Gumashta commented on HIVE-6907:
[~thejas] Thanks for fixing the issue.
Here is the root cause: ideally when a foreground thread creates a background
thread, the thread's Access Control Context is inherited
(http://docs.oracle.com/javase/7/docs/technotes/guides/security/spec/security-spec.doc4.html).
However, in our case, we're using a thread pool; so threads are not
necessarily created by the foreground thread. Therefore, an explicit doAs is
necessary when switching to background thread.
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968945#comment-13968945
]
Jitendra Nath Pandey commented on HIVE-6907:
+1. looks good to me.
A minor comment.
bq. runInternal(getConfigForOperation());
If opConfig is final, the call to get the config may not be needed.
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6891) Alter rename partition Perm inheritance and general partition/table group inheritance
[ https://issues.apache.org/jira/browse/HIVE-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968946#comment-13968946 ] Brock Noland commented on HIVE-6891: +1 Alter rename partition Perm inheritance and general partition/table group inheritance - Key: HIVE-6891 URL: https://issues.apache.org/jira/browse/HIVE-6891 Project: Hive Issue Type: Bug Reporter: Szehon Ho Assignee: Szehon Ho Attachments: HIVE-6891.2.patch, HIVE-6891.3.patch, HIVE-6891.4.patch, HIVE-6891.patch Found this issue while looking at the method mentioned by HIVE-6648. 'alter table .. partition .. rename to ..' and other commands calling Warehouse.mkdirs() doesn't inherit permission on the partition directories and consequently the data, when hive.warehouse.subdir.inherit.perms is set. Also, in these scenarios of directory creation, group is not being inherited. Data files are already inheriting group by HIVE-3756. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 20340: HIVE-6907 - HiveServer2 - wrong user gets used for metastore operation with embedded metastore
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20340/
---
(Updated April 14, 2014, 10:36 p.m.)
Review request for hive.
Changes
---
HIVE-6907.3.patch
Bugs: HIVE-6907
https://issues.apache.org/jira/browse/HIVE-6907
Repository: hive-git
Description
---
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
Diffs (updated)
-
service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
a09590a
Diff: https://reviews.apache.org/r/20340/diff/
Testing
---
Testing done manually with several concurrent queries that also perform
metastore operations, each thread uses different users.
Thanks,
Thejas Nair
[jira] [Updated] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-6907:
Attachment: HIVE-6907.3.patch
HIVE-6907.3.patch - addressing review comment
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch, HIVE-6907.3.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968954#comment-13968954
]
Ashutosh Chauhan commented on HIVE-6907:
+1 Root cause identified by [~vgumashta] seems like plausible explanation.
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch, HIVE-6907.3.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968955#comment-13968955
]
Harish Butani commented on HIVE-6907:
-
+1 for 0.13
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch, HIVE-6907.3.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6905) Implement Auto increment, primary-foreign Key, not null constraints and default value in Hive Table columns
[
https://issues.apache.org/jira/browse/HIVE-6905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968986#comment-13968986
]
Damien Carol commented on HIVE-6905:
The sequence based approach to generate auto increment IDs is a must have
feature (part of [SQL:2003|http://www.wiscorp.com/SQL2003Features.pdf]).
{code}
CREATE SEQUENCE sequence name [AS data type]
[START WITH num] [INCREMENT BY num]
[MINVALUE num] [MAXVALUE num]
[CYCLE | NO CYCLE]
{code}
Implement Auto increment, primary-foreign Key, not null constraints and
default value in Hive Table columns
Key: HIVE-6905
URL: https://issues.apache.org/jira/browse/HIVE-6905
Project: Hive
Issue Type: New Feature
Reporter: Pardeep Kumar
For Hive to replace a modern datawarehouse based on RDBMS, it must have
support for keys, constraints, auto-increment values, surrogate keys and not
null features etc. Many customers do not move their EDW to Hive due to these
reasons as these have been challenging to maintain in Hive.
This must be implemented once https://issues.apache.org/jira/browse/HIVE-5317
for Updates, Deletes and Inserts are done in Hive. This should be next stop
for Hive enhancement to take it closer to a very wide mainstream adoption..
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-6908) TestThriftBinaryCLIService.testExecuteStatementAsync has intermittent failures
Szehon Ho created HIVE-6908: --- Summary: TestThriftBinaryCLIService.testExecuteStatementAsync has intermittent failures Key: HIVE-6908 URL: https://issues.apache.org/jira/browse/HIVE-6908 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.13.0 Reporter: Szehon Ho Assignee: Szehon Ho This has failed sometimes in the pre-commit tests. ThriftCLIServiceTest.testExecuteStatementAsync runs two statements. They are given 100 second timeout total, not sure if its by intention. As the first is a select query, it will take a majority of the time. The second statement (create table) should be quicker, but it fails sometimes because timeout is already mostly used up. The timeout should probably be reset after the first statement. If the operation finishes before the timeout, it wont have any effect as it'll break out. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6908) TestThriftBinaryCLIService.testExecuteStatementAsync has intermittent failures
[ https://issues.apache.org/jira/browse/HIVE-6908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-6908: Status: Patch Available (was: Open) TestThriftBinaryCLIService.testExecuteStatementAsync has intermittent failures -- Key: HIVE-6908 URL: https://issues.apache.org/jira/browse/HIVE-6908 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.13.0 Reporter: Szehon Ho Assignee: Szehon Ho Attachments: HIVE-6908.patch This has failed sometimes in the pre-commit tests. ThriftCLIServiceTest.testExecuteStatementAsync runs two statements. They are given 100 second timeout total, not sure if its by intention. As the first is a select query, it will take a majority of the time. The second statement (create table) should be quicker, but it fails sometimes because timeout is already mostly used up. The timeout should probably be reset after the first statement. If the operation finishes before the timeout, it wont have any effect as it'll break out. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6908) TestThriftBinaryCLIService.testExecuteStatementAsync has intermittent failures
[ https://issues.apache.org/jira/browse/HIVE-6908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-6908: Attachment: HIVE-6908.patch Attaching a fix. TestThriftBinaryCLIService.testExecuteStatementAsync has intermittent failures -- Key: HIVE-6908 URL: https://issues.apache.org/jira/browse/HIVE-6908 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.13.0 Reporter: Szehon Ho Assignee: Szehon Ho Attachments: HIVE-6908.patch This has failed sometimes in the pre-commit tests. ThriftCLIServiceTest.testExecuteStatementAsync runs two statements. They are given 100 second timeout total, not sure if its by intention. As the first is a select query, it will take a majority of the time. The second statement (create table) should be quicker, but it fails sometimes because timeout is already mostly used up. The timeout should probably be reset after the first statement. If the operation finishes before the timeout, it wont have any effect as it'll break out. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6906) Fix assembly/src.xml so that sr tar ball contains top level pom.xml
[ https://issues.apache.org/jira/browse/HIVE-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969000#comment-13969000 ] Ashutosh Chauhan commented on HIVE-6906: +1 Fix assembly/src.xml so that sr tar ball contains top level pom.xml --- Key: HIVE-6906 URL: https://issues.apache.org/jira/browse/HIVE-6906 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Fix For: 0.13.0 Attachments: HIVE-6906.1.patch This affects the tarballs we package with Hive 0.13. Current src tarballs being generated are not buildable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6867) Bucketized Table feature fails in some cases
[ https://issues.apache.org/jira/browse/HIVE-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969001#comment-13969001 ] Ashutosh Chauhan commented on HIVE-6867: Dupe of HIVE-3077 / HIVE-3244 ? Bucketized Table feature fails in some cases Key: HIVE-6867 URL: https://issues.apache.org/jira/browse/HIVE-6867 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran Bucketized Table feature fails in some cases. if src destination is bucketed on same key, and if actual data in the src is not bucketed (because data got loaded using LOAD DATA LOCAL INPATH ) then the data won't be bucketed while writing to destination. Example -- CREATE TABLE P1(key STRING, val STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE; LOAD DATA LOCAL INPATH '/Users/jp/apache-hive1/data/files/P1.txt' INTO TABLE P1; – perform an insert to make sure there are 2 files INSERT OVERWRITE TABLE P1 select key, val from P1; -- This is not a regression. This has never worked. This got only discovered due to Hadoop2 changes. In Hadoop1, in local mode, number of reducers will always be 1, regardless of what is requested by app. Hadoop2 now honors the number of reducer setting in local mode (by spawning threads). Long term solution seems to be to prevent load data for bucketed table. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-3077) Insert overwrite table doesn't fail for bucketed tables and breaks bucketing
[ https://issues.apache.org/jira/browse/HIVE-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969002#comment-13969002 ] Ashutosh Chauhan commented on HIVE-3077: Dupe of HIVE-6867 ? Insert overwrite table doesn't fail for bucketed tables and breaks bucketing Key: HIVE-3077 URL: https://issues.apache.org/jira/browse/HIVE-3077 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.10.0 Environment: java version 1.6.0_30 hive version 0.9.0 hadoop version 0.20.205.0 Reporter: Mark Grover Assignee: Xuefu Zhang If table my_table is bucketed, the command insert into table my_table ... is supposed to give an error stating Bucketized tables do not support INSERT INTO. However, it doesn't seem to do that in all cases. Consider the following example on Hive 0.9.0: create table src(x string) clustered by( x ) sorted by ( x ) into 32 buckets; create table dest(x string) clustered by( x ) sorted by ( x ) into 32 buckets; Now, put some data into x (after enable hive.enforce.bucketing and hive.enforce.sorting to be true). Then, do: insert into table dest select * from src; This should fail since dest is a bucketized table. However, this succeeds creating a 33rd file inside the HDFS folder for the table, thereby corrupting it. This happens regardless of whether the src table is bucketed or not. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6888) Hive leaks MapWork objects via Utilities::gWorkMap
[ https://issues.apache.org/jira/browse/HIVE-6888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969019#comment-13969019 ] Sergey Shelukhin commented on HIVE-6888: [~rhbutani] can you let me know if there's another RC? I'd like to get this patch in if possible Hive leaks MapWork objects via Utilities::gWorkMap -- Key: HIVE-6888 URL: https://issues.apache.org/jira/browse/HIVE-6888 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Gunther Hagleitner Fix For: 0.14.0 Attachments: HIVE-6888.patch When running multiple queries with hive on a single Application Master, we found that hive leaks a large number of MapWork objects which accumulate in the AM -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6908) TestThriftBinaryCLIService.testExecuteStatementAsync has intermittent failures
[ https://issues.apache.org/jira/browse/HIVE-6908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969025#comment-13969025 ] Ashutosh Chauhan commented on HIVE-6908: Doesn't this mean test is poorly written if it is relying on timing of operations ? May be test can be rewritten in a different way so it doesnt rely on timeouts. TestThriftBinaryCLIService.testExecuteStatementAsync has intermittent failures -- Key: HIVE-6908 URL: https://issues.apache.org/jira/browse/HIVE-6908 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.13.0 Reporter: Szehon Ho Assignee: Szehon Ho Attachments: HIVE-6908.patch This has failed sometimes in the pre-commit tests. ThriftCLIServiceTest.testExecuteStatementAsync runs two statements. They are given 100 second timeout total, not sure if its by intention. As the first is a select query, it will take a majority of the time. The second statement (create table) should be quicker, but it fails sometimes because timeout is already mostly used up. The timeout should probably be reset after the first statement. If the operation finishes before the timeout, it wont have any effect as it'll break out. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969049#comment-13969049
]
Thejas M Nair commented on HIVE-6907:
-
I have run the tests and results look good.
[~ashutoshc] Can you commit this patch ? I have trouble doing that after the
password reset.
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch, HIVE-6907.3.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6908) TestThriftBinaryCLIService.testExecuteStatementAsync has intermittent failures
[ https://issues.apache.org/jira/browse/HIVE-6908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969053#comment-13969053 ] Szehon Ho commented on HIVE-6908: - We can eliminate the polling timeout, but I guess the author was thinking that timeout is better than hang for unit-test purpose, if the status is not updated successfully. Or did you have some other thoughts? TestThriftBinaryCLIService.testExecuteStatementAsync has intermittent failures -- Key: HIVE-6908 URL: https://issues.apache.org/jira/browse/HIVE-6908 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.13.0 Reporter: Szehon Ho Assignee: Szehon Ho Attachments: HIVE-6908.patch This has failed sometimes in the pre-commit tests. ThriftCLIServiceTest.testExecuteStatementAsync runs two statements. They are given 100 second timeout total, not sure if its by intention. As the first is a select query, it will take a majority of the time. The second statement (create table) should be quicker, but it fails sometimes because timeout is already mostly used up. The timeout should probably be reset after the first statement. If the operation finishes before the timeout, it wont have any effect as it'll break out. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HIVE-6906) Fix assembly/src.xml so that sr tar ball contains top level pom.xml
[ https://issues.apache.org/jira/browse/HIVE-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani resolved HIVE-6906. - Resolution: Fixed Committed to trunk and 0.13 Thanks Gunther, Ashutosh. Fix assembly/src.xml so that sr tar ball contains top level pom.xml --- Key: HIVE-6906 URL: https://issues.apache.org/jira/browse/HIVE-6906 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Fix For: 0.13.0 Attachments: HIVE-6906.1.patch This affects the tarballs we package with Hive 0.13. Current src tarballs being generated are not buildable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harish Butani resolved HIVE-6907.
-
Resolution: Fixed
Committed to trunk and 0.13
Thanks Thejas, Vaibhav, Jitendra, Ashutosh.
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch, HIVE-6907.3.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Reopened] (HIVE-6732) Update Release Notes for Hive 0.13
[ https://issues.apache.org/jira/browse/HIVE-6732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani reopened HIVE-6732: - Add in jiras committed to 0.13 today. Update Release Notes for Hive 0.13 -- Key: HIVE-6732 URL: https://issues.apache.org/jira/browse/HIVE-6732 Project: Hive Issue Type: Bug Affects Versions: 0.13.0, 0.14.0 Reporter: Harish Butani Assignee: Harish Butani Priority: Trivial Fix For: 0.13.0 Attachments: HIVE-6732.1.patch, HIVE-6732.2.patch, HIVE-6732.3.patch NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harish Butani updated HIVE-6907:
Fix Version/s: 0.13.0
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Fix For: 0.13.0
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch, HIVE-6907.3.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HIVE-6732) Update Release Notes for Hive 0.13
[ https://issues.apache.org/jira/browse/HIVE-6732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani resolved HIVE-6732. - Resolution: Fixed Just going to open a new jira for the delta change. Update Release Notes for Hive 0.13 -- Key: HIVE-6732 URL: https://issues.apache.org/jira/browse/HIVE-6732 Project: Hive Issue Type: Bug Affects Versions: 0.13.0, 0.14.0 Reporter: Harish Butani Assignee: Harish Butani Priority: Trivial Fix For: 0.13.0 Attachments: HIVE-6732.1.patch, HIVE-6732.2.patch, HIVE-6732.3.patch NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6909) Update Release Note for Hive 0.13 RC1
Harish Butani created HIVE-6909: --- Summary: Update Release Note for Hive 0.13 RC1 Key: HIVE-6909 URL: https://issues.apache.org/jira/browse/HIVE-6909 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Priority: Trivial -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6909) Update Release Note for Hive 0.13 RC1
[ https://issues.apache.org/jira/browse/HIVE-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-6909: Attachment: HIVE-6909.1.patch Update Release Note for Hive 0.13 RC1 - Key: HIVE-6909 URL: https://issues.apache.org/jira/browse/HIVE-6909 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Priority: Trivial Attachments: HIVE-6909.1.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6907) HiveServer2 - wrong user gets used for metastore operation with embedded metastore
[
https://issues.apache.org/jira/browse/HIVE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969107#comment-13969107
]
Navis commented on HIVE-6907:
-
[~thejas] Doesn't this break use case for doAs disabled?
HiveServer2 - wrong user gets used for metastore operation with embedded
metastore
--
Key: HIVE-6907
URL: https://issues.apache.org/jira/browse/HIVE-6907
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Priority: Blocker
Fix For: 0.13.0
Attachments: HIVE-6907.1.patch, HIVE-6907.2.patch, HIVE-6907.3.patch
When queries are being run concurrently against HS2, sometimes the wrong user
ends performing the metastore action and you get an error like -
{code}
..INFO|java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.net:8020/apps/hive/warehouse/tbl_4eeulg9zp4
for user hrt_qa)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-5072) [WebHCat]Enable directly invoke Sqoop job through Templeton
[ https://issues.apache.org/jira/browse/HIVE-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuaishuai Nie updated HIVE-5072: - Attachment: HIVE-5072.4.patch Updated the patch based on the comment. [~ekoifman], here are the comments for the questions: Hi [~ekoifman]. 5. Your implementation supports automatically shipping of Sqoop tar file to the target node where the job will be executed. The tar file doesn't contain JDBC drivers for all DB types so you can't expect it to be in the lib folder on target node. If the user is using the archive, They need to add the jdbc in the archive file before passing it. JDBC driver jar is a third party add-on for Sqoop,and also provide the flexibility for Sqoop to connect to any DB supporting JDBC, so user need to provide the JDBC driver jar based on the DB they use. 7. Since this API would mostly be used by Sqoop users, I think they'll find it confusing that items with the same name in Sqoop have different meaning in WebHCat/Sqoop. I would try to either change the names of parameters or make the definition in the 2 systems match. This make sense, I will change the name of the parameter. Also I will create a separate JIRA for the documentation after the patch got committed. 9. Does your implementation work with Sqoop 1 and Sqoop 2? If not, it should probably assert the right version is installed/tar file is available and produce a meaningful error to client. The implementation only works with Sqoop 1. Since the usage of Sqoop 1 and Sqoop 2 differs a lot (Sqoop 2 need to create connection, create job for import/export) and the way to pass command in Sqoop 1 is not working in Sqoop 2, so it is not likely user will use this function against Sqoop 2. I think the best way is to specify that this endpoint only support Sqoop 1 in the documentation. [WebHCat]Enable directly invoke Sqoop job through Templeton --- Key: HIVE-5072 URL: https://issues.apache.org/jira/browse/HIVE-5072 Project: Hive Issue Type: Improvement Components: WebHCat Affects Versions: 0.12.0 Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5072.1.patch, HIVE-5072.2.patch, HIVE-5072.3.patch, HIVE-5072.4.patch, Templeton-Sqoop-Action.pdf Now it is hard to invoke a Sqoop job through templeton. The only way is to use the classpath jar generated by a sqoop job and use the jar delegator in Templeton. We should implement Sqoop Delegator to enable directly invoke Sqoop job through Templeton. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5235) Infinite loop with ORC file and Hive 0.11
[
https://issues.apache.org/jira/browse/HIVE-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969149#comment-13969149
]
Edwin Chiu commented on HIVE-5235:
--
Seeing this issue with Hive 0.12 on HDP 2.0 as well...
Infinite loop with ORC file and Hive 0.11
-
Key: HIVE-5235
URL: https://issues.apache.org/jira/browse/HIVE-5235
Project: Hive
Issue Type: Bug
Affects Versions: 0.11.0, 0.12.0
Environment: Gentoo linux with Hortonworks Hadoop
hadoop-1.1.2.23.tar.gz and Apache Hive 0.11d
Reporter: Iván de Prado
Priority: Blocker
Attachments: gendata.py
We are using Hive 0.11 with ORC file format and we get some tasks blocked in
some kind of infinite loop. They keep working indefinitely when we set a huge
task expiry timeout. If we the expiry time to 600 second, the taks fail
because of not reporting progress, and finally, the Job fails.
That is not consistent, and some times between jobs executions the behavior
changes. It happen for different queries.
We are using Hive 0.11 with Hadoop hadoop-1.1.2.23 from Hortonworks. The taks
that is blocked keeps consuming 100% of CPU usage, and the stack trace is
always the same consistently. Everything points to some kind of infinite
loop. My guessing is that it has some relation to the ORC file. Maybe some
pointer is not right when writing generating some kind of infinite loop when
reading. Or maybe there is a bug in the reading stage.
More information below. The stack trace:
{noformat}
main prio=10 tid=0x7f20a000a800 nid=0x1ed2 runnable [0x7f20a8136000]
java.lang.Thread.State: RUNNABLE
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Inflater.java:256)
- locked 0xf42a6ca0 (a java.util.zip.ZStreamRef)
at
org.apache.hadoop.hive.ql.io.orc.ZlibCodec.decompress(ZlibCodec.java:64)
at
org.apache.hadoop.hive.ql.io.orc.InStream$CompressedStream.readHeader(InStream.java:128)
at
org.apache.hadoop.hive.ql.io.orc.InStream$CompressedStream.read(InStream.java:143)
at
org.apache.hadoop.hive.ql.io.orc.SerializationUtils.readVulong(SerializationUtils.java:54)
at
org.apache.hadoop.hive.ql.io.orc.SerializationUtils.readVslong(SerializationUtils.java:65)
at
org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReader.readValues(RunLengthIntegerReader.java:66)
at
org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReader.next(RunLengthIntegerReader.java:81)
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$IntTreeReader.next(RecordReaderImpl.java:332)
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StructTreeReader.next(RecordReaderImpl.java:802)
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.next(RecordReaderImpl.java:1214)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:71)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:46)
at
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:274)
at
org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:101)
at
org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:41)
at
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:108)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:300)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:218)
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:236)
- eliminated 0xe1459700 (a
org.apache.hadoop.mapred.MapTask$TrackedRecordReader)
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:216)
- locked 0xe1459700 (a
org.apache.hadoop.mapred.MapTask$TrackedRecordReader)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1178)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
{noformat}
We have seen the same stack trace repeatedly for several executions of jstack.
The log file for this kind of task is the
[jira] [Updated] (HIVE-5235) Infinite loop with ORC file and Hive 0.11
[
https://issues.apache.org/jira/browse/HIVE-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edwin Chiu updated HIVE-5235:
-
Affects Version/s: 0.12.0
Infinite loop with ORC file and Hive 0.11
-
Key: HIVE-5235
URL: https://issues.apache.org/jira/browse/HIVE-5235
Project: Hive
Issue Type: Bug
Affects Versions: 0.11.0, 0.12.0
Environment: Gentoo linux with Hortonworks Hadoop
hadoop-1.1.2.23.tar.gz and Apache Hive 0.11d
Reporter: Iván de Prado
Priority: Blocker
Attachments: gendata.py
We are using Hive 0.11 with ORC file format and we get some tasks blocked in
some kind of infinite loop. They keep working indefinitely when we set a huge
task expiry timeout. If we the expiry time to 600 second, the taks fail
because of not reporting progress, and finally, the Job fails.
That is not consistent, and some times between jobs executions the behavior
changes. It happen for different queries.
We are using Hive 0.11 with Hadoop hadoop-1.1.2.23 from Hortonworks. The taks
that is blocked keeps consuming 100% of CPU usage, and the stack trace is
always the same consistently. Everything points to some kind of infinite
loop. My guessing is that it has some relation to the ORC file. Maybe some
pointer is not right when writing generating some kind of infinite loop when
reading. Or maybe there is a bug in the reading stage.
More information below. The stack trace:
{noformat}
main prio=10 tid=0x7f20a000a800 nid=0x1ed2 runnable [0x7f20a8136000]
java.lang.Thread.State: RUNNABLE
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Inflater.java:256)
- locked 0xf42a6ca0 (a java.util.zip.ZStreamRef)
at
org.apache.hadoop.hive.ql.io.orc.ZlibCodec.decompress(ZlibCodec.java:64)
at
org.apache.hadoop.hive.ql.io.orc.InStream$CompressedStream.readHeader(InStream.java:128)
at
org.apache.hadoop.hive.ql.io.orc.InStream$CompressedStream.read(InStream.java:143)
at
org.apache.hadoop.hive.ql.io.orc.SerializationUtils.readVulong(SerializationUtils.java:54)
at
org.apache.hadoop.hive.ql.io.orc.SerializationUtils.readVslong(SerializationUtils.java:65)
at
org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReader.readValues(RunLengthIntegerReader.java:66)
at
org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReader.next(RunLengthIntegerReader.java:81)
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$IntTreeReader.next(RecordReaderImpl.java:332)
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StructTreeReader.next(RecordReaderImpl.java:802)
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.next(RecordReaderImpl.java:1214)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:71)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:46)
at
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:274)
at
org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:101)
at
org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:41)
at
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:108)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:300)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:218)
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:236)
- eliminated 0xe1459700 (a
org.apache.hadoop.mapred.MapTask$TrackedRecordReader)
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:216)
- locked 0xe1459700 (a
org.apache.hadoop.mapred.MapTask$TrackedRecordReader)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1178)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
{noformat}
We have seen the same stack trace repeatedly for several executions of jstack.
The log file for this kind of task is the following:
{noformat}
2013-09-04 23:12:34,332 INFO
