[jira] [Assigned] (HIVE-25650) Make WorkerId and WorkerVersionId optional in the FindNextCompactRequest

2021-10-26 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csomor Viktor reassigned HIVE-25650:


Assignee: Csomor Viktor

> Make WorkerId and WorkerVersionId optional in the FindNextCompactRequest
> 
>
> Key: HIVE-25650
> URL: https://issues.apache.org/jira/browse/HIVE-25650
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Minor
>
> In hive_metastore.thrift the FindNextCompactRequest struct's fields are 
> required:
> {code}
> struct FindNextCompactRequest {
> 1: required string workerId,
> 2: required string workerVersion
> }{code}
> these should probably be made optional, to avoid breaking compaction if 
> they're not available.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-25642) Log a warning if multiple Compaction Worker versions are running compactions

2021-10-25 Thread Csomor Viktor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-25642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17433791#comment-17433791
 ] 

Csomor Viktor commented on HIVE-25642:
--

Pull request: https://github.com/apache/hive/pull/2743

> Log a warning if multiple Compaction Worker versions are running compactions
> 
>
> Key: HIVE-25642
> URL: https://issues.apache.org/jira/browse/HIVE-25642
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Minor
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Log a warning if multiple versions of a Compaction Workers are running 
> compactions.
> The start time of the individual HMS services are not stored at the moment, 
> however this information could proved a good baseline of detecting multiple 
> Worker versions. 
> Due to the lack of this information we can check periodically in the past N 
> hours to detect the versions.
> The N hours can be configured by 
> {{metastore.compactor.worker.detect_multiple_versions.threshold}} property.
> This periodical check only make sense if the Compaction are enabled.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-25642) Log a warning if multiple Compaction Worker versions are running compactions

2021-10-25 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csomor Viktor updated HIVE-25642:
-
Description: 
Log a warning if multiple versions of a Compaction Workers are running 
compactions.

The start time of the individual HMS services are not stored at the moment, 
however this information could proved a good baseline of detecting multiple 
Worker versions. 
Due to the lack of this information we can check periodically in the past N 
hours to detect the versions.
The N hours can be configured by 
{{metastore.compactor.worker.detect_multiple_versions.threshold}} property.

This periodical check only make sense if the Compaction are enabled.

  was:
Log a warning if multiple versions of a Compaction Workers are running.

The start time of the individual HMS services are not stored at the moment, 
however this information could proved a good baseline of detecting multiple 
Worker versions. 
Due to the lack of this information we can check periodically in the past N 
hours to detect the versions.
The N hours can be configured by 
{{metastore.compactor.worker.detect_multiple_versions.threshold}} property.

This periodical check only make sense if the Compaction are enabled.


> Log a warning if multiple Compaction Worker versions are running compactions
> 
>
> Key: HIVE-25642
> URL: https://issues.apache.org/jira/browse/HIVE-25642
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Minor
>
> Log a warning if multiple versions of a Compaction Workers are running 
> compactions.
> The start time of the individual HMS services are not stored at the moment, 
> however this information could proved a good baseline of detecting multiple 
> Worker versions. 
> Due to the lack of this information we can check periodically in the past N 
> hours to detect the versions.
> The N hours can be configured by 
> {{metastore.compactor.worker.detect_multiple_versions.threshold}} property.
> This periodical check only make sense if the Compaction are enabled.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work started] (HIVE-25642) Log a warning if multiple Compaction Worker versions are running compactions

2021-10-25 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-25642 started by Csomor Viktor.

> Log a warning if multiple Compaction Worker versions are running compactions
> 
>
> Key: HIVE-25642
> URL: https://issues.apache.org/jira/browse/HIVE-25642
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Minor
>
> Log a warning if multiple versions of a Compaction Workers are running.
> The start time of the individual HMS services are not stored at the moment, 
> however this information could proved a good baseline of detecting multiple 
> Worker versions. 
> Due to the lack of this information we can check periodically in the past N 
> hours to detect the versions.
> The N hours can be configured by 
> {{metastore.compactor.worker.detect_multiple_versions.threshold}} property.
> This periodical check only make sense if the Compaction are enabled.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-25642) Log a warning if multiple Compaction Worker versions are running compactions

2021-10-25 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csomor Viktor reassigned HIVE-25642:



> Log a warning if multiple Compaction Worker versions are running compactions
> 
>
> Key: HIVE-25642
> URL: https://issues.apache.org/jira/browse/HIVE-25642
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Minor
>
> Log a warning if multiple versions of a Compaction Workers are running.
> The start time of the individual HMS services are not stored at the moment, 
> however this information could proved a good baseline of detecting multiple 
> Worker versions. 
> Due to the lack of this information we can check periodically in the past N 
> hours to detect the versions.
> The N hours can be configured by 
> {{metastore.compactor.worker.detect_multiple_versions.threshold}} property.
> This periodical check only make sense if the Compaction are enabled.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (HIVE-25518) CompactionTxHandler NPE if no CompactionInfo

2021-09-16 Thread Csomor Viktor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-25518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17416465#comment-17416465
 ] 

Csomor Viktor edited comment on HIVE-25518 at 9/17/21, 5:54 AM:


This caused a wrong Intellij jar assembly. can't reproduce by using the single 
binary


was (Author: vcsomor):
This caused a wrong Intellij jar assembly

> CompactionTxHandler NPE if no CompactionInfo
> 
>
> Key: HIVE-25518
> URL: https://issues.apache.org/jira/browse/HIVE-25518
> Project: Hive
>  Issue Type: Bug
>  Components: Standalone Metastore
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE_25518_CompactionTxnHandle_NPE.txt
>
>
> If no {{CompactionInfo}} is provided to the 
> {{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
> the method. No information inside the COMPLETED_COMPACTION info.
> Stacktrace:
> {noformat}
> [TThreadPoolServer WorkerProcess-%d] ERROR 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler - 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.markFailed(CompactionTxnHandler.java:1116)
>   at 
> org.apache.hadoop.hive.metastore.HMSHandler.mark_failed(HMSHandler.java:8716)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
>   at com.sun.proxy.$Proxy13.mark_failed(Unknown Source)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23846)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23825)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
>   at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (HIVE-25518) CompactionTxHandler NPE if no CompactionInfo

2021-09-16 Thread Csomor Viktor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-25518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17416465#comment-17416465
 ] 

Csomor Viktor edited comment on HIVE-25518 at 9/17/21, 5:54 AM:


This caused a wrong Intellij jar assembly. can't reproduce by using the binary


was (Author: vcsomor):
This caused a wrong Intellij jar assembly. can't reproduce by using the single 
binary

> CompactionTxHandler NPE if no CompactionInfo
> 
>
> Key: HIVE-25518
> URL: https://issues.apache.org/jira/browse/HIVE-25518
> Project: Hive
>  Issue Type: Bug
>  Components: Standalone Metastore
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE_25518_CompactionTxnHandle_NPE.txt
>
>
> If no {{CompactionInfo}} is provided to the 
> {{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
> the method. No information inside the COMPLETED_COMPACTION info.
> Stacktrace:
> {noformat}
> [TThreadPoolServer WorkerProcess-%d] ERROR 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler - 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.markFailed(CompactionTxnHandler.java:1116)
>   at 
> org.apache.hadoop.hive.metastore.HMSHandler.mark_failed(HMSHandler.java:8716)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
>   at com.sun.proxy.$Proxy13.mark_failed(Unknown Source)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23846)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23825)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
>   at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HIVE-25518) CompactionTxHandler NPE if no CompactionInfo

2021-09-16 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csomor Viktor resolved HIVE-25518.
--
Resolution: Won't Fix

This caused a wrong Intellij jar assembly

> CompactionTxHandler NPE if no CompactionInfo
> 
>
> Key: HIVE-25518
> URL: https://issues.apache.org/jira/browse/HIVE-25518
> Project: Hive
>  Issue Type: Bug
>  Components: Standalone Metastore
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE_25518_CompactionTxnHandle_NPE.txt
>
>
> If no {{CompactionInfo}} is provided to the 
> {{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
> the method. No information inside the COMPLETED_COMPACTION info.
> Stacktrace:
> {noformat}
> [TThreadPoolServer WorkerProcess-%d] ERROR 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler - 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.markFailed(CompactionTxnHandler.java:1116)
>   at 
> org.apache.hadoop.hive.metastore.HMSHandler.mark_failed(HMSHandler.java:8716)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
>   at com.sun.proxy.$Proxy13.mark_failed(Unknown Source)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23846)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23825)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
>   at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work stopped] (HIVE-25518) CompactionTxHandler NPE if no CompactionInfo

2021-09-16 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-25518 stopped by Csomor Viktor.

> CompactionTxHandler NPE if no CompactionInfo
> 
>
> Key: HIVE-25518
> URL: https://issues.apache.org/jira/browse/HIVE-25518
> Project: Hive
>  Issue Type: Bug
>  Components: Standalone Metastore
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE_25518_CompactionTxnHandle_NPE.txt
>
>
> If no {{CompactionInfo}} is provided to the 
> {{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
> the method. No information inside the COMPLETED_COMPACTION info.
> Stacktrace:
> {noformat}
> [TThreadPoolServer WorkerProcess-%d] ERROR 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler - 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.markFailed(CompactionTxnHandler.java:1116)
>   at 
> org.apache.hadoop.hive.metastore.HMSHandler.mark_failed(HMSHandler.java:8716)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
>   at com.sun.proxy.$Proxy13.mark_failed(Unknown Source)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23846)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23825)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
>   at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-25518) CompactionTxHandler NPE if no CompactionInfo

2021-09-15 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csomor Viktor updated HIVE-25518:
-
Attachment: HIVE_25518_CompactionTxnHandle_NPE.txt

> CompactionTxHandler NPE if no CompactionInfo
> 
>
> Key: HIVE-25518
> URL: https://issues.apache.org/jira/browse/HIVE-25518
> Project: Hive
>  Issue Type: Bug
>  Components: Standalone Metastore
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE_25518_CompactionTxnHandle_NPE.txt
>
>
> If no {{CompactionInfo}} is provided to the 
> {{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
> the method. No information inside the COMPLETED_COMPACTION info.
> Stacktrace:
> {noformat}
> [TThreadPoolServer WorkerProcess-%d] ERROR 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler - 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.markFailed(CompactionTxnHandler.java:1116)
>   at 
> org.apache.hadoop.hive.metastore.HMSHandler.mark_failed(HMSHandler.java:8716)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
>   at com.sun.proxy.$Proxy13.mark_failed(Unknown Source)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23846)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23825)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
>   at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-25518) CompactionTxHandler NPE if no CompactionInfo

2021-09-15 Thread Csomor Viktor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-25518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17415691#comment-17415691
 ] 

Csomor Viktor commented on HIVE-25518:
--

[~zabetak] thanks for letting me know! I've attached the stacktrace. 
(/)  [^HIVE_25518_CompactionTxnHandle_NPE.txt] 

> CompactionTxHandler NPE if no CompactionInfo
> 
>
> Key: HIVE-25518
> URL: https://issues.apache.org/jira/browse/HIVE-25518
> Project: Hive
>  Issue Type: Bug
>  Components: Standalone Metastore
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE_25518_CompactionTxnHandle_NPE.txt
>
>
> If no {{CompactionInfo}} is provided to the 
> {{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
> the method. No information inside the COMPLETED_COMPACTION info.
> Stacktrace:
> {noformat}
> [TThreadPoolServer WorkerProcess-%d] ERROR 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler - 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.markFailed(CompactionTxnHandler.java:1116)
>   at 
> org.apache.hadoop.hive.metastore.HMSHandler.mark_failed(HMSHandler.java:8716)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
>   at com.sun.proxy.$Proxy13.mark_failed(Unknown Source)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23846)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23825)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
>   at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-25518) CompactionTxHandler NPE if no CompactionInfo

2021-09-15 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csomor Viktor updated HIVE-25518:
-
Description: 
If no {{CompactionInfo}} is provided to the 
{{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
the method. No information inside the COMPLETED_COMPACTION info.
Stacktrace:
{noformat}
20:47:44.916 [TThreadPoolServer WorkerProcess-%d] ERROR 
org.apache.hadoop.hive.metastore.RetryingHMSHandler - 
java.lang.NullPointerException
at 
org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.markFailed(CompactionTxnHandler.java:1116)
at 
org.apache.hadoop.hive.metastore.HMSHandler.mark_failed(HMSHandler.java:8716)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy13.mark_failed(Unknown Source)
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23846)
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23825)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}

  was:If no {{CompactionInfo}} is provided to the 
{{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
the method. No information inside the COMPLETED_COMPACTION info.


> CompactionTxHandler NPE if no CompactionInfo
> 
>
> Key: HIVE-25518
> URL: https://issues.apache.org/jira/browse/HIVE-25518
> Project: Hive
>  Issue Type: Bug
>  Components: Standalone Metastore
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Major
> Fix For: 4.0.0
>
>
> If no {{CompactionInfo}} is provided to the 
> {{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
> the method. No information inside the COMPLETED_COMPACTION info.
> Stacktrace:
> {noformat}
> 20:47:44.916 [TThreadPoolServer WorkerProcess-%d] ERROR 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler - 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.markFailed(CompactionTxnHandler.java:1116)
>   at 
> org.apache.hadoop.hive.metastore.HMSHandler.mark_failed(HMSHandler.java:8716)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
>   at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
>   at com.sun.proxy.$Proxy13.mark_failed(Unknown Source)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23846)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23825)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
>   at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
>   at java.security.AccessController.doPrivileged(Native M

[jira] [Updated] (HIVE-25518) CompactionTxHandler NPE if no CompactionInfo

2021-09-15 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csomor Viktor updated HIVE-25518:
-
Description: 
If no {{CompactionInfo}} is provided to the 
{{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
the method. No information inside the COMPLETED_COMPACTION info.
Stacktrace:
{noformat}
[TThreadPoolServer WorkerProcess-%d] ERROR 
org.apache.hadoop.hive.metastore.RetryingHMSHandler - 
java.lang.NullPointerException
at 
org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.markFailed(CompactionTxnHandler.java:1116)
at 
org.apache.hadoop.hive.metastore.HMSHandler.mark_failed(HMSHandler.java:8716)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy13.mark_failed(Unknown Source)
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23846)
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23825)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}

  was:
If no {{CompactionInfo}} is provided to the 
{{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
the method. No information inside the COMPLETED_COMPACTION info.
Stacktrace:
{noformat}
20:47:44.916 [TThreadPoolServer WorkerProcess-%d] ERROR 
org.apache.hadoop.hive.metastore.RetryingHMSHandler - 
java.lang.NullPointerException
at 
org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.markFailed(CompactionTxnHandler.java:1116)
at 
org.apache.hadoop.hive.metastore.HMSHandler.mark_failed(HMSHandler.java:8716)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy13.mark_failed(Unknown Source)
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23846)
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$mark_failed.getResult(ThriftHiveMetastore.java:23825)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
at 
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}


> CompactionTxHandler NPE if no CompactionInfo
>

[jira] [Updated] (HIVE-25518) CompactionTxHandler NPE if no CompactionInfo

2021-09-14 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csomor Viktor updated HIVE-25518:
-
Summary: CompactionTxHandler NPE if no CompactionInfo  (was: 
CompactionTxHandler NPE if no ComaptionInfo)

> CompactionTxHandler NPE if no CompactionInfo
> 
>
> Key: HIVE-25518
> URL: https://issues.apache.org/jira/browse/HIVE-25518
> Project: Hive
>  Issue Type: Bug
>  Components: Standalone Metastore
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Major
> Fix For: 4.0.0
>
>
> If no {{CompactionInfo}} is provided to the 
> {{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
> the method. No information inside the COMPLETED_COMPACTION info.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work started] (HIVE-25518) CompactionTxHandler NPE if no ComaptionInfo

2021-09-14 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-25518 started by Csomor Viktor.

> CompactionTxHandler NPE if no ComaptionInfo
> ---
>
> Key: HIVE-25518
> URL: https://issues.apache.org/jira/browse/HIVE-25518
> Project: Hive
>  Issue Type: Bug
>  Components: Standalone Metastore
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Major
> Fix For: 4.0.0
>
>
> If no {{CompactionInfo}} is provided to the 
> {{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
> the method. No information inside the COMPLETED_COMPACTION info.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-25518) CompactionTxHandler NPE if no ComaptionInfo

2021-09-14 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csomor Viktor updated HIVE-25518:
-
Description: If no {{CompactionInfo}} is provided to the 
{{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
the method. No information inside the COMPLETED_COMPACTION info.  (was: If no 
{{CompactionInfo}} provided to the {{CompcationTxHandler#markFailed()}} then an 
NPE happens at the beginning of the method. No information inside the 
COMPLETED_COMPACTION info.)

> CompactionTxHandler NPE if no ComaptionInfo
> ---
>
> Key: HIVE-25518
> URL: https://issues.apache.org/jira/browse/HIVE-25518
> Project: Hive
>  Issue Type: Bug
>  Components: Standalone Metastore
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Major
> Fix For: 4.0.0
>
>
> If no {{CompactionInfo}} is provided to the 
> {{CompactionTxHandler#markFailed()}} then an NPE happens at the beginning of 
> the method. No information inside the COMPLETED_COMPACTION info.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-25518) CompactionTxHandler NPE if no ComaptionInfo

2021-09-14 Thread Csomor Viktor (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-25518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csomor Viktor reassigned HIVE-25518:



> CompactionTxHandler NPE if no ComaptionInfo
> ---
>
> Key: HIVE-25518
> URL: https://issues.apache.org/jira/browse/HIVE-25518
> Project: Hive
>  Issue Type: Bug
>  Components: Standalone Metastore
>Reporter: Csomor Viktor
>Assignee: Csomor Viktor
>Priority: Major
> Fix For: 4.0.0
>
>
> If no {{CompactionInfo}} provided to the {{CompcationTxHandler#markFailed()}} 
> then an NPE happens at the beginning of the method. No information inside the 
> COMPLETED_COMPACTION info.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)