[jira] [Work logged] (HIVE-25223) Select with limit returns no rows on non native table
[
https://issues.apache.org/jira/browse/HIVE-25223?focusedWorklogId=609465=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609465
]
ASF GitHub Bot logged work on HIVE-25223:
-
Author: ASF GitHub Bot
Created on: 10/Jun/21 04:49
Start Date: 10/Jun/21 04:49
Worklog Time Spent: 10m
Work Description: kasakrisz commented on pull request #2375:
URL: https://github.com/apache/hive/pull/2375#issuecomment-858296921
+1, pending tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609465)
Time Spent: 20m (was: 10m)
> Select with limit returns no rows on non native table
> -
>
> Key: HIVE-25223
> URL: https://issues.apache.org/jira/browse/HIVE-25223
> Project: Hive
> Issue Type: Bug
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Str:
> {code:java}
> CREATE EXTERNAL TABLE hht (key string, value int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "hht", "hbase.mapred.output.outputtable"
> = "hht");
> insert into hht select uuid(), cast((rand() * 100) as int);
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> set hive.fetch.task.conversion=none;
> select * from hht limit 10;
> +--++
> | hht.key | hht.value |
> +--++
> +--++
> No rows selected (5.22 seconds) {code}
>
> This is caused by GlobalLimitOptimizer. The table directory is always empty
> with a non native table since the data is not managed by hive (but hbase in
> this case).
> The optimizer scans the directory and sets the file list to an empty list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25220) Query with union fails CBO with OOM
[
https://issues.apache.org/jira/browse/HIVE-25220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-25220:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master. Thanks [~jcamachorodriguez] for review.
> Query with union fails CBO with OOM
> ---
>
> Key: HIVE-25220
> URL: https://issues.apache.org/jira/browse/HIVE-25220
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code}
> 2021-06-08T08:15:14,450 ERROR [6241f234-77e0-4e63-9873-6eb9d655421c
> HiveServer2-Handler-Pool: Thread-79] parse.CalcitePlanner: CBO failed,
> skipping CBO.
> java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.rethrowCalciteException(CalcitePlanner.java:1728)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1564)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:538)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12680)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:428)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:170)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:221)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:188)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:600)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:546)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:540)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:127)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:199)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:260)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:274)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:565)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:551)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_262]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_262]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_262]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_262]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
>
[jira] [Work logged] (HIVE-25220) Query with union fails CBO with OOM
[
https://issues.apache.org/jira/browse/HIVE-25220?focusedWorklogId=609461=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609461
]
ASF GitHub Bot logged work on HIVE-25220:
-
Author: ASF GitHub Bot
Created on: 10/Jun/21 04:17
Start Date: 10/Jun/21 04:17
Worklog Time Spent: 10m
Work Description: kasakrisz merged pull request #2372:
URL: https://github.com/apache/hive/pull/2372
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609461)
Time Spent: 20m (was: 10m)
> Query with union fails CBO with OOM
> ---
>
> Key: HIVE-25220
> URL: https://issues.apache.org/jira/browse/HIVE-25220
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code}
> 2021-06-08T08:15:14,450 ERROR [6241f234-77e0-4e63-9873-6eb9d655421c
> HiveServer2-Handler-Pool: Thread-79] parse.CalcitePlanner: CBO failed,
> skipping CBO.
> java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.rethrowCalciteException(CalcitePlanner.java:1728)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1564)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:538)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12680)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:428)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:170)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:221)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:188)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:600)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:546)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:540)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:127)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:199)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:260)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:274)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:565)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:551)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_262]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_262]
> at
>
[jira] [Assigned] (HIVE-25230) add position and occurrence to instr()
[
https://issues.apache.org/jira/browse/HIVE-25230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang reassigned HIVE-25230:
-
> add position and occurrence to instr()
> --
>
> Key: HIVE-25230
> URL: https://issues.apache.org/jira/browse/HIVE-25230
> Project: Hive
> Issue Type: New Feature
> Components: UDF
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
>
> Current instr() only supports two arguments:
> {code:java}
> instr(str, substr) - Returns the index of the first occurance of substr in str
> {code}
> Other systems (Vertica, Oracle, Impala etc) support additional position and
> occurrence arguments:
> {code:java}
> instr(str, substr[, pos[, occurrence]])
> {code}
> Oracle doc:
> [https://docs.oracle.com/database/121/SQLRF/functions089.htm#SQLRF00651]
> It'd be nice to support this as well. Otherwise, it's a SQL difference
> between Impala and Hive.
> Impala supports this in IMPALA-3973
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25221) Hive Beeline & CLI: execute alter table comamnd With Error: Unable to alter partitions because table or database does not exist
[
https://issues.apache.org/jira/browse/HIVE-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-25221:
--
Labels: pull-request-available (was: )
> Hive Beeline & CLI: execute alter table comamnd With Error: Unable to alter
> partitions because table or database does not exist
> ---
>
> Key: HIVE-25221
> URL: https://issues.apache.org/jira/browse/HIVE-25221
> Project: Hive
> Issue Type: Bug
> Components: Beeline, CLI
>Affects Versions: 2.3.7, 2.3.8
>Reporter: XixiHua
>Assignee: XixiHua
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Use Hive Beeline and CLI to execute the following command:
>
> {code:java}
> alter table xxx.xxx partition(xxx) set location 'xxx';{code}
>
> If don't execute +*use *+ first, there will fail with the error:
> Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> Unable to alter partition. Unable to alter partitions because table or
> database does not exist. even if the table does exist.
> and more info:
> {code:java}
> 2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not
> exist.)2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not exist.) at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3908)
> at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy27.alter_partitions_with_environment_context(Unknown
> Source) at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12598)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12582)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> but when execute add/drop partition, don't need to follow this mode.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25221) Hive Beeline & CLI: execute alter table comamnd With Error: Unable to alter partitions because table or database does not exist
[
https://issues.apache.org/jira/browse/HIVE-25221?focusedWorklogId=609443=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609443
]
ASF GitHub Bot logged work on HIVE-25221:
-
Author: ASF GitHub Bot
Created on: 10/Jun/21 02:14
Start Date: 10/Jun/21 02:14
Worklog Time Spent: 10m
Work Description: honeyaya commented on pull request #2374:
URL: https://github.com/apache/hive/pull/2374#issuecomment-858226467
cc @szehon-ho @sunchao
Could you help to review this? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609443)
Remaining Estimate: 0h
Time Spent: 10m
> Hive Beeline & CLI: execute alter table comamnd With Error: Unable to alter
> partitions because table or database does not exist
> ---
>
> Key: HIVE-25221
> URL: https://issues.apache.org/jira/browse/HIVE-25221
> Project: Hive
> Issue Type: Bug
> Components: Beeline, CLI
>Affects Versions: 2.3.7, 2.3.8
>Reporter: XixiHua
>Assignee: XixiHua
>Priority: Major
> Fix For: 2.3.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Use Hive Beeline and CLI to execute the following command:
>
> {code:java}
> alter table xxx.xxx partition(xxx) set location 'xxx';{code}
>
> If don't execute +*use *+ first, there will fail with the error:
> Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> Unable to alter partition. Unable to alter partitions because table or
> database does not exist. even if the table does exist.
> and more info:
> {code:java}
> 2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not
> exist.)2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not exist.) at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3908)
> at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy27.alter_partitions_with_environment_context(Unknown
> Source) at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12598)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12582)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> but when execute add/drop partition, don't need to follow this mode.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25229) Hive lineage is not generated for columns on CREATE MATERIALIZED VIEW
[ https://issues.apache.org/jira/browse/HIVE-25229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25229: -- Labels: pull-request-available (was: ) > Hive lineage is not generated for columns on CREATE MATERIALIZED VIEW > - > > Key: HIVE-25229 > URL: https://issues.apache.org/jira/browse/HIVE-25229 > Project: Hive > Issue Type: Bug >Reporter: Soumyakanti Das >Assignee: Soumyakanti Das >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > While creating materialized view HookContext is supposed to send lineage info > which is missing. > CREATE MATERIALIZED VIEW tbl1_view as select * from tbl1; > Hook Context passed from hive.ql.Driver to Hive Hook of Atlas through > hookRunner.runPostExecHooks call doesn't have lineage info. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25229) Hive lineage is not generated for columns on CREATE MATERIALIZED VIEW
[ https://issues.apache.org/jira/browse/HIVE-25229?focusedWorklogId=609441=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609441 ] ASF GitHub Bot logged work on HIVE-25229: - Author: ASF GitHub Bot Created on: 10/Jun/21 02:08 Start Date: 10/Jun/21 02:08 Worklog Time Spent: 10m Work Description: soumyakanti3578 opened a new pull request #2377: URL: https://github.com/apache/hive/pull/2377 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 609441) Remaining Estimate: 0h Time Spent: 10m > Hive lineage is not generated for columns on CREATE MATERIALIZED VIEW > - > > Key: HIVE-25229 > URL: https://issues.apache.org/jira/browse/HIVE-25229 > Project: Hive > Issue Type: Bug >Reporter: Soumyakanti Das >Assignee: Soumyakanti Das >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > While creating materialized view HookContext is supposed to send lineage info > which is missing. > CREATE MATERIALIZED VIEW tbl1_view as select * from tbl1; > Hook Context passed from hive.ql.Driver to Hive Hook of Atlas through > hookRunner.runPostExecHooks call doesn't have lineage info. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25229) Hive lineage is not generated for columns on CREATE MATERIALIZED VIEW
[ https://issues.apache.org/jira/browse/HIVE-25229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyakanti Das reassigned HIVE-25229: -- > Hive lineage is not generated for columns on CREATE MATERIALIZED VIEW > - > > Key: HIVE-25229 > URL: https://issues.apache.org/jira/browse/HIVE-25229 > Project: Hive > Issue Type: Bug >Reporter: Soumyakanti Das >Assignee: Soumyakanti Das >Priority: Major > > While creating materialized view HookContext is supposed to send lineage info > which is missing. > CREATE MATERIALIZED VIEW tbl1_view as select * from tbl1; > Hook Context passed from hive.ql.Driver to Hive Hook of Atlas through > hookRunner.runPostExecHooks call doesn't have lineage info. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25225) Update column stat throws NPE if direct sql is disabled
[ https://issues.apache.org/jira/browse/HIVE-25225?focusedWorklogId=609402=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609402 ] ASF GitHub Bot logged work on HIVE-25225: - Author: ASF GitHub Bot Created on: 09/Jun/21 21:30 Start Date: 09/Jun/21 21:30 Worklog Time Spent: 10m Work Description: pvary commented on pull request #2376: URL: https://github.com/apache/hive/pull/2376#issuecomment-858115682 Thanks for the quick fix @maheshk114! Left some comments on the code. Also we could definitely use some additional tests which could make sure that someone else later does not break this functionality. Thanks, Peter -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 609402) Time Spent: 40m (was: 0.5h) > Update column stat throws NPE if direct sql is disabled > --- > > Key: HIVE-25225 > URL: https://issues.apache.org/jira/browse/HIVE-25225 > Project: Hive > Issue Type: Sub-task >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > In case direct sql is disabled, the MetaStoreDirectSql object is not > initialised and thats causing NPE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25225) Update column stat throws NPE if direct sql is disabled
[
https://issues.apache.org/jira/browse/HIVE-25225?focusedWorklogId=609400=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609400
]
ASF GitHub Bot logged work on HIVE-25225:
-
Author: ASF GitHub Bot
Created on: 09/Jun/21 21:28
Start Date: 09/Jun/21 21:28
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #2376:
URL: https://github.com/apache/hive/pull/2376#discussion_r648697220
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java
##
@@ -9037,8 +9061,16 @@ public boolean
set_aggr_stats_for(SetPartitionsStatsRequest request) throws TExc
} else { // No merge.
Table t = getTable(catName, dbName, tableName);
// We don't short-circuit on errors here anymore. That can leave acid
stats invalid.
-ret = updatePartitionColStatsInBatch(t, newStatsMap,
-request.getValidWriteIdList(), request.getWriteId());
+if (MetastoreConf.getBoolVar(getConf(), ConfVars.TRY_DIRECT_SQL)) {
Review comment:
It is very strange to use the `ConfVars.TRY_DIRECT_SQL` configuration in
client side.
We could have used the same patterns like in `get_table_statistics_req`:
- HMSHandler has only a single method to call on `getMS()`
- ObjectStore could use some `GetHelper` implementation to decide on the
actual implementation (directsql/datanucleus)
- If we want to make sure that the size of the Thrift request is not too
much we might have another configuration on client size like we do with
`BATCH_RETRIEVE_MAX`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609400)
Time Spent: 0.5h (was: 20m)
> Update column stat throws NPE if direct sql is disabled
> ---
>
> Key: HIVE-25225
> URL: https://issues.apache.org/jira/browse/HIVE-25225
> Project: Hive
> Issue Type: Sub-task
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> In case direct sql is disabled, the MetaStoreDirectSql object is not
> initialised and thats causing NPE.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25225) Update column stat throws NPE if direct sql is disabled
[
https://issues.apache.org/jira/browse/HIVE-25225?focusedWorklogId=609384=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609384
]
ASF GitHub Bot logged work on HIVE-25225:
-
Author: ASF GitHub Bot
Created on: 09/Jun/21 21:00
Start Date: 09/Jun/21 21:00
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #2376:
URL: https://github.com/apache/hive/pull/2376#discussion_r648681379
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java
##
@@ -7006,16 +7006,40 @@ private boolean updatePartitonColStatsInternal(Table
tbl, ColumnStatistics colSt
+ " part=" + csd.getPartName());
boolean ret = false;
+
+Map parameters;
+List partVals;
+boolean committed = false;
+getMS().openTransaction();
+
try {
if (tbl == null) {
tbl = getTable(catName, dbName, tableName);
}
- ret = updatePartitionColStatsInBatch(tbl,
Collections.singletonMap(csd.getPartName(), colStats),
- validWriteIds, writeId);
+ partVals = getPartValsFromName(tbl, csd.getPartName());
+ parameters = getMS().updatePartitionColumnStatistics(colStats, partVals,
validWriteIds, writeId);
+ if (parameters != null) {
+if (transactionalListeners != null &&
!transactionalListeners.isEmpty()) {
+ MetaStoreListenerNotifier.notifyEvent(transactionalListeners,
+ EventType.UPDATE_PARTITION_COLUMN_STAT,
+ new UpdatePartitionColumnStatEvent(colStats, partVals,
parameters, tbl,
+ writeId, this));
+}
+if (!listeners.isEmpty()) {
Review comment:
We usually put the non-transactional listeners after the
`commitTransaction` so if there is an error there then it will not roll back
the changes done in the transaction
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609384)
Time Spent: 20m (was: 10m)
> Update column stat throws NPE if direct sql is disabled
> ---
>
> Key: HIVE-25225
> URL: https://issues.apache.org/jira/browse/HIVE-25225
> Project: Hive
> Issue Type: Sub-task
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In case direct sql is disabled, the MetaStoreDirectSql object is not
> initialised and thats causing NPE.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-25191) Modernize Hive Thrift CLI Service Protocol
[ https://issues.apache.org/jira/browse/HIVE-25191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline resolved HIVE-25191. - Resolution: Invalid > Modernize Hive Thrift CLI Service Protocol > -- > > Key: HIVE-25191 > URL: https://issues.apache.org/jira/browse/HIVE-25191 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Unnecessary errors are occurring with the advent of proxy use such as > Gateways between the Hive client and Hive Server 2. Query failures can be due > to arbitrary proxy timeouts. This proposal avoids the timeouts by changing > the protocol to do regular polling. Currently, the Hive client uses one > request for the query compile request. Long query compile times make those > requests vulnerable to the arbitrary proxy timeouts. > Another issue is Hive Server 2 sometimes does not notice the client has > failed or has lost interest in a potentially long running query. This causes > Hive locks and Big Data query resources to be held unnecessarily. The > assumption is the client issues a cancel query request when it gets an error. > This assumption does not always hold. If the proxy returned an error itself, > that proxy may reject the subsequent cancel request, too. And, if the client > is killed or the network is down, the client cannot complete a cancel > request. The proposed solution here is for Hive Server 2 to watch that the > client is sending regular polling requests for status. If a client ceases > those requests, then Hive Server 2 will cancel the query. > Hive owns the JDBC path (i.e. HiveDriver). The ODBC path may be more > challenging because vendors provide ODBC drivers and Hive does not own the > ODBC protocol. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25191) Modernize Hive Thrift CLI Service Protocol
[ https://issues.apache.org/jira/browse/HIVE-25191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360326#comment-17360326 ] Matt McCline commented on HIVE-25191: - Closing this Jira for more specific JIRAs: [HIVE-25227] Thrift CLI Service Protocol: Eliminate long compile requests than proxies can timeout - ASF JIRA (apache.org) [HIVE-25228] Thrift CLI Service Protocol: Watch for lack of interest by client and kill queries faster - ASF JIRA (apache.org) > Modernize Hive Thrift CLI Service Protocol > -- > > Key: HIVE-25191 > URL: https://issues.apache.org/jira/browse/HIVE-25191 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Unnecessary errors are occurring with the advent of proxy use such as > Gateways between the Hive client and Hive Server 2. Query failures can be due > to arbitrary proxy timeouts. This proposal avoids the timeouts by changing > the protocol to do regular polling. Currently, the Hive client uses one > request for the query compile request. Long query compile times make those > requests vulnerable to the arbitrary proxy timeouts. > Another issue is Hive Server 2 sometimes does not notice the client has > failed or has lost interest in a potentially long running query. This causes > Hive locks and Big Data query resources to be held unnecessarily. The > assumption is the client issues a cancel query request when it gets an error. > This assumption does not always hold. If the proxy returned an error itself, > that proxy may reject the subsequent cancel request, too. And, if the client > is killed or the network is down, the client cannot complete a cancel > request. The proposed solution here is for Hive Server 2 to watch that the > client is sending regular polling requests for status. If a client ceases > those requests, then Hive Server 2 will cancel the query. > Hive owns the JDBC path (i.e. HiveDriver). The ODBC path may be more > challenging because vendors provide ODBC drivers and Hive does not own the > ODBC protocol. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25228) Thrift CLI Service Protocol: Watch for lack of interest by client and kill queries faster
[ https://issues.apache.org/jira/browse/HIVE-25228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25228: --- > Thrift CLI Service Protocol: Watch for lack of interest by client and kill > queries faster > - > > Key: HIVE-25228 > URL: https://issues.apache.org/jira/browse/HIVE-25228 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > CONSIDER: Have Hive Server 2 monitor operations (queries) for continuing > client interest. If a client does not ask for status every 15 seconds, then > automatically kill a query and release its txn locks and job resources. > > Users will experience queries cleaning up much faster (15 to 30 seconds > instead of minutes and possibly many minutes) when client communication is > lost. Cleaning up those queries prevents other queries from being blocked on > EXCLUSIVE txn locks and blocking of scheduling of their queries including > retries of the original query. Today, users can get timeouts when they retry > a query that got a connection error causing understandably upset users. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25227) Thrift CLI Service Protocol: Eliminate long compile requests than proxies can timeout
[ https://issues.apache.org/jira/browse/HIVE-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25227: Description: CONSIDER: Avoid proxy (GW) timeouts on long Hive query compiles. Use request to start the operation; then poll for status like we do for execution. E.g. a HTTP Gateway can have arbitrary timeouts and cause queries with long compile times to fail. was:CONSIDER: Avoid proxy (GW) timeouts on long Hive query compiles. Use request to start the operation; then poll for status like we do for execution. > Thrift CLI Service Protocol: Eliminate long compile requests than proxies can > timeout > - > > Key: HIVE-25227 > URL: https://issues.apache.org/jira/browse/HIVE-25227 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > CONSIDER: Avoid proxy (GW) timeouts on long Hive query compiles. Use request > to start the operation; then poll for status like we do for execution. > E.g. a HTTP Gateway can have arbitrary timeouts and cause queries with long > compile times to fail. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25227) Thrift CLI Service Protocol: Eliminate long compile requests than proxies can timeout
[ https://issues.apache.org/jira/browse/HIVE-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25227: --- > Thrift CLI Service Protocol: Eliminate long compile requests than proxies can > timeout > - > > Key: HIVE-25227 > URL: https://issues.apache.org/jira/browse/HIVE-25227 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > CONSIDER: Avoid proxy (GW) timeouts on long Hive query compiles. Use request > to start the operation; then poll for status like we do for execution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25226) Hive changes 'storage_handler' for existing Iceberg table when hive.engine.enabled is false
[
https://issues.apache.org/jira/browse/HIVE-25226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod reassigned HIVE-25226:
-
Assignee: Marton Bod
> Hive changes 'storage_handler' for existing Iceberg table when
> hive.engine.enabled is false
> ---
>
> Key: HIVE-25226
> URL: https://issues.apache.org/jira/browse/HIVE-25226
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltán Borók-Nagy
>Assignee: Marton Bod
>Priority: Major
> Labels: iceberg
>
> If Hive writes to an existing Iceberg table but property
> 'hive.engine.enabled' is not set, then Hive rewrites the table metadata with
> different SerDe/Input/Output format than it had before.
> E.g. there's an existing table with the following metadata:
> {noformat}
> storage_handler |
> org.apache.iceberg.mr.hive.HiveIcebergStorageHandler
> | SerDe Library: | org.apache.iceberg.mr.hive.HiveIcebergSerDe | NULL |
> | InputFormat: | org.apache.iceberg.mr.hive.HiveIcebergInputFormat | NULL |
> | OutputFormat: | org.apache.iceberg.mr.hive.HiveIcebergOutputFormat | NULL |
> {noformat}
> Now when Hive inserts to this table it clears 'storage_handler' and rewrites
> the rest:
> {noformat}
> | SerDe Library:|
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL
> |
> | InputFormat: | org.apache.hadoop.mapred.FileInputFormat
>| NULL |
> | OutputFormat: | org.apache.hadoop.mapred.FileOutputFormat
>| NULL |
> {noformat}
> This means the table becomes unreadable:
> {noformat}
> Error: java.io.IOException: java.io.IOException: Cannot create an instance of
> InputFormat class org.apache.hadoop.mapred.FileInputFormat as specified in
> mapredWork! (state=,code=0)
> {noformat}
> I think Hive should always set 'hive.engine.enabled' for Iceberg.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HIVE-25214) Add hive authorization support for Data connectors.
[ https://issues.apache.org/jira/browse/HIVE-25214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-25214 started by Dantong Dong. --- > Add hive authorization support for Data connectors. > --- > > Key: HIVE-25214 > URL: https://issues.apache.org/jira/browse/HIVE-25214 > Project: Hive > Issue Type: Sub-task >Reporter: Naveen Gangam >Assignee: Dantong Dong >Priority: Major > > We need to add authorization support for data connectors in hive. The default > behavior should be > 1) Connectors can be create/dropped by users in admin role. > 2) Connectors have READ and WRITE permissions. > * READ permissions are required to fetch a connector object or fetch all > connector names. So to create a REMOTE database using a connector, users will > need READ permission on the connector. DDL queries like "show connectors" and > "describe " will check for read access on the connector as well. > * WRITE permissions are required to alter/drop a connector. DDL queries > like "alter connector" and "drop connector" will need WRITE access on the > connector. > Adding this support, Ranger can integrate with this. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-25213) Implement List getTables() for existing connectors.
[ https://issues.apache.org/jira/browse/HIVE-25213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-25213 started by Dantong Dong. --- > Implement List getTables() for existing connectors. > -- > > Key: HIVE-25213 > URL: https://issues.apache.org/jira/browse/HIVE-25213 > Project: Hive > Issue Type: Sub-task >Reporter: Naveen Gangam >Assignee: Dantong Dong >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > In the initial implementation, connector providers do not implement the > getTables(string pattern) spi. We had deferred it for later. Only > getTableNames() and getTable() were implemented. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23633) Metastore some JDO query objects do not close properly
[
https://issues.apache.org/jira/browse/HIVE-23633?focusedWorklogId=609266=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609266
]
ASF GitHub Bot logged work on HIVE-23633:
-
Author: ASF GitHub Bot
Created on: 09/Jun/21 17:13
Start Date: 09/Jun/21 17:13
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #2344:
URL: https://github.com/apache/hive/pull/2344#discussion_r648509685
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
##
@@ -914,6 +914,7 @@ private boolean isViewTable(String catName, String dbName,
String tblName) throw
long queryTime = doTrace ? System.nanoTime() : 0;
MetastoreDirectSqlUtils.timingTrace(doTrace, queryText, start, queryTime);
if (sqlResult.isEmpty()) {
+ query.closeAll();
return Collections.emptyList(); // no partitions, bail early.
}
Review comment:
Can you just remove this entire block? This will allow for a single
return statement at the bottom and also allows the cleanup to happen.
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
##
@@ -1453,6 +1454,7 @@ public ColumnStatistics getTableStats(final String
catName, final String dbName,
};
List list = Batchable.runBatched(batchSize, colNames, b);
if (list.isEmpty()) {
+ b.closeAllQueries();
return null;
}
Review comment:
Same here
```java
List list = Batchable.runBatched(batchSize, colNames, b);
final ColumnStatistics result;
if (list.isEmpty()) {
result = null;
} else {
ColumnStatisticsDesc csd = new ColumnStatisticsDesc(true, dbName,
tableName);
csd.setCatName(catName);
result = makeColumnStats(list, csd, 0, engine);
}
b.closeAllQueries();
return result;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609266)
Time Spent: 4h 50m (was: 4h 40m)
> Metastore some JDO query objects do not close properly
> --
>
> Key: HIVE-23633
> URL: https://issues.apache.org/jira/browse/HIVE-23633
> Project: Hive
> Issue Type: Bug
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-23633.01.patch
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> After patched [HIVE-10895|https://issues.apache.org/jira/browse/HIVE-10895],
> The metastore still has seen a memory leak on db resources: many
> StatementImpls left unclosed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25086) Create Ranger Deny Policy for replication db in all cases if hive.repl.ranger.target.deny.policy is set to true.
[
https://issues.apache.org/jira/browse/HIVE-25086?focusedWorklogId=609265=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609265
]
ASF GitHub Bot logged work on HIVE-25086:
-
Author: ASF GitHub Bot
Created on: 09/Jun/21 17:10
Start Date: 09/Jun/21 17:10
Worklog Time Spent: 10m
Work Description: aasha commented on a change in pull request #2240:
URL: https://github.com/apache/hive/pull/2240#discussion_r648504479
##
File path: common/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java
##
@@ -627,7 +627,9 @@
REPL_INVALID_ARGUMENTS(40012, "Invalid arguments error : {0}.", true),
REPL_INVALID_ALTER_TABLE(40013, "{0}Unable to alter table{1}", true),
REPL_PERMISSION_DENIED(40014,
"{0}org.apache.hadoop.security.AccessControlException{1}", true),
- REPL_DISTCP_SNAPSHOT_EXCEPTION(40015, "SNAPSHOT_ERROR", true)
+ REPL_DISTCP_SNAPSHOT_EXCEPTION(40015, "SNAPSHOT_ERROR", true),
+ RANGER_AUTHORISATION_FAILED(40016, "Authorisation Failure while
communicating to Ranger admin", true),
Review comment:
type : authorization
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ranger/RangerRestClientImpl.java
##
@@ -202,12 +238,14 @@ public RangerExportPolicyList
importRangerPolicies(RangerExportPolicyList ranger
String finalUrl = getRangerImportUrl(baseUrl, dbName);
LOG.debug("URL to import policies on target Ranger: {}", finalUrl);
Retryable retryable = Retryable.builder()
- .withHiveConf(hiveConf)
- .withRetryOnException(Exception.class).build();
+ .withHiveConf(hiveConf).withFailOnException(RuntimeException.class)
+ .withRetryOnException(SemanticException.class).build();
try {
return retryable.executeCallable(() ->
importRangerPoliciesPlain(jsonRangerExportPolicyList,
rangerPoliciesJsonFileName,
serviceMapJsonFileName, jsonServiceMap, finalUrl,
rangerExportPolicyList, hiveConf));
+} catch (RuntimeException e) {
Review comment:
you are catching a runtime exception and throwing a semantic exception.
why are you doing that?
##
File path: common/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java
##
@@ -627,7 +627,9 @@
REPL_INVALID_ARGUMENTS(40012, "Invalid arguments error : {0}.", true),
REPL_INVALID_ALTER_TABLE(40013, "{0}Unable to alter table{1}", true),
REPL_PERMISSION_DENIED(40014,
"{0}org.apache.hadoop.security.AccessControlException{1}", true),
- REPL_DISTCP_SNAPSHOT_EXCEPTION(40015, "SNAPSHOT_ERROR", true)
+ REPL_DISTCP_SNAPSHOT_EXCEPTION(40015, "SNAPSHOT_ERROR", true),
+ RANGER_AUTHORISATION_FAILED(40016, "Authorisation Failure while
communicating to Ranger admin", true),
Review comment:
error code is same. it should be different. Check if any of the existing
errror codes that can be reused.
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ranger/RangerRestClientImpl.java
##
@@ -90,11 +86,13 @@ public RangerExportPolicyList exportRangerPolicies(String
sourceRangerEndpoint,
"cannot be empty", ReplUtils.REPL_RANGER_SERVICE));
}
Retryable retryable = Retryable.builder()
- .withHiveConf(hiveConf)
- .withRetryOnException(Exception.class).build();
+ .withHiveConf(hiveConf).withFailOnException(RuntimeException.class)
+ .withRetryOnException(URISyntaxException.class).build();
Review comment:
what is the use of retry in case of URISyntaxException?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609265)
Time Spent: 4h (was: 3h 50m)
> Create Ranger Deny Policy for replication db in all cases if
> hive.repl.ranger.target.deny.policy is set to true.
>
>
> Key: HIVE-25086
> URL: https://issues.apache.org/jira/browse/HIVE-25086
> Project: Hive
> Issue Type: Improvement
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23571) [CachedStore] Add ValidWriteIdList to SharedCache.TableWrapper
[
https://issues.apache.org/jira/browse/HIVE-23571?focusedWorklogId=609252=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609252
]
ASF GitHub Bot logged work on HIVE-23571:
-
Author: ASF GitHub Bot
Created on: 09/Jun/21 16:59
Start Date: 09/Jun/21 16:59
Worklog Time Spent: 10m
Work Description: sankarh commented on a change in pull request #2128:
URL: https://github.com/apache/hive/pull/2128#discussion_r648501960
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java
##
@@ -2476,6 +2498,25 @@ public boolean isTableConstraintValid(String catName,
String dbName, String tblN
return isValid;
}
+ public boolean isTableCacheStale(String catName, String dbName, String
tblName, String validWriteIdList) {
+boolean isStale = true;
+
+if (StringUtils.isEmpty(validWriteIdList))
+ return isStale;
Review comment:
Background thread is the current logic to refresh the cache at regular
intervals. Synchronized cache work shouldn't rely on that as it should provide
strict consistency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609252)
Time Spent: 4h 10m (was: 4h)
> [CachedStore] Add ValidWriteIdList to SharedCache.TableWrapper
> --
>
> Key: HIVE-23571
> URL: https://issues.apache.org/jira/browse/HIVE-23571
> Project: Hive
> Issue Type: Sub-task
>Reporter: Kishen Das
>Assignee: Ashish Sharma
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Add ValidWriteIdList to SharedCache.TableWrapper. This would be used in
> deciding whether a given read request can be served from the cache or we have
> to reload it from the backing database.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23571) [CachedStore] Add ValidWriteIdList to SharedCache.TableWrapper
[
https://issues.apache.org/jira/browse/HIVE-23571?focusedWorklogId=609247=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609247
]
ASF GitHub Bot logged work on HIVE-23571:
-
Author: ASF GitHub Bot
Created on: 09/Jun/21 16:54
Start Date: 09/Jun/21 16:54
Worklog Time Spent: 10m
Work Description: sankarh commented on a change in pull request #2128:
URL: https://github.com/apache/hive/pull/2128#discussion_r648497223
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
##
@@ -918,6 +938,29 @@ private void updateAllTableConstraints(RawStore rawStore,
String catName, String
}
}
+private void updateValidWriteIdList(TxnStore txnStore, String catName,
String dbName, String tblName) {
+ catName = StringUtils.normalizeIdentifier(catName);
+ dbName = StringUtils.normalizeIdentifier(dbName);
+ tblName = StringUtils.normalizeIdentifier(tblName);
+ LOG.debug("CachedStore: updating cached table validWriteIdlist for
catalog: {}, database: {}, table: {}", catName,
+ dbName, tblName);
+ ValidWriteIdList validWriteIdList = null;
+ try {
+Deadline.startTimer("getValidWriteIds");
+validWriteIdList =
TxnCommonUtils.createValidReaderWriteIdList(txnStore.getValidWriteIds(
Review comment:
AFAIK, we are targeting strict consistency with "Synchronized metastore
cache" work. Eventual consistency is supported even today with regular refresh
of cache with the interval of 1 min.
The problem is, it returns from cache even if it is invalid which is not
right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609247)
Time Spent: 4h (was: 3h 50m)
> [CachedStore] Add ValidWriteIdList to SharedCache.TableWrapper
> --
>
> Key: HIVE-23571
> URL: https://issues.apache.org/jira/browse/HIVE-23571
> Project: Hive
> Issue Type: Sub-task
>Reporter: Kishen Das
>Assignee: Ashish Sharma
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
> Add ValidWriteIdList to SharedCache.TableWrapper. This would be used in
> deciding whether a given read request can be served from the cache or we have
> to reload it from the backing database.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23571) [CachedStore] Add ValidWriteIdList to SharedCache.TableWrapper
[
https://issues.apache.org/jira/browse/HIVE-23571?focusedWorklogId=609246=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609246
]
ASF GitHub Bot logged work on HIVE-23571:
-
Author: ASF GitHub Bot
Created on: 09/Jun/21 16:52
Start Date: 09/Jun/21 16:52
Worklog Time Spent: 10m
Work Description: sankarh commented on a change in pull request #2128:
URL: https://github.com/apache/hive/pull/2128#discussion_r648497223
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
##
@@ -918,6 +938,29 @@ private void updateAllTableConstraints(RawStore rawStore,
String catName, String
}
}
+private void updateValidWriteIdList(TxnStore txnStore, String catName,
String dbName, String tblName) {
+ catName = StringUtils.normalizeIdentifier(catName);
+ dbName = StringUtils.normalizeIdentifier(dbName);
+ tblName = StringUtils.normalizeIdentifier(tblName);
+ LOG.debug("CachedStore: updating cached table validWriteIdlist for
catalog: {}, database: {}, table: {}", catName,
+ dbName, tblName);
+ ValidWriteIdList validWriteIdList = null;
+ try {
+Deadline.startTimer("getValidWriteIds");
+validWriteIdList =
TxnCommonUtils.createValidReaderWriteIdList(txnStore.getValidWriteIds(
Review comment:
AFAIK, we are targeting strict consistency with "Synchronized metastore
cache" work. Eventual consistency is supported even today with regular refresh
of cache with the interval of 1 min. Pls recheck this point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609246)
Time Spent: 3h 50m (was: 3h 40m)
> [CachedStore] Add ValidWriteIdList to SharedCache.TableWrapper
> --
>
> Key: HIVE-23571
> URL: https://issues.apache.org/jira/browse/HIVE-23571
> Project: Hive
> Issue Type: Sub-task
>Reporter: Kishen Das
>Assignee: Ashish Sharma
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Add ValidWriteIdList to SharedCache.TableWrapper. This would be used in
> deciding whether a given read request can be served from the cache or we have
> to reload it from the backing database.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25225) Update column stat throws NPE if direct sql is disabled
[ https://issues.apache.org/jira/browse/HIVE-25225?focusedWorklogId=609231=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609231 ] ASF GitHub Bot logged work on HIVE-25225: - Author: ASF GitHub Bot Created on: 09/Jun/21 16:35 Start Date: 09/Jun/21 16:35 Worklog Time Spent: 10m Work Description: maheshk114 opened a new pull request #2376: URL: https://github.com/apache/hive/pull/2376 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 609231) Remaining Estimate: 0h Time Spent: 10m > Update column stat throws NPE if direct sql is disabled > --- > > Key: HIVE-25225 > URL: https://issues.apache.org/jira/browse/HIVE-25225 > Project: Hive > Issue Type: Sub-task >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > In case direct sql is disabled, the MetaStoreDirectSql object is not > initialised and thats causing NPE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25225) Update column stat throws NPE if direct sql is disabled
[ https://issues.apache.org/jira/browse/HIVE-25225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25225: -- Labels: pull-request-available (was: ) > Update column stat throws NPE if direct sql is disabled > --- > > Key: HIVE-25225 > URL: https://issues.apache.org/jira/browse/HIVE-25225 > Project: Hive > Issue Type: Sub-task >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > In case direct sql is disabled, the MetaStoreDirectSql object is not > initialised and thats causing NPE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25225) Update column stat throws NPE if direct sql is disabled
[ https://issues.apache.org/jira/browse/HIVE-25225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera reassigned HIVE-25225: -- > Update column stat throws NPE if direct sql is disabled > --- > > Key: HIVE-25225 > URL: https://issues.apache.org/jira/browse/HIVE-25225 > Project: Hive > Issue Type: Sub-task >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > > In case direct sql is disabled, the MetaStoreDirectSql object is not > initialised and thats causing NPE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25098) [CVE-2020-13949] Upgrade thrift from 0.13.0 to 0.14.1
[ https://issues.apache.org/jira/browse/HIVE-25098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan resolved HIVE-25098. - Fix Version/s: 4.0.0 Target Version/s: 4.0.0 Resolution: Fixed Thanks [~ashish-kumar-sharma] for the patch and [~mattmccline] for the review! > [CVE-2020-13949] Upgrade thrift from 0.13.0 to 0.14.1 > - > > Key: HIVE-25098 > URL: https://issues.apache.org/jira/browse/HIVE-25098 > Project: Hive > Issue Type: Task >Affects Versions: All Versions >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrading thrift from 0.13.0 to 0.14.0 due to > https://nvd.nist.gov/vuln/detail/CVE-2020-13949 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25098) [CVE-2020-13949] Upgrade thrift from 0.13.0 to 0.14.1
[ https://issues.apache.org/jira/browse/HIVE-25098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25098: -- Labels: pull-request-available (was: ) > [CVE-2020-13949] Upgrade thrift from 0.13.0 to 0.14.1 > - > > Key: HIVE-25098 > URL: https://issues.apache.org/jira/browse/HIVE-25098 > Project: Hive > Issue Type: Task >Affects Versions: All Versions >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Upgrading thrift from 0.13.0 to 0.14.0 due to > https://nvd.nist.gov/vuln/detail/CVE-2020-13949 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25098) [CVE-2020-13949] Upgrade thrift from 0.13.0 to 0.14.1
[ https://issues.apache.org/jira/browse/HIVE-25098?focusedWorklogId=609224=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609224 ] ASF GitHub Bot logged work on HIVE-25098: - Author: ASF GitHub Bot Created on: 09/Jun/21 16:16 Start Date: 09/Jun/21 16:16 Worklog Time Spent: 10m Work Description: sankarh merged pull request #2330: URL: https://github.com/apache/hive/pull/2330 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 609224) Remaining Estimate: 0h Time Spent: 10m > [CVE-2020-13949] Upgrade thrift from 0.13.0 to 0.14.1 > - > > Key: HIVE-25098 > URL: https://issues.apache.org/jira/browse/HIVE-25098 > Project: Hive > Issue Type: Task >Affects Versions: All Versions >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Upgrading thrift from 0.13.0 to 0.14.0 due to > https://nvd.nist.gov/vuln/detail/CVE-2020-13949 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25144) Remove RetryingHMSHandler
[
https://issues.apache.org/jira/browse/HIVE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-25144:
--
Description:
I have recently seen an issue where a Hive {{CREATE TABLE}} method fails with
{{AlreadyExistsException}} even though the table does absolutely not exist.
I believe the issue is that there is a timeout/transient error with HMS and the
backend database. If the client submits a {{create table}} request to HMS, and
the request to the backend database fails, the request to the DB may be retried
even though the HMS has lost state of the DB. When the HMS Handler "retry"
functionality kicks in, the second time the request is submitted, the table
looks like it already exists.
If something goes wrong during a HMS CREATE operation, we do not know the state
of the operation and therefore it should just fail.
It would certainly be more transparent to the end-user what is going on. An
{{AlreadyExistsException}} is confusing.
was:
I have recently seen an issue where a Hive {{CREATE TABLE}} method fails with
{{AlreadyExistsException}} even though the table does absolutely not exist.
I believe the issue is there there is a timeout/transient error with HMS and
the backend database. So, the client submits the request to HMS, and the
request does eventually succeed, but only after the connection to the client
connects. Therefore, when the HMS Client "retry" functionality kicks it, the
second time around, the table looks like it already exists.
If something goes wrong during a HMS CREATE operation, we do not know the state
of the operation and therefore it should just fail.
It would certainly be more transparent to the end-user what is going on. An
{{AlreadyExistsException}} is confusing.
> Remove RetryingHMSHandler
> -
>
> Key: HIVE-25144
> URL: https://issues.apache.org/jira/browse/HIVE-25144
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> I have recently seen an issue where a Hive {{CREATE TABLE}} method fails with
> {{AlreadyExistsException}} even though the table does absolutely not exist.
>
> I believe the issue is that there is a timeout/transient error with HMS and
> the backend database. If the client submits a {{create table}} request to
> HMS, and the request to the backend database fails, the request to the DB may
> be retried even though the HMS has lost state of the DB. When the HMS
> Handler "retry" functionality kicks in, the second time the request is
> submitted, the table looks like it already exists.
>
> If something goes wrong during a HMS CREATE operation, we do not know the
> state of the operation and therefore it should just fail.
>
> It would certainly be more transparent to the end-user what is going on. An
> {{AlreadyExistsException}} is confusing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25144) Remove RetryingHMSHandler
[
https://issues.apache.org/jira/browse/HIVE-25144?focusedWorklogId=609165=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609165
]
ASF GitHub Bot logged work on HIVE-25144:
-
Author: ASF GitHub Bot
Created on: 09/Jun/21 14:21
Start Date: 09/Jun/21 14:21
Worklog Time Spent: 10m
Work Description: belugabehr closed pull request #2303:
URL: https://github.com/apache/hive/pull/2303
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609165)
Time Spent: 1h (was: 50m)
> Remove RetryingHMSHandler
> -
>
> Key: HIVE-25144
> URL: https://issues.apache.org/jira/browse/HIVE-25144
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> I have recently seen an issue where a Hive {{CREATE TABLE}} method fails with
> {{AlreadyExistsException}} even though the table does absolutely not exist.
>
> I believe the issue is there there is a timeout/transient error with HMS and
> the backend database. So, the client submits the request to HMS, and the
> request does eventually succeed, but only after the connection to the client
> connects. Therefore, when the HMS Client "retry" functionality kicks it, the
> second time around, the table looks like it already exists.
>
> If something goes wrong during a HMS CREATE operation, we do not know the
> state of the operation and therefore it should just fail.
>
> It would certainly be more transparent to the end-user what is going on. An
> {{AlreadyExistsException}} is confusing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-25144) Remove RetryingHMSHandler
[
https://issues.apache.org/jira/browse/HIVE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360102#comment-17360102
]
David Mollitor edited comment on HIVE-25144 at 6/9/21, 2:08 PM:
OK, I've been looking at this all wrong. This is all happening internal to
HMS. I thought it was the HMS-client retrying, but that's not what this is.
This is a re-try capability ON TOP of the client re-try. This should be
removed. This is very confusing, hides transient errors, and looking at the
code, seems very error-prone and fragile as the project changes. If an HMS
error occurs, it should report back to the client and let the client decide how
it wants to handle the error. For example, the client may want to re-try the
request with a different instance of HMS instead of waiting for this one to try
N times and fail.
was (Author: belugabehr):
OK, I've been looking at this all wrong. This is all happening internal to
HMS. I thought it was the HMS-client retrying. This should be removed. This
is very confusing, hides transient errors, and looking at the code, seems very
error-prone and fragile as the project changes. If an HMS error occurs, it
should report back to the client and let the client decide how it wants to
handle the error. For example, the client may want to re-try the request with
a different instance of HMS instead of waiting for this one to try N times and
fail.
> Remove RetryingHMSHandler
> -
>
> Key: HIVE-25144
> URL: https://issues.apache.org/jira/browse/HIVE-25144
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> I have recently seen an issue where a Hive {{CREATE TABLE}} method fails with
> {{AlreadyExistsException}} even though the table does absolutely not exist.
>
> I believe the issue is there there is a timeout/transient error with HMS and
> the backend database. So, the client submits the request to HMS, and the
> request does eventually succeed, but only after the connection to the client
> connects. Therefore, when the HMS Client "retry" functionality kicks it, the
> second time around, the table looks like it already exists.
>
> If something goes wrong during a HMS CREATE operation, we do not know the
> state of the operation and therefore it should just fail.
>
> It would certainly be more transparent to the end-user what is going on. An
> {{AlreadyExistsException}} is confusing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-25144) Remove RetryingHMSHandler
[
https://issues.apache.org/jira/browse/HIVE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360102#comment-17360102
]
David Mollitor commented on HIVE-25144:
---
OK, I've been looking at this all wrong. This is all happening internal to
HMS. I thought it was the HMS-client retrying. This should be removed. This
is very confusing, hides transient errors, and looking at the code, seems very
error-prone and fragile as the project changes. If an HMS error occurs, it
should report back to the client and let the client decide how it wants to
handle the error. For example, the client may want to re-try the request with
a different instance of HMS instead of waiting for this one to try N times and
fail.
> Remove RetryingHMSHandler
> -
>
> Key: HIVE-25144
> URL: https://issues.apache.org/jira/browse/HIVE-25144
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> I have recently seen an issue where a Hive {{CREATE TABLE}} method fails with
> {{AlreadyExistsException}} even though the table does absolutely not exist.
>
> I believe the issue is there there is a timeout/transient error with HMS and
> the backend database. So, the client submits the request to HMS, and the
> request does eventually succeed, but only after the connection to the client
> connects. Therefore, when the HMS Client "retry" functionality kicks it, the
> second time around, the table looks like it already exists.
>
> If something goes wrong during a HMS CREATE operation, we do not know the
> state of the operation and therefore it should just fail.
>
> It would certainly be more transparent to the end-user what is going on. An
> {{AlreadyExistsException}} is confusing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25144) Remove RetryingHMSHandler
[
https://issues.apache.org/jira/browse/HIVE-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-25144:
--
Summary: Remove RetryingHMSHandler (was: Add NoReconnect Annotation to
CreateXXX Methods With AlreadyExistsException)
> Remove RetryingHMSHandler
> -
>
> Key: HIVE-25144
> URL: https://issues.apache.org/jira/browse/HIVE-25144
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> I have recently seen an issue where a Hive {{CREATE TABLE}} method fails with
> {{AlreadyExistsException}} even though the table does absolutely not exist.
>
> I believe the issue is there there is a timeout/transient error with HMS and
> the backend database. So, the client submits the request to HMS, and the
> request does eventually succeed, but only after the connection to the client
> connects. Therefore, when the HMS Client "retry" functionality kicks it, the
> second time around, the table looks like it already exists.
>
> If something goes wrong during a HMS CREATE operation, we do not know the
> state of the operation and therefore it should just fail.
>
> It would certainly be more transparent to the end-user what is going on. An
> {{AlreadyExistsException}} is confusing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25144) Add NoReconnect Annotation to CreateXXX Methods With AlreadyExistsException
[
https://issues.apache.org/jira/browse/HIVE-25144?focusedWorklogId=609147=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609147
]
ASF GitHub Bot logged work on HIVE-25144:
-
Author: ASF GitHub Bot
Created on: 09/Jun/21 13:56
Start Date: 09/Jun/21 13:56
Worklog Time Spent: 10m
Work Description: belugabehr commented on pull request #2303:
URL: https://github.com/apache/hive/pull/2303#issuecomment-857716299
@nrg4878 Thanks for the gut-check. Ignore everything I've said thus far. I
was confusing things. I thought it was the client that was re-trying. It's
actually the HMS Handler. Ick. Ugh. I need to dig deeper on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609147)
Time Spent: 50m (was: 40m)
> Add NoReconnect Annotation to CreateXXX Methods With AlreadyExistsException
> ---
>
> Key: HIVE-25144
> URL: https://issues.apache.org/jira/browse/HIVE-25144
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> I have recently seen an issue where a Hive {{CREATE TABLE}} method fails with
> {{AlreadyExistsException}} even though the table does absolutely not exist.
>
> I believe the issue is there there is a timeout/transient error with HMS and
> the backend database. So, the client submits the request to HMS, and the
> request does eventually succeed, but only after the connection to the client
> connects. Therefore, when the HMS Client "retry" functionality kicks it, the
> second time around, the table looks like it already exists.
>
> If something goes wrong during a HMS CREATE operation, we do not know the
> state of the operation and therefore it should just fail.
>
> It would certainly be more transparent to the end-user what is going on. An
> {{AlreadyExistsException}} is confusing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25223) Select with limit returns no rows on non native table
[
https://issues.apache.org/jira/browse/HIVE-25223?focusedWorklogId=609144=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609144
]
ASF GitHub Bot logged work on HIVE-25223:
-
Author: ASF GitHub Bot
Created on: 09/Jun/21 13:41
Start Date: 09/Jun/21 13:41
Worklog Time Spent: 10m
Work Description: zeroflag opened a new pull request #2375:
URL: https://github.com/apache/hive/pull/2375
GlobalLimitOptimizer doesn't work well with non native tables. The table
directory is always empty since the data is not managed by hive (but hbase in
this case). The optimizer scans the directory and sets the file list to an
empty list. In the end the select + limit returns no rows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609144)
Remaining Estimate: 0h
Time Spent: 10m
> Select with limit returns no rows on non native table
> -
>
> Key: HIVE-25223
> URL: https://issues.apache.org/jira/browse/HIVE-25223
> Project: Hive
> Issue Type: Bug
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Str:
> {code:java}
> CREATE EXTERNAL TABLE hht (key string, value int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "hht", "hbase.mapred.output.outputtable"
> = "hht");
> insert into hht select uuid(), cast((rand() * 100) as int);
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> set hive.fetch.task.conversion=none;
> select * from hht limit 10;
> +--++
> | hht.key | hht.value |
> +--++
> +--++
> No rows selected (5.22 seconds) {code}
>
> This is caused by GlobalLimitOptimizer. The table directory is always empty
> with a non native table since the data is not managed by hive (but hbase in
> this case).
> The optimizer scans the directory and sets the file list to an empty list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25223) Select with limit returns no rows on non native table
[
https://issues.apache.org/jira/browse/HIVE-25223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-25223:
--
Labels: pull-request-available (was: )
> Select with limit returns no rows on non native table
> -
>
> Key: HIVE-25223
> URL: https://issues.apache.org/jira/browse/HIVE-25223
> Project: Hive
> Issue Type: Bug
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Str:
> {code:java}
> CREATE EXTERNAL TABLE hht (key string, value int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "hht", "hbase.mapred.output.outputtable"
> = "hht");
> insert into hht select uuid(), cast((rand() * 100) as int);
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> set hive.fetch.task.conversion=none;
> select * from hht limit 10;
> +--++
> | hht.key | hht.value |
> +--++
> +--++
> No rows selected (5.22 seconds) {code}
>
> This is caused by GlobalLimitOptimizer. The table directory is always empty
> with a non native table since the data is not managed by hive (but hbase in
> this case).
> The optimizer scans the directory and sets the file list to an empty list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25081) Put metrics collection behind a feature flag
[
https://issues.apache.org/jira/browse/HIVE-25081?focusedWorklogId=609141=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609141
]
ASF GitHub Bot logged work on HIVE-25081:
-
Author: ASF GitHub Bot
Created on: 09/Jun/21 13:28
Start Date: 09/Jun/21 13:28
Worklog Time Spent: 10m
Work Description: asinkovits commented on a change in pull request #2332:
URL: https://github.com/apache/hive/pull/2332#discussion_r648285067
##
File path: ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java
##
@@ -111,7 +111,7 @@ public void run() {
// so wrap it in a big catch Throwable statement.
try {
handle =
txnHandler.getMutexAPI().acquireLock(TxnStore.MUTEX_KEY.Cleaner.name());
- if (metricsEnabled) {
+ if (metricsEnabled && MetastoreConf.getBoolVar(conf,
MetastoreConf.ConfVars.METASTORE_ACIDMETRICS_EXT_ON)) {
Review comment:
yes. nice catch, fixed.
##
File path:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
##
@@ -454,6 +454,8 @@ public static ConfVars getMetaConf(String name) {
"hive.metastore.acidmetrics.check.interval", 300,
TimeUnit.SECONDS,
"Time in seconds between acid related metric collection runs."),
+METASTORE_ACIDMETRICS_EXT_ON("metastore.acidmetrics.ext.on",
"hive.metastore.acidmetrics.ext.on", true,
+"Whether to collect additional acid related metrics outside of the
acid metrics service."),
Review comment:
And/Or HIVE_SERVER2_METRICS_ENABLED. Shall I add both?
##
File path: ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java
##
@@ -111,7 +111,7 @@ public void run() {
// so wrap it in a big catch Throwable statement.
try {
handle =
txnHandler.getMutexAPI().acquireLock(TxnStore.MUTEX_KEY.Cleaner.name());
- if (metricsEnabled) {
+ if (metricsEnabled && MetastoreConf.getBoolVar(conf,
MetastoreConf.ConfVars.METASTORE_ACIDMETRICS_EXT_ON)) {
Review comment:
fixed.
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/metrics/DeltaFilesMetricReporter.java
##
@@ -115,41 +117,45 @@ public static DeltaFilesMetricReporter getInstance() {
return InstanceHolder.instance;
}
- public static synchronized void init(HiveConf conf){
+ public static synchronized void init(HiveConf conf) {
getInstance().configure(conf);
}
public void submit(TezCounters counters) {
-updateMetrics(NUM_OBSOLETE_DELTAS,
-obsoleteDeltaCache, obsoleteDeltaTopN, obsoleteDeltasThreshold,
counters);
-updateMetrics(NUM_DELTAS,
-deltaCache, deltaTopN, deltasThreshold, counters);
-updateMetrics(NUM_SMALL_DELTAS,
-smallDeltaCache, smallDeltaTopN, deltasThreshold, counters);
+if(acidMetricsExtEnabled) {
Review comment:
yeah.. So first issue is that this metrics is collected in HS2 but the
conf (METASTORE_ACIDMETRICS_EXT_ON) for the metrics collection is defined in
the MetastoreConf, so I wanted to minimize the exposure of that.
Second is in the end I put it here, because it seamed to make the class more
resilient. Here is my reasoning: This is a singleton class, so you can access
it basically anywhere, but you need to call the init method on it so that it
works properly. The acidMetricsExtEnabled flag is indeed set in the init
(configure), so if it was not called it will skip this code part which
otherwise would throw a NPE.
But I'm open to a better solution :)
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/metrics/DeltaFilesMetricReporter.java
##
@@ -115,41 +117,45 @@ public static DeltaFilesMetricReporter getInstance() {
return InstanceHolder.instance;
}
- public static synchronized void init(HiveConf conf){
+ public static synchronized void init(HiveConf conf) {
getInstance().configure(conf);
}
public void submit(TezCounters counters) {
-updateMetrics(NUM_OBSOLETE_DELTAS,
-obsoleteDeltaCache, obsoleteDeltaTopN, obsoleteDeltasThreshold,
counters);
-updateMetrics(NUM_DELTAS,
-deltaCache, deltaTopN, deltasThreshold, counters);
-updateMetrics(NUM_SMALL_DELTAS,
-smallDeltaCache, smallDeltaTopN, deltasThreshold, counters);
+if(acidMetricsExtEnabled) {
+ updateMetrics(NUM_OBSOLETE_DELTAS,
+ obsoleteDeltaCache, obsoleteDeltaTopN, obsoleteDeltasThreshold,
counters);
+ updateMetrics(NUM_DELTAS,
+ deltaCache, deltaTopN, deltasThreshold, counters);
+ updateMetrics(NUM_SMALL_DELTAS,
+ smallDeltaCache, smallDeltaTopN, deltasThreshold, counters);
+}
}
- public static void mergeDeltaFilesStats(AcidDirectory dir, long
checkThresholdInSec,
-float deltaPctThreshold, EnumMap>
[jira] [Work logged] (HIVE-25144) Add NoReconnect Annotation to CreateXXX Methods With AlreadyExistsException
[
https://issues.apache.org/jira/browse/HIVE-25144?focusedWorklogId=609139=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609139
]
ASF GitHub Bot logged work on HIVE-25144:
-
Author: ASF GitHub Bot
Created on: 09/Jun/21 13:26
Start Date: 09/Jun/21 13:26
Worklog Time Spent: 10m
Work Description: belugabehr commented on pull request #2303:
URL: https://github.com/apache/hive/pull/2303#issuecomment-857692863
@nrg4878 Here is the code I'm looking at:
https://github.com/apache/hive/blob/0c7903d7fd4675d82f653f8d305d96509a84e186/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/RetryingMetaStoreClient.java#L167-L176
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 609139)
Time Spent: 40m (was: 0.5h)
> Add NoReconnect Annotation to CreateXXX Methods With AlreadyExistsException
> ---
>
> Key: HIVE-25144
> URL: https://issues.apache.org/jira/browse/HIVE-25144
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I have recently seen an issue where a Hive {{CREATE TABLE}} method fails with
> {{AlreadyExistsException}} even though the table does absolutely not exist.
>
> I believe the issue is there there is a timeout/transient error with HMS and
> the backend database. So, the client submits the request to HMS, and the
> request does eventually succeed, but only after the connection to the client
> connects. Therefore, when the HMS Client "retry" functionality kicks it, the
> second time around, the table looks like it already exists.
>
> If something goes wrong during a HMS CREATE operation, we do not know the
> state of the operation and therefore it should just fail.
>
> It would certainly be more transparent to the end-user what is going on. An
> {{AlreadyExistsException}} is confusing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-25185) Improve Logging On Polling Tez Session from Pool
[ https://issues.apache.org/jira/browse/HIVE-25185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor resolved HIVE-25185. --- Fix Version/s: 4.0.0 Resolution: Fixed Pushed to master. Thanks [~pgaref] and [~mgergely] for the reviews! > Improve Logging On Polling Tez Session from Pool > > > Key: HIVE-25185 > URL: https://issues.apache.org/jira/browse/HIVE-25185 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25185) Improve Logging On Polling Tez Session from Pool
[ https://issues.apache.org/jira/browse/HIVE-25185?focusedWorklogId=609135=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609135 ] ASF GitHub Bot logged work on HIVE-25185: - Author: ASF GitHub Bot Created on: 09/Jun/21 13:16 Start Date: 09/Jun/21 13:16 Worklog Time Spent: 10m Work Description: belugabehr merged pull request #2339: URL: https://github.com/apache/hive/pull/2339 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 609135) Time Spent: 1.5h (was: 1h 20m) > Improve Logging On Polling Tez Session from Pool > > > Key: HIVE-25185 > URL: https://issues.apache.org/jira/browse/HIVE-25185 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25221) Hive Beeline & CLI: execute alter table comamnd With Error: Unable to alter partitions because table or database does not exist
[
https://issues.apache.org/jira/browse/HIVE-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XixiHua updated HIVE-25221:
---
Summary: Hive Beeline & CLI: execute alter table comamnd With Error: Unable
to alter partitions because table or database does not exist (was: Hive
Beeline: execute alter table comamnd With Error: Unable to alter partitions
because table or database does not exist)
> Hive Beeline & CLI: execute alter table comamnd With Error: Unable to alter
> partitions because table or database does not exist
> ---
>
> Key: HIVE-25221
> URL: https://issues.apache.org/jira/browse/HIVE-25221
> Project: Hive
> Issue Type: Bug
> Components: Beeline, CLI
>Affects Versions: 2.3.7, 2.3.8
>Reporter: XixiHua
>Assignee: XixiHua
>Priority: Major
> Fix For: 2.3.9
>
>
> Use Hive CLI to execute the following command:
>
> {code:java}
> alter table xxx.xxx partition(xxx) set location 'xxx';{code}
>
> If don't execute +*use *+ first, there will fail with the error:
> Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> Unable to alter partition. Unable to alter partitions because table or
> database does not exist. even if the table does exist.
> and more info:
> {code:java}
> 2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not
> exist.)2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not exist.) at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3908)
> at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy27.alter_partitions_with_environment_context(Unknown
> Source) at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12598)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12582)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> but when execute add/drop partition, don't need to follow this mode.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25221) Hive Beeline: execute alter table comamnd With Error: Unable to alter partitions because table or database does not exist
[
https://issues.apache.org/jira/browse/HIVE-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XixiHua updated HIVE-25221:
---
Component/s: Beeline
> Hive Beeline: execute alter table comamnd With Error: Unable to alter
> partitions because table or database does not exist
> -
>
> Key: HIVE-25221
> URL: https://issues.apache.org/jira/browse/HIVE-25221
> Project: Hive
> Issue Type: Bug
> Components: Beeline, CLI
>Affects Versions: 2.3.7, 2.3.8
>Reporter: XixiHua
>Assignee: XixiHua
>Priority: Major
> Fix For: 2.3.9
>
>
> Use Hive CLI to execute the following command:
>
> {code:java}
> alter table xxx.xxx partition(xxx) set location 'xxx';{code}
>
> If don't execute +*use *+ first, there will fail with the error:
> Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> Unable to alter partition. Unable to alter partitions because table or
> database does not exist. even if the table does exist.
> and more info:
> {code:java}
> 2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not
> exist.)2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not exist.) at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3908)
> at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy27.alter_partitions_with_environment_context(Unknown
> Source) at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12598)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12582)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> but when execute add/drop partition, don't need to follow this mode.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25221) Hive Beeline & CLI: execute alter table comamnd With Error: Unable to alter partitions because table or database does not exist
[
https://issues.apache.org/jira/browse/HIVE-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XixiHua updated HIVE-25221:
---
Description:
Use Hive Beeline and CLI to execute the following command:
{code:java}
alter table xxx.xxx partition(xxx) set location 'xxx';{code}
If don't execute +*use *+ first, there will fail with the error:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
Unable to alter partition. Unable to alter partitions because table or database
does not exist. even if the table does exist.
and more info:
{code:java}
2021-06-08T07:37:21,596 ERROR [pool-6-thread-173] metastore.RetryingHMSHandler:
InvalidOperationException(message:Unable to alter partitions because table or
database does not exist.)2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to alter
partitions because table or database does not exist.) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3908)
at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
at com.sun.proxy.$Proxy27.alter_partitions_with_environment_context(Unknown
Source) at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12598)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12582)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
at java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
but when execute add/drop partition, don't need to follow this mode.
was:
Use Hive CLI to execute the following command:
{code:java}
alter table xxx.xxx partition(xxx) set location 'xxx';{code}
If don't execute +*use *+ first, there will fail with the error:
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
Unable to alter partition. Unable to alter partitions because table or database
does not exist. even if the table does exist.
and more info:
{code:java}
2021-06-08T07:37:21,596 ERROR [pool-6-thread-173] metastore.RetryingHMSHandler:
InvalidOperationException(message:Unable to alter partitions because table or
database does not exist.)2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to alter
partitions because table or database does not exist.) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3908)
at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
at com.sun.proxy.$Proxy27.alter_partitions_with_environment_context(Unknown
Source) at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12598)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12582)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
at java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
[jira] [Updated] (HIVE-25221) Hive Beeline: execute alter table comamnd With Error: Unable to alter partitions because table or database does not exist
[
https://issues.apache.org/jira/browse/HIVE-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XixiHua updated HIVE-25221:
---
Summary: Hive Beeline: execute alter table comamnd With Error: Unable to
alter partitions because table or database does not exist (was: Hive CLI:
execute alter table comamnd With Error: Unable to alter partitions because
table or database does not exist)
> Hive Beeline: execute alter table comamnd With Error: Unable to alter
> partitions because table or database does not exist
> -
>
> Key: HIVE-25221
> URL: https://issues.apache.org/jira/browse/HIVE-25221
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.3.7, 2.3.8
>Reporter: XixiHua
>Assignee: XixiHua
>Priority: Major
> Fix For: 2.3.9
>
>
> Use Hive CLI to execute the following command:
>
> {code:java}
> alter table xxx.xxx partition(xxx) set location 'xxx';{code}
>
> If don't execute +*use *+ first, there will fail with the error:
> Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> Unable to alter partition. Unable to alter partitions because table or
> database does not exist. even if the table does exist.
> and more info:
> {code:java}
> 2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not
> exist.)2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not exist.) at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3908)
> at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy27.alter_partitions_with_environment_context(Unknown
> Source) at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12598)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12582)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> but when execute add/drop partition, don't need to follow this mode.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-25224) Multi insert statements involving tables with different bucketing_versions results in error
[
https://issues.apache.org/jira/browse/HIVE-25224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359946#comment-17359946
]
Zoltan Haindrich commented on HIVE-25224:
-
this exception should be expected in case the tables are bucketed into >1
buckets; however if they are not bucketed - then it could be suppressed.
> Multi insert statements involving tables with different bucketing_versions
> results in error
> ---
>
> Key: HIVE-25224
> URL: https://issues.apache.org/jira/browse/HIVE-25224
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>
> {code}
> drop table if exists t;
> drop table if exists t2;
> drop table if exists t3;
> create table t (a integer);
> create table t2 (a integer);
> create table t3 (a integer);
> alter table t set tblproperties ('bucketing_version'='1');
> explain from t3 insert into t select a insert into t2 select a;
> {code}
> results in
> {code}
> Error: Error while compiling statement: FAILED: RuntimeException Error
> setting bucketingVersion for group: [[op: FS[2], bucketingVersion=1], [op:
> FS[11], bucketingVersion=2]] (state=42000,code=4)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-25224) Multi insert statements involving tables with different bucketing_versions results in error
[
https://issues.apache.org/jira/browse/HIVE-25224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich reassigned HIVE-25224:
---
> Multi insert statements involving tables with different bucketing_versions
> results in error
> ---
>
> Key: HIVE-25224
> URL: https://issues.apache.org/jira/browse/HIVE-25224
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>
> {code}
> drop table if exists t;
> drop table if exists t2;
> drop table if exists t3;
> create table t (a integer);
> create table t2 (a integer);
> create table t3 (a integer);
> alter table t set tblproperties ('bucketing_version'='1');
> explain from t3 insert into t select a insert into t2 select a;
> {code}
> results in
> {code}
> Error: Error while compiling statement: FAILED: RuntimeException Error
> setting bucketingVersion for group: [[op: FS[2], bucketingVersion=1], [op:
> FS[11], bucketingVersion=2]] (state=42000,code=4)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-25223) Select with limit returns no rows on non native table
[
https://issues.apache.org/jira/browse/HIVE-25223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar reassigned HIVE-25223:
> Select with limit returns no rows on non native table
> -
>
> Key: HIVE-25223
> URL: https://issues.apache.org/jira/browse/HIVE-25223
> Project: Hive
> Issue Type: Bug
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
>
> Str:
> {code:java}
> CREATE EXTERNAL TABLE hht (key string, value int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "hht", "hbase.mapred.output.outputtable"
> = "hht");
> insert into hht select uuid(), cast((rand() * 100) as int);
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> insert into hht select uuid(), cast((rand() * 100) as int) from hht;
> set hive.fetch.task.conversion=none;
> select * from hht limit 10;
> +--++
> | hht.key | hht.value |
> +--++
> +--++
> No rows selected (5.22 seconds) {code}
>
> This is caused by GlobalLimitOptimizer. The table directory is always empty
> with a non native table since the data is not managed by hive (but hbase in
> this case).
> The optimizer scans the directory and sets the file list to an empty list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-25222) Fix reading Iceberg tables with a comma in column names
[ https://issues.apache.org/jira/browse/HIVE-25222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod reassigned HIVE-25222: - > Fix reading Iceberg tables with a comma in column names > --- > > Key: HIVE-25222 > URL: https://issues.apache.org/jira/browse/HIVE-25222 > Project: Hive > Issue Type: Bug >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > > When using a table with a column name containing a comma (e.g. `employ,ee`), > reading an Iceberg table fails because we rely on the property > "hive.io.file.readcolumn.names" which encodes the read columns in a > comma-separated list, put together by the ColumnProjectionUtils class. > Because it's comma-separated in all cases, it will produce a string like: > "id,birth_date,employ,ee" which can cause problems for Iceberg readers which > use this string list to construct their expected read schema. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25195) Store Iceberg write commit and ctas information in QueryState
[ https://issues.apache.org/jira/browse/HIVE-25195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod resolved HIVE-25195. --- Resolution: Fixed > Store Iceberg write commit and ctas information in QueryState > -- > > Key: HIVE-25195 > URL: https://issues.apache.org/jira/browse/HIVE-25195 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > We should replace the current method of passing Iceberg write commit-related > information (jobID, task num) and CTAS info via the session conf using > prefixed keys. We have a new way of doing that more cleanly, using the > QueryState object. This should make the code easier to maintain and guard > against accidental session conf pollution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25195) Store Iceberg write commit and ctas information in QueryState
[ https://issues.apache.org/jira/browse/HIVE-25195?focusedWorklogId=608969=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-608969 ] ASF GitHub Bot logged work on HIVE-25195: - Author: ASF GitHub Bot Created on: 09/Jun/21 07:13 Start Date: 09/Jun/21 07:13 Worklog Time Spent: 10m Work Description: lcspinter merged pull request #2347: URL: https://github.com/apache/hive/pull/2347 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 608969) Time Spent: 3h 40m (was: 3.5h) > Store Iceberg write commit and ctas information in QueryState > -- > > Key: HIVE-25195 > URL: https://issues.apache.org/jira/browse/HIVE-25195 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > We should replace the current method of passing Iceberg write commit-related > information (jobID, task num) and CTAS info via the session conf using > prefixed keys. We have a new way of doing that more cleanly, using the > QueryState object. This should make the code easier to maintain and guard > against accidental session conf pollution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22915) java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument
[ https://issues.apache.org/jira/browse/HIVE-22915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359824#comment-17359824 ] Jay Sen commented on HIVE-22915: +1 to the James's point. > java.lang.NoSuchMethodError: > com.google.common.base.Preconditions.checkArgument > --- > > Key: HIVE-22915 > URL: https://issues.apache.org/jira/browse/HIVE-22915 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.4 > Environment: Ubuntu 16.04 >Reporter: pradeepkumar >Priority: Critical > > Hi Team, > I am Not able to run hive. Getting following error on hive version above 3.X, > i tried all the versions. It is very critical issue.SLF4J: Class path > contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/home/sreeramadasu/Downloads/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/home/sreeramadasu/Downloads/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See [http://www.slf4j.org/codes.html#multiple_bindings] for an > explanation. > SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory] > Exception in thread "main" java.lang.NoSuchMethodError: > com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V > at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357) > at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338) > at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:536) > at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:554) > at org.apache.hadoop.mapred.JobConf.(JobConf.java:448) > at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:4045) > at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:4003) > at > org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:81) > at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:65) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:702) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.hadoop.util.RunJar.run(RunJar.java:323) > at org.apache.hadoop.util.RunJar.main(RunJar.java:236) > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25221) Hive CLI: execute alter table comamnd With Error: Unable to alter partitions because table or database does not exist
[
https://issues.apache.org/jira/browse/HIVE-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XixiHua updated HIVE-25221:
---
Summary: Hive CLI: execute alter table comamnd With Error: Unable to alter
partitions because table or database does not exist (was: Hive ClI: execute
alter table comamnd With Error: Unable to alter partitions because table or
database does not exist)
> Hive CLI: execute alter table comamnd With Error: Unable to alter partitions
> because table or database does not exist
> -
>
> Key: HIVE-25221
> URL: https://issues.apache.org/jira/browse/HIVE-25221
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.3.7, 2.3.8
>Reporter: XixiHua
>Assignee: XixiHua
>Priority: Major
> Fix For: 2.3.9
>
>
> Use Hive CLI to execute the following command:
>
> {code:java}
> alter table xxx.xxx partition(xxx) set location 'xxx';{code}
>
> If don't execute +*use *+ first, there will fail with the error:
> Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> Unable to alter partition. Unable to alter partitions because table or
> database does not exist. even if the table does exist.
> and more info:
> {code:java}
> 2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not
> exist.)2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not exist.) at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3908)
> at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy27.alter_partitions_with_environment_context(Unknown
> Source) at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12598)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12582)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> but when execute add/drop partition, don't need to follow this mode.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25221) Hive ClI: execute alter table comamnd With Error: Unable to alter partitions because table or database does not exist
[
https://issues.apache.org/jira/browse/HIVE-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XixiHua updated HIVE-25221:
---
Fix Version/s: 2.3.9
> Hive ClI: execute alter table comamnd With Error: Unable to alter partitions
> because table or database does not exist
> -
>
> Key: HIVE-25221
> URL: https://issues.apache.org/jira/browse/HIVE-25221
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.3.7, 2.3.8
>Reporter: XixiHua
>Assignee: XixiHua
>Priority: Major
> Fix For: 2.3.9
>
>
> Use Hive CLI to execute the following command:
>
> {code:java}
> alter table xxx.xxx partition(xxx) set location 'xxx';{code}
>
> If don't execute +*use *+ first, there will fail with the error:
> Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> Unable to alter partition. Unable to alter partitions because table or
> database does not exist. even if the table does exist.
> and more info:
> {code:java}
> 2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not
> exist.)2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not exist.) at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3908)
> at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy27.alter_partitions_with_environment_context(Unknown
> Source) at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12598)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12582)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> but when execute add/drop partition, don't need to follow this mode.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HIVE-25221) Hive ClI: execute alter table comamnd With Error: Unable to alter partitions because table or database does not exist
[
https://issues.apache.org/jira/browse/HIVE-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-25221 started by XixiHua.
--
> Hive ClI: execute alter table comamnd With Error: Unable to alter partitions
> because table or database does not exist
> -
>
> Key: HIVE-25221
> URL: https://issues.apache.org/jira/browse/HIVE-25221
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.3.7, 2.3.8
>Reporter: XixiHua
>Assignee: XixiHua
>Priority: Major
>
> Use Hive CLI to execute the following command:
>
> {code:java}
> alter table xxx.xxx partition(xxx) set location 'xxx';{code}
>
> If don't execute +*use *+ first, there will fail with the error:
> Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> Unable to alter partition. Unable to alter partitions because table or
> database does not exist. even if the table does exist.
> and more info:
> {code:java}
> 2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not
> exist.)2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not exist.) at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3908)
> at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy27.alter_partitions_with_environment_context(Unknown
> Source) at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12598)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12582)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> but when execute add/drop partition, don't need to follow this mode.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25220) Query with union fails CBO with OOM
[
https://issues.apache.org/jira/browse/HIVE-25220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-25220:
--
Status: Patch Available (was: Open)
> Query with union fails CBO with OOM
> ---
>
> Key: HIVE-25220
> URL: https://issues.apache.org/jira/browse/HIVE-25220
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> 2021-06-08T08:15:14,450 ERROR [6241f234-77e0-4e63-9873-6eb9d655421c
> HiveServer2-Handler-Pool: Thread-79] parse.CalcitePlanner: CBO failed,
> skipping CBO.
> java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.rethrowCalciteException(CalcitePlanner.java:1728)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1564)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:538)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12680)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:428)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:170)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:221)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:188)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:600)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:546)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:540)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:127)
> ~[hive-exec-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:199)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:260)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:274)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:565)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:551)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_262]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_262]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_262]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_262]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
> ~[hive-service-3.1.3000.7.2.6.3-1.jar:3.1.3000.7.2.6.3-1]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
>
[jira] [Assigned] (HIVE-25221) Hive ClI: execute alter table comamnd With Error: Unable to alter partitions because table or database does not exist
[
https://issues.apache.org/jira/browse/HIVE-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XixiHua reassigned HIVE-25221:
--
> Hive ClI: execute alter table comamnd With Error: Unable to alter partitions
> because table or database does not exist
> -
>
> Key: HIVE-25221
> URL: https://issues.apache.org/jira/browse/HIVE-25221
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.3.8, 2.3.7
>Reporter: XixiHua
>Assignee: XixiHua
>Priority: Major
>
> Use Hive CLI to execute the following command:
>
> {code:java}
> alter table xxx.xxx partition(xxx) set location 'xxx';{code}
>
> If don't execute +*use *+ first, there will fail with the error:
> Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> Unable to alter partition. Unable to alter partitions because table or
> database does not exist. even if the table does exist.
> and more info:
> {code:java}
> 2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not
> exist.)2021-06-08T07:37:21,596 ERROR [pool-6-thread-173]
> metastore.RetryingHMSHandler: InvalidOperationException(message:Unable to
> alter partitions because table or database does not exist.) at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3908)
> at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy27.alter_partitions_with_environment_context(Unknown
> Source) at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12598)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions_with_environment_context.getResult(ThriftHiveMetastore.java:12582)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> but when execute add/drop partition, don't need to follow this mode.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21489) EXPLAIN command throws ClassCastException in Hive
[ https://issues.apache.org/jira/browse/HIVE-21489?focusedWorklogId=608936=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-608936 ] ASF GitHub Bot logged work on HIVE-21489: - Author: ASF GitHub Bot Created on: 09/Jun/21 06:12 Start Date: 09/Jun/21 06:12 Worklog Time Spent: 10m Work Description: ramesh0201 opened a new pull request #2373: URL: https://github.com/apache/hive/pull/2373 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 608936) Remaining Estimate: 0h Time Spent: 10m > EXPLAIN command throws ClassCastException in Hive > - > > Key: HIVE-21489 > URL: https://issues.apache.org/jira/browse/HIVE-21489 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.4 >Reporter: Ping Lu >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-21489.1.patch, HIVE-21489.2.patch > > Time Spent: 10m > Remaining Estimate: 0h > > I'm trying to run commands like explain select * from src in hive-2.3.4,but > it falls with the ClassCastException: > org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer cannot be cast to > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer > Steps to reproduce: > 1）hive.execution.engine is the default value mr > 2）hive.security.authorization.enabled is set to true, and > hive.security.authorization.manager is set to > org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider > 3）start hivecli to run command：explain select * from src > I debug the code and find the issue HIVE-18778 causing the above > ClassCastException.If I set hive.in.test to true，the explain command can be > successfully executed。 > Now,I have one question,due to hive.in.test cann't be modified at runtime.how > to run explain command with using default authorization in hive-2.3.4, -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21489) EXPLAIN command throws ClassCastException in Hive
[ https://issues.apache.org/jira/browse/HIVE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-21489: -- Labels: pull-request-available (was: ) > EXPLAIN command throws ClassCastException in Hive > - > > Key: HIVE-21489 > URL: https://issues.apache.org/jira/browse/HIVE-21489 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.4 >Reporter: Ping Lu >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21489.1.patch, HIVE-21489.2.patch > > Time Spent: 10m > Remaining Estimate: 0h > > I'm trying to run commands like explain select * from src in hive-2.3.4,but > it falls with the ClassCastException: > org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer cannot be cast to > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer > Steps to reproduce: > 1）hive.execution.engine is the default value mr > 2）hive.security.authorization.enabled is set to true, and > hive.security.authorization.manager is set to > org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider > 3）start hivecli to run command：explain select * from src > I debug the code and find the issue HIVE-18778 causing the above > ClassCastException.If I set hive.in.test to true，the explain command can be > successfully executed。 > Now,I have one question,due to hive.in.test cann't be modified at runtime.how > to run explain command with using default authorization in hive-2.3.4, -- This message was sent by Atlassian Jira (v8.3.4#803005)
