Re: What is the latest version of JDK supported by hive 3.1.2?
https://issues.apache.org/jira/browse/HIVE-22415 On Thu, Sep 24, 2020 at 2:05 AM Zoltán Haindrich wrote: > Hello, > > For Hive3 I don't think you can go above java8 - there are changes in the > classloader area which will cause all kind of exceptions with java >=9 > > cheers, > Zoltan > > On September 18, 2020 12:36:56 PM GMT+02:00, qq <987626...@qq.com> wrote: > >Hello: > > What is the latest version of JDK supported by > >hive 3.1.2? > > > >thinks. > >I am looking forward to your reply！ > > -- > Zoltán Haindrich
[jira] [Created] (HIVE-24202) Clean up local HS2 HMS cache code (II)
Jesus Camacho Rodriguez created HIVE-24202: -- Summary: Clean up local HS2 HMS cache code (II) Key: HIVE-24202 URL: https://issues.apache.org/jira/browse/HIVE-24202 Project: Hive Issue Type: Improvement Components: HiveServer2 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Follow-up for HIVE-24183 (split into different JIRAs). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24201) WorkloadManager kills query being moved to different pool if destination pool does not have enough sessions
Adesh Kumar Rao created HIVE-24201:
--
Summary: WorkloadManager kills query being moved to different pool
if destination pool does not have enough sessions
Key: HIVE-24201
URL: https://issues.apache.org/jira/browse/HIVE-24201
Project: Hive
Issue Type: Bug
Components: HiveServer2, llap
Affects Versions: 4.0.0
Reporter: Adesh Kumar Rao
Assignee: Nikhil Gupta
To reproduce, create a resource plan with move trigger, like below:
{code:java}
++
|line|
++
| experiment[status=DISABLED,parallelism=null,defaultPool=default] |
| + default[allocFraction=0.888,schedulingPolicy=null,parallelism=1] |
| | mapped for default |
| + pool2[allocFraction=0.1,schedulingPolicy=fair,parallelism=1] |
| | trigger t1: if (ELAPSED_TIME > 20) { MOVE TO pool1 } |
| | mapped for users: abcd |
| + pool1[allocFraction=0.012,schedulingPolicy=null,parallelism=1] |
| | mapped for users: efgh |
{code}
Now, run two queries in pool1 and pool2 using different users. The query
running in pool2 will tried to move to pool1 and it will get killed because
pool1 will not have session to handle the query.
Once killed this query needs to be re-run externally. It can be optimized and
should be retried to run in destination pool directly(it will get queued and
run once the session is alive).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24200) MSCK repair table is not working
stephbat created HIVE-24200:
---
Summary: MSCK repair table is not working
Key: HIVE-24200
URL: https://issues.apache.org/jira/browse/HIVE-24200
Project: Hive
Issue Type: Bug
Affects Versions: 3.1.0
Reporter: stephbat
*+steps to reproduce :+*
create external table test_sync_part (name string) partitioned by (id int)
location '/projects/PTEST/dev/hive/test_sync_part';
insert into table test_sync_part values ('nom1',1),('nom2',2);
delete the sub-folder of one partition on the folder
/projects/PTEST/dev/hive/test_sync_part
msck repair table test_sync_part drop partitions;
{code:java}
2020-09-24T14:45:57,419 INFO [HiveServer2-Background-Pool: Thread-208]:
metastore.Msck (:()) - Tables not in metastore: []
2020-09-24T14:45:57,419 INFO [HiveServer2-Background-Pool: Thread-208]:
metastore.Msck (:()) - Tables missing on filesystem: []
2020-09-24T14:45:57,419 INFO [HiveServer2-Background-Pool: Thread-208]:
metastore.Msck (:()) - Partitions not in metastore: []
2020-09-24T14:45:57,419 INFO [HiveServer2-Background-Pool: Thread-208]:
metastore.Msck (:()) - Partitions missing from filesystem: [test_sync_part:id=2]
2020-09-24T14:45:57,419 INFO [HiveServer2-Background-Pool: Thread-208]:
metastore.Msck (:()) - Expired partitions: []
2020-09-24T14:45:57,420 INFO [HiveServer2-Background-Pool: Thread-208]:
metastore.HiveMetaStoreClient (:()) - Closed a connection to metastore, current
connections: 8
2020-09-24T14:45:57,420 INFO [HiveServer2-Background-Pool: Thread-208]:
reexec.ReOptimizePlugin (:()) - ReOptimization: retryPossible: false
2020-09-24T14:45:57,420 INFO [HiveServer2-Background-Pool: Thread-208]:
hooks.HiveProtoLoggingHook (:()) - Received post-hook notification for:
hive_20200924144557_3e164203-720a-4e4a-bbdd-b65f53901e15
2020-09-24T14:45:57,421 ERROR [HiveServer2-Background-Pool: Thread-208]:
ql.Driver (:()) - FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
2020-09-24T14:45:57,421 INFO [HiveServer2-Background-Pool: Thread-208]:
ql.Driver (:()) - Completed executing
command(queryId=hive_20200924144557_3e164203-720a-4e4a-bbdd-b65f53901e15); Time
taken: 0.289 seconds
2020-09-24T14:45:57,421 INFO [HiveServer2-Background-Pool: Thread-208]:
lockmgr.DbTxnManager (:()) - Stopped heartbeat for query:
hive_20200924144557_3e164203-720a-4e4a-bbdd-b65f53901e15
2020-09-24T14:45:57,458 ERROR [HiveServer2-Background-Pool: Thread-208]:
operation.Operation (:()) - Error running hive query:
org.apache.hive.service.cli.HiveSQLException: Error while processing statement:
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
at
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:348)
~[hive-service-3.1.0.3.1.4.0-315.jar:3.1.0.3.1.4.0-315]
at
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:228)
~[hive-service-3.1.0.3.1.4.0-315.jar:3.1.0.3.1.4.0-315]
at
org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)
~[hive-service-3.1.0.3.1.4.0-315.jar:3.1.0.3.1.4.0-315]
at
org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:324)
~[hive-service-3.1.0.3.1.4.0-315.jar:3.1.0.3.1.4.0-315]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_112]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
~[hadoop-common-3.1.1.3.1.4.0-315.jar:?]
at
org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:342)
~[hive-service-3.1.0.3.1.4.0-315.jar:3.1.0.3.1.4.0-315]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_112]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[?:1.8.0_112]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_112]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[?:1.8.0_112]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[?:1.8.0_112]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[?:1.8.0_112]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24199) Incorrect result when subquey in exists contains limit
Krisztian Kasa created HIVE-24199:
-
Summary: Incorrect result when subquey in exists contains limit
Key: HIVE-24199
URL: https://issues.apache.org/jira/browse/HIVE-24199
Project: Hive
Issue Type: Bug
Reporter: Krisztian Kasa
Assignee: Krisztian Kasa
{code:java}
create table web_sales (ws_order_number int, ws_warehouse_sk int) stored as orc;
insert into web_sales values
(1, 1),
(1, 2),
(2, 1),
(2, 2);
select * from web_sales ws1
where exists (select 1 from web_sales ws2 where ws1.ws_order_number =
ws2.ws_order_number limit 1);
1 1
1 2
{code}
{code:java}
CBO PLAN:
HiveSemiJoin(condition=[=($0, $2)], joinType=[semi])
HiveProject(ws_order_number=[$0], ws_warehouse_sk=[$1])
HiveFilter(condition=[IS NOT NULL($0)])
HiveTableScan(table=[[default, web_sales]], table:alias=[ws1])
HiveProject(ws_order_number=[$0])
HiveSortLimit(fetch=[1]) <-- This shouldn't be added
HiveProject(ws_order_number=[$0])
HiveFilter(condition=[IS NOT NULL($0)])
HiveTableScan(table=[[default, web_sales]], table:alias=[ws2])
{code}
Limit n on the right side of the join reduces the result set coming from the
right to only n record hence not all the ws_order_number values are included
which leads to correctness issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24198) Map side SMB join produceing wrong result
mahesh kumar behera created HIVE-24198: -- Summary: Map side SMB join produceing wrong result Key: HIVE-24198 URL: https://issues.apache.org/jira/browse/HIVE-24198 Project: Hive Issue Type: Bug Components: Hive Reporter: mahesh kumar behera Assignee: mahesh kumar behera CREATE TABLE tbl1_n5(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS ; CREATE TABLE tbl2_n4(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; set hive.auto.convert.join=true; set hive.optimize.bucketmapjoin = true; set hive.optimize.bucketmapjoin.sortedmerge = true; set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; set hive.auto.convert.sortmerge.join=true; set hive.auto.convert.sortmerge.join.to.mapjoin=false; set hive.auto.convert.join.noconditionaltask.ize=1; set hive.optimize.semijoin.conversion = false; insert into tbl2_n4 values (2, 'val_2'), (0, 'val_0'), (0, 'val_0'), (0, 'val_0'), (4, 'val_4') ,(5, 'val_5') ,(5, 'val_5') , (5, 'val_5'), (8, 'val_8'), (9, 'val_9'); insert into tbl1_n5 values (2, 'val_2'), (0, 'val_0'), (0, 'val_0'), (0, 'val_0'), (4, 'val_4') ,(5, 'val_5') ,(5, 'val_5') , (5, 'val_5'), (8, 'val_8'), (9, 'val_9'); Select * from (select b.key as key, count(*) as value from tbl1_n5 b where key < 6 group by b.key) subq1 join (select a.key as key, a.value as value from tbl2_n4 a where key < 6) subq2 on subq1.key = subq2.key; The above select is producing 0,0,0,2,4,5,5,5,5,5,5 instead of 0,0,0,2,4,5,5,5 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24197) Check for write transactions for the db under replication at a frequent interval
Aasha Medhi created HIVE-24197: -- Summary: Check for write transactions for the db under replication at a frequent interval Key: HIVE-24197 URL: https://issues.apache.org/jira/browse/HIVE-24197 Project: Hive Issue Type: Task Reporter: Aasha Medhi Assignee: Aasha Medhi -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24196) Refactor getAcidState in AcidUtils to use HMS endpoint
Barnabas Maidics created HIVE-24196: --- Summary: Refactor getAcidState in AcidUtils to use HMS endpoint Key: HIVE-24196 URL: https://issues.apache.org/jira/browse/HIVE-24196 Project: Hive Issue Type: Improvement Components: HiveServer2 Reporter: Barnabas Maidics Assignee: Barnabas Maidics -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24195) Avoid reallocation of the arrays in the lateral view explode of complex types
Ramesh Kumar Thangarajan created HIVE-24195: --- Summary: Avoid reallocation of the arrays in the lateral view explode of complex types Key: HIVE-24195 URL: https://issues.apache.org/jira/browse/HIVE-24195 Project: Hive Issue Type: Bug Reporter: Ramesh Kumar Thangarajan Assignee: Ramesh Kumar Thangarajan -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: What is the latest version of JDK supported by hive 3.1.2?
Hello, For Hive3 I don't think you can go above java8 - there are changes in the classloader area which will cause all kind of exceptions with java >=9 cheers, Zoltan On September 18, 2020 12:36:56 PM GMT+02:00, qq <987626...@qq.com> wrote: >Hello: > What is the latest version of JDK supported by >hive 3.1.2? > >thinks. >I am looking forward to your reply！ -- Zoltán Haindrich
