Which lock manager are you using?
Do you have acid enabled and if so are these tables transactional?
Eugene
From: Igor Kuzmenko <f1she...@gmail.com>
Reply-To: "user@hive.apache.org" <user@hive.apache.org>
Date: Thursday, October 12, 2017 at 3:58 AM
To: "user@hive.apache.org" <user@hive.apache.org>
Subject: Hive locking mechanism on read partition.
Hello, I'm using HDP 2.5.0.0 with included hive 1.2.1. And I have problem with
locking mechanism.
Most of my queries to hive looks like this.
(1) insert into table results_table partition(task_id=${task_id})
select * from data_table where ....;
results_table partitioned by task_id field and i expect to get exclusive lock
on corresponding partition.Which is true:
Lock ID Database
Table Partition
State Blocked By
Type Transaction ID
136639682.4 default
results_table task_id=5556
ACQUIRED
EXCLUSIVE NULL
Another type of query is fetching data from results_table:
(2) select * from results_table where task_id = ${task_id}
This select doesn't require any map reduce and executes fast. This is exactly
what I want.
But if I execute this two queries at the same time I can't perform read from
result_table partition while inserting data into another.
Locks looks like this:
Lock ID Database
Table Partition
State Blocked By
Type Transaction ID
136639682.4 default
results_table task_id=5556
ACQUIRED
EXCLUSIVE NULL
136639700.1 default
results_table NULL
WAITING 136639682.4
SHARED_READ NULL
136639700.2 default
results_table task_id=5535
WAITING
SHARED_READ NULL
Reading data from specified partition requires shared lock on whole table. This
prevents me to get data untill first query completes.
As I can see on this page
<https://cwiki.apache.org/confluence/display/Hive/Locking#Locking-UseCases>
this is expected behaivor. But I don't understand why we need lock on table.
Can I get rid of shared lock on whole table, while still having shared lock on
specific partition?
