[
https://issues.apache.org/jira/browse/HIVE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-15572:
--------------------------------
Description:
When query canceling command sent during Hive Acquire locks (from zookeeper),
hive will finish acquiring all the locks and release them. As it is shown in
the following log:
It took 165 s to finish acquire the lock,then spend 81s to release them.
We can improve the performance by not acquiring any more locks and releasing
held locks when the query canceling command is received.
{noformat}
Background-Pool: Thread-224]: <PERFLOG method=TimeToSubmit
from=org.apache.hadoop.hive.ql.Driver>
2017-01-03 10:50:35,413 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
[HiveServer2-Background-Pool: Thread-224]: <PERFLOG
method=acquireReadWriteLocks from=org.apache.hadoop.hive.ql.Driver>
2017-01-03 10:51:00,671 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
[HiveServer2-Background-Pool: Thread-218]: </PERFLOG
method=acquireReadWriteLocks start=1483469295080 end=1483469460671
duration=165591 from=org.apache.hadoop.hive.ql.Driver>
2017-01-03 10:51:00,672 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
[HiveServer2-Background-Pool: Thread-218]: <PERFLOG method=Driver.execute
from=org.apache.hadoop.hive.ql.Driver>
2017-01-03 10:51:00,672 ERROR org.apache.hadoop.hive.ql.Driver:
[HiveServer2-Background-Pool: Thread-218]: FAILED: query select count(*) from
manyparttbl has been cancelled
2017-01-03 10:51:00,673 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
[HiveServer2-Background-Pool: Thread-218]: <PERFLOG method=releaseLocks
from=org.apache.hadoop.hive.ql.Driver>
2017-01-03 10:51:40,755 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
[HiveServer2-Background-Pool: Thread-215]: </PERFLOG method=releaseLocks
start=1483469419487 end=1483469500755 duration=81268
from=org.apache.hadoop.hive.ql.Driver>
{noformat}
was:
When query canceling command sent during Hive Acquire locks (from zookeeper),
hive will finish acquiring all the locks and release them. As it is shown in
the following log:
It took 165 s to finish acquire the lock,then spend 81s to release them.
We can improve the performance by not acquiring any more locks and releasing
held locks when the query canceling command is received.
<noformat>
Background-Pool: Thread-224]: <PERFLOG method=TimeToSubmit
from=org.apache.hadoop.hive.ql.Driver>
2017-01-03 10:50:35,413 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
[HiveServer2-Background-Pool: Thread-224]: <PERFLOG
method=acquireReadWriteLocks from=org.apache.hadoop.hive.ql.Driver>
2017-01-03 10:51:00,671 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
[HiveServer2-Background-Pool: Thread-218]: </PERFLOG
method=acquireReadWriteLocks start=1483469295080 end=1483469460671
duration=165591 from=org.apache.hadoop.hive.ql.Driver>
2017-01-03 10:51:00,672 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
[HiveServer2-Background-Pool: Thread-218]: <PERFLOG method=Driver.execute
from=org.apache.hadoop.hive.ql.Driver>
2017-01-03 10:51:00,672 ERROR org.apache.hadoop.hive.ql.Driver:
[HiveServer2-Background-Pool: Thread-218]: FAILED: query select count(*) from
manyparttbl has been cancelled
2017-01-03 10:51:00,673 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
[HiveServer2-Background-Pool: Thread-218]: <PERFLOG method=releaseLocks
from=org.apache.hadoop.hive.ql.Driver>
2017-01-03 10:51:40,755 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
[HiveServer2-Background-Pool: Thread-215]: </PERFLOG method=releaseLocks
start=1483469419487 end=1483469500755 duration=81268
from=org.apache.hadoop.hive.ql.Driver>
<noformat>
> Improve the response time for query canceling when it happens during
> acquiring locks
> ------------------------------------------------------------------------------------
>
> Key: HIVE-15572
> URL: https://issues.apache.org/jira/browse/HIVE-15572
> Project: Hive
> Issue Type: Improvement
> Reporter: Yongzhi Chen
> Assignee: Yongzhi Chen
>
> When query canceling command sent during Hive Acquire locks (from zookeeper),
> hive will finish acquiring all the locks and release them. As it is shown in
> the following log:
> It took 165 s to finish acquire the lock,then spend 81s to release them.
> We can improve the performance by not acquiring any more locks and releasing
> held locks when the query canceling command is received.
> {noformat}
> Background-Pool: Thread-224]: <PERFLOG method=TimeToSubmit
> from=org.apache.hadoop.hive.ql.Driver>
> 2017-01-03 10:50:35,413 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [HiveServer2-Background-Pool: Thread-224]: <PERFLOG
> method=acquireReadWriteLocks from=org.apache.hadoop.hive.ql.Driver>
> 2017-01-03 10:51:00,671 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [HiveServer2-Background-Pool: Thread-218]: </PERFLOG
> method=acquireReadWriteLocks start=1483469295080 end=1483469460671
> duration=165591 from=org.apache.hadoop.hive.ql.Driver>
> 2017-01-03 10:51:00,672 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [HiveServer2-Background-Pool: Thread-218]: <PERFLOG method=Driver.execute
> from=org.apache.hadoop.hive.ql.Driver>
> 2017-01-03 10:51:00,672 ERROR org.apache.hadoop.hive.ql.Driver:
> [HiveServer2-Background-Pool: Thread-218]: FAILED: query select count(*) from
> manyparttbl has been cancelled
> 2017-01-03 10:51:00,673 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [HiveServer2-Background-Pool: Thread-218]: <PERFLOG method=releaseLocks
> from=org.apache.hadoop.hive.ql.Driver>
> 2017-01-03 10:51:40,755 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [HiveServer2-Background-Pool: Thread-215]: </PERFLOG method=releaseLocks
> start=1483469419487 end=1483469500755 duration=81268
> from=org.apache.hadoop.hive.ql.Driver>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
