[
https://issues.apache.org/jira/browse/HIVE-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-15254:
------------------------------------
Description:
{noformat}
2016-11-18 20:20:01,472 .
[HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]: vertexName=Map
7,
2016-11-18 20:20:01,500 .
[HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]: vertexName=Map
4,
2016-11-18 20:20:01,503 .
[HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]: vertexName=Map
5,
2016-11-18 20:20:01,505 .
[HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]: vertexName=Map
8,
2016-11-18 20:20:01,530 .
[HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]: vertexName=Map
6,
2016-11-18 20:20:01,870 .
[HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]: vertexName=Map
3,
...
2016-11-18T20:26:59,410 INFO [Wait-Queue-Scheduler-0 ()]
org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorService: Attempting to
execute TaskWrapper{task=attempt_1478967587833_2622_1_06_000024_1,
inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true,
canFinish=false,
firstAttemptStartTime=1479500401952, dagStartTime=1479500397795,
withinDagPriority=64, vertexParallelism= 107, selfAndUpstreamParallelism= 131,
selfAndUpstreamComplete= 87}
2016-11-18T20:26:59,411 INFO [Wait-Queue-Scheduler-0 ()]
org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorService:
attempt_1478967587833_2622_1_06_000024_1 is not finishable. Adding it to
pre-emption queue
2016-11-18T20:26:59,415 INFO [Task-Executor-14
(1478967587833_2622_1_06_000024_1)]
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask: Initializing
LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ... inputSpecList=[
{{ sourceVertexName=a11, physicalEdgeCount=1,
inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }},
{{ sourceVertexName=Map 5, physicalEdgeCount=1,
inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
{{ sourceVertexName=Map 6, physicalEdgeCount=3,
inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
{{ sourceVertexName=Map 8, physicalEdgeCount=1,
inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
{{ sourceVertexName=Map 4, physicalEdgeCount=1,
inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
{{ sourceVertexName=Map 7, physicalEdgeCount=1,
inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
{{ sourceVertexName=Map 3, physicalEdgeCount=17,
inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ],
{noformat}
The task should be finishable but isn't.
That can be a problem when adding to wait queue and then rejecting the least
important task (the one being submitted, or an old evicted one) - wait queue
comparator takes finishable state into account, so while this task should be
able to kick any non-finishable task out of the queue, it might actually get
rejected because it "looks" non-finishable at submission time.
was:
{noformat}
2016-11-18 20:20:01,472 .
[HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]: vertexName=Map
7,
2016-11-18 20:20:01,500 .
[HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]: vertexName=Map
4,
2016-11-18 20:20:01,503 .
[HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]: vertexName=Map
5,
2016-11-18 20:20:01,505 .
[HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]: vertexName=Map
8,
2016-11-18 20:20:01,530 .
[HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]: vertexName=Map
6,
2016-11-18 20:20:01,870 .
[HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]: vertexName=Map
3,
...
2016-11-18T20:26:59,410 INFO [Wait-Queue-Scheduler-0 ()]
org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorService: Attempting to
execute TaskWrapper{task=attempt_1478967587833_2622_1_06_000024_1,
inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true,
canFinish=false, firstAttemptStartTime=1479500401952,
dagStartTime=1479500397795, withinDagPriority=64, vertexParallelism= 107,
selfAndUpstreamParallelism= 131, selfAndUpstreamComplete= 87}
2016-11-18T20:26:59,411 INFO [Wait-Queue-Scheduler-0 ()]
org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorService:
attempt_1478967587833_2622_1_06_000024_1 is not finishable. Adding it to
pre-emption queue
2016-11-18T20:26:59,415 INFO [Task-Executor-14
(1478967587833_2622_1_06_000024_1)]
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask: Initializing
LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ... inputSpecList=[
{{ sourceVertexName=a11, physicalEdgeCount=1,
inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }},
{{ sourceVertexName=Map 5, physicalEdgeCount=1,
inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
{{ sourceVertexName=Map 6, physicalEdgeCount=3,
inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
{{ sourceVertexName=Map 8, physicalEdgeCount=1,
inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
{{ sourceVertexName=Map 4, physicalEdgeCount=1,
inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
{{ sourceVertexName=Map 7, physicalEdgeCount=1,
inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
{{ sourceVertexName=Map 3, physicalEdgeCount=17,
inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ],
{noformat}
The task should be finishable but isn't.
That can be a problem when adding to wait queue and then rejecting the least
important task (the one being submitted, or an old evicted one) - wait queue
comparator takes finishable state into account, so while this task should be
able to kick any non-finishable task out of the queue, it might actually get
rejected because it "looks" non-finishable at submission time.
> LLAP: send finishable state with the task, not (just) as a message
> ------------------------------------------------------------------
>
> Key: HIVE-15254
> URL: https://issues.apache.org/jira/browse/HIVE-15254
> Project: Hive
> Issue Type: Bug
> Reporter: Sergey Shelukhin
>
> {noformat}
> 2016-11-18 20:20:01,472 .
> [HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]:
> vertexName=Map 7,
> 2016-11-18 20:20:01,500 .
> [HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]:
> vertexName=Map 4,
> 2016-11-18 20:20:01,503 .
> [HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]:
> vertexName=Map 5,
> 2016-11-18 20:20:01,505 .
> [HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]:
> vertexName=Map 8,
> 2016-11-18 20:20:01,530 .
> [HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]:
> vertexName=Map 6,
> 2016-11-18 20:20:01,870 .
> [HISTORY][DAG:dag_1478967587833_2622_1][Event:VERTEX_FINISHED]:
> vertexName=Map 3,
> ...
> 2016-11-18T20:26:59,410 INFO [Wait-Queue-Scheduler-0 ()]
> org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorService: Attempting to
> execute TaskWrapper{task=attempt_1478967587833_2622_1_06_000024_1,
> inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true,
> canFinish=false,
> firstAttemptStartTime=1479500401952, dagStartTime=1479500397795,
> withinDagPriority=64, vertexParallelism= 107, selfAndUpstreamParallelism=
> 131, selfAndUpstreamComplete= 87}
> 2016-11-18T20:26:59,411 INFO [Wait-Queue-Scheduler-0 ()]
> org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorService:
> attempt_1478967587833_2622_1_06_000024_1 is not finishable. Adding it to
> pre-emption queue
> 2016-11-18T20:26:59,415 INFO [Task-Executor-14
> (1478967587833_2622_1_06_000024_1)]
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask: Initializing
> LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ... inputSpecList=[
> {{ sourceVertexName=a11, physicalEdgeCount=1,
> inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }},
> {{ sourceVertexName=Map 5, physicalEdgeCount=1,
> inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
> {{ sourceVertexName=Map 6, physicalEdgeCount=3,
> inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
> {{ sourceVertexName=Map 8, physicalEdgeCount=1,
> inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
> {{ sourceVertexName=Map 4, physicalEdgeCount=1,
> inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
> {{ sourceVertexName=Map 7, physicalEdgeCount=1,
> inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }},
> {{ sourceVertexName=Map 3, physicalEdgeCount=17,
> inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ],
> {noformat}
> The task should be finishable but isn't.
> That can be a problem when adding to wait queue and then rejecting the least
> important task (the one being submitted, or an old evicted one) - wait queue
> comparator takes finishable state into account, so while this task should be
> able to kick any non-finishable task out of the queue, it might actually get
> rejected because it "looks" non-finishable at submission time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
