[
https://issues.apache.org/jira/browse/HIVE-16967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064439#comment-16064439
]
xujie commented on HIVE-16967:
------------------------------
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
Edges:
Reducer 11 <- Map 10 (SIMPLE_EDGE), Map 12 (SIMPLE_EDGE), Map 13
(BROADCAST_EDGE), Map 14 (BROADCAST_EDGE), Map 15 (BROADCAST_EDGE), Map 16
(BROADCAST_EDGE), Map 17 (BROADCAST_EDGE), Union 3 (CONTAINS)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5
(BROADCAST_EDGE), Map 6 (BROADCAST_EDGE), Map 7 (BROADCAST_EDGE), Map 8
(BROADCAST_EDGE), Map 9 (BROADCAST_EDGE), Union 3 (CONTAINS)
DagName: hadoop_20170627161010_072d8ca5-d7c5-4048-aeab-4683f226d56f:1
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: vp_ambs_tmp_credit_acct_h_01
Statistics: Num rows: 47016953 Data size: 80577838747 Basic
stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (org = '242') (type: boolean)
Statistics: Num rows: 23508476 Data size: 40288918516 Basic
stats: COMPLETE Column stats: NONE
Select Operator
expressions: acct (type: string), COALESCE(block_code_1,'
') (type: string), COALESCE(block_code_2,' ') (type: string),
block_code_1_set_date (type: string), block_code_2_set_date (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 23508476 Data size: 40288918516
Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 23508476 Data size: 40288918516
Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string), _col2 (type:
string), _col3 (type: string), _col4 (type: string)
Map 10
Map Operator Tree:
TableScan
alias: vp_ambs_tmp_credit_acct_h_01
Statistics: Num rows: 47016953 Data size: 80577838747 Basic
stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (org = '242') (type: boolean)
Statistics: Num rows: 23508476 Data size: 40288918516 Basic
stats: COMPLETE Column stats: NONE
Select Operator
expressions: acct (type: string), COALESCE(block_code_1,'
') (type: string), COALESCE(block_code_2,' ') (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 23508476 Data size: 40288918516
Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 23508476 Data size: 40288918516
Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string), _col2 (type:
string)
Map 12
Map Operator Tree:
TableScan
alias: vp_ambs_tmp_credit_acct_h_01
Statistics: Num rows: 47016953 Data size: 80577838747 Basic
stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (org = '241') (type: boolean)
Statistics: Num rows: 23508476 Data size: 40288918516 Basic
stats: COMPLETE Column stats: NONE
Select Operator
expressions: acct (type: string), COALESCE(block_code_1,'
') (type: string), CASE WHEN (((block_code_2 = 'Z') and
(COALESCE(block_code_2_memo,' ')) IN ('UCF', 'CUCF'))) THEN (' ') ELSE
(COALESCE(block_code_2,' ')) END (type: string), block_code_1_set_date (type:
string), block_code_2_set_date (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 23508476 Data size: 40288918516
Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 23508476 Data size: 40288918516
Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string), _col2 (type:
string), _col3 (type: string), _col4 (type: string)
Map 13
Map Operator Tree:
TableScan
alias: g3
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: COALESCE(block_code,' ') (type: string)
sort order: +
Map-reduce partition columns: COALESCE(block_code,' ')
(type: string)
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Map 14
Map Operator Tree:
TableScan
alias: g4
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: COALESCE(block_code,' ') (type: string)
sort order: +
Map-reduce partition columns: COALESCE(block_code,' ')
(type: string)
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Map 15
Map Operator Tree:
TableScan
alias: g5
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: COALESCE(block_code,' ') (type: string)
sort order: +
Map-reduce partition columns: COALESCE(block_code,' ')
(type: string)
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
value expressions: pri (type: bigint)
Map 16
Map Operator Tree:
TableScan
alias: g6
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: COALESCE(block_code,' ') (type: string)
sort order: +
Map-reduce partition columns: COALESCE(block_code,' ')
(type: string)
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
value expressions: pri (type: bigint)
Map 17
Map Operator Tree:
TableScan
alias: t2
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: pri (type: bigint)
sort order: +
Map-reduce partition columns: pri (type: bigint)
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
value expressions: block_code (type: string)
Map 4
Map Operator Tree:
TableScan
alias: vp_ambs_tmp_credit_acct_h_01
Statistics: Num rows: 47016953 Data size: 80577838747 Basic
stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (org = '241') (type: boolean)
Statistics: Num rows: 23508476 Data size: 40288918516 Basic
stats: COMPLETE Column stats: NONE
Select Operator
expressions: acct (type: string), COALESCE(block_code_1,'
') (type: string), CASE WHEN (((block_code_2 = 'Z') and
(COALESCE(block_code_2_memo,' ')) IN ('UCF', 'CUCF'))) THEN (' ') ELSE
(COALESCE(block_code_2,' ')) END (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 23508476 Data size: 40288918516
Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 23508476 Data size: 40288918516
Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string), _col2 (type:
string)
Map 5
Map Operator Tree:
TableScan
alias: g3
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: COALESCE(block_code,' ') (type: string)
sort order: +
Map-reduce partition columns: COALESCE(block_code,' ')
(type: string)
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
value expressions: pri (type: bigint)
Map 6
Map Operator Tree:
TableScan
alias: g4
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: COALESCE(block_code,' ') (type: string)
sort order: +
Map-reduce partition columns: COALESCE(block_code,' ')
(type: string)
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
value expressions: pri (type: bigint)
Map 7
Map Operator Tree:
TableScan
alias: g5
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: COALESCE(block_code,' ') (type: string)
sort order: +
Map-reduce partition columns: COALESCE(block_code,' ')
(type: string)
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Map 8
Map Operator Tree:
TableScan
alias: g6
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: COALESCE(block_code,' ') (type: string)
sort order: +
Map-reduce partition columns: COALESCE(block_code,' ')
(type: string)
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Map 9
Map Operator Tree:
TableScan
alias: t2
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: pri (type: bigint)
sort order: +
Map-reduce partition columns: pri (type: bigint)
Statistics: Num rows: 22 Data size: 4158 Basic stats:
COMPLETE Column stats: NONE
value expressions: block_code (type: string)
Reducer 11
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col7, _col8, _col9,
_col10
Statistics: Num rows: 25859324 Data size: 44317811328 Basic
stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Left Outer Join0 to 1
keys:
0 COALESCE(_col1,' ') (type: string)
1 COALESCE(block_code,' ') (type: string)
outputColumnNames: _col0, _col2, _col7, _col8, _col9, _col10
input vertices:
1 Map 13
Statistics: Num rows: 28445257 Data size: 48749593517 Basic
stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Left Outer Join0 to 1
keys:
0 COALESCE(_col2,' ') (type: string)
1 COALESCE(block_code,' ') (type: string)
outputColumnNames: _col0, _col7, _col8, _col9, _col10
input vertices:
1 Map 14
Statistics: Num rows: 31289783 Data size: 53624554030 Basic
stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Left Outer Join0 to 1
keys:
0 COALESCE(_col7,' ') (type: string)
1 COALESCE(block_code,' ') (type: string)
outputColumnNames: _col0, _col8, _col9, _col10, _col26
input vertices:
1 Map 15
Statistics: Num rows: 34418762 Data size: 58987010711
Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Left Outer Join0 to 1
keys:
0 COALESCE(_col8,' ') (type: string)
1 COALESCE(block_code,' ') (type: string)
outputColumnNames: _col0, _col9, _col10, _col26, _col32
input vertices:
1 Map 16
Statistics: Num rows: 37860639 Data size: 64885713188
Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), COALESCE(_col26,0)
(type: bigint), COALESCE(_col32,0) (type: bigint), COALESCE(_col9,'29991231')
(type: string), COALESCE(_col10,'29991231') (type: string)
outputColumnNames: _col0, _col8, _col11, _col9, _col12
Statistics: Num rows: 37860639 Data size: 64885713188
Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), CASE WHEN
((_col8 = _col11)) THEN (_col8) WHEN ((_col8 > _col11)) THEN (_col8) WHEN
((_col8 < _col11)) THEN (_col11) END (type: bigint), CASE WHEN ((_col8 =
_col11)) THEN (least(_col9,_col12)) WHEN ((_col8 > _col11)) THEN (_col9) WHEN
((_col8 < _col11)) THEN (_col12) END (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 37860639 Data size:
64885713188 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Left Outer Join0 to 1
keys:
0 _col1 (type: bigint)
1 pri (type: bigint)
outputColumnNames: _col0, _col2, _col3
input vertices:
1 Map 17
Statistics: Num rows: 41646703 Data size:
71374286053 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), '241' (type:
string), _col3 (type: string), _col2 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 41646703 Data size:
71374286053 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
Statistics: Num rows: 83293406 Data size:
142748572106 Basic stats: COMPLETE Column stats: NONE
table:
input format:
org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde:
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 2
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col7,
_col8
Statistics: Num rows: 25859324 Data size: 44317811328 Basic
stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Left Outer Join0 to 1
keys:
0 COALESCE(_col1,' ') (type: string)
1 COALESCE(block_code,' ') (type: string)
outputColumnNames: _col0, _col2, _col3, _col4, _col7, _col8,
_col14
input vertices:
1 Map 5
Statistics: Num rows: 28445257 Data size: 48749593517 Basic
stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Left Outer Join0 to 1
keys:
0 COALESCE(_col2,' ') (type: string)
1 COALESCE(block_code,' ') (type: string)
outputColumnNames: _col0, _col3, _col4, _col7, _col8,
_col14, _col20
input vertices:
1 Map 6
Statistics: Num rows: 31289783 Data size: 53624554030 Basic
stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Left Outer Join0 to 1
keys:
0 COALESCE(_col7,' ') (type: string)
1 COALESCE(block_code,' ') (type: string)
outputColumnNames: _col0, _col3, _col4, _col8, _col14,
_col20
input vertices:
1 Map 7
Statistics: Num rows: 34418762 Data size: 58987010711
Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Left Outer Join0 to 1
keys:
0 COALESCE(_col8,' ') (type: string)
1 COALESCE(block_code,' ') (type: string)
outputColumnNames: _col0, _col3, _col4, _col14, _col20
input vertices:
1 Map 8
Statistics: Num rows: 37860639 Data size: 64885713188
Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), COALESCE(_col14,0)
(type: bigint), COALESCE(_col20,0) (type: bigint), COALESCE(_col3,'29991231')
(type: string), COALESCE(_col4,'29991231') (type: string)
outputColumnNames: _col0, _col2, _col5, _col3, _col6
Statistics: Num rows: 37860639 Data size: 64885713188
Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), CASE WHEN
((_col2 = _col5)) THEN (_col2) WHEN ((_col2 > _col5)) THEN (_col2) WHEN ((_col2
< _col5)) THEN (_col5) END (type: bigint), CASE WHEN ((_col2 = _col5)) THEN
(least(_col3,_col6)) WHEN ((_col2 > _col5)) THEN (_col3) WHEN ((_col2 < _col5))
THEN (_col6) END (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 37860639 Data size:
64885713188 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Left Outer Join0 to 1
keys:
0 _col1 (type: bigint)
1 pri (type: bigint)
outputColumnNames: _col0, _col2, _col3
input vertices:
1 Map 9
Statistics: Num rows: 41646703 Data size:
71374286053 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), '242' (type:
string), _col3 (type: string), _col2 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 41646703 Data size:
71374286053 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
Statistics: Num rows: 83293406 Data size:
142748572106 Basic stats: COMPLETE Column stats: NONE
table:
input format:
org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde:
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 3
Vertex: Union 3
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
> hive use tez engine occur org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row (tag=0)
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-16967
> URL: https://issues.apache.org/jira/browse/HIVE-16967
> Project: Hive
> Issue Type: Bug
> Components: Tez
> Affects Versions: 1.1.0
> Environment: hive version:1.1.0
> tez version:0.85
> hadoop version:hadoop2.6 -cdh-5.10.0
> Reporter: xujie
>
> Status: Running (Executing on YARN cluster with App id
> application_1494499520849_1063145)
> --------------------------------------------------------------------------------
> VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED
> KILLED
> --------------------------------------------------------------------------------
> Map 1 ......... KILLED 15 14 0 1 0
> 1
> Map 10 ......... SUCCEEDED 15 15 0 0 0
> 0
> Map 12 ......... SUCCEEDED 15 15 0 0 0
> 0
> Map 13 ......... SUCCEEDED 1 1 0 0 0
> 0
> Map 14 ......... SUCCEEDED 1 1 0 0 0
> 0
> Map 15 ......... SUCCEEDED 1 1 0 0 0
> 0
> Map 16 ......... SUCCEEDED 1 1 0 0 0
> 0
> Map 17 ......... SUCCEEDED 1 1 0 0 0
> 0
> Map 4 .......... SUCCEEDED 15 15 0 0 0
> 0
> Map 5 .......... SUCCEEDED 1 1 0 0 0
> 0
> Map 6 .......... SUCCEEDED 1 1 0 0 0
> 0
> Map 7 .......... SUCCEEDED 1 1 0 0 0
> 0
> Map 8 .......... SUCCEEDED 1 1 0 0 0
> 0
> Map 9 .......... SUCCEEDED 1 1 0 0 0
> 0
> Reducer 11 FAILED 315 0 0 315 774
> 314
> Reducer 2 KILLED 315 0 0 315 0
> 315
> --------------------------------------------------------------------------------
> VERTICES: 14/16 [==>>------------------------] 9% ELAPSED TIME: 130.89 s
>
> --------------------------------------------------------------------------------
> Status: Failed
> Vertex re-running, vertexName=Map 12,
> vertexId=vertex_1494499520849_1063145_1_07
> Vertex re-running, vertexName=Map 1,
> vertexId=vertex_1494499520849_1063145_1_11
> Vertex re-running, vertexName=Map 10,
> vertexId=vertex_1494499520849_1063145_1_06
> Vertex re-running, vertexName=Map 4,
> vertexId=vertex_1494499520849_1063145_1_03
> Vertex re-running, vertexName=Map 10,
> vertexId=vertex_1494499520849_1063145_1_06
> Vertex re-running, vertexName=Map 4,
> vertexId=vertex_1494499520849_1063145_1_03
> Vertex re-running, vertexName=Map 12,
> vertexId=vertex_1494499520849_1063145_1_07
> Vertex re-running, vertexName=Map 4,
> vertexId=vertex_1494499520849_1063145_1_03
> Vertex re-running, vertexName=Map 10,
> vertexId=vertex_1494499520849_1063145_1_06
> Vertex re-running, vertexName=Map 5,
> vertexId=vertex_1494499520849_1063145_1_02
> Vertex failed, vertexName=Reducer 11,
> vertexId=vertex_1494499520849_1063145_1_14, diagnostics=[Task failed,
> taskId=task_1494499520849_1063145_1_14_000013, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1494499520849_1063145_1_14_000013_0:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row (tag=0)
> {"key":{"reducesinkkey0":"0002991000585141671"},"value":{"_col0":"","_col1":""}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:187)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1796)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":"0002991000585141671"},"value":{"_col0":"","_col1":""}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:270)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{"reducesinkkey0":"0002991000585141671"},"value":{"_col0":"","_col1":""}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
> ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected
> exception: null
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:318)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:642)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.createForwardJoinObject(CommonJoinOperator.java:437)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:529)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:526)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genJoinObject(CommonJoinOperator.java:455)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:761)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinObject(CommonMergeJoinOperator.java:248)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:213)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.processOp(CommonMergeJoinOperator.java:196)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
> ... 17 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:257)
> ... 28 more
> ], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) :
> attempt_1494499520849_1063145_1_14_000013_1:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row (tag=0)
> {"key":{"reducesinkkey0":"0002991000585141671"},"value":{"_col0":"","_col1":""}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:187)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1796)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":"0002991000585141671"},"value":{"_col0":"","_col1":""}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:270)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{"reducesinkkey0":"0002991000585141671"},"value":{"_col0":"","_col1":""}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
> ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected
> exception: null
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:318)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:638)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.createForwardJoinObject(CommonJoinOperator.java:433)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:525)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:522)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genJoinObject(CommonJoinOperator.java:451)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:752)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinObject(CommonMergeJoinOperator.java:248)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:213)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.processOp(CommonMergeJoinOperator.java:196)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
> ... 17 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:257)
> ... 28 more
> ], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) :
> attempt_1494499520849_1063145_1_14_000013_2:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row (tag=0)
> {"key":{"reducesinkkey0":"0002991000585141671"},"value":{"_col0":"","_col1":""}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:187)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1796)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":"0002991000585141671"},"value":{"_col0":"","_col1":""}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:270)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{"reducesinkkey0":"0002991000585141671"},"value":{"_col0":"","_col1":""}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
> ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected
> exception: null
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:318)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:642)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.createForwardJoinObject(CommonJoinOperator.java:437)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:529)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:526)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genJoinObject(CommonJoinOperator.java:455)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:761)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinObject(CommonMergeJoinOperator.java:248)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:213)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.processOp(CommonMergeJoinOperator.java:196)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
> ... 17 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:257)
> ... 28 more
> ], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) :
> attempt_1494499520849_1063145_1_14_000013_3:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row (tag=0)
> {"key":{"reducesinkkey0":"0002991000585141671"},"value":{"_col0":"","_col1":""}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:187)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1796)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":"0002991000585141671"},"value":{"_col0":"","_col1":""}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:270)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{"reducesinkkey0":"0002991000585141671"},"value":{"_col0":"","_col1":""}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
> ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected
> exception: null
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:318)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:642)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.createForwardJoinObject(CommonJoinOperator.java:437)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:529)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:526)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genJoinObject(CommonJoinOperator.java:455)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:761)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinObject(CommonMergeJoinOperator.java:248)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:213)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.processOp(CommonMergeJoinOperator.java:196)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
> ... 17 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:257)
> ... 28 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:314, Vertex vertex_1494499520849_1063145_1_14 [Reducer 11]
> killed/failed due to:OWN_TASK_FAILURE]
> Vertex killed, vertexName=Map 1, vertexId=vertex_1494499520849_1063145_1_11,
> diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not
> succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex
> vertex_1494499520849_1063145_1_11 [Map 1] killed/failed due
> to:OTHER_VERTEX_FAILURE]
> Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1494499520849_1063145_1_15, diagnostics=[Vertex received Kill
> while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE,
> failedTasks:0 killedTasks:315, Vertex vertex_1494499520849_1063145_1_15
> [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]
> DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:2
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
