[
https://issues.apache.org/jira/browse/HIVE-26828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis resolved HIVE-26828.
----------------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
Fixed in
https://github.com/apache/hive/commit/c126422a91be695c75ec4a750638a0aa4d1ba6cd.
> Fix OOM for hybridgrace_hashjoin_2.q
> ------------------------------------
>
> Key: HIVE-26828
> URL: https://issues.apache.org/jira/browse/HIVE-26828
> Project: Hive
> Issue Type: Bug
> Components: Test, Tez
> Affects Versions: 4.0.0-alpha-2
> Reporter: Alessandro Solimando
> Assignee: Stamatis Zampetakis
> Priority: Major
> Fix For: 4.0.0
>
>
> _hybridgrace_hashjoin_2.q_ test was disabled because it was failing with OOM
> transiently (from [flaky_test
> output|http://ci.hive.apache.org/blue/organizations/jenkins/hive-flaky-check/detail/hive-flaky-check/597/tests/],
> in case it disappears):
> {quote}< Status: Failed
> < Vertex failed, vertexName=Map 2, vertexId=vertex_#ID#, diagnostics=[Vertex
> vertex_#ID# [Map 2] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex
> Input: z1 initializer failed, vertex=vertex_#ID# [Map 2],
> java.lang.RuntimeException: Failed to load plan:
> hdfs://localhost:45033/home/jenkins/agent/workspace/hive-flaky-check/itests/qtest/target/tmp/scratchdir/jenkins/88f705a8-2d67-4d0a-92fd-d9617faf4e46/hive_2022-12-08_02-25-15_569_4666093830564098399-1/jenkins/_tez_scratch_dir/5b786380-b362-45e0-ac10-0f835ef1d8d7/map.xml
> < #### A masked pattern was here ####
> < Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException:
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> < Serialization trace:
> < childOperators (org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator)
> < childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> < aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> < #### A masked pattern was here ####
> < Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
> < #### A masked pattern was here ####
> < ]
> < [Masked Vertex killed due to OTHER_VERTEX_FAILURE]
> < [Masked Vertex killed due to OTHER_VERTEX_FAILURE]
> < [Masked Vertex killed due to OTHER_VERTEX_FAILURE]
> < [Masked Vertex killed due to OTHER_VERTEX_FAILURE]
> < [Masked Vertex killed due to OTHER_VERTEX_FAILURE]
> < DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:5
> < FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 2,
> vertexId=vertex_#ID#, diagnostics=[Vertex vertex_#ID# [Map 2] killed/failed
> due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: z1 initializer failed,
> vertex=vertex_#ID# [Map 2], java.lang.RuntimeException: Failed to load plan:
> hdfs://localhost:45033/home/jenkins/agent/workspace/hive-flaky-check/itests/qtest/target/tmp/scratchdir/jenkins/88f705a8-2d67-4d0a-92fd-d9617faf4e46/hive_2022-12-08_02-25-15_569_4666093830564098399-1/jenkins/_tez_scratch_dir/5b786380-b362-45e0-ac10-0f835ef1d8d7/map.xml
> < #### A masked pattern was here ####
> < Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException:
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> < Serialization trace:
> < childOperators (org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator)
> < childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> < aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> < #### A masked pattern was here ####
> < Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
> < #### A masked pattern was here ####
> < ][Masked Vertex killed due to OTHER_VERTEX_FAILURE][Masked Vertex killed
> due to OTHER_VERTEX_FAILURE][Masked Vertex killed due to
> OTHER_VERTEX_FAILURE][Masked Vertex killed due to
> OTHER_VERTEX_FAILURE][Masked Vertex killed due to OTHER_VERTEX_FAILURE]DAG
> did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:5
> < PREHOOK: query: SELECT COUNT( * )
> < FROM src1 x
> < JOIN srcpart z1 ON (x.key = z1.key)
> < JOIN src y1 ON (x.key = y1.key)
> < JOIN srcpart z2 ON (x.value = z2.value)
> < JOIN src y2 ON (x.value = y2.value)
> < WHERE z1.key < 'zzzzzzzz' AND z2.key < 'zzzzzzzzzz'
> < AND y1.value < 'zzzzzzzz' AND y2.value < 'zzzzzzzzzz'
> < PREHOOK: type: QUERY
> < PREHOOK: Input: default@src
> < PREHOOK: Input: default@src1
> < PREHOOK: Input: default@srcpart
> < PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11
> < PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> < PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> < PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> < PREHOOK: Output: hdfs://### HDFS PATH ###
> {quote}
> The aim of this ticket is to investigate the issue, fix it and re-enable the
> test.
> The problem seems to lie in the deserialization of the computed tez dag plan.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
