[
https://issues.apache.org/jira/browse/HDDS-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17627218#comment-17627218
]
Sadanand Shenoy commented on HDDS-6126:
---------------------------------------
Ran the test locally and it turns out that only the hadoop27 tests of the
acceptance MR suite are failing, As mentioned in the description the issue is
that yarn doesn’t release the AM containers even after the MR job completes,
There is some liveliness monitor that tracks the activity and it times out
after 10 min which triggers a kill/release of the AM container. This is not
occurring in hadoop3 tests which means the issue got fixed in later versions
however still not sure how it’s related to ozone streaming even OM, SCM , DN
doesn't log anything meaningful for the 10 min period.
> MR Acceptance test failure on Streaming Branch (HDDS-4454)
> ----------------------------------------------------------
>
> Key: HDDS-6126
> URL: https://issues.apache.org/jira/browse/HDDS-6126
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sadanand Shenoy
> Assignee: Sadanand Shenoy
> Priority: Major
> Attachments: NM-jstack.txt
>
>
> The ozone MR Acceptance fails on the streaming branch if
> ozone.fs.datastream.enable is set to true.
> The test basically runs a yarn wordcount job and the observation here is
> that the job doesn’t release the Application master (AM container ) as soon
> as the job is done and the job counters are displayed which is why the second
> job waits for a long time ~10 min for the AM container to be released.
> Tried with enabling streaming and disabling streaming and verified that the
> problem occurs only when ozone.fs.datastream.enable=true
> {code:java}
> yarn jar
> /opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar pi -D
> fs.defaultFS=o3fs://bucket1.volume1.om/ -D ozone.fs.datastream.enable=true
> 3 3{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
