[
https://issues.apache.org/jira/browse/HDDS-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632538#comment-17632538
]
Sadanand Shenoy commented on HDDS-6126:
---------------------------------------
{code:java}
~ % cd
hadoop-ozone/dist/target/ozone-1.3.0-SNAPSHOT/compose/ozone-mr/hadoop27
~ % docker-compose up --scale datanode=3 -d
Creating network "hadoop27_default" with the default driver
Creating hadoop27_nm_1 ... done
Creating hadoop27_scm_1 ... done
Creating hadoop27_om_1 ... done
Creating hadoop27_datanode_1 ... done
Creating hadoop27_datanode_2 ... done
Creating hadoop27_datanode_3 ... done
Creating hadoop27_rm_1 ... done
Creating hadoop27_s3g_1 ... done
~ % docker exec -it hadoop27_om_1 /bin/bash
bash-4.2$ ozone sh volume create volume1
bash-4.2$ ozone sh bucket create volume1/bucket1
bash-4.2$ exit
exit
~ % docker exec -it hadoop27_nm_1 /bin/bash
[hadoop@nm ~]$ yarn jar
/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar pi -D
fs.defaultFS=o3fs://bucket1.volume1.om/ -D ozone.fs.datastream.enable=true 3 3
{code}
The last command if run twice will hang on the second run waiting for 10 min
for AM container to be freed up.
> MR Acceptance test failure on Streaming Branch (HDDS-4454)
> ----------------------------------------------------------
>
> Key: HDDS-6126
> URL: https://issues.apache.org/jira/browse/HDDS-6126
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sadanand Shenoy
> Assignee: Sadanand Shenoy
> Priority: Major
> Attachments: NM-jstack.txt, nm_logs.log
>
>
> The ozone MR Acceptance fails on the streaming branch if
> ozone.fs.datastream.enable is set to true.
> The test basically runs a yarn wordcount job and the observation here is
> that the job doesn’t release the Application master (AM container ) as soon
> as the job is done and the job counters are displayed which is why the second
> job waits for a long time ~10 min for the AM container to be released.
> Tried with enabling streaming and disabling streaming and verified that the
> problem occurs only when ozone.fs.datastream.enable=true
> {code:java}
> yarn jar
> /opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar pi -D
> fs.defaultFS=o3fs://bucket1.volume1.om/ -D ozone.fs.datastream.enable=true
> 3 3{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
