Aggarwal-Raghav commented on PR #456: URL: https://github.com/apache/tez/pull/456#issuecomment-4092591297
1. I have used official hadoop docker image 3.4.2-lean 2. namenode and datanode will be separate service otherwise we need to use a custom entrypoint to start in a single container as happening in minimal-hadoop docker image. I think it is better to have separate. I have added a wait also to ensure namenode is not in safemode before datanode and tez-am is up. 3. Handled the review comments regarding docs and file naming conventions. 4. Added a sample program to run in tez-docker am and here it get tricky ⚠️. It is throwing same error as mentioned in TEZ-4686 with standalone program. Steps: ``` 1. cd tez-dist/src/docker/tez-am/ 2. docker compose up -d --build ``` <img width="1231" height="278" alt="Screenshot 2026-03-20 at 12 26 15 AM" src="https://github.com/user-attachments/assets/4019dec9-6ee0-48c8-ab44-d42677281ad6"; /> **Everything should be running at this point** ``` 3. docker exec -it tez-am bash 4. echo "Hello world Hello" > /tmp/input.txt 5. java -cp ./*:./lib/*:tez-examples-1.0.0-SNAPSHOT.jar org.apache.tez.examples.ExternalAmWordCount /tmp/input.txt /tmp/output ``` **With TEZ-4686 cherrypick:** ``` ❯ docker exec -it tez-am bash bash-5.1$ echo "Hello world Hello" > /tmp/input.txt java -cp ./*:./lib/*:tez-examples-1.0.0-SNAPSHOT.jar org.apache.tez.examples.ExternalAmWordCount /tmp/input.txt /tmp/output log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. bash-5.1$ ls /tmp/output attempt_1773945902335_0000_1_00_000000_0_10003 attempt_1773945902335_0000_1_00_000000_0_10003_0 part-v001-o000-r-00000 _SUCCESS bash-5.1$ cat /tmp/output/part-v001-o000-r-00000 Hello 2 world 1 bash-5.1$ ``` **Without TEZ-4686 cherrypick:** ``` ❯ docker exec -it tez-am bash bash-5.1$ echo "Hello world Hello" > /tmp/input.txt java -cp ./*:./lib/*:tez-examples-1.0.0-SNAPSHOT.jar org.apache.tez.examples.ExternalAmWordCount /tmp/input.txt /tmp/output log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. Exception in thread "main" java.lang.NullPointerException: Cannot invoke "org.apache.tez.client.registry.AMRecord.getApplicationId()" because "this.amRecord" is null at org.apache.tez.client.registry.zookeeper.ZkFrameworkClient.createApplication(ZkFrameworkClient.java:114) at org.apache.tez.client.TezClient.createApplication(TezClient.java:1103) at org.apache.tez.client.TezClient.start(TezClient.java:399) at org.apache.tez.examples.ExternalAmWordCount.main(ExternalAmWordCount.java:74)src/docker/tez-am on TEZ-4682 [⇡] took 20s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
