I suspect the env issue? I reprovision the cluster again and it still works. I've it recorded for your reference: https://youtu.be/7fL6tEYZcNA
I'm on mac:. Docker version 18.06.1-ce, build e68fc7a docker-compose version 1.22.0, build f46880f Also, did you try to curl the web ui inside the container? Is it reachable? Evans Jason Wen <zhenshan....@workday.com> 於 2021年2月20日 週六 上午7:30寫道： > I see the hadoop processes are running in the container, and I am using > 1.5.0 release. > > > > [root@0a92e78b6832 /]# jps > > 1748 NameNode > > 1912 DataNode > > 2745 Jps > > 1499 NodeManager > > 988 WebAppProxyServer > > 1372 JobHistoryServer > > 1086 ResourceManager > > > > Curl returns following result: > > http://localhost:57001 > > curl: (52) Empty reply from server > > > > Thanks, > > Jason > > > > *From: *Evans Ye <evan...@apache.org> > *Reply-To: *"user@bigtop.apache.org" <user@bigtop.apache.org> > *Date: *Friday, February 19, 2021 at 11:37 AM > *To: *"user@bigtop.apache.org" <user@bigtop.apache.org> > *Subject: *Re: docker-provisioner question > > > > Yes. the config for docker-compose.yaml is the only thing that needs to be > added. > > Make sure you've the daemons up and running: > > > > $ ./docker-hadoop.sh -e 1 bash > > WARNING: The DOCKER_IMAGE variable is not set. Defaulting to a blank > string. > > WARNING: The MEM_LIMIT variable is not set. Defaulting to a blank string. > > [root@36eb2f2533c4 /]# jps > > 2290 Jps > > 1941 DataNode > > 1001 WebAppProxyServer > > 1101 ResourceManager > > 1389 JobHistoryServer > > 1518 NodeManager > > 1775 NameNode > > > > If the namenode/resourcemanager are not up. The provisioning might has an > issue. Make sure you've switched to branch-1.5 because the master branch is > currently in transition upgrading to Hadoop 3 and the provisioner might > fail. > > > > - Evans > > > > Jason Wen <zhenshan....@workday.com> 於 2021年2月20日 週六 上午12:18寫道： > > Hi Evans, > > > > Thanks for your reply. > > I modified docker-compose.yaml as you suggested and run `./gradlew > docker-provisioner -Pprefix=1.5.0 -Pnum_instances=3` > > I can see the port mapping on docker containers: > > CONTAINER ID IMAGE COMMAND > CREATED STATUS > PORTS > NAMES > > d97eada3f01d bigtop/puppet:1.5.0-centos-7 "/sbin/init" 9 > hours ago Up 9 hours 0.0.0.0:18001->8088/tcp, > 0.0.0.0:57001->50070/tcp, 0.0.0.0:57501->50075/tcp > 20210218_232241_r25597_bigtop_3 > > 20a7fbadd5f8 bigtop/puppet:1.5.0-centos-7 "/sbin/init" 9 > hours ago Up 9 hours 0.0.0.0:18000->8088/tcp, > 0.0.0.0:57000->50070/tcp, 0.0.0.0:57500->50075/tcp > 20210218_232241_r25597_bigtop_2 > > 0a92e78b6832 bigtop/puppet:1.5.0-centos-7 "/sbin/init" 9 > hours ago Up 9 hours 0.0.0.0:18002->8088/tcp, > 0.0.0.0:57002->50070/tcp, 0.0.0.0:57502->50075/tcp > 20210218_232241_r25597_bigtop_1 > > > > However it does not work when I try to access http://localhost:57000 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A57000&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=cp-l8GE6gVhlReZ92oxBuSBinEwCtGcCCPxO9zZAYgI&s=TQgifr-OVsB3jPD1wjb6lvhvr-a7kbtSz_fIXZapEPw&e=> > or http://localhost:57001 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A57001&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=cp-l8GE6gVhlReZ92oxBuSBinEwCtGcCCPxO9zZAYgI&s=Aog6ht98dRjfWzKdfGEN5p-PRMmUCzh8evfXTJz8ahs&e=> > or http://localhost:57002 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A57002&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=cp-l8GE6gVhlReZ92oxBuSBinEwCtGcCCPxO9zZAYgI&s=mrRp8-npBVxHzqLfhYA9tg8bXOHT8D3Nv32Yw3lChrE&e=> > > Is there any other step that I missed? > > > > Thanks, > > Jason > > > > *From: *Evans Ye <evan...@apache.org> > *Reply-To: *"user@bigtop.apache.org" <user@bigtop.apache.org> > *Date: *Wednesday, February 17, 2021 at 7:48 PM > *To: *"user@bigtop.apache.org" <user@bigtop.apache.org> > *Subject: *Re: docker-provisioner question > > > > Hi Jason, > > > > Out of the box it does not support viewing Web UIs. However, you can > easily make that available by adding one line to config port exposing in > provisioner's docker-compose.yaml[1]. Try to add this: > > > > bigtop: > > image: ${DOCKER_IMAGE} > > command: /sbin/init > > domainname: bigtop.apache.org > <https://urldefense.proofpoint.com/v2/url?u=http-3A__bigtop.apache.org&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=-4GOcb1jNZJyz3xuMVFK0x2SMwIuOz7bmR8Trk_rtik&s=vDb_YQxDhJZJ8kj7X1SeVM0QefVztf4Qd6a2HKw7c58&e=> > > privileged: true > > mem_limit: ${MEM_LIMIT} > > volumes: > > - ../../:/bigtop-home > > - ./config/hiera.yaml:/etc/puppet/hiera.yaml > > - ./config/hieradata:/etc/puppet/hieradata > > - ./config/hosts:/etc/hosts > > - /sys/fs/cgroup:/sys/fs/cgroup:ro > > * ports:* > > * - 57000-57099:50070* > > * - 57500-57599:50075* > > * - 18000-18099:8088* > > > > See [2][3] for reference. > > For example, containers' 50070 are mapped host's 50070~50099. The mapping > is automatically done by docker. > > You can use the -l command to see actual mapping: > > > > ./docker-hadoop.sh -l > > Name Command State > Ports > > > -------------------------------------------------------------------------------------------------------------------------------- > > 20210218_113240_r104_bigtop_1 /sbin/init Up 0.0.0.0:57003->50070/tcp, > 0.0.0.0:57503->50075/tcp, 0.0.0.0:18003->8088/tcp > > 20210218_113240_r104_bigtop_2 /sbin/init Up 0.0.0.0:57002->50070/tcp, > 0.0.0.0:57502->50075/tcp, 0.0.0.0:18002->8088/tcp > > > > I think it would be great to open up a PR to add the feature after you > have it tested on your end. What do you think? > > > > Best, > > Evans Ye > > > > > > [1] > https://github.com/apache/bigtop/blob/master/provisioner/docker/docker-compose.yml > <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_bigtop_blob_master_provisioner_docker_docker-2Dcompose.yml&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=-4GOcb1jNZJyz3xuMVFK0x2SMwIuOz7bmR8Trk_rtik&s=Bn4eJbjimODq_lblNY-w0G18jsUFlV5KhyYwRJ-dCpA&e=> > > [2] https://docs.docker.com/engine/reference/run/#expose-incoming-ports > <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.docker.com_engine_reference_run_-23expose-2Dincoming-2Dports&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=-4GOcb1jNZJyz3xuMVFK0x2SMwIuOz7bmR8Trk_rtik&s=b-xqfN5yad-JycWdF2rfroo18KjeE24vBaqWITE4wD4&e=> > > [3] https://github.com/compose-spec/compose-spec/blob/master/spec.md#ports > <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_compose-2Dspec_compose-2Dspec_blob_master_spec.md-23ports&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=-4GOcb1jNZJyz3xuMVFK0x2SMwIuOz7bmR8Trk_rtik&s=hg4Lg2RWu4vLvigN1O1Fq0IEoGzGSa_4zUXg2E8aJKQ&e=> > > > > Jason Wen <zhenshan....@workday.com> 於 2021年2月18日 週四 上午12:44寫道： > > Hi, > > > > I am following > https://cwiki.apache.org/confluence/display/BIGTOP/Quickstart+Guide%3A+Bigtop+Integration+Test+Framework+2.0 > <https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_BIGTOP_Quickstart-2BGuide-253A-2BBigtop-2BIntegration-2BTest-2BFramework-2B2.0&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=-4GOcb1jNZJyz3xuMVFK0x2SMwIuOz7bmR8Trk_rtik&s=UgN_KP5zcmTmhk70LyUTB1xgvE_4NHcuz5QL9QRVzlg&e=> > to use bigtop. I have deployed a Hadoop cluster successfully using ` > ./gradlew docker-provisioner`. How can I access the > Namenode/ResourceManager web UI from my local machine? > > > > Thanks, > > Jason > >
