Yes. the config for docker-compose.yaml is the only thing that needs to be added. Make sure you've the daemons up and running:
$ ./docker-hadoop.sh -e 1 bash WARNING: The DOCKER_IMAGE variable is not set. Defaulting to a blank string. WARNING: The MEM_LIMIT variable is not set. Defaulting to a blank string. [root@36eb2f2533c4 /]# jps 2290 Jps 1941 DataNode 1001 WebAppProxyServer 1101 ResourceManager 1389 JobHistoryServer 1518 NodeManager 1775 NameNode If the namenode/resourcemanager are not up. The provisioning might has an issue. Make sure you've switched to branch-1.5 because the master branch is currently in transition upgrading to Hadoop 3 and the provisioner might fail. - Evans Jason Wen <zhenshan....@workday.com> 於 2021年2月20日 週六 上午12:18寫道： > Hi Evans, > > > > Thanks for your reply. > > I modified docker-compose.yaml as you suggested and run `./gradlew > docker-provisioner -Pprefix=1.5.0 -Pnum_instances=3` > > I can see the port mapping on docker containers: > > CONTAINER ID IMAGE COMMAND > CREATED STATUS > PORTS > NAMES > > d97eada3f01d bigtop/puppet:1.5.0-centos-7 "/sbin/init" 9 > hours ago Up 9 hours 0.0.0.0:18001->8088/tcp, > 0.0.0.0:57001->50070/tcp, 0.0.0.0:57501->50075/tcp > 20210218_232241_r25597_bigtop_3 > > 20a7fbadd5f8 bigtop/puppet:1.5.0-centos-7 "/sbin/init" 9 > hours ago Up 9 hours 0.0.0.0:18000->8088/tcp, > 0.0.0.0:57000->50070/tcp, 0.0.0.0:57500->50075/tcp > 20210218_232241_r25597_bigtop_2 > > 0a92e78b6832 bigtop/puppet:1.5.0-centos-7 "/sbin/init" 9 > hours ago Up 9 hours 0.0.0.0:18002->8088/tcp, > 0.0.0.0:57002->50070/tcp, 0.0.0.0:57502->50075/tcp > 20210218_232241_r25597_bigtop_1 > > > > However it does not work when I try to access http://localhost:57000 or > http://localhost:57001 or http://localhost:57002 > > Is there any other step that I missed? > > > > Thanks, > > Jason > > > > *From: *Evans Ye <evan...@apache.org> > *Reply-To: *"user@bigtop.apache.org" <user@bigtop.apache.org> > *Date: *Wednesday, February 17, 2021 at 7:48 PM > *To: *"user@bigtop.apache.org" <user@bigtop.apache.org> > *Subject: *Re: docker-provisioner question > > > > Hi Jason, > > > > Out of the box it does not support viewing Web UIs. However, you can > easily make that available by adding one line to config port exposing in > provisioner's docker-compose.yaml[1]. Try to add this: > > > > bigtop: > > image: ${DOCKER_IMAGE} > > command: /sbin/init > > domainname: bigtop.apache.org > <https://urldefense.proofpoint.com/v2/url?u=http-3A__bigtop.apache.org&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=-4GOcb1jNZJyz3xuMVFK0x2SMwIuOz7bmR8Trk_rtik&s=vDb_YQxDhJZJ8kj7X1SeVM0QefVztf4Qd6a2HKw7c58&e=> > > privileged: true > > mem_limit: ${MEM_LIMIT} > > volumes: > > - ../../:/bigtop-home > > - ./config/hiera.yaml:/etc/puppet/hiera.yaml > > - ./config/hieradata:/etc/puppet/hieradata > > - ./config/hosts:/etc/hosts > > - /sys/fs/cgroup:/sys/fs/cgroup:ro > > * ports:* > > * - 57000-57099:50070* > > * - 57500-57599:50075* > > * - 18000-18099:8088* > > > > See [2][3] for reference. > > For example, containers' 50070 are mapped host's 50070~50099. The mapping > is automatically done by docker. > > You can use the -l command to see actual mapping: > > > > ./docker-hadoop.sh -l > > Name Command State > Ports > > > -------------------------------------------------------------------------------------------------------------------------------- > > 20210218_113240_r104_bigtop_1 /sbin/init Up 0.0.0.0:57003->50070/tcp, > 0.0.0.0:57503->50075/tcp, 0.0.0.0:18003->8088/tcp > > 20210218_113240_r104_bigtop_2 /sbin/init Up 0.0.0.0:57002->50070/tcp, > 0.0.0.0:57502->50075/tcp, 0.0.0.0:18002->8088/tcp > > > > I think it would be great to open up a PR to add the feature after you > have it tested on your end. What do you think? > > > > Best, > > Evans Ye > > > > > > [1] > https://github.com/apache/bigtop/blob/master/provisioner/docker/docker-compose.yml > <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_bigtop_blob_master_provisioner_docker_docker-2Dcompose.yml&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=-4GOcb1jNZJyz3xuMVFK0x2SMwIuOz7bmR8Trk_rtik&s=Bn4eJbjimODq_lblNY-w0G18jsUFlV5KhyYwRJ-dCpA&e=> > > [2] https://docs.docker.com/engine/reference/run/#expose-incoming-ports > <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.docker.com_engine_reference_run_-23expose-2Dincoming-2Dports&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=-4GOcb1jNZJyz3xuMVFK0x2SMwIuOz7bmR8Trk_rtik&s=b-xqfN5yad-JycWdF2rfroo18KjeE24vBaqWITE4wD4&e=> > > [3] https://github.com/compose-spec/compose-spec/blob/master/spec.md#ports > <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_compose-2Dspec_compose-2Dspec_blob_master_spec.md-23ports&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=-4GOcb1jNZJyz3xuMVFK0x2SMwIuOz7bmR8Trk_rtik&s=hg4Lg2RWu4vLvigN1O1Fq0IEoGzGSa_4zUXg2E8aJKQ&e=> > > > > Jason Wen <zhenshan....@workday.com> 於 2021年2月18日 週四 上午12:44寫道： > > Hi, > > > > I am following > https://cwiki.apache.org/confluence/display/BIGTOP/Quickstart+Guide%3A+Bigtop+Integration+Test+Framework+2.0 > <https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_BIGTOP_Quickstart-2BGuide-253A-2BBigtop-2BIntegration-2BTest-2BFramework-2B2.0&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=UflFQf1BWcrVtfjfN1LUqWWh-UBP5XtRGMdcDC-0P7o&m=-4GOcb1jNZJyz3xuMVFK0x2SMwIuOz7bmR8Trk_rtik&s=UgN_KP5zcmTmhk70LyUTB1xgvE_4NHcuz5QL9QRVzlg&e=> > to use bigtop. I have deployed a Hadoop cluster successfully using ` > ./gradlew docker-provisioner`. How can I access the > Namenode/ResourceManager web UI from my local machine? > > > > Thanks, > > Jason > >
