Nice work Kai! Quick comment: have you considered Bigtop? It allows creating a multi-node docker cluster too. But I'm not sure if Bigtop takes a custom built Hadoop (from the instructions I read, it builds from a certain git branch; I'm not sure if that customizable).
On Wed, Nov 9, 2016 at 5:39 AM Sasaki Kai <lewua...@me.com> wrote: > Hi Hadoop developers > > The other day I created a tool for launching multiple node hadoop cluster > on docker container. > You can easily launch multiple node hadoop cluster from your Hadoop source > code. > It is useful for testing and debugging. Actually I often use it before > submitting a patch to Hadoop project. > https://github.com/Lewuathe/docker-hadoop-cluster < > https://github.com/Lewuathe/docker-hadoop-cluster> > > And I also updated to build the latest trunk image automatically and > upload onto Docker Hub. > So you can easily check and test the latest trunk branch in the > environment which is more close to actual usage. > > If you already installed docker and docker-compose, what needed is > docker-compose.yml like this. > > version: '2' > > services: > master: > image: lewuathe/hadoop-master > ports: > - "9870:9870" > - "8088:8088" > - "19888:19888" > - "8188:8188" > container_name: "master" > slave1: > image: lewuathe/hadoop-slave > container_name: "slave1" > depends_on: > - master > ports: > - "9901:9864" > - "8041:8042" > slave2: > image: lewuathe/hadoop-slave > container_name: "slave2" > depends_on: > - master > ports: > - "9902:9864" > - "8042:8042" > > The usage in detail is described in the repository. > https://github.com/Lewuathe/docker-hadoop-cluster/blob/master/README.md < > https://github.com/Lewuathe/docker-hadoop-cluster/blob/master/README.md> > > I would be glad if you use this tool for developing and debugging and make > our development more efficient. > Please give me any feedbacks to me. Thanks you! > > ———————— > Kai Sasaki > mail: lewua...@me.com <mailto:lewua...@me.com> > github: https://github.com/Lewuathe <https://github.com/Lewuathe> > > > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap
