Marton, thank you for working on this. I think Official Docker images for Hadoop would be very useful for a lot of reasons. I think that it is better to have a coordinated effort with production ready base images with dependent images for prototyping. Does anyone else have an opinion about this?
Thank you, Miklos On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <[email protected]> wrote: > > TL;DR: I propose to create official hadoop images and upload them to the > dockerhub. > > GOAL/SCOPE: I would like improve the existing documentation with > easy-to-use docker based recipes to start hadoop clusters with various > configuration. > > The images also could be used to test experimental features. For example > ozone could be tested easily with these compose file and configuration: > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6 > > Or even the configuration could be included in the compose file: > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc > ker-compose.yaml > > I would like to create separated example compose files for federation, ha, > metrics usage, etc. to make it easier to try out and understand the > features. > > CONTEXT: There is an existing Jira https://issues.apache.org/jira > /browse/HADOOP-13397 > But it’s about a tool to generate production quality docker images > (multiple types, in a flexible way). If no objections, I will create a > separated issue to create simplified docker images for rapid prototyping > and investigating new features. And register the branch to the dockerhub to > create the images automatically. > > MY BACKGROUND: I am working with docker based hadoop/spark clusters quite > a while and run them succesfully in different environments (kubernetes, > docker-swarm, nomad-based scheduling, etc.) My work is available from here: > https://github.com/flokkr but they could handle more complex use cases > (eg. instrumenting java processes with btrace, or read/reload configuration > from consul). > And IMHO in the official hadoop documentation it’s better to suggest to > use official apache docker images and not external ones (which could be > changed). > > Please let me know if you have any comments. > > Marton > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
