Hi,

1. People are using docker for packaging and deployment instead of rpm/deb.

You can package everything as a container, but it may or may not make sense.

2. Hadoop 3's messy layout don't fit into rpm/deb conventions. Maybe it's
because they think it's the right design for the container era.

There is nothing prepared to run hadoop in a container. BTW it would be messy since yarn is supporting compute loads to run in a container. That would mean a nested container runtime. Feasible but unnecessary mind twisting.

Hadoop3 does fit in rpm/deb, one only would have to change the packaging layout to a monolith approach. Can be done, not nice, but feasible.

I am exhausted in explaining the points over and over again. Go and try, I tried it. For a secure hadoop service you need a known IP address, and a FQDN hostname resolution. You will find ton's of docker bug reports like https://github.com/moby/moby/issues/14282 or https://github.com/moby/moby/issues/29100 only to pick up two. I used lxc to demonstrate setting up a secure hadoop environment at Apache Conference in Budapest for a reason.

I am out of this project now. Do whatever you like.

Olaf

Reply via email to