I am 100% up to go container. We need to embrace the changes and right now container seems to be a dominating solution.
Breaking down into a concrete plan, we got some choices: 1. Leverage docker provisiner to provision cluster on swarm using compose 2. Develop docker based packaging solution 2a. Reuse our RPM/DEB packages to create docker images 2b. Develop a brand new packaging solution for docker and drop puppet deploy I would favor 2a. Any other thoughts? Olaf Flebbe <[email protected]>於 2017年12月17日 週日，上午1:57寫道： > Hi Jeff, > > Regarding your comment on *https://issues.apache.org/jira/browse/BIGTOP-2754 > <https://issues.apache.org/jira/browse/BIGTOP-2754>* > > Well, pretty ugly story. The thread leading to reverting zookeeper is > here: > > > https://lists.apache.org/thread.html/c2e9705ee7bdc8afb7b9e79683dda18b0079aec215079b608862c99b@%3Cdev.bigtop.apache.org%3E > > Included you will find the unresolved Hadoop JIRA tickets > > https://issues.apache.org/jira/browse/HADOOP-12928 > > https://issues.apache.org/jira/browse/HADOOP-13413 > > > However, Jay flashed a proposal several times that we should "go > container". I am now ready to see this necessary. > > I think the hadoop ecosystem should interoperate on stable network > protocols, not by sharing Java artifacts. > > This would put the final nail in the coffin of the attempt harmonising > protocols by symlinking client and server packages in order to enforce same > versions of client and server. We would give up most of our inter-package > dependencies, every package get its own dependency packaged, pulled from > whatever the upstream project decided. > > This may solve issues like the zookeeper / hadoop interdependency . > Supplying a zookeeper-3.11 server package would be not an issue any more. > Packages needing zookeeper client would use their zookeeper client to > access the server. And than think "Hadoop-3" as a further example. > > For dependencies with native libraries this may lead to issues, which can > be solved IMHO. > > I am now out of the position to control a reasonable large hadoop bigtop > infra, so I am not sure this path will lead to disaster area, since I > cannot test the long time stability any more. > > Other Bigtoppers, please comment, since this will have pretty big impact . > > I like to see traditional Bigtop bare metal -- no containers -- supported > as well. At least as a corner case. > > Cheers > Olaf > > > > > Am 16.12.2017 um 02:54 schrieb Jeff Widman (JIRA) <[email protected]>: > > > [ > https://issues.apache.org/jira/browse/BIGTOP-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16293553#comment-16293553 > ] > > Jeff Widman commented on BIGTOP-2754: > ------------------------------------- > > Is there a ticket in Hadoop tracking this? > > Zookeeper 3.4.6-->3.4.9 have a couple of bugs affecting snapshot garbage > collection (ZOOKEEPER-1797 and ZOOKEEPER-2574), so we want to upgrade to > 3.4.11. > > However, when I tried pulling that from Zookeeper repo, I noticed that it > pins Java 6 which breaks on Ubuntu 16.04: > https://github.com/apache/zookeeper/blob/release-3.4.6/src/packages/update-zookeeper-env.sh#L152 > > I looked at the Zookeeper master branch, and looks like Zookeeper decided > in ZOOKEEPER-1604 to delegate packaging to the Bigtop project. However, > when I came here, I was disappointed to see that the newest Zookeeper > version that Bigtop packages is 3.4.6.... > > For those of us who use Zookeeper apart from Hadoop but don't want to deal > with the headaches of packaging it'd be nice to figure out a way to get a > newer zookeeper package released. I'm happy to take a look at the Hadoop > project to see if they can upgrade to 3.4.11, but not sure where to find > their rationale for pinning to 3.4.6... > > Revert BIGTOP-2730: Upgrade Zookeeper to version 3.4.10 > ------------------------------------------------------- > > Key: BIGTOP-2754 > URL: https://issues.apache.org/jira/browse/BIGTOP-2754 > Project: Bigtop > Issue Type: Bug > Affects Versions: 1.2.0 > Reporter: Olaf Flebbe > Assignee: Olaf Flebbe > Fix For: 1.2.1, 1.3.0 > > Attachments: BIGTOP-2754.patch > > > Since Hadoop enforces zookeeper-3.6 we will revert BIGTOP-2730 in order > to wait for upstream to deal with this issue. > > > > > -- > This message was sent by Atlassian JIRA > (v6.4.14#64029) > >
