makes sense to me; This should help me to picture where bigtop is headed for the next several months.
So I guess the answer is "yes : we still beleive in multitenant packaging and systems". Thanks for all the feedback! > On Feb 11, 2015, at 3:13 AM, Bruno Mahé <[email protected]> wrote: > >> On 02/10/2015 10:05 PM, Roman Shaposhnik wrote: >>> On Tue, Feb 10, 2015 at 6:00 PM, RJ Nowling <[email protected]> wrote: >>> Can we articulate the value of packages over tarballs? In my view, packages >>> are useful for managing dependencies and in-place updates. >> In my view packages are the only way to get into the traditional IT >> deployment >> infrastructures. These are the same infrastructures that don't want to touch >> Ambari at all, since they are all standardized on Puppet/Chef and traditional >> Linux packaging. >> >> There's quite a few of them out there still, despite all the push of >> Silicon Valley >> to get everybody to things like Docker, etc. > > +1. > > I like docker and it is a very nice project. But it is not going to be an end > in itself. > Companies will continue to have various hosts, going from bare metal to > different clouds providers (SaaS, PaaS...), docker included. > > Aside from that, using packages provide so many benefits over tarballs: > * Packages have some metadata so I know what file belong where and how and > what version > * all the dependencies are specified in it. Which makes it easier to reuse > even across docker files. This includes system dependencies as well (ex: who > depends on psmisc? why? can it be removed now that we updated Apache Hadoop?) > * it enables us to respect the Single Responsibility Principe and to satisfy > everyone, folks using bare metal as well as cloud technologies users > * some patches may still need to be applied for compatibility/build reasons. > Using packages makes that easier > * It provides a deep integration with the system so "it just works". Users > are created, initscripts setup, alternatives setup, everything has the right > permissions... > * It makes it dead easy when I want to build multiple variants of the same > image since everything is pulled and setup correctly. If I were to manually > unpack tarballs, I would have to take care of that manually and also it would > take a lot more space than the package equivalent unless I spend a lot of > time deleting internal parts of each component. Example: I want hadoop client > and fuse only for a variant. > > Note that this could also be done with tarballs as well, but that would > require a lot of duplication of command lines, trials and errors and wouldn't > be as maintainable. > > In conclusion, even if Apache Bigtop was to focus on docker, building > packages would be much better than dropping them and going toward a 'tarball' > approach. Packages would not only be more maintainable, satisfy more use > cases but would also provide an abstraction layer so the docker files could > focus on the image itself instead of setting up the various combinations of > Apache Hadoop components. > From a 10 000 ft view and in the big lines, docker is not much different than > vagrant or boxgrinder. For those tools, having the recipe using the packages > was simplifying a lot of things and I don't see why it would be different > with docker. > > >>> Related question: what are BigTop's goals? Just integration testing? >>> Full blown distro targeted at end users? Packaging for others to build >>> distros on top of? >> All of the above? ;-) Seriously, I think we need to provide a way for >> consumers >> of bigdata technology to be able to deploy it in the most efficient >> way. This means >> that we are likely to need to embrace different ways of packaging our stuff. >> >> Thanks, >> Roman. > +1 again > > Another way to put it is to make the Apache Hadoop ecosystem usable. > That includes making it consumable as well as verifying that it all works > together. > Packages have been the main way to consume such artifacts, but we have always > been opened to other ways (see vagrant and boxgrinder). We even had at some > point a kickstart image to build bootable usb keys with an out of the box > working Apache Hadoop environment :) > > If tomorrow packages become obsolete, I don't see why we could not drop them. > But I think we are still far from that. > > > Thanks, > Bruno
