I'm mostly working with researchers and people developing early prototypes. I can't blame them for using technologies that aren't production ready. That said, I attended pragmatic docker days a while back and there were some companies, like Yelp, who found a good way to run Docker in production so it is possible, given you have a boatload of good ops people.
Big Data Europe <https://www.big-data-europe.eu/> seems to be going towards Docker containers for scalable Hadoop setups <https://www.big-data-europe.eu/scalable-sparkhdfs-workbench-using-docker/>. Not that it's a production ready setup, but with a name like that and H2020 funding, we (big data researchers) can't really ignore them... Juju is awesome for us (big data researchers) because we have a bunch of short-lived projects that use Hadoop etc. in a bunch of different setups, and 1. we don't want to be writing a new wrapper around the Hadoop chef cookbook for every project; 2. we don't want to be creating a new "Hadoop + X" Docker container for every setup. However, we can't ignore the advantages of Docker vs Juju: 1. image-based so the same setup is 100% reproducible if you have the image; 2. auto scaling and failure recovery. So we want the stateless, auto-scalable, auto-recoverable images from Docker and we want Juju's relations and automatic configuration. So how to we get docker containers that can be configured at run-time by Juju? Ben is working on something to configure containers <https://github.com/bcsaller/layercake>, but afaik, no integration with Juju planned. PS: We're interested in Mesos but, as always, our time-to-put-into-it is limited.. 2016-11-18 12:27 GMT+01:00 Tom Barber <[email protected]>: > I'll fork this so we're not hijacking another thread. > > Mesos runs Mesos tasks via frameworks or Docker/Rocket containers > currently. Annoyingly they used to have a scriptable container endpoint I > was hoping to knock up a POC against but they removed it, and my C is > woeful so implementing it will take some time. I also asked on the Mesos > mailing lists and they couldn't grok the use case, apparently docker does > everything anyone needs ;) > > When I was at the Pentaho meetup last week, there's already a bunch of > data folk who run DC/OS or Mesos to manage their workloads which sorta > validated my use case as prior to that it was only theoretical. > > There are certainly a bunch of useful docker containers, I wouldn't deny > that for a second, but the docker reality in production is often a lot like > the Big Data stuff a few years back, it works but does it work well enough. > In some places certainly, but in others not so much. We make a lot of use > of Docker recently on some NASA projects, but I'm under no illusions that > in reality Juju running containers would be a much improved plan, but they > already have Mesos etc running so why upset the apple cart? Similarly at > ApacheCon we had developers praising Docker and Systems Administrators > saying its the bane of their life. > > That said, you don't really see people spin up a scalable hadoop setup in > Docker because it would be annoying to maintain, but you could easily do > that in Juju on whatever, or Puppet etc. Of course you can do it, and it > will become more common over time especially with k8s auto scaling etc for > sure. > > Who said Mesos or DC/OS providers and charms wouldn't get official > support? That said currently we're just lacking bandwidth to build them(I > speak entirely as an impartial observer I have no real idea if they'd get > Canonical support, but why not?) ;) > > Tom > > On Fri, Nov 18, 2016 at 9:59 AM, Merlijn Sebrechts < > [email protected]> wrote: > >> Wait, wouldn't this require juju to have an "mesos" provider, so juju can >> request lxc containers from mesos? I've heard something like this mentioned >> at the Summit, will this become a reality? [that would be awesome!] >> >> We want support for Docker containers because: >> - A lot devs we work with create their prototypes in docker >> - There are a bunch of useful docker containers with stuff that isn't >> charmed yet >> >> We want Kubernetes because: >> - Auto scaling >> - Auto failure recovery >> - It has a future beyond Docker >> - The Charms are officially supported by Canonical (hence Kubernetes > >> Mesos) >> >> >> 2016-11-18 10:41 GMT+01:00 Tom Barber <[email protected]>: >> >>> What you want Merlijn is LXC on Apache Mesos so you can provision a >>> Mesos cluster on MAAS and then provision Juju Charms into LXC on the >>> infinitely scalable cluster! Docker is cool but until it releases the >>> proper orchestration stuff, it comes a poor second to deploying workloads >>> with Juju ;) >>> >>> That's not a slight at the great work Adam, Chuck and co are doing, but >>> feedback I got from people at the Pentaho User meetup last weekend and >>> ApacheCon this week who all get 'stuck' with Docker once the convenience >>> factor has gone away. Anyway, I digress.... Amazing getting proper Docker >>> running on LXD as well. >>> >>> Tom >>> >>> >>>
-- Juju mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
