Re: Official Mesos/Marathon Docker images - please share your thoughts/plans

[Michal Rostecki]

Hi Artem,

First of all, we are experimenting now with creating 
Mesos+Marathon+Chronos images which satisfy our needs under the Kolla 
project umbrella. My change about this is in review and you can see it here:


https://review.openstack.org/#/c/261331/

So, these images I'm introducing to kolla have the following features:
- are using newest version of every Mesos+Marathon component
- are running processes by non-root users (except the mesos-slave container)
- have support for both CentOS and Ubuntu
- everything comes from Mesosphere yum/apt repository

If you're open for bringing such changes (or pull requests) into the 
Dockerfiles under the mesosphere/ namespace, let us know.


And well, I have one new problem, which is more about packaging of 
Marathon. I see that Marathon package on CentOS works well with OpenJDK 
and doesn't require Oracle Java. Unfortunately, Ubuntu packages require 
only Oracle Java packages and there is no way to use OpenJDK without hacks.


So, my question here - is there any specific reason why Marathon on 
Ubuntu can't be installed with OpenJDK like on CentOS?


For now, we probably need to "hack" these requirements and use OpenJDK 
anyway, because we shoudn't provide anything on the non-free license for 
the OpenStack users.


Thank you in advance.

Also, one comment inline.

On 01/11/2016 08:26 AM, Artem Harutyunyan wrote:

Hi Marek,

Thanks for the feedback! The issues that you've discovered with images at
https://hub.docker.com/r/mesosphere
 all seem to be solvable.
I'll forward your feedback to the maintainer of ZooKeeper and Chronos
images. We'll also look into adding Docker into Mesos slave image. I am not
sure whether Docker in Docker works out of the box (I know it doesn't for
earlier versions of Docker).



Docker in Docker image works well. There are some examples / proofs for 
that.


1. In my patch to kolla which I linked above, it's working in 
mesos-slave container.
2. Docker provides a Dockerfile for itself :) 
https://github.com/docker/docker/blob/master/Dockerfile
3. Kolla provides a Dockerfile for Docker as well - 
https://github.com/openstack/kolla/blob/master/docker/dind/Dockerfile.j2


Cheers,
Michal

Re: Official Mesos/Marathon Docker images - please share your thoughts/plans

[Michal Rostecki]

On 01/11/2016 05:13 AM, Dave Lester wrote:

Hi Marek,

Great to hear that you're doing work to deploy OpenStack on Mesos! I'm sure 
there are many others on the mailing list who would be interested, so feel free 
to share more when you have the time. :)



To shortly sum it up.

There is a Kolla project under the OpenStack umbrella which provides 
Dockerfiles for OpenStack services. You can see it here:


https://github.com/openstack/kolla
http://docs.openstack.org/developer/kolla/

By default, it's using Ansible for running these containers on the bare 
metal nodes.


Now we're trying to run these containers on Mesos+Marathon(+Chronos) and 
we have another project for that:


https://github.com/openstack/kolla
http://docs.openstack.org/developer/kolla-mesos/

This project uses Kolla dockerfiles and provides the two things:
- application which pushes OpenStack configuration to ZooKeeper and runs 
Kolla containers via Marathon and Chronos API
- extended start scripts which generates configuration for each 
OpenStack service from what's stored in ZooKeeper


About the second point - there will be probably a long-term plan to 
teach OpenStack services to get configuration from ZooKeeper, without 
using any configuration files.


The whole Kolla-mesos project is under development now, but of course 
any contribution is welcome.



Apologies for the brief Sunday night response, but I wanted to add a few (and 
IMO important) things in response to topics that have come up so far:

1) mesoscloud as a name conflicts with the Apache trademark guidelines and is 
likely infringing. http://www.apache.org/foundation/marks/#guidelines It's also 
unclear to me who has created this GitHub org.

2) The Docker files linked to are not official, they are created by Mesosphere. 
It's possible for Mesos to make Docker files part of the official release 
process, though, and I'd encourage that!



OK. Then if there will be any plans to create Docker images maintained 
by Apache, we'll stay tuned.



3) Yes, Apache proejcts can create binaries for Apache releases, they just 
require separate vote. Apache Aurora has been doing this, for example. 
http://markmail.org/thread/4gxaiuzlo7r5vigk

Dave



Cheers,
Michal

Re: Official Mesos/Marathon Docker images - please share your thoughts/plans

[Michal Rostecki]

On 01/11/2016 08:07 PM, Michal Rostecki wrote:

On 01/11/2016 05:13 AM, Dave Lester wrote:

Hi Marek,

Great to hear that you're doing work to deploy OpenStack on Mesos! I'm
sure there are many others on the mailing list who would be
interested, so feel free to share more when you have the time. :)



To shortly sum it up.

There is a Kolla project under the OpenStack umbrella which provides
Dockerfiles for OpenStack services. You can see it here:

https://github.com/openstack/kolla
http://docs.openstack.org/developer/kolla/

By default, it's using Ansible for running these containers on the bare
metal nodes.

Now we're trying to run these containers on Mesos+Marathon(+Chronos) and
we have another project for that:

https://github.com/openstack/kolla
http://docs.openstack.org/developer/kolla-mesos/

This project uses Kolla dockerfiles and provides the two things:
- application which pushes OpenStack configuration to ZooKeeper and runs
Kolla containers via Marathon and Chronos API
- extended start scripts which generates configuration for each
OpenStack service from what's stored in ZooKeeper

About the second point - there will be probably a long-term plan to
teach OpenStack services to get configuration from ZooKeeper, without
using any configuration files.

The whole Kolla-mesos project is under development now, but of course
any contribution is welcome.


Apologies for the brief Sunday night response, but I wanted to add a
few (and IMO important) things in response to topics that have come up
so far:

1) mesoscloud as a name conflicts with the Apache trademark guidelines
and is likely infringing.
http://www.apache.org/foundation/marks/#guidelines It's also unclear
to me who has created this GitHub org.

2) The Docker files linked to are not official, they are created by
Mesosphere. It's possible for Mesos to make Docker files part of the
official release process, though, and I'd encourage that!



OK. Then if there will be any plans to create Docker images maintained
by Apache, we'll stay tuned.


3) Yes, Apache proejcts can create binaries for Apache releases, they
just require separate vote. Apache Aurora has been doing this, for
example. http://markmail.org/thread/4gxaiuzlo7r5vigk

Dave



Cheers,
Michal


Of course a correct link to kolla-mesos repo is: 
https://github.com/openstack/kolla-mesos


My mistake.

Cheers,
Michal

Re: Official Mesos/Marathon Docker images - please share your thoughts/plans

[Dave Lester]

Hi Marek,

Great to hear that you're doing work to deploy OpenStack on Mesos! I'm sure 
there are many others on the mailing list who would be interested, so feel free 
to share more when you have the time. :)

Apologies for the brief Sunday night response, but I wanted to add a few (and 
IMO important) things in response to topics that have come up so far:

1) mesoscloud as a name conflicts with the Apache trademark guidelines and is 
likely infringing. http://www.apache.org/foundation/marks/#guidelines It's also 
unclear to me who has created this GitHub org.

2) The Docker files linked to are not official, they are created by Mesosphere. 
It's possible for Mesos to make Docker files part of the official release 
process, though, and I'd encourage that!

3) Yes, Apache proejcts can create binaries for Apache releases, they just 
require separate vote. Apache Aurora has been doing this, for example. 
http://markmail.org/thread/4gxaiuzlo7r5vigk

Dave

> On Jan 8, 2016, at 12:48 AM, Marek Zawadzki  wrote:
> 
> Adam - thank you.
> Issue filed (https://github.com/mesosphere/docker-containers/issues/37).
> 
> -marek
> 
> On 08.01.2016 01:41, Adam Bordelon wrote:
>> Officially, Apache prefers to distribute source instead of binaries, which
>> is why Mesosphere volunteered to build and distribute rpm/deb packages as
>> well as docker images. Apache Mesos could own the Docker files for Mesos
>> though.
>> That said, if you have problems with the Mesosphere (or mesoscloud) docker
>> images for Mesos/Marathon, file an issue with the source repo, and we'd be
>> happy to address your issues.
>> https://github.com/mesosphere/docker-containers/issues
>> 
>> On Thu, Jan 7, 2016 at 3:00 AM, haosdent  wrote:
>> 
>>> hmm, mesoscloud seems not keep updating since 0.24.1
>>> 
>>> On Thu, Jan 7, 2016 at 5:55 PM, Michal Rostecki 
>>> wrote:
>>> 
 On 01/07/2016 10:17 AM, haosdent wrote:
 
> Mesoscloud docker file https://github.com/mesoscloud
> 
> 
 Hello haosdent,
 
 These Dockerfiles don't include the newest versions of
 
 - Marathon (the newest is 0.13.0, mesoscloud maintains 0.11.0)
 - Mesos (the newest is 0.26.0, mesoscloud maintains 0.24.1)
 
 I recently discovered also a few disadvantages of the mesoscloud images
 that are not mentioned in Marek's mail:
 
 - ZooKeeper is installed from some tarball, while it can be easily
 installed from Mesosphere package repository like the other components
 - all containers are running as root - there is no need for that except
 mesos-slave repo and IMO it's a good practice to run applications as
 non-root user if possible
 
 Cheers,
 Michal
 
>>> 
>>> 
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>> 
> 

Re: Official Mesos/Marathon Docker images - please share your thoughts/plans

[Artem Harutyunyan]

Hi Marek,

Thanks for the feedback! The issues that you've discovered with images at
https://hub.docker.com/r/mesosphere
 all seem to be solvable.
I'll forward your feedback to the maintainer of ZooKeeper and Chronos
images. We'll also look into adding Docker into Mesos slave image. I am not
sure whether Docker in Docker works out of the box (I know it doesn't for
earlier versions of Docker).

Cheers,
Artem.

On Thu, Jan 7, 2016 at 1:09 AM, Marek Zawadzki 
wrote:

> Hello all.
>
> This is Marek Zawadzki from Mirantis Kolla Team - I've met Brenden
> Matthews & Ben Lin at Mesosphere SF office this October.
> We work on deploying OpenStack on Mesos/Marathon cluster using Kolla <
> https://wiki.openstack.org/wiki/Kolla> and we find Docker a great way for
> quickly setting up our environment.
> Could you please share your thoughts about Docker images for
> Mesos/Marathon?
>
> TL; DR: the images that can be found on DockerHub are either out of date
> or not working. We would like to know your plans about maintaining them.
>
> You can find details in Michal Rostecki's (Kolla core reviewer) email
> below.
>
> Thanks!
>
> -marek
>
> --
> Marek Zawadzki
> Mirantis Kolla Team
>
>
>  Forwarded Message 
> Subject:Mesosphere Docker images
> Date:   Wed, 23 Dec 2015 14:11:40 +0100
> From:   Michal Rostecki 
>
>
>
>
>
> [...]
> I recently found the official Docker images from Mesosphere:
>
> https://hub.docker.com/u/mesosphere/
> https://github.com/mesosphere/docker-containers
>
> In theory, we should be able to set up a Mesos cluster using these
> images. In practice, I ran into the following problems:
>
> 1. ZooKeeper -https://hub.docker.com/r/mesosphere/zookeeper/
>
> First of all - I don't see any source code of this image. Dockerfile for
> it just doesn't exists om their Github (mesosphere/docker-containers).
> To figure out how their start script works, I had to access the file
> inside the container and search for it.
>
> Second thing - it binds to the localhost address, instead of 0.0.0.0 or
> concrete IP address. I didn't see any option in their start scripts to
> configure that. That makes multinode installation impossible, but it
> even makes working on all-in-one hard too - because a good practice is
> to point Mesos master and Marathon to the public IP of ZooKeeper, even
> if all of these services are running on the same node.
>
> 2. Mesos slave -https://hub.docker.com/r/mesosphere/mesos-slave/
>
> It doesn't have Docker installed. The comments on dockerhub confirm
> this. It makes using docker containerized impossible. This image works
> only with cgroups containerized, so it's useless for us.
>
> 3. Chronos -https://hub.docker.com/r/mesosphere/chronos/
>
> It requires configuring via volume containing config file. It's in
> contradiction with the other images which are configured via env
> variables. We would like to configure all Mesos stuff via env.
>
> That's all about Mesosphere's images.
>
> In kolla-mesos, we're currently using images from
> https://hub.docker.com/r/mesoscloud/  - this account doesn't seem to be
> associated with any company.
>
> Images from mesoscloud work perfectly, but are outdated - especially
> about Marathon which is released often.
>
> On the other hand, mesosphere images have the newest versions, are last
> built no earlier than a week ago, but well, they just don't work good
> for us.
>
> That's why we would like Mesosphere to improve quality of their images.
> This may block our work.
>
> Regards,
> Michal
>
>
>
>
>
>

Re: Official Mesos/Marathon Docker images - please share your thoughts/plans

[Marek Zawadzki]

Adam - thank you.
Issue filed (https://github.com/mesosphere/docker-containers/issues/37).

-marek

On 08.01.2016 01:41, Adam Bordelon wrote:

Officially, Apache prefers to distribute source instead of binaries, which
is why Mesosphere volunteered to build and distribute rpm/deb packages as
well as docker images. Apache Mesos could own the Docker files for Mesos
though.
That said, if you have problems with the Mesosphere (or mesoscloud) docker
images for Mesos/Marathon, file an issue with the source repo, and we'd be
happy to address your issues.
https://github.com/mesosphere/docker-containers/issues

On Thu, Jan 7, 2016 at 3:00 AM, haosdent  wrote:


hmm, mesoscloud seems not keep updating since 0.24.1

On Thu, Jan 7, 2016 at 5:55 PM, Michal Rostecki 
wrote:


On 01/07/2016 10:17 AM, haosdent wrote:


Mesoscloud docker file https://github.com/mesoscloud



Hello haosdent,

These Dockerfiles don't include the newest versions of

- Marathon (the newest is 0.13.0, mesoscloud maintains 0.11.0)
- Mesos (the newest is 0.26.0, mesoscloud maintains 0.24.1)

I recently discovered also a few disadvantages of the mesoscloud images
that are not mentioned in Marek's mail:

- ZooKeeper is installed from some tarball, while it can be easily
installed from Mesosphere package repository like the other components
- all containers are running as root - there is no need for that except
mesos-slave repo and IMO it's a good practice to run applications as
non-root user if possible

Cheers,
Michal




--
Best Regards,
Haosdent Huang

Re: Official Mesos/Marathon Docker images - please share your thoughts/plans

[Adam Bordelon]

Officially, Apache prefers to distribute source instead of binaries, which
is why Mesosphere volunteered to build and distribute rpm/deb packages as
well as docker images. Apache Mesos could own the Docker files for Mesos
though.
That said, if you have problems with the Mesosphere (or mesoscloud) docker
images for Mesos/Marathon, file an issue with the source repo, and we'd be
happy to address your issues.
https://github.com/mesosphere/docker-containers/issues

On Thu, Jan 7, 2016 at 3:00 AM, haosdent  wrote:

> hmm, mesoscloud seems not keep updating since 0.24.1
>
> On Thu, Jan 7, 2016 at 5:55 PM, Michal Rostecki 
> wrote:
>
> > On 01/07/2016 10:17 AM, haosdent wrote:
> >
> >> Mesoscloud docker file https://github.com/mesoscloud
> >>
> >>
> > Hello haosdent,
> >
> > These Dockerfiles don't include the newest versions of
> >
> > - Marathon (the newest is 0.13.0, mesoscloud maintains 0.11.0)
> > - Mesos (the newest is 0.26.0, mesoscloud maintains 0.24.1)
> >
> > I recently discovered also a few disadvantages of the mesoscloud images
> > that are not mentioned in Marek's mail:
> >
> > - ZooKeeper is installed from some tarball, while it can be easily
> > installed from Mesosphere package repository like the other components
> > - all containers are running as root - there is no need for that except
> > mesos-slave repo and IMO it's a good practice to run applications as
> > non-root user if possible
> >
> > Cheers,
> > Michal
> >
>
>
>
> --
> Best Regards,
> Haosdent Huang
>

Re: Official Mesos/Marathon Docker images - please share your thoughts/plans

[haosdent]

hmm, mesoscloud seems not keep updating since 0.24.1

On Thu, Jan 7, 2016 at 5:55 PM, Michal Rostecki 
wrote:

> On 01/07/2016 10:17 AM, haosdent wrote:
>
>> Mesoscloud docker file https://github.com/mesoscloud
>>
>>
> Hello haosdent,
>
> These Dockerfiles don't include the newest versions of
>
> - Marathon (the newest is 0.13.0, mesoscloud maintains 0.11.0)
> - Mesos (the newest is 0.26.0, mesoscloud maintains 0.24.1)
>
> I recently discovered also a few disadvantages of the mesoscloud images
> that are not mentioned in Marek's mail:
>
> - ZooKeeper is installed from some tarball, while it can be easily
> installed from Mesosphere package repository like the other components
> - all containers are running as root - there is no need for that except
> mesos-slave repo and IMO it's a good practice to run applications as
> non-root user if possible
>
> Cheers,
> Michal
>



-- 
Best Regards,
Haosdent Huang

Re: Official Mesos/Marathon Docker images - please share your thoughts/plans

[haosdent]

Mesoscloud docker file https://github.com/mesoscloud

On Thu, Jan 7, 2016 at 5:09 PM, Marek Zawadzki 
wrote:

> Hello all.
>
> This is Marek Zawadzki from Mirantis Kolla Team - I've met Brenden
> Matthews & Ben Lin at Mesosphere SF office this October.
> We work on deploying OpenStack on Mesos/Marathon cluster using Kolla <
> https://wiki.openstack.org/wiki/Kolla> and we find Docker a great way for
> quickly setting up our environment.
> Could you please share your thoughts about Docker images for
> Mesos/Marathon?
>
> TL; DR: the images that can be found on DockerHub are either out of date
> or not working. We would like to know your plans about maintaining them.
>
> You can find details in Michal Rostecki's (Kolla core reviewer) email
> below.
>
> Thanks!
>
> -marek
>
> --
> Marek Zawadzki
> Mirantis Kolla Team
>
>
>  Forwarded Message 
> Subject:Mesosphere Docker images
> Date:   Wed, 23 Dec 2015 14:11:40 +0100
> From:   Michal Rostecki 
>
>
>
>
>
> [...]
> I recently found the official Docker images from Mesosphere:
>
> https://hub.docker.com/u/mesosphere/
> https://github.com/mesosphere/docker-containers
>
> In theory, we should be able to set up a Mesos cluster using these
> images. In practice, I ran into the following problems:
>
> 1. ZooKeeper -https://hub.docker.com/r/mesosphere/zookeeper/
>
> First of all - I don't see any source code of this image. Dockerfile for
> it just doesn't exists om their Github (mesosphere/docker-containers).
> To figure out how their start script works, I had to access the file
> inside the container and search for it.
>
> Second thing - it binds to the localhost address, instead of 0.0.0.0 or
> concrete IP address. I didn't see any option in their start scripts to
> configure that. That makes multinode installation impossible, but it
> even makes working on all-in-one hard too - because a good practice is
> to point Mesos master and Marathon to the public IP of ZooKeeper, even
> if all of these services are running on the same node.
>
> 2. Mesos slave -https://hub.docker.com/r/mesosphere/mesos-slave/
>
> It doesn't have Docker installed. The comments on dockerhub confirm
> this. It makes using docker containerized impossible. This image works
> only with cgroups containerized, so it's useless for us.
>
> 3. Chronos -https://hub.docker.com/r/mesosphere/chronos/
>
> It requires configuring via volume containing config file. It's in
> contradiction with the other images which are configured via env
> variables. We would like to configure all Mesos stuff via env.
>
> That's all about Mesosphere's images.
>
> In kolla-mesos, we're currently using images from
> https://hub.docker.com/r/mesoscloud/  - this account doesn't seem to be
> associated with any company.
>
> Images from mesoscloud work perfectly, but are outdated - especially
> about Marathon which is released often.
>
> On the other hand, mesosphere images have the newest versions, are last
> built no earlier than a week ago, but well, they just don't work good
> for us.
>
> That's why we would like Mesosphere to improve quality of their images.
> This may block our work.
>
> Regards,
> Michal
>
>
>
>
>
>


-- 
Best Regards,
Haosdent Huang

Official Mesos/Marathon Docker images - please share your thoughts/plans

[Marek Zawadzki]

Hello all.

This is Marek Zawadzki from Mirantis Kolla Team - I've met Brenden 
Matthews & Ben Lin at Mesosphere SF office this October.
We work on deploying OpenStack on Mesos/Marathon cluster using Kolla 
 and we find Docker a great way 
for quickly setting up our environment.

Could you please share your thoughts about Docker images for Mesos/Marathon?

TL; DR: the images that can be found on DockerHub are either out of date 
or not working. We would like to know your plans about maintaining them.


You can find details in Michal Rostecki's (Kolla core reviewer) email below.

Thanks!

-marek

--
Marek Zawadzki
Mirantis Kolla Team


 Forwarded Message 
Subject:Mesosphere Docker images
Date:   Wed, 23 Dec 2015 14:11:40 +0100
From:   Michal Rostecki 





[...]
I recently found the official Docker images from Mesosphere:

https://hub.docker.com/u/mesosphere/
https://github.com/mesosphere/docker-containers

In theory, we should be able to set up a Mesos cluster using these
images. In practice, I ran into the following problems:

1. ZooKeeper -https://hub.docker.com/r/mesosphere/zookeeper/

First of all - I don't see any source code of this image. Dockerfile for
it just doesn't exists om their Github (mesosphere/docker-containers).
To figure out how their start script works, I had to access the file
inside the container and search for it.

Second thing - it binds to the localhost address, instead of 0.0.0.0 or
concrete IP address. I didn't see any option in their start scripts to
configure that. That makes multinode installation impossible, but it
even makes working on all-in-one hard too - because a good practice is
to point Mesos master and Marathon to the public IP of ZooKeeper, even
if all of these services are running on the same node.

2. Mesos slave -https://hub.docker.com/r/mesosphere/mesos-slave/

It doesn't have Docker installed. The comments on dockerhub confirm
this. It makes using docker containerized impossible. This image works
only with cgroups containerized, so it's useless for us.

3. Chronos -https://hub.docker.com/r/mesosphere/chronos/

It requires configuring via volume containing config file. It's in
contradiction with the other images which are configured via env
variables. We would like to configure all Mesos stuff via env.

That's all about Mesosphere's images.

In kolla-mesos, we're currently using images from
https://hub.docker.com/r/mesoscloud/  - this account doesn't seem to be
associated with any company.

Images from mesoscloud work perfectly, but are outdated - especially
about Marathon which is released often.

On the other hand, mesosphere images have the newest versions, are last
built no earlier than a week ago, but well, they just don't work good
for us.

That's why we would like Mesosphere to improve quality of their images.
This may block our work.

Regards,
Michal