Re: [Wikitech-l] [Engineering] Announcing mediawiki-containers, a Docker-based MediaWiki installer

[Gabriel Wicke]

I see containers as one ingredient in a more automated and tested
pipeline from development through CI to production.
mediawiki-containers could expand to cover the development use case,
but I think we can and should move from there into CI, and finally
production.

Right now, Yuvi is evaluating the Kubernetes cluster manager in labs.
Its features include scheduling of "pods" (groups of containers) to
hardware nodes, networking, rolling deploys and more. While all these
features provide a very high degree of automation, they also mean that
failures in Kubernetes can have grave consequences. I think operations
are wise to wait for Kubernetes to mature a bit further before
considering it for critical production use cases.

Rather than waiting until one-stop cluster managers are mature, we
could also start with a more traditional config / deploy system. I
have played with this approach using Ansible [1] a while ago, and the
ergonomics are pretty much the same as git-based deploys. There is
also some support to run docker images in systemd, which could be an
alternative if we want to avoid the dependency on the docker runtime
in production. This older task lists some options:
https://phabricator.wikimedia.org/T93439

Lets get together and figure out a plan.

Gabriel

[1]: http://docs.ansible.com/ansible/docker_module.html

On Fri, Jan 29, 2016 at 8:23 PM, James Forrester
 wrote:
> On 27 December 2015 at 11:52, Ori Livneh  wrote:
>>
>> On Thu, Dec 24, 2015 at 3:57 PM, Gabriel Wicke 
>> wrote:
>>>
>>> I am writing to announce mediawiki-containers [1], a simple installer for
>>> MediaWiki with VisualEditor, Parsoid, RESTBase and other services, using
>>> Linux containers.
>>
>>
>> This is very nice work -- kudos. Is it too soon to envision running this
>> (or rather, some future iteration) in production at Wikimedia? What would
>> need to happen?
>
>
> Ping on this. I for one would be interested too. :-)
>
> J.
> --
> James D. Forrester
> Lead Product Manager, Editing
> Wikimedia Foundation, Inc.
>
> jforres...@wikimedia.org | @jdforrester



-- 
Gabriel Wicke
Principal Engineer, Wikimedia Foundation

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Engineering] Announcing mediawiki-containers, a Docker-based MediaWiki installer

[Giuseppe Lavagetto]

On Sat, Jan 30, 2016 at 9:59 AM, Gabriel Wicke  wrote:
> Right now, Yuvi is evaluating the Kubernetes cluster manager in labs.

Just a clarification: Yuvi has already evaluated kubernetes and it's
being actively used to build an awesome replacement for at least part
of what toollabs does right now. A handful of tools are already
running, with success, on it for quite a long time.

> Its features include scheduling of "pods" (groups of containers) to
> hardware nodes, networking, rolling deploys and more. While all these
> features provide a very high degree of automation, they also mean that
> failures in Kubernetes can have grave consequences. I think operations
> are wise to wait for Kubernetes to mature a bit further before
> considering it for critical production use cases.
>

Failures in any complex system are surely scary, but kubernetes seems
stable enough to be evaluated for production use. We also had an
unconference session at the WMDS about this - or better what we want
to achieve by using kubernetes as a tool.

I will also stress that there are more "mature" cluster/container
framework like Apache Mesos/Aurora/Marathon, but after taking a hard
look at them me and Yuvi evaluated that kubernetes is way more
promising for any of our use cases.

This is still a bit further away in the future, anyways. There is
already a phabricator task for this, which is anyways sitting idle at
the moment as it's not in our immediate roadmap. The task is by the
way trying to be independent of the specific technology in describing
what we actually want to achieve. Kubernetes, as any other product we
might use, is just a mean to an end, and we should never be in love
with any specific technology.

https://phabricator.wikimedia.org/T122822

>  There is
> also some support to run docker images in systemd, which could be an
> alternative if we want to avoid the dependency on the docker runtime
> in production.

I guess you mean containers can run within systemd, but I don't think
just running containers instead of firejail would give us any
practical advantage at the moment from any operational prespective,
but I might miss the point.

> Lets get together and figure out a plan.

Let's do it! maybe next quarter when ops are not mostly focused on the
datacenter switch it will be easier, I guess :)

Cheers,

Giuseppe

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Engineering] Announcing mediawiki-containers, a Docker-based MediaWiki installer

[James Forrester]

On 27 December 2015 at 11:52, Ori Livneh  wrote:

> On Thu, Dec 24, 2015 at 3:57 PM, Gabriel Wicke 
> wrote:
>
>> I am writing to announce mediawiki-containers [1], a simple installer for
>> MediaWiki with VisualEditor, Parsoid, RESTBase and other services, using
>> Linux containers.
>>
>
> This is very nice work -- kudos. Is it too soon to envision running this
> (or rather, some future iteration) in production at Wikimedia? What would
> need to happen?
>

​Ping on this. I for one would be interested too. :-)​

J.
-- 
James D. Forrester
Lead Product Manager, Editing
Wikimedia Foundation, Inc.

jforres...@wikimedia.org | @jdforrester
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Engineering] Announcing mediawiki-containers, a Docker-based MediaWiki installer

[Ori Livneh]

On Thu, Dec 24, 2015 at 3:57 PM, Gabriel Wicke  wrote:

> I am writing to announce mediawiki-containers [1], a simple installer for
> MediaWiki with VisualEditor, Parsoid, RESTBase and other services, using
> Linux containers.
>

This is very nice work -- kudos. Is it too soon to envision running this
(or rather, some future iteration) in production at Wikimedia? What would
need to happen?
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l