Persistent resource volumes

2015-04-01 Thread Maxime Brugidou
Hi, We have been talking more and more about the long-awaited feature at https://issues.apache.org/jira/browse/MESOS-1554 I started looking at what have been done in the APIs and as far as I understand framework will be able to reserved and unreserve resources with DiskInfo. Each DiskInfo has a

Re: Persistent resource volumes

2015-04-01 Thread Adam Bordelon
So far the disk resource reported by Mesos refers to the disk where sandboxes will be created. By default, persistent volumes will be created under the same slave work_dir. We could pretty easily provide a separate flag so that all persistent volumes can go on a different disk than all sandboxes,

Re: Apache Mesos Community Sync

2015-04-01 Thread Adam Bordelon
Reminder: We're having another Mesos Developer Community Sync this Thursday, April 2nd from 3-5pm Pacific. Agenda: https://docs.google.com/document/d/153CUCj5LOJCFAVpdDZC7COJDwKh9RDjxaTA0S7lzwDA/edit?usp=sharing To Join: follow the BlueJeans instructions from the recurring meeting invite at the

Current State of Service Discovery

2015-04-01 Thread John Omernik
I have been researching service discovery on Mesos quite a bit lately, and due to my background, may be making assumptions that don't apply to a Mesos Datacenter. I've read through docs, and I have come up with two main approaches to service discovery, and both appear to have strengths and

Custom python executor with Docker

2015-04-01 Thread Tom Fordon
Hi. I'm trying to understand using docker within a custom executor. For each of my tasks, I would like to perform some steps on the node before launching a docker container. I was planning on writing a custom python executor for this, but I wasn't sure how to launch docker from within this

Re: Current State of Service Discovery

2015-04-01 Thread Zia Bhatti
John, I will throw in my 2 cents: There are a number of open-sourced offerings done by well known companies in this area and I have tried a few of them. Generally my opinion is that they are a little restrictive. We are using the HAProxy service on each slave as you described. We have

Re: Current State of Service Discovery

2015-04-01 Thread Adam Shannon
I figured I would comment on how Banno is setting up service discovery with mesos. We've built everything around docker containers and then a wrapper around that which we call sidecar that handles service discovery, basic process supervision, and hot reloads of the underlying app config.

Re: Current State of Service Discovery

2015-04-01 Thread Adam Shannon
David, Smartstack was one of the inspirations we used to decide how we wanted to build out service discovery. The one thing we decided on was that we wanted the haproxy instances to be the front line load balancers. (The ones directly open to the internet.) The one thing from the Smartstack post

RE: Current State of Service Discovery

2015-04-01 Thread David Kesler
That approach sounds similar to Smartstack (http://nerds.airbnb.com/smartstack-service-discovery-cloud/). From: Adam Shannon [mailto:adam.shan...@banno.com] Sent: Wednesday, April 01, 2015 10:58 AM To: mesos-users Subject: Re: Current State of Service Discovery I figured I would comment on how

Re: Persistent resource volumes

2015-04-01 Thread Maxime Brugidou
Thanks i didn't know about MESOS-191. I think that it is indeed not easy to manage the disk resource. The simplest solution would be in addition to offer space on the workspace volume, to simply dedicate other volumes/disks/spindles (configured in the slave) to some frameworks. The DiskInfo would

Re: Current State of Service Discovery

2015-04-01 Thread craig w
We're going with HAProxy on every node + haproxy-marathon-bridge (since we're leveraging Marathon). We deployed mesos-dns but it didn't make seem to make sense to have both solutions. -craig On Wed, Apr 1, 2015 at 12:33 PM, Adam Shannon adam.shan...@banno.com wrote: David, Smartstack was one

Re: Current State of Service Discovery

2015-04-01 Thread CCAAT
On 04/01/2015 11:20 AM, Christos Kozyrakis wrote: Service discovery is a topic where it's unlikely that a single solution will satisfy every need and every constraint. It's also good for the Mesos community to have multiple successful alternatives, even when they overlap in some ways. I will