Re: 0.28.2 has been released

[craig w]

Jie,

Thanks for the updates. When will the packages be available (in particular
RPM)?

-craig

On Sun, Jun 5, 2016 at 2:31 PM, Jie Yu  wrote:

> Hi folks,
>
> I just released Mesos 0.28.2 and updated the website.
>
> It includes some important bug fixes. The change log can be found here:
>
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.28.2
>
> If you are considering using 0.28, please use 0.28.2!
>
> Thanks!
> - Jie
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Set LIBPROCESS_IP for frameworks launched with marathon

[craig w]

Hi, marathon sets the HOST env var. If it's not the ip address you can use
getent with the value from HOST to figure it out.
>However, in order for the frameworks to receive resource offers I need to
set the LIBPROCESS_IP environment variable to the hosts IP address for the
docker container running the frameworks.

Hi, @Gmail. Could you provide more details about this?

On Sun, Apr 3, 2016 at 10:40 PM, Rad Gruchalski 
wrote:

> Hi Gmail,
>
> AFAIK not. The only way to do so is setting up the env variable as you do
> now.
>
> Kind regards,
> Radek Gruchalski
> ra...@gruchalski.com 
> de.linkedin.com/in/radgruchalski/
>
>
> *Confidentiality:*This communication is intended for the above-named
> person and may be confidential and/or legally privileged.
> If it has come to you in error you must take no action based on it, nor
> must you copy or show it to anyone; please delete/destroy and inform the
> sender immediately.
>
> On Sunday, 3 April 2016 at 16:09, Gmail wrote:
>
> I'm pretty new to mesos and marathon, and I'm running a couple of
> frameworks with marathon (Kafka and elastic search). However, in order for
> the frameworks to receive resource offers I need to set the LIBPROCESS_IP
> environment variable to the hosts IP address for the docker container
> running the frameworks. Currently I am working around me this by using a
> constraint to hard wire the slave that the framework gets launched on, so
> then I can put the slaves ip in the marathon json file.
>
> Obviously this is not ideal. Is there a better way to define the host ip
> Inside the docker container?
>
> Sent from my iPad
>
>
>


-- 
Best Regards,
Haosdent Huang

Re: [RESULT][VOTE] Release Apache Mesos 0.28.0 (rc2)

[craig w]

Great news. Do the rpm's get automatically built and released or will they
come later this week?

On Thu, Mar 17, 2016 at 1:28 PM, Vinod Kone  wrote:

> Hi all,
>
>
> The vote for Mesos 0.28.0 (rc2) has passed with the
>
> following votes.
>
>
> +1 (Binding)
>
> --
>
> Vinod Kone
>
> Michael Park
>
> Kapil Arya
>
>
> +1 (Non-binding)
>
> --
>
> Greg Mann
>
> Daniel Osborne
>
> Jorg Schad
>
> Zhitao Li
>
>
> There were no 0 or -1 votes.
>
>
> Please find the release at:
>
> https://dist.apache.org/repos/dist/release/mesos/0.28.0
>
>
> It is recommended to use a mirror to download the release:
>
> http://www.apache.org/dyn/closer.cgi
>
>
> The CHANGELOG for the release is available at:
>
>
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.28.0
>
>
> The mesos-0.28.0.jar has been released to:
>
> https://repository.apache.org
>
>
> The website (http://mesos.apache.org) will be updated shortly to reflect
> this release.
>
>
> Thanks,
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: 0.27.2 RPMs

[craig w]

I thought so based on the website:
http://mesos.apache.org/blog/mesos-0-27-2-released/

0.27.2 RPMs

[craig w]

When will the 0.27.2 RPMs be available?

Re: marathon-lb at scale

[craig w]

Alfredo -- i'll be curious to hear how it goes if you scale it up. I had
initially tested with 12 nodes and it seemed fine, then when i went to 90
it became an issue. Again, this was with an older Marathon, so things could
be much better now.

On Thu, Mar 3, 2016 at 11:02 AM, Alfredo Carneiro <
alfr...@simbioseventures.com> wrote:

> I've just started to use it on production with 14 nodes and it is scaling
> well! I had no problems yet. And different from haproxy-marathon-bridge, it
> supports natively a loop every second, so you don't have any delay, at
> least I haven't seen.
>
> On Thu, Mar 3, 2016 at 12:23 PM, craig w  wrote:
>
>> That was using the marathon-haproxy-bridge script, so it was polling
>> marathon every minute to try and keep up to date. Though a minute lag in
>> updating the proxy wasn't sufficient.
>>
>
>
>
> --
> Alfredo Miranda
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: marathon-lb at scale

[craig w]

That was using the marathon-haproxy-bridge script, so it was polling
marathon every minute to try and keep up to date. Though a minute lag in
updating the proxy wasn't sufficient.

Re: marathon-lb at scale

[craig w]

For what it's worth, I had tried the haproxy bridge with marathon 0.11 back
when that was the latest release. I had the bridge running on 90 nodes and
it crushed the marathon leader.

Marathon had made lots of improvements so it might be better now, but
figured I'd share.
On Mar 2, 2016 7:59 PM, "Brenden Matthews"  wrote:

> I'd suggest you also try posting on the Marathon group:
> https://groups.google.com/forum/#!forum/marathon-framework
>
> On Tue, Mar 1, 2016 at 6:21 PM, Bill Green  wrote:
>
>> For folks using marathon-lb, how far have you scaled it? I’d be very
>> interested to hear your experiences with it, especially in the area of
>> partition tolerance.
>>
>> Any insights would be greatly appreciated, thanks!
>>
>> --
>> Bill Green
>> SRE, New Relic
>> @cloudangst
>>
>>
>>
>>
>

Re: Mesos metrics -> influxdb

[craig w]

Niklas -- is the mesos collector ready? it's in the "wish list" on that
page.

On Fri, Feb 26, 2016 at 11:12 AM, Niklas Nielsen  wrote:

> Or take SNAP for a spin:
> https://github.com/intelsdi-x/snap/blob/master/docs/PLUGIN_CATALOG.md
> Can collect metrics from Mesos and output in several TSDBs and you can do
> transformations on the metrics in flight.
>
> Niklas
>
> On Fri, Feb 26, 2016 at 1:05 AM, Aaron Carey  wrote:
>
>> Thanks Alberto, those are the two projects I was having issues with. I
>> was hoping not to have to fork the first one (to add 0.24 functionality)
>> and the second just keeps timing out and crashing (!)
>>
>> I'll have a look at telegraph!
>>
>> Thanks all for your help,
>>
>> Aaron
>>
>> --
>> *From:* Alberto del Barrio [alberto.delbarrio.albe...@gmail.com]
>> *Sent:* 26 February 2016 08:04
>> *To:* user@mesos.apache.org
>> *Subject:* Re: Mesos metrics -> influxdb
>>
>> I am using collectd to gather those metrics and I am pretty happy with it.
>> I use two plugins:
>>  - To gather mesos metrics (System metrics, number of slaves connected...
>> )
>> https://github.com/rayrod2030/collectd-mesos
>>  - To gather CPU and memory used per mesos tasks:
>> https://github.com/bobrik/collectd-mesos-tasks
>>
>> The setup is stable, it never crashed. Regarding the version 0.24 it can
>> work with few modifications.
>>
>>
>> On 02/25/16 18:32, Aaron Carey wrote:
>>
>> Has anyone had a good experience recording mesos metrics into influxdb?
>>
>> I've found a couple of options, a collectd plugin which doesn't appear to
>> work with version 0.24.x and a more up to date containerised option which
>> randomly crashes regularly and doesn't appear to actually post any stats.
>>
>> Anyone have any good solutions?
>>
>>
>>
>>
>
>
> --
> Niklas
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: "Chaos monkey" for mesos?

[craig w]

here's a direct link in the video
https://youtu.be/0I6qG9RQUnY?t=389

On Thu, Feb 25, 2016 at 12:17 PM, David Wood  wrote:

> The DCOS tutorial mentions a chaos tool at the end of the video.  Not sure
> if that's what your looking for, but it might be something to follow up on
> somehow.
>
> https://mesosphere.com/learn/
>
> David Wood
> Computing Systems for Wireless Networks
> IBM TJ Watson Research Center
> daw...@us.ibm.com
> 914-945-4923 (office), 914-396-6515 (mobile)
>
>
>
>
> From:Srikanth Viswanathan 
> To:user@mesos.apache.org
> Date:02/25/2016 12:01 PM
> Subject:"Chaos monkey" for mesos?
> --
>
>
>
> Has there been any work done to develop a "*chaos monkey*
> " analogue for
> Mesos? I have been researching on how to write one, but I wanted to know if
> there's any work already available that I can take a look at for
> comparison, and possibly re-use.
>
> The end goal would be something loaded into Mesos or separate from Mesos
> that randomly kills tasks. Could it be something as simple as an
> application that uses the KILL HTTP request from the scheduler API to kill
> tasks?
>
> Thanks.
>
> Srikanth
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Resources clarification in Mesos UI

[craig w]

Done - https://issues.apache.org/jira/browse/MESOS-4586

thanks

On Wed, Feb 3, 2016 at 9:16 AM, haosdent  wrote:

> hmm, I think it is correct. Could you help create a ticket about this
> display problem in mesos jira?
> On Feb 3, 2016 9:28 PM, "craig w"  wrote:
>
>> Ah ok. That seems like it could be misleading. For example, if I have a
>> single mesos-slave and it has 100gb of memory. Say it is running tasks what
>> are currently using 10gb but have allocated 80gb. the mesos front page
>> would show there is 90gb free (100gb total - 10gb actively being used), but
>> in reality the offers from that slave would say they only have 20gb free
>> (since 80gb is allocated). Is that correct?
>>
>> On Wed, Feb 3, 2016 at 8:10 AM, haosdent  wrote:
>>
>>> I think "used" is calculated by sum up every executor cpu and memory
>>> statistics. It should <= "allocated" resources.
>>>
>>> On Wed, Feb 3, 2016 at 9:02 PM, craig w  wrote:
>>>
>>>> On the Mesos UI (v 0.26.0) under the "resources" section when it lists
>>>> CPUs and Mem, is "used" the actual amount of memory being used or the
>>>> amount that's been allocated?
>>>>
>>>> I ask, because on the page that displays information for a slave it
>>>> shows the CPUs and Mem used and allocated.
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>>
>>
>>
>>
>> --
>>
>> https://github.com/mindscratch
>> https://www.google.com/+CraigWickesser
>> https://twitter.com/mind_scratch
>> https://twitter.com/craig_links
>>
>>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Resources clarification in Mesos UI

[craig w]

Ah ok. That seems like it could be misleading. For example, if I have a
single mesos-slave and it has 100gb of memory. Say it is running tasks what
are currently using 10gb but have allocated 80gb. the mesos front page
would show there is 90gb free (100gb total - 10gb actively being used), but
in reality the offers from that slave would say they only have 20gb free
(since 80gb is allocated). Is that correct?

On Wed, Feb 3, 2016 at 8:10 AM, haosdent  wrote:

> I think "used" is calculated by sum up every executor cpu and memory
> statistics. It should <= "allocated" resources.
>
> On Wed, Feb 3, 2016 at 9:02 PM, craig w  wrote:
>
>> On the Mesos UI (v 0.26.0) under the "resources" section when it lists
>> CPUs and Mem, is "used" the actual amount of memory being used or the
>> amount that's been allocated?
>>
>> I ask, because on the page that displays information for a slave it shows
>> the CPUs and Mem used and allocated.
>>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Resources clarification in Mesos UI

[craig w]

On the Mesos UI (v 0.26.0) under the "resources" section when it lists CPUs
and Mem, is "used" the actual amount of memory being used or the amount
that's been allocated?

I ask, because on the page that displays information for a slave it shows
the CPUs and Mem used and allocated.

blog post for 0.27.0 release

[craig w]

Will there be a blog post for the 0.27.0 release?

Re: mesos-elasticsearch vs Elasticsearch with Marathon

[craig w]

We have elasticsearch in a docker container. We configure elasticsearch to
have ports 9200 and other port 9300 (this is for "transport.tcp.port"). We
modify the elasticsearch configuration file on startup to add the IP
addresses of other elasticsearch instances (we query mesos-dns for these ip
addresses).

On Mon, Dec 28, 2015 at 5:51 PM, Alex Rukletsov  wrote:

> Craig,
>
> mind elaborating, how exactly do you run elasticsearch in Marathon?
>
> On Mon, Dec 28, 2015 at 8:36 PM, craig w  wrote:
> > In terms of discovery, elasticsearch provides that out of the box
> >
> https://www.elastic.co/guide/en/elasticsearch/reference/1.4/modules-discovery.html
> .
> > We deploy elasticsearch via Marathon and it works great.
> >
> > On Mon, Dec 28, 2015 at 2:17 PM, Eric LEMOINE 
> wrote:
> >>
> >> On Mon, Dec 28, 2015 at 7:55 PM, Alex Rukletsov 
> >> wrote:
> >> > Eric—
> >> >
> >> > give me a chance to answer that before you fall into frustration : ).
> >> > Also, you can directly write to framework developers
> >> > (mesos...@container-solutions.com) and they either confirm or bust my
> >> > guess. Or maybe one of the authors — Frank — will chime in in this
> >> > thread.
> >> >
> >> > Marathon has no idea about application logic, hence a "scale"
> >> > operation just starts more application instances. But sometimes you
> >> > may want to do extra job (track instances, report ip:port of a new
> >> > instance to existing instances, and so on). That's when a dedicated
> >> > framework makes sense. Each framework has a scheduler that is able to
> >> > track each instance and do all aforementioned actions.
> >> >
> >> > How this maps to your question? AFAIK, all Elasticsearch nodes should
> >> > see each other, hence once a new node is started, it should be somehow
> >> > advertised to other nodes. You can do it by wrapping Elasticsearch
> >> > command in a shell script and maintain some sort of an out-of-band
> >> > registry, take a look at one of the first efforts [1] to run
> >> > Elasticsearch on Mesos to get an impression how it may look like. But
> >> > you can use a dedicated framework instead : ).
> >> >
> >> > [1] https://github.com/mesosphere/elasticsearch-mesos
> >>
> >>
> >> That makes great sense Alex. Thanks for chiming in.
> >
> >
> >
> >
> > --
> >
> > https://github.com/mindscratch
> > https://www.google.com/+CraigWickesser
> > https://twitter.com/mind_scratch
> > https://twitter.com/craig_links
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: mesos-elasticsearch vs Elasticsearch with Marathon

[craig w]

In terms of discovery, elasticsearch provides that out of the box
https://www.elastic.co/guide/en/elasticsearch/reference/1.4/modules-discovery.html.
We deploy elasticsearch via Marathon and it works great.

On Mon, Dec 28, 2015 at 2:17 PM, Eric LEMOINE  wrote:

> On Mon, Dec 28, 2015 at 7:55 PM, Alex Rukletsov 
> wrote:
> > Eric—
> >
> > give me a chance to answer that before you fall into frustration : ).
> > Also, you can directly write to framework developers
> > (mesos...@container-solutions.com) and they either confirm or bust my
> > guess. Or maybe one of the authors — Frank — will chime in in this
> > thread.
> >
> > Marathon has no idea about application logic, hence a "scale"
> > operation just starts more application instances. But sometimes you
> > may want to do extra job (track instances, report ip:port of a new
> > instance to existing instances, and so on). That's when a dedicated
> > framework makes sense. Each framework has a scheduler that is able to
> > track each instance and do all aforementioned actions.
> >
> > How this maps to your question? AFAIK, all Elasticsearch nodes should
> > see each other, hence once a new node is started, it should be somehow
> > advertised to other nodes. You can do it by wrapping Elasticsearch
> > command in a shell script and maintain some sort of an out-of-band
> > registry, take a look at one of the first efforts [1] to run
> > Elasticsearch on Mesos to get an impression how it may look like. But
> > you can use a dedicated framework instead : ).
> >
> > [1] https://github.com/mesosphere/elasticsearch-mesos
>
>
> That makes great sense Alex. Thanks for chiming in.
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Mesos masters and zookeeper running together?

[craig w]

What specs do you have for the zookeeper servers? Ram, cpu, disk, OS.
On Dec 24, 2015 5:13 PM, "Dick Davies"  wrote:

> zookeeper really wants a dedicated cluster IMO; preferably with SSD
> under it - if zookeeper
> starts to run slow then everything else will start to bog down. I've
> co-hosted it with mesos masters
> before now for demo purposes etc. but for production it's probably
> worth choosing dedicated hosts.
>
> On 24 December 2015 at 20:36, Rodrick Brown 
> wrote:
> > With our design we end up building out a stand alone zookeeper cluster 3
> > nodes.  Zookeeper seems to be the default dumping ground for many Apache
> > based products these days. You will eventually see many services and
> > frameworks require a zk instance for leader election, coordination, Kv
> store
> > etc.. I've seen situations where the masters can become extremely busy
> and
> > cause performance problem with Zk which can be huge issue for mesos.
> >
> > Sent from Outlook Mobile
> >
> >
> >
> >
> > On Thu, Dec 24, 2015 at 8:01 AM -0800, "Ron Lipke" 
> wrote:
> >
> >> Hello, I've been working on setting up a mesos cluster for eventual
> >> production use and I have a question on configuring zookeeper alongside
> >> the mesos masters.
> >> Is it best practice to run zookeeper/exhibitor as a separate cluster (in
> >> our case, three nodes) or on the same machines as the mesos masters?  I
> >> understand the drawbacks of increased cost for compute resources that
> >> will just be running a single service and most of the reference docs
> >> have them running together, but just wondering if it's beneficial to
> >> have them uncoupled.
> >>
> >> Thanks in advance for any input.
> >>
> >> Ron Lipke
> >> @neverminding
> >
> >
> > NOTICE TO RECIPIENTS: This communication is confidential and intended for
> > the use of the addressee only. If you are not an intended recipient of
> this
> > communication, please delete it immediately and notify the sender by
> return
> > email. Unauthorized reading, dissemination, distribution or copying of
> this
> > communication is prohibited. This communication does not constitute an
> offer
> > to sell or a solicitation of an indication of interest to purchase any
> loan,
> > security or any other financial product or instrument, nor is it an
> offer to
> > sell or a solicitation of an indication of interest to purchase any
> products
> > or services to any persons who are prohibited from receiving such
> > information under applicable law. The contents of this communication may
> not
> > be accurate or complete and are subject to change without notice. As
> such,
> > Orchard App, Inc. (including its subsidiaries and affiliates, "Orchard")
> > makes no representation regarding the accuracy or completeness of the
> > information contained herein. The intended recipient is advised to
> consult
> > its own professional advisors, including those specializing in legal, tax
> > and accounting matters. Orchard does not provide legal, tax or accounting
> > advice.
>

Re: Mesos masters and zookeeper running together?

[craig w]

We've run the mesos-master and marathon processes in the same servers as
zookeeper.
On Dec 24, 2015 11:01 AM, "Ron Lipke"  wrote:

> Hello, I've been working on setting up a mesos cluster for eventual
> production use and I have a question on configuring zookeeper alongside
> the mesos masters.
> Is it best practice to run zookeeper/exhibitor as a separate cluster (in
> our case, three nodes) or on the same machines as the mesos masters?  I
> understand the drawbacks of increased cost for compute resources that
> will just be running a single service and most of the reference docs
> have them running together, but just wondering if it's beneficial to
> have them uncoupled.
>
> Thanks in advance for any input.
>
> Ron Lipke
> @neverminding
>

Re: mesos-elasticsearch vs Elasticsearch with Marathon

[craig w]

We'd like to use the framework once some more features are available (see
the road map).

Currently we deploy ES in docker using marathon.
On Dec 21, 2015 11:14 PM, "Weitao"  wrote:

> IMO, es as a framework running on Mesos is our future, atst, we can deploy
> the es framework over Marathon.
>
> 在 2015年12月21日，17:11，Eric LEMOINE  写道：
>
> Hi
>
> I am new to Mesos and I have a naive question related to Elasticsearch,
> Mesos and Marathon.
>
> So there's the mesos-elasticsearch [*] project which provides a Mesos
> framework/scheduler for Elasticsearch. I guess it's also possible to run
> Elasticsearch with Marathon. What are the fundamental differences between
> the two approaches? When should one favor one approach over the other one?
> What are the reasons for using mesos-elasticsearch instead of just running
> Elasticsearch on top of Marathon?
>
> Thanks for any insight.
>
> [*] 
>
>

Re: mesos 0.26.0 website updates

[craig w]

I see it now, not sure why I didn't earlier. Thanks

On Mon, Dec 21, 2015 at 6:41 AM, Jörg Schad  wrote:

> Hi,
> looking at the website (http://mesos.apache.org/) I see 0.26.0 already
> mentioned e.g. under news.
> At which place are you missing it?
> News
>
>- *December 16, 2015* - Mesos 0.26.0 is released! See the CHANGELOG
>
> <https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.26.0>
> and blog post <http://mesos.apache.org/blog/mesos-0-26-0-released/> for
>    details.
>
>
> On Mon, Dec 21, 2015 at 12:16 PM, craig w  wrote:
>
>> When will the mesos website and blog be updated to mention the 0.26.0
>> release?
>>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

mesos 0.26.0 website updates

[craig w]

When will the mesos website and blog be updated to mention the 0.26.0
release?

Re: mesos-slave configuration file with Ports Offer

[craig w]

Miguel, create a file named: /etc/mesos-slave/resources/ports(*)

It should contain: [21000-24000]

On Fri, Nov 13, 2015 at 6:30 AM, Miguel Ángel Ausó 
wrote:

> Hi
>
> I'm trying configure "port offers" in mesos-slave using a configuration
> file (I'm using mesosphere package), I know that I can use the "--resource"
> option in start command, but I need to know if it's possible using a
> configuration file, the same that other option, for exemple gc-delay or
> master configuration
>
> I tried with differents formats in
> /etc/mesos-slave/resources/ports
> [21000-24000]
> ports:[21000-24000]
> *:[21000-24000]
> 21000-24000
>
> thanks!!
>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: many outstanding chronos offers

[craig w]

You're correct, "5mins" or "30secs", etc.
thank

On Thu, Nov 5, 2015 at 9:24 AM, Guangya Liu  wrote:

> I think it should be "5mins" or "30secs", please refer to
> https://github.com/mesosphere/mesos/blob/master/3rdparty/libprocess/3rdparty/stout/include/stout/duration.hpp#L55-L69
> for detail, thanks!
>
>
>
> On Thu, Nov 5, 2015 at 9:04 PM, craig w  wrote:
>
>> Looks like it would be "5min" or "30s"
>>
>>
>> https://github.com/mesosphere/mesos/blob/master/3rdparty/libprocess/3rdparty/stout/include/stout/duration.hpp#L37
>>
>>
>> On Thu, Nov 5, 2015 at 7:55 AM, craig w  wrote:
>>
>>> What's the format for that value?
>>>
>>> "5 minutes" or "5m"
>>> "30 seconds" or "30s"
>>>
>>>
>>> On Thu, Nov 5, 2015 at 7:12 AM, Anand Mazumdar 
>>> wrote:
>>>
>>>> Can you try setting the flag `—offer_timeout` on the mesos master to
>>>> some small value e.g. 5 minutes ? The default behavior is that a framework
>>>> can keep hoarding the offered resources forever.
>>>>
>>>> http://mesos.apache.org/documentation/latest/configuration/
>>>>
>>>> -anand
>>>>
>>>> On Nov 5, 2015, at 5:15 PM, craig w  wrote:
>>>>
>>>> I'm running Mesos 0.24.1, Marathon 0.11.1 and Chronos 2.4.0.
>>>>
>>>> It seems I'm unable to launch a new app in marathon b/c resources
>>>> aren't available to it. I say that b/c the "offers" ui page, which are
>>>> "outstanding offers" (https://issues.apache.org/jira/browse/MESOS-3817),
>>>> shows most resources have been offered to chronos which hasn't
>>>> accepted/declined them.
>>>>
>>>> Is there some way to prevent this from happening? How do I get the
>>>> offers to be "unstuck"? Perhaps, stop chronos for a bit?
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> https://github.com/mindscratch
>>> https://www.google.com/+CraigWickesser
>>> https://twitter.com/mind_scratch
>>> https://twitter.com/craig_links
>>>
>>>
>>
>>
>> --
>>
>> https://github.com/mindscratch
>> https://www.google.com/+CraigWickesser
>> https://twitter.com/mind_scratch
>> https://twitter.com/craig_links
>>
>>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: many outstanding chronos offers

[craig w]

Looks like it would be "5min" or "30s"

https://github.com/mesosphere/mesos/blob/master/3rdparty/libprocess/3rdparty/stout/include/stout/duration.hpp#L37


On Thu, Nov 5, 2015 at 7:55 AM, craig w  wrote:

> What's the format for that value?
>
> "5 minutes" or "5m"
> "30 seconds" or "30s"
>
>
> On Thu, Nov 5, 2015 at 7:12 AM, Anand Mazumdar 
> wrote:
>
>> Can you try setting the flag `—offer_timeout` on the mesos master to some
>> small value e.g. 5 minutes ? The default behavior is that a framework can
>> keep hoarding the offered resources forever.
>>
>> http://mesos.apache.org/documentation/latest/configuration/
>>
>> -anand
>>
>> On Nov 5, 2015, at 5:15 PM, craig w  wrote:
>>
>> I'm running Mesos 0.24.1, Marathon 0.11.1 and Chronos 2.4.0.
>>
>> It seems I'm unable to launch a new app in marathon b/c resources aren't
>> available to it. I say that b/c the "offers" ui page, which are
>> "outstanding offers" (https://issues.apache.org/jira/browse/MESOS-3817),
>> shows most resources have been offered to chronos which hasn't
>> accepted/declined them.
>>
>> Is there some way to prevent this from happening? How do I get the offers
>> to be "unstuck"? Perhaps, stop chronos for a bit?
>>
>> Thanks
>>
>>
>>
>
>
> --
>
> https://github.com/mindscratch
> https://www.google.com/+CraigWickesser
> https://twitter.com/mind_scratch
> https://twitter.com/craig_links
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: many outstanding chronos offers

[craig w]

What's the format for that value?

"5 minutes" or "5m"
"30 seconds" or "30s"


On Thu, Nov 5, 2015 at 7:12 AM, Anand Mazumdar  wrote:

> Can you try setting the flag `—offer_timeout` on the mesos master to some
> small value e.g. 5 minutes ? The default behavior is that a framework can
> keep hoarding the offered resources forever.
>
> http://mesos.apache.org/documentation/latest/configuration/
>
> -anand
>
> On Nov 5, 2015, at 5:15 PM, craig w  wrote:
>
> I'm running Mesos 0.24.1, Marathon 0.11.1 and Chronos 2.4.0.
>
> It seems I'm unable to launch a new app in marathon b/c resources aren't
> available to it. I say that b/c the "offers" ui page, which are
> "outstanding offers" (https://issues.apache.org/jira/browse/MESOS-3817),
> shows most resources have been offered to chronos which hasn't
> accepted/declined them.
>
> Is there some way to prevent this from happening? How do I get the offers
> to be "unstuck"? Perhaps, stop chronos for a bit?
>
> Thanks
>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

many outstanding chronos offers

[craig w]

I'm running Mesos 0.24.1, Marathon 0.11.1 and Chronos 2.4.0.

It seems I'm unable to launch a new app in marathon b/c resources aren't
available to it. I say that b/c the "offers" ui page, which are
"outstanding offers" (https://issues.apache.org/jira/browse/MESOS-3817),
shows most resources have been offered to chronos which hasn't
accepted/declined them.

Is there some way to prevent this from happening? How do I get the offers
to be "unstuck"? Perhaps, stop chronos for a bit?

Thanks

Re: Marathon 0.11.1 - Mesos 0.25 - Mesos-DNS 0.4.0

[craig w]

I've had no issue with the following combination:

MesosDNS 0.4.0
Marathon 0.11.0
Mesos 0.24.1

I've been waiting to upgrade to Mesos 0.25.0 because of issues mentioned in
the mesos mailing list regarding Marathon 0.11.x and Mesos 0.25.0

On Wed, Oct 28, 2015 at 10:38 AM, John Omernik  wrote:

> Hey all -
>
> I am cross posting this because it's a number of moving parts that could
> be at issue here (Mesos, Mesos-dns, and/or Marathon).
>
> Basically: At the version combination in Subject, the IP that is
> registered in mesos-dns for Docker containers running in Marathon is the
> internal (container) IP address of the docker (in bridged mode) not the
> nodes. This obviously causes issues.  Note this doesn't happen when the
> Marathon application is non-Docker.
>
> I was running Mesos-dns 0.4.0 on a cluster running Mesos 0.24.0 and
> Marathon 0.10.0 and I upgraded to Mesos 0.25.0 and Marathon 0.11.1 and
> noticed this behavior happening.
>
> I thought that was odd because I have another cluster that was running
> Mesos 0.25.0 and Marathon 0.11.1 and it wasn't happening, until I realized
> that I hadn't upgraded Mesos-dns lately, I upgraded to Mesos-dns 0.4.0 and
> the problem started occurring.
>
> Is there a setting that I need to use the external IP of the container? Is
> this issue known? Is there a workaround? This is pretty major for Docker
> running on Marathon and using Mesos-dns for service discovery.
>
> John Omernik
>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Docker Private Registry problem when update from 0.23 to 0.25

[craig w]

@Luke - Good to know. I have a similar setup, only different being our
timeout is at 5 minutes. We had problems using docker 1.8.x with a private
registry that was backed by NFS, apparently we're not alone (
https://github.com/docker/docker/issues/15833).

On Fri, Oct 23, 2015 at 9:52 AM, Luke Amdor  wrote:

> We currently run Mesos 0.25.0 with Marathon 0.11.1 with a private docker
> registry (distribution v2.1.1, no creds required) and haven't had any weird
> problems running docker containers. We however do run our marathon with a
> really high task_launch_confirm_timeout of 60 (10 minutes) as we were
> running into problems with docker pulls and staged tasks. We also run
> docker version 1.7.1 as we ran into problems with pulls on docker 1.8.0. We
> haven't yet tried the newest docker release to see if it's helped.
>
> On Fri, Oct 23, 2015 at 6:50 AM, craig w  wrote:
>
>> I wasn't sure because of this comment:
>> https://github.com/mesosphere/marathon/pull/2462#issuecomment-148703383
>>
>> On Fri, Oct 23, 2015 at 7:41 AM, haosdent  wrote:
>>
>>> According https://github.com/mesosphere/marathon/pull/2462 , I think
>>> the patches merge into 0.12.0-RC1 and 0.11.1 doesn't contain yet.
>>>
>>> On Fri, Oct 23, 2015 at 7:32 PM, craig w  wrote:
>>>
>>>> Has anyone confirmed that Mesos 0.25.0 with Marathon 0.11.1 with a
>>>> private docker registry (without credentials) works?
>>>>
>>>> On Thu, Oct 15, 2015 at 4:51 PM, Jan Stabenow <
>>>> j.stabe...@livespottingmedia.com> wrote:
>>>>
>>>>> Marathon 0.11.1 + Mesos 0.25.0 + private Docker-Reg (without
>>>>> credentials) works, but with a new error. Marathon (or Mesos) finished 
>>>>> each
>>>>> task with an "finished-state" immediately.
>>>>>
>>>>> So let’s wait…
>>>>> https://github.com/mesosphere/marathon/pull/2462#issuecomment-148482471
>>>>>
>>>>>
>>>>> Am 15.10.2015 um 21:29 schrieb Jan Stabenow <
>>>>> j.stabe...@livespottingmedia.com>:
>>>>>
>>>>> Hey Brian,
>>>>>
>>>>> sorry for my late response.
>>>>>
>>>>> I'll try the latest 0.11.1.
>>>>>
>>>>> This may be one of the problems:
>>>>> https://github.com/mesosphere/marathon/pull/2415
>>>>>
>>>>>
>>>>> Am 15.10.2015 um 21:14 schrieb craig w :
>>>>>
>>>>> I've successfully pulled images from a private registry using Mesos
>>>>> 0.24.1 and marathon 0.11. I use forcePullImage: true.
>>>>> On Oct 15, 2015 2:25 PM, "Brian Devins"  wrote:
>>>>>
>>>>>> Has anyone found some resolution for this?
>>>>>>
>>>>>> I have the mesosphere repo versions of mesos and marathon. 0.24.1 and
>>>>>> 0.11.0 respectively. I have also tried 0.10.1 for marathon but still no
>>>>>> dice. I can manually pull the images if I log into the box.
>>>>>>
>>>>>> On Wed, Oct 14, 2015 at 2:30 AM, Joachim Andersson <
>>>>>> joachim.anders...@resurs.se> wrote:
>>>>>>
>>>>>>>
>>>>>>> Those images I have tried with Marathon 0.11
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I use "disk": 512,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Here is my app config.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> {
>>>>>>>
>>>>>>> "id":
>>>>>>> "/marathonkempaffb060830d6ad074c66923b77f75e199a4c8a23",
>>>>>>>
>>>>>>> "env": {
>>>>>>>
>>>>>>> "GIT_CALCULATED_BRANCH":
>>>>>>> "affb060830d6ad074c66923b77f75e199a4c8a23",
>>>>>>>
>>>>>>> "marathonkemp_kemp_password": "!12345!m",
>>>>>>>
>>>>>>> },
>>>>>>>
>>>>>>> "instances": 1,
>&g

Re: Docker Private Registry problem when update from 0.23 to 0.25

[craig w]

I wasn't sure because of this comment:
https://github.com/mesosphere/marathon/pull/2462#issuecomment-148703383

On Fri, Oct 23, 2015 at 7:41 AM, haosdent  wrote:

> According https://github.com/mesosphere/marathon/pull/2462 , I think the
> patches merge into 0.12.0-RC1 and 0.11.1 doesn't contain yet.
>
> On Fri, Oct 23, 2015 at 7:32 PM, craig w  wrote:
>
>> Has anyone confirmed that Mesos 0.25.0 with Marathon 0.11.1 with a
>> private docker registry (without credentials) works?
>>
>> On Thu, Oct 15, 2015 at 4:51 PM, Jan Stabenow <
>> j.stabe...@livespottingmedia.com> wrote:
>>
>>> Marathon 0.11.1 + Mesos 0.25.0 + private Docker-Reg (without
>>> credentials) works, but with a new error. Marathon (or Mesos) finished each
>>> task with an "finished-state" immediately.
>>>
>>> So let’s wait…
>>> https://github.com/mesosphere/marathon/pull/2462#issuecomment-148482471
>>>
>>>
>>> Am 15.10.2015 um 21:29 schrieb Jan Stabenow <
>>> j.stabe...@livespottingmedia.com>:
>>>
>>> Hey Brian,
>>>
>>> sorry for my late response.
>>>
>>> I'll try the latest 0.11.1.
>>>
>>> This may be one of the problems:
>>> https://github.com/mesosphere/marathon/pull/2415
>>>
>>>
>>> Am 15.10.2015 um 21:14 schrieb craig w :
>>>
>>> I've successfully pulled images from a private registry using Mesos
>>> 0.24.1 and marathon 0.11. I use forcePullImage: true.
>>> On Oct 15, 2015 2:25 PM, "Brian Devins"  wrote:
>>>
>>>> Has anyone found some resolution for this?
>>>>
>>>> I have the mesosphere repo versions of mesos and marathon. 0.24.1 and
>>>> 0.11.0 respectively. I have also tried 0.10.1 for marathon but still no
>>>> dice. I can manually pull the images if I log into the box.
>>>>
>>>> On Wed, Oct 14, 2015 at 2:30 AM, Joachim Andersson <
>>>> joachim.anders...@resurs.se> wrote:
>>>>
>>>>>
>>>>> Those images I have tried with Marathon 0.11
>>>>>
>>>>>
>>>>>
>>>>> I use "disk": 512,
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Here is my app config.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> {
>>>>>
>>>>> "id": "/marathonkempaffb060830d6ad074c66923b77f75e199a4c8a23",
>>>>>
>>>>> "env": {
>>>>>
>>>>> "GIT_CALCULATED_BRANCH":
>>>>> "affb060830d6ad074c66923b77f75e199a4c8a23",
>>>>>
>>>>> "marathonkemp_kemp_password": "!12345!m",
>>>>>
>>>>> },
>>>>>
>>>>> "instances": 1,
>>>>>
>>>>> "cpus": 0.5,
>>>>>
>>>>> "mem": 1024,
>>>>>
>>>>> "disk": 512,
>>>>>
>>>>> "executor": "",
>>>>>
>>>>> "constraints": [
>>>>>
>>>>> [
>>>>>
>>>>> "hostname",
>>>>>
>>>>> "UNIQUE"
>>>>>
>>>>> ]
>>>>>
>>>>> ],
>>>>>
>>>>> "container": {
>>>>>
>>>>> "type": "DOCKER",
>>>>>
>>>>> "volumes": [],
>>>>>
>>>>> "docker": {
>>>>>
>>>>> "image":
>>>>> "dockerregistry.dte.loc:80/marathonkemp:affb060830d6ad074c66923b77f75e199a4c8a23",
>>>>>
>>>>> "network": "BRIDGE",
>>>>>
>>>>> "portMappings": [
>>>>>
>>>>> {
>>>>>
>>>>> "containerPort": 8080,
>>>>>
>>>>> "hostPort": 0,
>

Re: Docker Private Registry problem when update from 0.23 to 0.25

[craig w]

Has anyone confirmed that Mesos 0.25.0 with Marathon 0.11.1 with a private
docker registry (without credentials) works?

On Thu, Oct 15, 2015 at 4:51 PM, Jan Stabenow <
j.stabe...@livespottingmedia.com> wrote:

> Marathon 0.11.1 + Mesos 0.25.0 + private Docker-Reg (without credentials)
> works, but with a new error. Marathon (or Mesos) finished each task with an
> "finished-state" immediately.
>
> So let’s wait…
> https://github.com/mesosphere/marathon/pull/2462#issuecomment-148482471
>
>
> Am 15.10.2015 um 21:29 schrieb Jan Stabenow <
> j.stabe...@livespottingmedia.com>:
>
> Hey Brian,
>
> sorry for my late response.
>
> I'll try the latest 0.11.1.
>
> This may be one of the problems:
> https://github.com/mesosphere/marathon/pull/2415
>
>
> Am 15.10.2015 um 21:14 schrieb craig w :
>
> I've successfully pulled images from a private registry using Mesos 0.24.1
> and marathon 0.11. I use forcePullImage: true.
> On Oct 15, 2015 2:25 PM, "Brian Devins"  wrote:
>
>> Has anyone found some resolution for this?
>>
>> I have the mesosphere repo versions of mesos and marathon. 0.24.1 and
>> 0.11.0 respectively. I have also tried 0.10.1 for marathon but still no
>> dice. I can manually pull the images if I log into the box.
>>
>> On Wed, Oct 14, 2015 at 2:30 AM, Joachim Andersson <
>> joachim.anders...@resurs.se> wrote:
>>
>>>
>>> Those images I have tried with Marathon 0.11
>>>
>>>
>>>
>>> I use "disk": 512,
>>>
>>>
>>>
>>>
>>>
>>> Here is my app config.
>>>
>>>
>>>
>>>
>>>
>>> {
>>>
>>> "id": "/marathonkempaffb060830d6ad074c66923b77f75e199a4c8a23",
>>>
>>> "env": {
>>>
>>> "GIT_CALCULATED_BRANCH":
>>> "affb060830d6ad074c66923b77f75e199a4c8a23",
>>>
>>> "marathonkemp_kemp_password": "!12345!m",
>>>
>>> },
>>>
>>> "instances": 1,
>>>
>>> "cpus": 0.5,
>>>
>>> "mem": 1024,
>>>
>>> "disk": 512,
>>>
>>> "executor": "",
>>>
>>> "constraints": [
>>>
>>> [
>>>
>>> "hostname",
>>>
>>> "UNIQUE"
>>>
>>> ]
>>>
>>> ],
>>>
>>> "container": {
>>>
>>> "type": "DOCKER",
>>>
>>> "volumes": [],
>>>
>>> "docker": {
>>>
>>> "image":
>>> "dockerregistry.dte.loc:80/marathonkemp:affb060830d6ad074c66923b77f75e199a4c8a23",
>>>
>>> "network": "BRIDGE",
>>>
>>> "portMappings": [
>>>
>>> {
>>>
>>> "containerPort": 8080,
>>>
>>> "hostPort": 0,
>>>
>>> "servicePort": 1,
>>>
>>> "protocol": "tcp"
>>>
>>> },
>>>
>>> {
>>>
>>> "containerPort": 8000,
>>>
>>> "hostPort": 0,
>>>
>>> "servicePort": 10001,
>>>
>>> "protocol": "tcp"
>>>
>>> },
>>>
>>> {
>>>
>>> "containerPort": 9010,
>>>
>>> "hostPort": 0,
>>>
>>> "servicePort": 10002,
>>>
>>> "protocol": "tcp"
>>>
>>> }
>>>
>>> ],
>>>
>>> "privileged": false,
>>>
>>> "parameters": [],
>>>
>>> "f

Re: Docker Private Registry problem when update from 0.23 to 0.25

[craig w]

I've successfully pulled images from a private registry using Mesos 0.24.1
and marathon 0.11. I use forcePullImage: true.
On Oct 15, 2015 2:25 PM, "Brian Devins"  wrote:

> Has anyone found some resolution for this?
>
> I have the mesosphere repo versions of mesos and marathon. 0.24.1 and
> 0.11.0 respectively. I have also tried 0.10.1 for marathon but still no
> dice. I can manually pull the images if I log into the box.
>
> On Wed, Oct 14, 2015 at 2:30 AM, Joachim Andersson <
> joachim.anders...@resurs.se> wrote:
>
>> Those images I have tried with Marathon 0.11
>>
>>
>>
>> I use "disk": 512,
>>
>>
>>
>>
>>
>> Here is my app config.
>>
>>
>>
>>
>>
>> {
>>
>> "id": "/marathonkempaffb060830d6ad074c66923b77f75e199a4c8a23",
>>
>> "env": {
>>
>> "GIT_CALCULATED_BRANCH":
>> "affb060830d6ad074c66923b77f75e199a4c8a23",
>>
>> "marathonkemp_kemp_password": "!12345!m",
>>
>> },
>>
>> "instances": 1,
>>
>> "cpus": 0.5,
>>
>> "mem": 1024,
>>
>> "disk": 512,
>>
>> "executor": "",
>>
>> "constraints": [
>>
>> [
>>
>> "hostname",
>>
>> "UNIQUE"
>>
>> ]
>>
>> ],
>>
>> "container": {
>>
>> "type": "DOCKER",
>>
>> "volumes": [],
>>
>> "docker": {
>>
>> "image":
>> "dockerregistry.dte.loc:80/marathonkemp:affb060830d6ad074c66923b77f75e199a4c8a23",
>>
>> "network": "BRIDGE",
>>
>> "portMappings": [
>>
>> {
>>
>> "containerPort": 8080,
>>
>> "hostPort": 0,
>>
>> "servicePort": 1,
>>
>> "protocol": "tcp"
>>
>> },
>>
>> {
>>
>> "containerPort": 8000,
>>
>> "hostPort": 0,
>>
>> "servicePort": 10001,
>>
>> "protocol": "tcp"
>>
>> },
>>
>> {
>>
>> "containerPort": 9010,
>>
>> "hostPort": 0,
>>
>> "servicePort": 10002,
>>
>> "protocol": "tcp"
>>
>> }
>>
>> ],
>>
>> "privileged": false,
>>
>> "parameters": [],
>>
>> "forcePullImage": false
>>
>> }
>>
>> },
>>
>> "healthChecks": [
>>
>> {
>>
>> "path": ".",
>>
>> "protocol": "HTTP",
>>
>> "portIndex": 0,
>>
>> "gracePeriodSeconds": 360,
>>
>> "intervalSeconds": 30,
>>
>> "timeoutSeconds": 10,
>>
>> "maxConsecutiveFailures": 3,
>>
>> "ignoreHttp1xx": false
>>
>> }
>>
>> ]
>>
>>
>>
>> }
>>
>>
>>
>>
>>
>>
>>
>> Remove a env parameters
>>
>>
>>
>> Joachim Andersson Systemutvecklare IT | Utveckling
>>
>>
>>
>> Resurs Bank AB Ekslingan 9 Box 22209, SE-250 24 Helsingborg
>>
>>
>>
>> Direkt Tfn: +46 700 528171 Mobil: +46 700 528171 Växel: +46 42 38 20 00
>> E-post: joachim.anders...@resurs.se Webb: www.resursbank.se
>>
>>
>>
>> *Från:* Jan Stabenow [mailto:j.stabe...@livespottingmedia.com]
>> *Skickat:* den 13 oktober 2015 15:52
>> *Till:* user@mesos.apache.org
>> *Ämne:* Re: Docker Private Registry problem when update from 0.23 to 0.25
>>
>>
>>
>> Hey!
>>
>>
>>
>> I had the same problems when upgrading 0.23 > 0.24.1 +! marathon 0.10 >
>> 0.11
>>
>> Have you set the disk-limit in your app-config? If not, set this ("disk":
>> "value") and try it again.
>>
>>
>>
>> Looks like there's a problem with Marathon, but not sure.
>>
>>
>>
>> Best Regards,
>>
>> Jan
>>
>>
>>
>>
>>
>> Am 13.10.2015 um 14:14 schrieb Joachim Andersson <
>> joachim.anders...@resurs.se>:
>>
>>
>>
>>
>>
>> When i deploed with forcePullImage=true it dosn't work in Marathon 0.11
>> but when I downgraded to Marathon 0.10 it works,
>>
>>
>>
>>
>>
>>
>>
>> 
>>
>> *Joachim Andersson*
>>
>> *Systemutvecklare*
>>
>> *IT | Utveckling*
>>
>>
>>
>> Resurs Bank AB
>>
>> Ekslingan 9
>>
>> Box 22209, SE-250 24 Helsingborg
>>
>>
>>
>> Direkt Tfn:
>>
>> +46 700 528171
>>
>> Mobil:
>>
>> +46 700 528171
>>
>> Växel:
>>
>> +46 42 38 20 00
>>
>> E-post:
>>
>> joachim.anders...@resurs.se
>>
>> Webb:
>>
>> www.resursbank.se
>>
>>
>>
>>
>>
>> *Från:* Joachim Andersson [mailto:joachim.anders...@resurs.se
>> ]
>> *Skickat:* den 13 oktober 2015 13:24
>> *Till:* user@mesos.apache.org
>> *Ämne:* Docker Private Registry problem when update from 0.23 to 0.25
>>
>>
>>
>> Hi, I updated from 0.23 to 0.25 of mesos and to Marathon 0.11
>>
>>
>>
>> When running docker containers from the public repository it works fine.
>>
>> But when I try to run from my local repository the images don't start.
>>
>>
>>
>> If I start them on my slave with docker run 

Re: upgrade from 0.24.1 to 0.25

[craig w]

Itamar,

Where did you see Marathon 0.11.0 was not ready for production? I ran into
this issue (https://github.com/mesosphere/marathon/issues/2369), however,
it turned out I just needed to update a configuration parameter. So far,
Mesos 0.24.1 and Marathon 0.11.0 are working great. I'm hoping to upgrade
to 0.25.0 soon.

On Thu, Oct 15, 2015 at 4:16 AM, Itamar Ostricher 
wrote:

> Also note that you're on a problematic Marathon version.
> I was thinking to upgrade from 0.23&0.10.1 to latest versions, but decided
> to wait after I saw an announcement on the marathon users list that 0.11 is
> not recommended for production from last week (Oct 9).
> I'm waiting until 0.11.1 is out... (hopefully soon?)
>
> On Wed, Oct 14, 2015, 10:16 Adam Bordelon  wrote:
>
>> > When restartng the masters, should they be restarted gradually (restart
>> a master, wait 30 seconds, restart the next)?
>> Craig, ideally you should roll the masters one at a time (maintain a
>> --quorum of masters up at all times), providing enough time for a new
>> master to recover the replicated log and be ready for a failover. This
>> reduces downtime and guarantees a continuous connection to
>> frameworks/agents. If you restart all the masters at once, there's a period
>> when they're all inaccessible, and you don't maintain quorum.
>>
>> On Tue, Oct 13, 2015 at 9:34 AM, craig w  wrote:
>>
>>> I have not tried to upgrade to 0.25.0 yet. I'm hoping to try the upgrade
>>> this week. We're currently on mesos 0.24.1 and marathon 0.11.0 (just
>>> upgraded this today).
>>>
>>> I'll give the upgrade to 0.25.0 a shot in a test environment, in the
>>> meantime if you perform an experiment I'd be interested in your findings.
>>>
>>> Thanks,
>>> Craig
>>>
>>> On Tue, Oct 13, 2015 at 12:25 PM, Niklas Nielsen 
>>> wrote:
>>>
>>>> Hi Craig,
>>>>
>>>> That should definitely not happen; did you try to upgrade to 0.25.0
>>>> already? If not, we can try to run an upgrade experiment with that marathon
>>>> version.
>>>>
>>>> Niklas
>>>>
>>>> On 13 October 2015 at 02:39, craig w  wrote:
>>>>
>>>>> When upgrading from 0.23.0 to 0.24.1, I installed the new binaries and
>>>>> restarted the masters (all at once), then restarted all of the slaves.
>>>>>
>>>>> I then observed all of the tasks that were running (via Marathon
>>>>> 0.10.x) were restarted. I had expected "no downtime" or restarts, did I
>>>>> misunderstand the upgrade instructions or did I perhaps do something
>>>>> incorrectly?
>>>>>
>>>>> When restartng the masters, should they be restarted gradually
>>>>> (restart a master, wait 30 seconds, restart the next)?
>>>>>
>>>>> I'm looking to upgrade from 0.24.1 to 0.25.0 but want to avoid having
>>>>> all of the tasks restart again.
>>>>>
>>>>> Thanks,
>>>>> Craig
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> https://github.com/mindscratch
>>> https://www.google.com/+CraigWickesser
>>> https://twitter.com/mind_scratch
>>> https://twitter.com/craig_links
>>>
>>>
>>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Mesos-Dns Masters List

[craig w]

If you can, try mesos-dns 0.4.0, there were a handful of fixes and
improvements that might help. 0.4.0 is working great for me.

On Wed, Oct 14, 2015 at 9:10 AM, John Omernik  wrote:

> Hey all,
>
> I was using mesos-dns, and I filled in my zk field based on the HA mesos
> cluster I have.  Mesos dns is up, but in the stderr, I keep seeing
> "generator.go:342 warning: leader "master@10.0.0.1:5050" is not in master
> list.
>
> I don't have a master list in my config.json, instead I am using the zk
> list.   but the 10.0.0.1 is a valid master which is running properly. I am
> running Mesos 0.24 and mesos-dns 0.3.0.   Any thoughts on this?
>
> John
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: upgrade from 0.24.1 to 0.25

[craig w]

I have not tried to upgrade to 0.25.0 yet. I'm hoping to try the upgrade
this week. We're currently on mesos 0.24.1 and marathon 0.11.0 (just
upgraded this today).

I'll give the upgrade to 0.25.0 a shot in a test environment, in the
meantime if you perform an experiment I'd be interested in your findings.

Thanks,
Craig

On Tue, Oct 13, 2015 at 12:25 PM, Niklas Nielsen 
wrote:

> Hi Craig,
>
> That should definitely not happen; did you try to upgrade to 0.25.0
> already? If not, we can try to run an upgrade experiment with that marathon
> version.
>
> Niklas
>
> On 13 October 2015 at 02:39, craig w  wrote:
>
>> When upgrading from 0.23.0 to 0.24.1, I installed the new binaries and
>> restarted the masters (all at once), then restarted all of the slaves.
>>
>> I then observed all of the tasks that were running (via Marathon 0.10.x)
>> were restarted. I had expected "no downtime" or restarts, did I
>> misunderstand the upgrade instructions or did I perhaps do something
>> incorrectly?
>>
>> When restartng the masters, should they be restarted gradually (restart a
>> master, wait 30 seconds, restart the next)?
>>
>> I'm looking to upgrade from 0.24.1 to 0.25.0 but want to avoid having all
>> of the tasks restart again.
>>
>> Thanks,
>> Craig
>>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

upgrade from 0.24.1 to 0.25

[craig w]

When upgrading from 0.23.0 to 0.24.1, I installed the new binaries and
restarted the masters (all at once), then restarted all of the slaves.

I then observed all of the tasks that were running (via Marathon 0.10.x)
were restarted. I had expected "no downtime" or restarts, did I
misunderstand the upgrade instructions or did I perhaps do something
incorrectly?

When restartng the masters, should they be restarted gradually (restart a
master, wait 30 seconds, restart the next)?

I'm looking to upgrade from 0.24.1 to 0.25.0 but want to avoid having all
of the tasks restart again.

Thanks,
Craig

Re: How production un-ready are Mesos Cassandra, Spark and Kafka Frameworks?

[craig w]

So far the Kafka framework has worked well in production. We launch the
framework using marathon, then execute a few command line statements to add
and start the brokers.
On Oct 12, 2015 1:38 PM, "Dick Davies"  wrote:

> Hi Chris
>
> 
>
> Spark is a Mesos native, I'd have no hesitation running it on Mesos.
>
> Cassandra not so much -
> that's not to disparage the work people are putting in there, I think
> it's really interesting. But personally with complex beasts like Cassandra
> I want to be running as 'stock' as possible, as it makes it easier to learn
> from other peoples experiences.
>
> On 12 October 2015 at 17:47, Chris Elsmore <
> chris.elsm...@demandlogic.co.uk> wrote:
>
>> Hi all,
>>
>> Have just got back from a brilliant MesosCon Europe in Dublin, I learnt a
>> huge amount and a big thank-you for putting on a great conference to all
>> involved!
>>
>>
>> I am looking to deploy a small (maybe 5 max) Cassandra & Spark cluster to
>> do some data analysis at my current employer, and am a little unsure of the
>> current status of the frameworks this would need to run on Mesos- both the
>> mesosphere docs (which I’m guessing use the frameworks of the same name
>> hosted on Github) and the Github ReadMes mention that these are not
>> production ready, and the rough timeline of Q1 2016.
>>
>> I’m just wondering how production un-ready these are!? I am looking at
>> using Mesos to deploy future stateless services in the next 6 months or so,
>> and so I like the idea of adding to that system and the look of the
>> configuration that is handled for you to bind nodes together in these
>> frameworks. However it feels like for a smallish cluster of production
>> ready machines it might be better to deploy them standalone and stay
>> observant on the status of such things in the near future, and the
>> configuration wins are not that large especially for a small cluster.
>>
>>
>> Any experience and advice on the above would be greatly received!
>>
>>
>> Chris
>>
>>
>>
>>
>

Re: After restarting cluster task disappeared

[craig w]

I'm not sure about #3. I have seen things go awry when restarting the whole
cluster. When doing an upgrade from mesos 0.23.0 to 0.24.1, I restarted all
of the mesos-masters. Waited a few moments for a leader to be elected, then
restarted the slaves. When I went back to look at Marathon all of the tasks
were being redeployed, as though they had all been killed off for some
reason. That wasn't what I expected to happen since the upgrade was suppose
to be as simple as install and restart. Perhaps you're experiencing a
similar issue?

On Fri, Oct 9, 2015 at 8:25 AM, Badal Naik  wrote:

> Any idea about #1 ?
>
> Any one has experienced #3 ?
>
> On 09-Oct-2015, at 5:53 pm, craig w  wrote:
>
> With regards to item #2, I saw the same issue. it's been fixed in mesos
> 0.25 (release candidates are out now), see
> https://issues.apache.org/jira/browse/MESOS-3282.
>
> On Fri, Oct 9, 2015 at 8:16 AM, Badal Naik  wrote:
>
>> Hello Mesos-Users,
>>
>> I have set up 3 node mess cluster with ubuntu 14.04. i have started
>> zookeeper,Mesos and marathon. Every thing working fine expect three things.
>>
>> 1) When i restart the whole cluster mesos does not show completed tasks.
>> is it expected behaviour? if not what i should do?
>>
>> 2) in mesos web ui i’m not able to see
>> staged/started/finished/killed/failed/lost task numbers even when tasks are
>> running.
>>
>> 3) Every zookeeper instance throws this exception regularly:
>>
>>  2015-10-09 17:27:26,302 [myid:3] - WARN
>> [SendWorker:1:QuorumCnxManager$SendWorker@679] - Interrupted while
>> waiting for message on queue
>> java.lang.InterruptedException
>> at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
>> at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088)
>> at
>> java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
>> at
>> org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:831)
>> at
>> org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:62)
>> at
>> org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:667)
>>
>>
>>
>>
>> *Here is my Mesos-master configuration:*
>>
>> mesos master --ip=10.1.0.72  --work_dir=/var/lib/mesos-master --zk=
>> file:///etc/mesos/conf/zk --quorum=file:///etc/mesos/conf/quorum
>> Where zk=zk://zoo.service.consul:2181/mesos
>>   quorum=2
>>
>>
>>
>> *Mesos-Slave Configuration:*
>>
>> mesos slave --work_dir=/var/lib/mesos-slave --ip=10.1.0.72
>> --hostname=10.1.0.72 --strict=false  --master=
>> file:///etc/mesos/conf/master FrameworkInfo.checkpoint=True
>>
>>
>>
>> *Marathon Configuration:*
>>
>> java -jar /opt/marathon.jar  --master zk://zoo.service.consul:2181/mesos
>> --zk zk://zoo.service.consul:2181/marathon  --ha --hostname 10.1.0.72
>> --checkpoint
>>
>>
>>
>>
>> *Zookeeper configs with java version *"1.8.0_45"*:*
>>
>>
>>
>> dataDir=/var/lib/zookeeper
>> clientPort=2181
>> tickTime=2000
>> initLimit=10
>> syncLimit=20
>>
>>
>> autopurge.purgeInterval=0
>>
>>
>> zookeeper.connection.timeout.ms=6000
>> server.1=10.1.0.70:2888:3888
>> server.2=10.1.0.71:2888:3888
>> server.3=10.1.0.72:2888:3888
>>
>> And different *myid* has been given.
>>
>>
>> Can Anyone Help!!!
>>
>>
>>
>
>
> --
>
> https://github.com/mindscratch
> https://www.google.com/+CraigWickesser
> https://twitter.com/mind_scratch
> https://twitter.com/craig_links
>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: After restarting cluster task disappeared

[craig w]

With regards to item #2, I saw the same issue. it's been fixed in mesos
0.25 (release candidates are out now), see
https://issues.apache.org/jira/browse/MESOS-3282.

On Fri, Oct 9, 2015 at 8:16 AM, Badal Naik  wrote:

> Hello Mesos-Users,
>
> I have set up 3 node mess cluster with ubuntu 14.04. i have started
> zookeeper,Mesos and marathon. Every thing working fine expect three things.
>
> 1) When i restart the whole cluster mesos does not show completed tasks.
> is it expected behaviour? if not what i should do?
>
> 2) in mesos web ui i’m not able to see
> staged/started/finished/killed/failed/lost task numbers even when tasks are
> running.
>
> 3) Every zookeeper instance throws this exception regularly:
>
>  2015-10-09 17:27:26,302 [myid:3] - WARN
> [SendWorker:1:QuorumCnxManager$SendWorker@679] - Interrupted while
> waiting for message on queue
> java.lang.InterruptedException
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088)
> at
> java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
> at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:831)
> at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:62)
> at
> org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:667)
>
>
>
>
> *Here is my Mesos-master configuration:*
>
> mesos master --ip=10.1.0.72  --work_dir=/var/lib/mesos-master --zk=
> file:///etc/mesos/conf/zk --quorum=file:///etc/mesos/conf/quorum
> Where zk=zk://zoo.service.consul:2181/mesos
>   quorum=2
>
>
>
> *Mesos-Slave Configuration:*
>
> mesos slave --work_dir=/var/lib/mesos-slave --ip=10.1.0.72
> --hostname=10.1.0.72 --strict=false  --master=
> file:///etc/mesos/conf/master FrameworkInfo.checkpoint=True
>
>
>
> *Marathon Configuration:*
>
> java -jar /opt/marathon.jar  --master zk://zoo.service.consul:2181/mesos
> --zk zk://zoo.service.consul:2181/marathon  --ha --hostname 10.1.0.72
> --checkpoint
>
>
>
>
> *Zookeeper configs with java version *"1.8.0_45"*:*
>
>
>
> dataDir=/var/lib/zookeeper
> clientPort=2181
> tickTime=2000
> initLimit=10
> syncLimit=20
>
>
> autopurge.purgeInterval=0
>
>
> zookeeper.connection.timeout.ms=6000
> server.1=10.1.0.70:2888:3888
> server.2=10.1.0.71:2888:3888
> server.3=10.1.0.72:2888:3888
>
> And different *myid* has been given.
>
>
> Can Anyone Help!!!
>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

0.24.1 blog post link broken

[craig w]

The link to the blog post about mesos 0.24.1 seems to be broken:
http://mesos.apache.org/blog/mesos-0-24-1-and-more-released/

Re: Changing mesos slave configuration

[craig w]

I believe Brian might be referring to the "KillMode" in the systemd unit
file:

# the default is cgroup, which means kill all processes
# in the control group of this process, which is not
# what you'd want
KillMode=process

On Wed, Sep 23, 2015 at 8:11 AM, Brian Devins  wrote:

> Are you using systemd? There is a known issue with slave recovery on
> systemd. I'm on mobile or I would link you to the last thread around this
> but there is a line you can add to the config that is supposed to fix it.
> Whether it will fix it is another matter. I am fighting this issue at work
> myself.
> On Sep 23, 2015 7:53 AM, "Paul Bell"  wrote:
>
>> Hi Pradeep,
>>
>> Perhaps I am speaking to a slightly different point, but when I change
>> /etc/default/mesos-slave to add a new attribute, I have to remove file
>> /tmp/mesos/meta/slaves/latest.
>>
>> IIRC, mesos-slave itself, in failing to start after such a change, tells
>> me to do this:
>>
>> rm -f /tmp/mesos/meta/slaves/latest
>>
>>
>> But I know of no way to make such configuration changes without downtime.
>> And I'd very much like it if Mesos supported such dynamic changes. I
>> suppose this would require that the agent consult its default file on
>> demand, rather than once at start-up.
>>
>> Cordially,
>>
>> Paul
>>
>> On Wed, Sep 23, 2015 at 4:41 AM, Pradeep Chhetri <
>> pradeep.chhetr...@gmail.com> wrote:
>>
>>> Hello all,
>>>
>>> I have often faced this problem that whenever i try to add some
>>> configuration parameter to mesos-slave or change any configuration (eg. add
>>> a new attribute in mesos-slave), the mesos slave doesnt come up on restart.
>>> I have to delete the slave.info file and then restart the slave but it
>>> ends up killing all the docker containers started using mesos.
>>>
>>> I was trying to figure out the best way to make such changes without
>>> making any downtime.
>>>
>>> Thank you.
>>>
>>> --
>>> Pradeep Chhetri
>>>
>>
>>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: mesos 0.24 released?

[craig w]

Thanks for following up.

On Mon, Sep 21, 2015 at 3:00 PM, Vinod Kone  wrote:

> Published the blogpost for 24.0.
>
> On Fri, Sep 18, 2015 at 9:42 PM, CCAAT  wrote:
>
>>
>> Oh,
>>
>> Here is a link that explains the Variable meanings for the packages
>> downloaded by gentoo's package manager, portage::
>>
>> https://devmanual.gentoo.org/ebuild-writing/variables/
>>
>> I really am at the stage that I want/need to test many tarball releases
>> and also to start testing on other architectures, like aarch64.
>> Please advise on the location(s) of these tarballs, even if they
>> are not formal releases or release candidates.
>>
>> Is anyone testing tarball compilations on other arches?
>> Do I need to start reading the dev list for this sort of granularity?
>> All suggestions are most welcome
>>
>>
>> curiously,
>> James
>>
>>
>>
>>
>>
>> I'm confused. Here at gentoo, we usually (almost always) follow the
>>> release numbers including the full minor release number designations
>>> of the tarball. So if I download a tarball, it's not references by minor
>>> release number?
>>>
>>> Here is a simple, quick example::
>>>
>>> www-client/seamonkeyAvailable versions:  2.33.1-r1 2.35
>>>
>>> This accuracy is quintessentially necessary for the gentoo community.
>>> Do I have to fix the name of the tarballs manually?  In gentoo ebuilds,
>>> the files that download, the tarballs directly from a source repo
>>> the exact and complete version number is necessary.   Should
>>> I use github?
>>>
>>> What are recommendations for this?
>>>
>>> Here is what I'm using for mesos' current ebuild::
>>>
>>> SRC_URI="http://apache.org/dist/${PN}/${PV}/${P}.tar.gz";
>>>
>>>
>>> curiously,
>>> James
>>>
>>>
>>>
>>>
>>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Official RPMs

[craig w]

Mesosphere provides packages, you can find more information here:
https://mesosphere.com/downloads/

As of right now, they don't seem to have a 0.24.0 package.

On Fri, Sep 18, 2015 at 8:51 PM, Brian Hicks  wrote:

> We've got some experimental packages at bintray.com/asteris/mantl-rpm,
> source is at github.com/asteris-llc/mesos-packaging. They can really use
> some testing if you wanted to give them a try. Configuration is a bit
> different than the Mesosphere packages, see the repo for details.
> On Sep 18, 2015 7:01 PM, "Zameer Manji"  wrote:
>
>> Hey,
>>
>> Does the Apache Mesos project provide OS packages for installation? I
>> haven't been able to find any for the 0.24 release and I think having them
>> would make installing Mesos a lot easier.
>>
>> --
>> Zameer Manji
>>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: mesos 0.24 released?

[craig w]

Gotcha will there be a blog post / release announcement on the website soon?
On Sep 18, 2015 2:12 PM, "Greg Mann"  wrote:

> Hi Craig,
> 0.24.0 has been released, the vote that selected 0.24.0-rc2 as the final
> release passed on Sept. 4, which matches the date from that tarball.
>
> Cheers,
> Greg
>
> On Fri, Sep 18, 2015 at 10:57 AM, craig w  wrote:
>
>> So RC releases are still going out, it's not yet "final"?
>> On Sep 18, 2015 12:15 PM, "Joris Van Remoortere" 
>> wrote:
>>
>>> Mesos 0.24.0 was indeed released.
>>> 0.24.0 rc2 was voted in on september 4th. You can see the voting e-mail
>>> chain on the dev list.
>>>
>>>
>>> On Fri, Sep 18, 2015 at 7:51 AM, craig w  wrote:
>>>
>>>> has mesos 0.24 been released? it's on the downloads page, but the
>>>> tarball is from september 4
>>>>
>>>> http://mesos.apache.org/downloads/
>>>>
>>>
>>>
>

Re: mesos 0.24 released?

[craig w]

So RC releases are still going out, it's not yet "final"?
On Sep 18, 2015 12:15 PM, "Joris Van Remoortere" 
wrote:

> Mesos 0.24.0 was indeed released.
> 0.24.0 rc2 was voted in on september 4th. You can see the voting e-mail
> chain on the dev list.
>
>
> On Fri, Sep 18, 2015 at 7:51 AM, craig w  wrote:
>
>> has mesos 0.24 been released? it's on the downloads page, but the tarball
>> is from september 4
>>
>> http://mesos.apache.org/downloads/
>>
>
>

mesos 0.24 released?

[craig w]

has mesos 0.24 been released? it's on the downloads page, but the tarball
is from september 4

http://mesos.apache.org/downloads/

Re: Basic installation question

[craig w]

No problem, they have a "downloads" link inn their menu:
https://mesosphere.com/downloads/
On Sep 4, 2015 5:43 PM, "Stephen Boesch"  wrote:

> @Craig . That is an incomplete answer - given that such links are not
> presented in an obvious manner .  Maybe you managed to find  a link on
> their site that provides prebuilt for Centos7: if so then please share it.
>
>
> I had previously found a link on their site for prebuilt binaries but is
> based on using CDH4 (which is not possible for my company). It is also old.
>
> https://docs.mesosphere.com/tutorials/install_centos_rhel/
>
>
> 2015-09-04 14:27 GMT-07:00 craig w :
>
>> Mesosphere has packages prebuilt, go to their site to find how to install
>> On Sep 4, 2015 5:11 PM, "Stephen Boesch"  wrote:
>>
>>>
>>> After following the directions here:
>>> http://mesos.apache.org/gettingstarted/
>>>
>>> Which for centos7 includes the following:
>>>
>>>
>>>
>>>
>>>   # Change working directory.
>>> $ cd mesos
>>>
>>> # Bootstrap (Only required if building from git repository).
>>> $ ./bootstrap
>>>
>>> # Configure and build.
>>> $ mkdir build
>>> $ cd build
>>> $ ../configure
>>> $ make
>>>
>>> In order to speed up the build and reduce verbosity of the logs, you can
>>> append-j  V=0 to make.
>>>
>>> # Run test suite.
>>> $ make check
>>>
>>> # Install (Optional).
>>> $ make install
>>>
>>>
>>>
>>> But the installation is not correct afterwards: here is the bin
>>> directory:
>>>
>>> $ ll bin
>>> total 92
>>> -rw-r--r--.  1 stack stack 1769 Jul 17 23:14 valgrind-mesos-tests.sh.in
>>> -rw-r--r--.  1 stack stack 1769 Jul 17 23:14 valgrind-mesos-slave.sh.in
>>> -rw-r--r--.  1 stack stack 1772 Jul 17 23:14 valgrind-mesos-master.sh.in
>>> -rw-r--r--.  1 stack stack 1769 Jul 17 23:14 valgrind-mesos-local.sh.in
>>> -rw-r--r--.  1 stack stack 1026 Jul 17 23:14 mesos-tests.sh.in
>>> -rw-r--r--.  1 stack stack  901 Jul 17 23:14 mesos-tests-flags.sh.in
>>> -rw-r--r--.  1 stack stack 1019 Jul 17 23:14 mesos-slave.sh.in
>>> -rw-r--r--.  1 stack stack 1721 Jul 17 23:14 mesos-slave-flags.sh.in
>>> -rw-r--r--.  1 stack stack 1366 Jul 17 23:14 mesos.sh.in
>>> -rw-r--r--.  1 stack stack 1026 Jul 17 23:14 mesos-master.sh.in
>>> -rw-r--r--.  1 stack stack  858 Jul 17 23:14 mesos-master-flags.sh.in
>>> -rw-r--r--.  1 stack stack 1023 Jul 17 23:14 mesos-local.sh.in
>>> -rw-r--r--.  1 stack stack  935 Jul 17 23:14 mesos-local-flags.sh.in
>>> -rw-r--r--.  1 stack stack 1466 Jul 17 23:14 lldb-mesos-tests.sh.in
>>> -rw-r--r--.  1 stack stack 1489 Jul 17 23:14 lldb-mesos-slave.sh.in
>>> -rw-r--r--.  1 stack stack 1492 Jul 17 23:14 lldb-mesos-master.sh.in
>>> -rw-r--r--.  1 stack stack 1489 Jul 17 23:14 lldb-mesos-local.sh.in
>>> -rw-r--r--.  1 stack stack 1498 Jul 17 23:14 gdb-mesos-tests.sh.in
>>> -rw-r--r--.  1 stack stack 1527 Jul 17 23:14 gdb-mesos-slave.sh.in
>>> -rw-r--r--.  1 stack stack 1530 Jul 17 23:14 gdb-mesos-master.sh.in
>>> -rw-r--r--.  1 stack stack 1521 Jul 17 23:14 gdb-mesos-local.sh.in
>>> drwxr-xr-x.  2 stack stack 4096 Jul 17 23:21 .
>>> drwxr-xr-x. 11 stack stack 4096 Sep  4 20:08 ..
>>>
>>> So .. two things:
>>>
>>> (a) what is missing from the installation instructions?
>>>
>>> (b) Is there an *up to date *rpm/yum installation for centos7?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>

Re: Basic installation question

[craig w]

Mesosphere has packages prebuilt, go to their site to find how to install
On Sep 4, 2015 5:11 PM, "Stephen Boesch"  wrote:

>
> After following the directions here:
> http://mesos.apache.org/gettingstarted/
>
> Which for centos7 includes the following:
>
>
>
>
>   # Change working directory.
> $ cd mesos
>
> # Bootstrap (Only required if building from git repository).
> $ ./bootstrap
>
> # Configure and build.
> $ mkdir build
> $ cd build
> $ ../configure
> $ make
>
> In order to speed up the build and reduce verbosity of the logs, you can
> append-j  V=0 to make.
>
> # Run test suite.
> $ make check
>
> # Install (Optional).
> $ make install
>
>
>
> But the installation is not correct afterwards: here is the bin directory:
>
> $ ll bin
> total 92
> -rw-r--r--.  1 stack stack 1769 Jul 17 23:14 valgrind-mesos-tests.sh.in
> -rw-r--r--.  1 stack stack 1769 Jul 17 23:14 valgrind-mesos-slave.sh.in
> -rw-r--r--.  1 stack stack 1772 Jul 17 23:14 valgrind-mesos-master.sh.in
> -rw-r--r--.  1 stack stack 1769 Jul 17 23:14 valgrind-mesos-local.sh.in
> -rw-r--r--.  1 stack stack 1026 Jul 17 23:14 mesos-tests.sh.in
> -rw-r--r--.  1 stack stack  901 Jul 17 23:14 mesos-tests-flags.sh.in
> -rw-r--r--.  1 stack stack 1019 Jul 17 23:14 mesos-slave.sh.in
> -rw-r--r--.  1 stack stack 1721 Jul 17 23:14 mesos-slave-flags.sh.in
> -rw-r--r--.  1 stack stack 1366 Jul 17 23:14 mesos.sh.in
> -rw-r--r--.  1 stack stack 1026 Jul 17 23:14 mesos-master.sh.in
> -rw-r--r--.  1 stack stack  858 Jul 17 23:14 mesos-master-flags.sh.in
> -rw-r--r--.  1 stack stack 1023 Jul 17 23:14 mesos-local.sh.in
> -rw-r--r--.  1 stack stack  935 Jul 17 23:14 mesos-local-flags.sh.in
> -rw-r--r--.  1 stack stack 1466 Jul 17 23:14 lldb-mesos-tests.sh.in
> -rw-r--r--.  1 stack stack 1489 Jul 17 23:14 lldb-mesos-slave.sh.in
> -rw-r--r--.  1 stack stack 1492 Jul 17 23:14 lldb-mesos-master.sh.in
> -rw-r--r--.  1 stack stack 1489 Jul 17 23:14 lldb-mesos-local.sh.in
> -rw-r--r--.  1 stack stack 1498 Jul 17 23:14 gdb-mesos-tests.sh.in
> -rw-r--r--.  1 stack stack 1527 Jul 17 23:14 gdb-mesos-slave.sh.in
> -rw-r--r--.  1 stack stack 1530 Jul 17 23:14 gdb-mesos-master.sh.in
> -rw-r--r--.  1 stack stack 1521 Jul 17 23:14 gdb-mesos-local.sh.in
> drwxr-xr-x.  2 stack stack 4096 Jul 17 23:21 .
> drwxr-xr-x. 11 stack stack 4096 Sep  4 20:08 ..
>
> So .. two things:
>
> (a) what is missing from the installation instructions?
>
> (b) Is there an *up to date *rpm/yum installation for centos7?
>
>
>
>
>
>
>

Are the resource options documented?

[craig w]

When configuring a mesos-slave with "--resources", I know "cpu", "mem" and
"ports" are available. Are there others? Are these documented somewhere?

I've found some examples here
https://open.mesosphere.com/reference/mesos-slave/ and the configuration
page (http://mesos.apache.org/documentation/latest/configuration/) is
generic with it's description of "--resources".

Thanks
craig

Re: Master UI - Tasks section is empty

[craig w]

See https://issues.apache.org/jira/browse/MESOS-3282
On Aug 23, 2015 12:28 PM, "Jeremy Olexa"  wrote:

> Hi all,
>
> On a new cluster, the tasks section of the left sidebar is populated as
> jobs are staged, started, killed, etc. I've noticed that after a rolling
> restart of the cluster, like taking a node out for maintenance - or
> restarted instances in an ASG, that the Tasks section of the UI stops
> working. There are no longer any value in the UI.
>
> It appears that this part of the UI is in js/controllers.js, but I don't
> understand the internals quite yet. Is this issue related to MESOS-527? Any
> other insight into this problem?
>
> Thanks,
> Jeremy
>

Re: Talks at MesosCon 2015

[craig w]

An earlier post said all videos will be online a few days after the
conference.

On Thu, Aug 20, 2015 at 1:29 PM, Kenneth Su  wrote:

> Yes, agree that. I am looking for live video today on the youtube, but
> nothing there.
>
> Definitely the forum will help to follow up and discuss from there
>
> Thanks!
>
> Kenneth
>
> On Thu, Aug 20, 2015 at 11:26 AM, Haripriya Ayyalasomayajula <
> aharipriy...@gmail.com> wrote:
>
>> Hi all,
>>
>> I'm at the MesosCon 2015 today and was just curious if all the talks/
>> presentations would be captured anywhere( mesosphere blog/ youtube). It
>> would be very helpful to have them recorded. There are multiple interesting
>> talks at the same time scheduled and Its not possible to cover all.
>> I strongly believe if we have a forum to follow up with these talks /
>> topics presented here will be helpful.
>>
>> Thanks.
>>
>>
>> --
>> Regards,
>> Haripriya Ayyalasomayajula
>>
>>
>>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: SSL in Mesos 0.23

[craig w]

I've run ldd on /usr/sbin/mesos-master (on CentOS 7 using mesos 0.23 from
mesosphere repo) and I see "libssl.3.so" and "libssl.so.10"

On Thu, Aug 6, 2015 at 12:20 PM, Jeff Schroeder 
wrote:

> Can you run ldd on the mesos-master or mesos-slave binaries? I believe you
> *should* see openssl libraries in the output if those packages are
> configured using --enable-ssl.
>
> On Thu, Aug 6, 2015 at 9:46 AM, Dharmit Shah 
> wrote:
>
>> Hello,
>>
>> I followed Mesos cluster setup guide on the Mesosphere website [1]. I
>> set it up on a CentOS 7 system. For installation of packages, I went
>> with Mesosphere provided repositories.
>>
>> Now that Mesos 0.23 has been released with SSL capabilities, I believe
>> it is possible to have communication between the master, slaves and
>> frameworks be secured by SSL. Am I right?
>>
>> I would like to set it up in my environment. I am using
>> `mesos-0.23.0-1.0.centos701406.x86_64`.
>>
>> The official Mesos documentation on the topic [2] illustrates how
>> things can be setup when building Mesos from source.
>>
>> I would like to know if Mesos package shipped by Mesosphere repo has
>> this feature or not yet? I tried setting
>> `/etc/mesos-master/ssl_enabled` on one of the master nodes. But
>> restarting `mesos-master` service failed stating that option
>> `ssl_enabled` is unknown.
>>
>> Thanks for your help!
>>
>> [1] http://open.mesosphere.com/getting-started/datacenter/install/
>> [2] http://mesos.apache.org/documentation/latest/mesos-ssl/
>>
>> Regards,
>> Dharmit.
>>
>
>
>
> --
> Jeff Schroeder
>
> Don't drink and derive, alcohol and analysis don't mix.
> http://www.digitalprognosis.com
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Mesos slave help

[craig w]

Have you tried clearing out the data in the slave's work_dir? for example
if work dir is /var/mesos, rm -rf /var/mesos/* then start the slave?

On Thu, Aug 6, 2015 at 6:55 AM, Stephen Knight  wrote:

> My system doesn't support cat with systemctl for some reason but here is
> the contents of /usr/lib/systemd/system/mesos-slave.service
>
> [Unit]
>
> Description=Mesos Slave
>
> After=network.target
>
> Wants=network.target
>
>
> [Service]
>
> ExecStart=/usr/bin/mesos-init-wrapper slave
>
> KillMode=process
>
> Restart=always
>
> RestartSec=20
>
> LimitNOFILE=16384
>
> CPUAccounting=true
>
> MemoryAccounting=true
>
>
> [Install]
>
> WantedBy=multi-user.target
>
>
> What are the required flags to start it manually?
>
> On Thu, Aug 6, 2015 at 2:51 PM, haosdent  wrote:
>
>> Or you could try "systemctl cat mesos-slave.service" and show us the
>> file content.
>>
>> On Thu, Aug 6, 2015 at 6:49 PM, haosdent  wrote:
>>
>>> From this message, I think "systemctl status mesos-slave.service -l" run
>>> mesos-slave with uncorrect flags. And the status out of it is the help
>>> message of slave. Could you try to start mesos-slave in manual way? Not
>>> through systemctl.
>>>
>>> On Thu, Aug 6, 2015 at 6:41 PM, Stephen Knight 
>>> wrote:
>>>
 systemctl gives me the following output on CentOS: The command to start
 I ran was "systemctl start mesos-slave.service"

 [root@ip-172-31-35-167 mesos]# systemctl status mesos-slave.service -l

 mesos-slave.service - Mesos Slave

Loaded: loaded (/usr/lib/systemd/system/mesos-slave.service; enabled)

   Drop-In: /etc/systemd/system/mesos-slave.service.d

└─mesos-slave-containerizers.conf

Active: activating (auto-restart) (Result: exit-code) since Thu
 2015-08-06 10:38:08 UTC; 2s ago

   Process: 1472 ExecStart=/usr/bin/mesos-init-wrapper slave *(code=exited,
 status=1/FAILURE)*

  Main PID: 1472 (code=exited, status=1/FAILURE)


 Aug 06 10:38:08 ip-172-31-35-167.ec2.internal mesos-slave[1483]: *If
 strict=false, any expected errors (e.g., slave cannot recover*

 Aug 06 10:38:08 ip-172-31-35-167.ec2.internal mesos-slave[1483]: 
 *information
 about an executor, because the slave died right before*

 Aug 06 10:38:08 ip-172-31-35-167.ec2.internal mesos-slave[1483]: *the
 executor registered.) during recovery are ignored and as much*

 Aug 06 10:38:08 ip-172-31-35-167.ec2.internal mesos-slave[1483]: *state
 as possible is recovered.*

 Aug 06 10:38:08 ip-172-31-35-167.ec2.internal mesos-slave[1483]: *(default:
 true)*

 Aug 06 10:38:08 ip-172-31-35-167.ec2.internal mesos-slave[1483]: 
 *--[no-]switch_user
   Whether to run tasks as the user who*

 Aug 06 10:38:08 ip-172-31-35-167.ec2.internal mesos-slave[1483]: *submitted
 them rather than the user running*

 Aug 06 10:38:08 ip-172-31-35-167.ec2.internal mesos-slave[1483]: *the
 slave (requires setuid permission) (default: true)*

 Aug 06 10:38:08 ip-172-31-35-167.ec2.internal mesos-slave[1483]: 
 *--[no-]version
   Show version and exit. (default: false)*

 Aug 06 10:38:08 ip-172-31-35-167.ec2.internal mesos-slave[1483]: 
 *--work_dir=VALUE
 Directory path to place framework work
 directories*



 I've also run strace against it, nothing sticks out:


 strace systemctl start mesos-slave.service

 execve("/bin/systemctl", ["systemctl", "start", "mesos-slave.service"],
 [/* 18 vars */]) = 0

 brk(0)  = 0x7f5c2af9f000

 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
 0) = 0x7f5c2a5c6000

 access("/etc/ld.so.preload", R_OK)  = -1 ENOENT (No such file or
 directory)

 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3

 fstat(3, {st_mode=S_IFREG|0644, st_size=20940, ...}) = 0

 mmap(NULL, 20940, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f5c2a5c

 close(3)= 0

 open("/lib64/libsystemd-daemon.so.0", O_RDONLY|O_CLOEXEC) = 3

 read(3,
 "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240\r\0\0\0\0\0\0"..., 832)
 = 832

 fstat(3, {st_mode=S_IFREG|0755, st_size=15216, ...}) = 0

 mmap(NULL, 2109448, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
 0) = 0x7f5c2a1a2000

 mprotect(0x7f5c2a1a4000, 2097152, PROT_NONE) = 0

 mmap(0x7f5c2a3a4000, 4096, PROT_READ|PROT_WRITE,
 MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f5c2a3a4000

 mmap(0x7f5c2a3a5000, 8, PROT_READ|PROT_WRITE,
 MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f5c2a3a5000

 close(3)= 0

 open("/lib64/libdbus-1.so.3", O_RDONLY|O_CLOEXEC) = 

Re: Get List of Active Slaves

[craig w]

If the information is going to be in Zookeeper, you could setup one of the
REST frontends and query Zookeeper over HTTP (check the zookeeper
src/contrib directory).
On Aug 5, 2015 1:26 AM, "Marco Massenzio"  wrote:

> Now that Mesos (0.24, to be released soon) publishes the Master info to
> ZooKeeper in JSON, it should be (relatively) easier to get the info about
> the leading master directly from there (or even set a Watcher on the znode
> to be alerted of leadership changes).
> Not as easy as hitting an HTTP endpoint, granted, but that's just a hard
> problem to solve anyway.
>
> I'm planning to provide sample code and a blog entry about this soon as I
> have time, but it won't be before this weekend at the earliest (and more
> likely the next one).
>
> *Marco Massenzio*
> *Distributed Systems Engineer*
>
> On Tue, Aug 4, 2015 at 5:04 PM, Steven Schlansker <
> sschlans...@opentable.com> wrote:
>
>> Unfortunately that sort of solution is also prone to races.
>> I do not think this is really possible (at least not even remotely
>> elegantly) to solve externally to Mesos itself.
>>
>> On Aug 4, 2015, at 4:49 PM, James DeFelice 
>> wrote:
>>
>> > If you're using mesos-dns I think you can query slave.mesos to get an a
>> record for each. I believe it responds to srv requests too.
>> >
>> > On Aug 4, 2015 7:29 PM, "Steven Schlansker" 
>> wrote:
>> > Unfortunately this is racey.  If you redirect to a master just as it is
>> removed from leadership, you can still get bogus data, with no indication
>> anything went wrong.  Some people are reporting that this breaks tools that
>> generate HTTP proxy configurations.
>> >
>> > I filed this issue a while ago as
>> https://issues.apache.org/jira/browse/MESOS-1865
>> >
>> > On Aug 4, 2015, at 3:49 PM, Vinod Kone  wrote:
>> >
>> > > Not today, no.
>> > >
>> > > But, you could either hit the "/redirect" endpoint on any master that
>> should redirect you to the leading master.
>> > >
>> > > On Tue, Aug 4, 2015 at 3:29 PM, Nastooh Avessta (navesta) <
>> nave...@cisco.com> wrote:
>> > > I see. Nope, and pointing to the leading master shows the proper
>> resultJ Thanks.
>> > >
>> > > Is there a REST equivalent to mesos-resolve, so that one can
>> ascertain who is the leader without having to point  to the leader?
>> > >
>> > > Cheers,
>> > >
>> > >
>> > >
>> > > 
>> > >
>> > > Nastooh Avessta
>> > > ENGINEER.SOFTWARE ENGINEERING
>> > > nave...@cisco.com
>> > > Phone: +1 604 647 1527
>> > >
>> > > Cisco Systems Limited
>> > > 595 Burrard Street, Suite 2123 Three Bentall Centre, PO Box 49121
>> > > VANCOUVER
>> > > BRITISH COLUMBIA
>> > > V7X 1J1
>> > > CA
>> > > Cisco.com
>> > >
>> > >
>> > >
>> > > Think before you print.
>> > >
>> > > This email may contain confidential and privileged material for the
>> sole use of the intended recipient. Any review, use, distribution or
>> disclosure by others is strictly prohibited. If you are not the intended
>> recipient (or authorized to receive for the recipient), please contact the
>> sender by reply email and delete all copies of this message.
>> > >
>> > > For corporate legal information go to:
>> > > http://www.cisco.com/web/about/doing_business/legal/cri/index.html
>> > >
>> > > Cisco Systems Canada Co, 181 Bay St., Suite 3400, Toronto, ON,
>> Canada, M5J 2T3. Phone: 416-306-7000; Fax: 416-306-7099. Preferences -
>> Unsubscribe – Privacy
>> > >
>> > >
>> > >
>> > > From: Vinod Kone [mailto:vinodk...@gmail.com]
>> > > Sent: Tuesday, August 04, 2015 3:19 PM
>> > > To: user@mesos.apache.org
>> > > Subject: Re: Get List of Active Slaves
>> > >
>> > >
>> > >
>> > > Is that the leading master?
>> > >
>> > >
>> > >
>> > > On Tue, Aug 4, 2015 at 3:09 PM, Nastooh Avessta (navesta) <
>> nave...@cisco.com> wrote:
>> > >
>> > > Hi
>> > >
>> > > Trying to get the list of active slaves, via cli, e.g. curl
>> http://10.4.50.80:5050/master/slaves | python -m json.tool and am not
>> getting the expected results.  The returned value is empty:
>> > >
>> > > {
>> > >
>> > > "slaves": []
>> > >
>> > > }
>> > >
>> > > , whereas, looking at web gui I can see that there are deployed
>> slaves. Am I missing something?
>> > >
>> > > Cheers,
>> > >
>> > >
>> > >
>> > > 
>> > >
>> > > Nastooh Avessta
>> > > ENGINEER.SOFTWARE ENGINEERING
>> > > nave...@cisco.com
>> > > Phone: +1 604 647 1527
>> > >
>> > > Cisco Systems Limited
>> > > 595 Burrard Street, Suite 2123 Three Bentall Centre, PO Box 49121
>> > > VANCOUVER
>> > > BRITISH COLUMBIA
>> > > V7X 1J1
>> > > CA
>> > > Cisco.com
>> > >
>> > >
>> > >
>> > > Think before you print.
>> > >
>> > > This email may contain confidential and privileged material for the
>> sole use of the intended recipient. Any review, use, distribution or
>> disclosure by others is strictly prohibited. If you are not the intended
>> recipient (or authorized to receive for the recipient), please contact the
>> sender by reply email and delete all copies of this message.
>> > >
>> > > For corporate legal information go to:
>> > 

Re: Is it possible to run mesos master/slave in private IP and exhibit mesos cluster status with public IP?

[craig w]

The mesos-slaves have their own UI separate from the master? If so, what's
the URL to get to it? I just tried http://:5051 and got a
blank page.

On Tue, Jul 28, 2015 at 6:51 AM, Adam Bordelon  wrote:

> A simple nginx reverse proxy will get you most of the way there, but only
> for the master webui. Since the tasks' sandboxes are hosted on each slave's
> webui, you would also have to reverse proxy each slave's webui in order for
> sandboxes to be publicly accessible. More complicated, but not impossible.
>
> Sounds like what you really want is:
> https://issues.apache.org/jira/browse/MESOS-2102
>
> On Mon, Jul 27, 2015 at 5:55 AM, haosdent  wrote:
>
>> Does nginx could satisfy your requirement? You requests to nginx, and
>> nginx reverse proxy the mesos master.
>>
>> On Mon, Jul 27, 2015 at 8:50 PM, sujz <43183...@qq.com> wrote:
>>
>>> Hi, all,
>>>
>>> I want to run mesos master/slave/framework in private IP, and exhibit
>>> mesos cluster status with a new public IP, this way can enhance safe and
>>> privacy protection of enterprise internal datacenter network.
>>>
>>> After reading through the code, unfortunately, I didn't find where
>>> mesos-master binds listening ip and port to http server(please give me some
>>> hints), so I want to know how to implement communicating with slaves and
>>> frameworks with private IP and building http server with public IP for
>>> serving external http requests?
>>>
>>> Any suggestions will be appreciated.
>>> Best regards!
>>
>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Docker 1.6 and Registry 2.0 support

[craig w]

That's good to know. I suppose Mesos/Marathon working with Docker Registry
2.0 is not ready, would like confirmation.

On Tue, Apr 21, 2015 at 6:40 AM, haosdent  wrote:

> My docker client version is 1.6 . When I use it to connect my private
> registry which only provide API version 1, it works as normal.
>
> On Tue, Apr 21, 2015 at 5:57 PM, craig w  wrote:
>
>> From what I can tell, Mesos and Marathon should still be able to work
>> with Docker 1.6, but will Mesos / Marathon be able to pull images from the
>> Registry 2.0?
>>
>> It seems like the new registry won't support the v1 API based on this
>> section of their "Deploying" page:
>> https://github.com/docker/distribution/blob/master/docs/deploying.md#configure-nginx-with-a-v1-and-v2-registry
>>
>>
>> Docker 1.6 and Registry 2.0 release article -
>> http://blog.docker.com/2015/04/faster-and-better-image-distribution-with-registry-2-0-and-engine-1-6/
>>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Docker 1.6 and Registry 2.0 support

[craig w]

>From what I can tell, Mesos and Marathon should still be able to work with
Docker 1.6, but will Mesos / Marathon be able to pull images from the
Registry 2.0?

It seems like the new registry won't support the v1 API based on this
section of their "Deploying" page:
https://github.com/docker/distribution/blob/master/docs/deploying.md#configure-nginx-with-a-v1-and-v2-registry


Docker 1.6 and Registry 2.0 release article -
http://blog.docker.com/2015/04/faster-and-better-image-distribution-with-registry-2-0-and-engine-1-6/

Re: Current State of Service Discovery

[craig w]

We're going with HAProxy on every node + haproxy-marathon-bridge (since
we're leveraging Marathon). We deployed mesos-dns but it didn't make seem
to make sense to have both solutions.

-craig

On Wed, Apr 1, 2015 at 12:33 PM, Adam Shannon 
wrote:

> David,
>
> Smartstack was one of the inspirations we used to decide how we wanted to
> build out service discovery. The one thing we decided on was that we wanted
> the haproxy instances to be the front line load balancers. (The ones
> directly open to the internet.)
>
> The one thing from the Smartstack post is that I don't see any mention of
> how their internal dashboards are routed and accessed. We are able to
> create backends which are propagated out to all instances of haproxy that
> proxy for dashboards. This allows us to scale those just as easily as any
> other service we run.
>
> On Wed, Apr 1, 2015 at 10:51 AM, David Kesler  wrote:
>
>>  That approach sounds similar to Smartstack (
>> http://nerds.airbnb.com/smartstack-service-discovery-cloud/).
>>
>>
>>
>> *From:* Adam Shannon [mailto:adam.shan...@banno.com]
>> *Sent:* Wednesday, April 01, 2015 10:58 AM
>> *To:* mesos-users
>> *Subject:* Re: Current State of Service Discovery
>>
>>
>>
>> I figured I would comment on how Banno is setting up service discovery
>> with mesos. We've built everything around docker containers and then a
>> wrapper around that which we call "sidecar" that handles service discovery,
>> basic process supervision, and hot reloads of the underlying app config.
>>
>>
>>
>> Basically sidecar wraps an existing docker image (with FROM) and runs the
>> underlying command but monitors it. From there sidecar also has a concept
>> of being able to format templates which are written to the filesystem (in
>> the container). When writing the template sidecar tracks the ports and
>> configs allocated and used. It uses that information to add watches into
>> zookeeper (where we store overrides from the default config options per
>> app).
>>
>>
>>
>> The nicest thing we've found is that we're able to use a large range of
>> ports per mesos slave. Further, because our port information is stored in
>> zookeeper any other app (and therefore sidecar) we write can lookup
>> host/port info for a service they need.
>>
>>
>>
>> From the host/port information in zookeeper we've created a sidecar for
>> haproxy which can have backends created on it which are tied to app names
>> from services registered in zookeeper. This allows haproxy to query (and
>> watch) for all instances of apps and proxy them from the known host/port of
>> haproxy. When changes occur (and thus watches fired from zookeeper) the
>> haproxy-sidecar instances are able to reload with the updates.
>>
>>
>>
>> We're still working to get this all fully deployed to production (and
>> then open sourced), but it seems to combine some of the best features of
>> other public options.
>>
>>
>>
>> On Wed, Apr 1, 2015 at 9:05 AM, John Omernik  wrote:
>>
>> I have been researching service discovery on Mesos quite a bit lately,
>> and due to my background, may be making assumptions that don't apply to a
>> Mesos Datacenter. I've read through docs, and I have come up with two main
>> approaches to service discovery, and both appear to have strengths and
>> weaknesses, and I wanted to describe what I've seen here, as well as the
>> challenges as I understand them to perhaps have any misconceptions I may
>> have corrected.
>>
>> Basically, I see two main approaches to the service discovery on Mesos.
>> You have the mesos-dns (https://github.com/mesosphere/mesos-dns) package
>> with is a DNS based service discovery, and then you have HAProxy based
>> discovery (which can be represented by both the haproxy-marathon-bridge (
>> https://github.com/mesosphere/marathon/blob/master/bin/haproxy-marathon-bridge)
>> script and the Bamboo project (https://github.com/QubitProducts/bamboo)).
>>
>> HAProxy
>>
>> With the HAProxy method, as I see it, you basically install HAProxy on
>> every node. The two above mentioned projects query marathon to determine
>> where the services are running, and then rewrite the haproxy config on
>> every node to allow basically every node to listen on a specific port, and
>> from there, that port will be forwarded, via round robin to the actual
>> node/port combinations where the services running.
>>
>> So, let's use the example of a Hive Thrift server running in a Docker
>> container on port 1.  Lets say you have a 5 node cluster, node1, node2,
>> etc. You spin that container up with instances = 3 in marathon, and
>> Marathon/docker run the container on node2, node3 and another on node2
>>  There is a bridged port to 1 inside the container, that is tied to an
>> available port on the physical node. Perhaps one instance on node2 gets
>> 3 and the other instance gets 30001.  node3's instance is tied to port
>> 30001.  So now you have 3 instances that exposed at
>>
>> node2:3  -> dockercontainer:1
>> node2:30

Re: [RESULT][VOTE] Release Apache Mesos 0.22.0 (rc4)

[craig w]

Congrats, I was working on a quick post summarizing what's new (based on
jira and the video from niklas) which I just posted (great timing)

http://craigwickesser.com/2015/03/mesos-022-release/

On Tue, Mar 24, 2015 at 8:30 PM, Paul Otto  wrote:

> This is awesome! Thanks for all the hard work you all have put into this!
> I am really excited to update to the latest stable version of Apache Mesos!
>
> Regards,
> Paul
>
>
> Paul Otto
> Principal DevOps Architect, Co-founder
> Otto Ops LLC | *OttoOps.com <http://OttoOps.com>*
> 970.343.4561 office
> 720.381.2383 cell
>
> On Tue, Mar 24, 2015 at 6:04 PM, Niklas Nielsen 
> wrote:
>
>> Hi all,
>>
>> The vote for Mesos 0.22.0 (rc4) has passed with the
>> following votes.
>>
>> +1 (Binding)
>> --
>> Ben Mahler
>> Tim St Clair
>> Adam Bordelon
>> Brenden Matthews
>>
>> +1 (Non-binding)
>> --
>> Alex Rukletsov
>> Craig W
>> Ben Whitehead
>> Elizabeth Lingg
>> Dario Rexin
>> Jeff Schroeder
>> Michael Park
>> Alexander Rojas
>> Andrew Langhorn
>>
>> There were no 0 or -1 votes.
>>
>> Please find the release at:
>> https://dist.apache.org/repos/dist/release/mesos/0.22.0
>>
>> It is recommended to use a mirror to download the release:
>> http://www.apache.org/dyn/closer.cgi
>>
>> The CHANGELOG for the release is available at:
>>
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.22.0
>>
>> The mesos-0.22.0.jar has been released to:
>> https://repository.apache.org
>>
>> The website (http://mesos.apache.org) will be updated shortly to reflect
>> this release.
>>
>> Thanks,
>> Niklas
>>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Zookeeper integration for Mesos-DNS

[craig w]

Keep in mind DNS will give you the ipaddress of the host, so
"rabbitmq.marathon.mesos" will resolve to some IP address. Do get port
information you have to query mesos-dns for its SRV records.

On Mon, Mar 23, 2015 at 11:29 AM, Ken Sipe  wrote:

> roger that
>
> On Mar 23, 2015, at 9:22 AM, Aaron Carey  wrote:
>
> Thanks Ken,
>
> So basically we just need to add mesos-dns to our /etc/resolv.conf on
> every machine and hey presto auto-service discovery (using DNS)? (Here I
> mean service discovery to be: hey where is rabbitmq? DNS says:
> 172.20.121.292:8393 or whatever)
>
> Aaron
>
> --
> *From:* Ken Sipe [kens...@gmail.com]
> *Sent:* 23 March 2015 14:29
> *To:* user@mesos.apache.org
> *Subject:* Re: Zookeeper integration for Mesos-DNS
>
> Aaron,
>
> Mesos-DNS is a DNS name server + a monitor of mesos-masters.  It listens
> to the mesos-master.  If a service is launched by mesos then mesos-dns
> conjures a service name (app_id + framework_id +.mesos) and associates it
> to the IP and PORT of the service.  Since Mesos-DNS is a name service, it
> needs to be in your list of name services for service discovery.  From a
> service discovery stand point there is no need to be in the cluster and
> there is no need to have a dependency on Mesos.
>
> Mesos-DNS is not a proxy.  It doesn’t provide any special services to
> clients or services inside the cluster.   more detail below.
>
>
> On Mar 23, 2015, at 7:52 AM, Aaron Carey  wrote:
>
> As I understood it, it provides a service for containers within the
> cluster to automatically find each other as it handles their dns calls?
>
>
> The way this is stated this doesn’t seem true.Mesos-DNS is a DNS name
> server.From a service discovery stand point, It doesn’t do anything
> different than a standard DNS naming server.
>
>
> However clients outside the cluster will not use the mesos-dns service by
> default, so won't have knowledge of anything running inside the cluster?
>
>
> This is all dependent on how /etc/resolv.conf is setup.  If mesos-dns is
> in the list… then this is not true.
>
>
> Is there an easy way to set this up to (for example) add records to AWS
> Route 53 when services get started in the cluster, so other clients can see
> them?
>
>
> This is outside of Mesos-DNS
>
> Good Luck!!
>
>
> Thanks!
> Aaron
>
> --
> *From:* Ken Sipe [kens...@gmail.com]
> *Sent:* 23 March 2015 13:31
> *To:* user@mesos.apache.org
> *Subject:* Re: Zookeeper integration for Mesos-DNS
>
> Aaron,
>
> It depends on what you mean however, Mesos-DNS works outside the cluster
> IMO. It is a bridge for things in the cluster (services launched by
> mesos)... But at that point it is DNS.  Any client in or out of the cluster
> that can query DNS that leverage the service.
>
> Sent from my iPhone
>
> On Mar 23, 2015, at 4:25 AM, Aaron Carey  wrote:
>
> Hey,
>
> I don't suppose there is anything like Mesos-DNS but for services/users
> outside the mesos cluster? So having a service which updates a DNS provider
> with task port/ips running inside the cluster so that external users are
> able to find those services? Am I correct in thinking Mesos-DNS only works
> inside the cluster?
>
> Currently we're using consul for this, but I'd be interested if there was
> some sort of magical plug and play solution?
>
> Thanks,
> Aaron
>
> --
> *From:* Christos Kozyrakis [kozyr...@gmail.com]
> *Sent:* 21 March 2015 00:18
> *To:* user@mesos.apache.org
> *Subject:* Zookeeper integration for Mesos-DNS
>
> Hi everybody,
>
> we have updated Mesos-DNS to integrate directly with Zookeeper. Instead of
> providing Mesos-DNS with a list of masters, you point it to the Zookeeper
> instances. Meson-DNS will watch Zookeeper to detect the current leading
> master. So, while the list of Zookeeper instances is configured in a static
> manner, Mesos masters can be added or removed freely without restarting
> Mesos-DNS.
>
> The integration with Zookeeper forced to switch from -v and -vv as the
> flags to control verbosity to -v=0 (default), -v=1 (verbose), and -v=2
> (very verbose).
>
> To reduce complications because of dependencies to other packages, we have
> also started using godep.
>
> Please take a look at the branch
> https://github.com/mesosphere/mesos-dns/tree/zk
> and provide us with any feedback on the code or the documentation.
>
> Thanks
>
> --
> Christos
>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Unable to install subversion-devel 1.8+ on CentOS 7

[craig w]

https://issues.apache.org/jira/browse/MESOS-2115

On Mon, Mar 23, 2015 at 8:30 AM, Aaron Carey  wrote:

>  ah interesting.. what causes this difference?
>
> I think this probably makes sense for our setup currently..
>
>  --
> *From:* craig w [codecr...@gmail.com]
> *Sent:* 23 March 2015 12:20
> *To:* user@mesos.apache.org
> *Subject:* Re: Unable to install subversion-devel 1.8+ on CentOS 7
>
>   I had considered running Mesos in docker containers, however with Mesos
> 0.21 if the slaves are running in a container and you have tasks running in
> containers, if the slave container were to exit/die, the tasks running in
> containers would also exit. If running mesos-slave on the host, any tasks
> that it had running will remain running even if the process dies. That's
> why I hadn't gone that route. Have you considered that?
>
> On Mon, Mar 23, 2015 at 8:15 AM, craig w  wrote:
>
>> What OS is your docker image based on?
>>
>> On Mon, Mar 23, 2015 at 8:11 AM, Aaron Carey  wrote:
>>
>>>  Not sure if this helps, but we've been using docker to run Mesos on
>>> Centos 7 hosts.
>>>
>>>  --
>>> *From:* craig w [codecr...@gmail.com]
>>> *Sent:* 23 March 2015 12:06
>>> *To:* user@mesos.apache.org
>>> *Subject:* Unable to install subversion-devel 1.8+ on CentOS 7
>>>
>>> Mesos 0.21.0+ requires subversion-devel 1.8+, which can be
>>> installed by adding the Wandisco yum repo. However, it appears that
>>> subversion-devel 1.8+ requires libsasl2.so.2, which is not available on
>>> CentOS7.
>>>
>>>  I've seen one person try to create a symlink to libsasl2.so.3 and it
>>> worked [1], while another person found it did not work [2].
>>>
>>>  I created a CentOS 7 droplet on DigitalOcean, added the Wandisco repo
>>> and tried to install subversion-devel and it failed b/c of the
>>> libsasl2.so.2 missing. I tried creating a symlink (ln -s
>>> /usr/lib64/libsasl2.so.3 /usr/lib64/libsasl2.so.2), restarting the server
>>> and installing subversion-devel still failed b/c of the libsasl issue:
>>>
>>>  Error: Package: subversion-1.8.11-1.x86_64 (WandiscoSVN)
>>>>Requires: libsasl2.so.2()(64bit)
>>>>
>>>
>>>  Anyone had any success on CentOS 7 with Mesos 0.21+?
>>>
>>>  [1] - did work: http://unix.stackexchange.com/a/178408
>>>
>>>  [2] - did not work:
>>> http://www.wandisco.com/svnforum/forum/smartsvn-community/smartsvn-help-and-support/69834-installing-subversion-command-line-client-on-centos-7
>>>
>>
>>
>>
>>  --
>>
>> https://github.com/mindscratch
>> https://www.google.com/+CraigWickesser
>> https://twitter.com/mind_scratch
>> https://twitter.com/craig_links
>>
>>
>
>
> --
>
> https://github.com/mindscratch
> https://www.google.com/+CraigWickesser
> https://twitter.com/mind_scratch
> https://twitter.com/craig_links
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Unable to install subversion-devel 1.8+ on CentOS 7

[craig w]

I had considered running Mesos in docker containers, however with Mesos
0.21 if the slaves are running in a container and you have tasks running in
containers, if the slave container were to exit/die, the tasks running in
containers would also exit. If running mesos-slave on the host, any tasks
that it had running will remain running even if the process dies. That's
why I hadn't gone that route. Have you considered that?

On Mon, Mar 23, 2015 at 8:15 AM, craig w  wrote:

> What OS is your docker image based on?
>
> On Mon, Mar 23, 2015 at 8:11 AM, Aaron Carey  wrote:
>
>>  Not sure if this helps, but we've been using docker to run Mesos on
>> Centos 7 hosts.
>>
>>  --
>> *From:* craig w [codecr...@gmail.com]
>> *Sent:* 23 March 2015 12:06
>> *To:* user@mesos.apache.org
>> *Subject:* Unable to install subversion-devel 1.8+ on CentOS 7
>>
>>Mesos 0.21.0+ requires subversion-devel 1.8+, which can be installed
>> by adding the Wandisco yum repo. However, it appears that subversion-devel
>> 1.8+ requires libsasl2.so.2, which is not available on CentOS7.
>>
>>  I've seen one person try to create a symlink to libsasl2.so.3 and it
>> worked [1], while another person found it did not work [2].
>>
>>  I created a CentOS 7 droplet on DigitalOcean, added the Wandisco repo
>> and tried to install subversion-devel and it failed b/c of the
>> libsasl2.so.2 missing. I tried creating a symlink (ln -s
>> /usr/lib64/libsasl2.so.3 /usr/lib64/libsasl2.so.2), restarting the server
>> and installing subversion-devel still failed b/c of the libsasl issue:
>>
>>  Error: Package: subversion-1.8.11-1.x86_64 (WandiscoSVN)
>>>Requires: libsasl2.so.2()(64bit)
>>>
>>
>>  Anyone had any success on CentOS 7 with Mesos 0.21+?
>>
>>  [1] - did work: http://unix.stackexchange.com/a/178408
>>
>>  [2] - did not work:
>> http://www.wandisco.com/svnforum/forum/smartsvn-community/smartsvn-help-and-support/69834-installing-subversion-command-line-client-on-centos-7
>>
>
>
>
> --
>
> https://github.com/mindscratch
> https://www.google.com/+CraigWickesser
> https://twitter.com/mind_scratch
> https://twitter.com/craig_links
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Unable to install subversion-devel 1.8+ on CentOS 7

[craig w]

What OS is your docker image based on?

On Mon, Mar 23, 2015 at 8:11 AM, Aaron Carey  wrote:

>  Not sure if this helps, but we've been using docker to run Mesos on
> Centos 7 hosts.
>
>  --
> *From:* craig w [codecr...@gmail.com]
> *Sent:* 23 March 2015 12:06
> *To:* user@mesos.apache.org
> *Subject:* Unable to install subversion-devel 1.8+ on CentOS 7
>
>Mesos 0.21.0+ requires subversion-devel 1.8+, which can be installed
> by adding the Wandisco yum repo. However, it appears that subversion-devel
> 1.8+ requires libsasl2.so.2, which is not available on CentOS7.
>
>  I've seen one person try to create a symlink to libsasl2.so.3 and it
> worked [1], while another person found it did not work [2].
>
>  I created a CentOS 7 droplet on DigitalOcean, added the Wandisco repo
> and tried to install subversion-devel and it failed b/c of the
> libsasl2.so.2 missing. I tried creating a symlink (ln -s
> /usr/lib64/libsasl2.so.3 /usr/lib64/libsasl2.so.2), restarting the server
> and installing subversion-devel still failed b/c of the libsasl issue:
>
>  Error: Package: subversion-1.8.11-1.x86_64 (WandiscoSVN)
>>Requires: libsasl2.so.2()(64bit)
>>
>
>  Anyone had any success on CentOS 7 with Mesos 0.21+?
>
>  [1] - did work: http://unix.stackexchange.com/a/178408
>
>  [2] - did not work:
> http://www.wandisco.com/svnforum/forum/smartsvn-community/smartsvn-help-and-support/69834-installing-subversion-command-line-client-on-centos-7
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Unable to install subversion-devel 1.8+ on CentOS 7

[craig w]

Mesos 0.21.0+ requires subversion-devel 1.8+, which can be installed by
adding the Wandisco yum repo. However, it appears that subversion-devel
1.8+ requires libsasl2.so.2, which is not available on CentOS7.

I've seen one person try to create a symlink to libsasl2.so.3 and it worked
[1], while another person found it did not work [2].

I created a CentOS 7 droplet on DigitalOcean, added the Wandisco repo and
tried to install subversion-devel and it failed b/c of the libsasl2.so.2
missing. I tried creating a symlink (ln -s /usr/lib64/libsasl2.so.3
/usr/lib64/libsasl2.so.2), restarting the server and installing
subversion-devel still failed b/c of the libsasl issue:

Error: Package: subversion-1.8.11-1.x86_64 (WandiscoSVN)
>Requires: libsasl2.so.2()(64bit)
>

Anyone had any success on CentOS 7 with Mesos 0.21+?

[1] - did work: http://unix.stackexchange.com/a/178408

[2] - did not work:
http://www.wandisco.com/svnforum/forum/smartsvn-community/smartsvn-help-and-support/69834-installing-subversion-command-line-client-on-centos-7

Re: [VOTE] Release Apache Mesos 0.22.0 (rc4)

[craig w]

+1

On Wed, Mar 18, 2015 at 3:52 PM, Niklas Nielsen 
wrote:

> Hi all,
>
> Please vote on releasing the following candidate as Apache Mesos 0.22.0.
>
>
> 0.22.0 includes the following:
>
> 
>
> * Support for explicitly sending status updates acknowledgements from
>   schedulers; refer to the upgrades document for upgrading schedulers.
> * Rate limiting slave removal, to safeguard against unforeseen bugs
> leading to
>   widespread slave removal.
> * Disk quota isolation in Mesos containerizer; refer to the
> containerization
>   documentation to enable disk quota monitoring and enforcement.
> * Support for module hooks in task launch sequence. Refer to the modules
>   documentation for more information.
> * Anonymous modules: a new kind of module that does not receive any
> callbacks
>   but coexists with its parent process.
> * New service discovery info in task info allows framework users to specify
>   discoverability of tasks for external service discovery systems. Refer to
>   the framework development guide for more information.
> * New '--external_log_file' flag to serve external logs through the Mesos
> web UI.
> * New '--gc_disk_headroom' flag to control maxmimum executor sandbox age.
>
>
> The CHANGELOG for the release is available at:
>
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.22.0-rc4
>
> 
>
> The candidate for Mesos 0.22.0 release is available at:
> https://dist.apache.org/repos/dist/dev/mesos/0.22.0-rc4/mesos-0.22.0.tar.gz
>
> The tag to be voted on is 0.22.0-rc4:
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.22.0-rc4
>
> The MD5 checksum of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.22.0-rc4/mesos-0.22.0.tar.gz.md5
>
> The signature of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.22.0-rc4/mesos-0.22.0.tar.gz.asc
>
> The PGP key used to sign the release is here:
> https://dist.apache.org/repos/dist/release/mesos/KEYS
>
> The JAR is up in Maven in a staging repository here:
> https://repository.apache.org/content/repositories/orgapachemesos-1048
>
> Please vote on releasing this package as Apache Mesos 0.22.0!
>
> The vote is open until Sat Mar 21 12:49:56 PDT 2015 and passes if a
> majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Mesos 0.22.0
> [ ] -1 Do not release this package because ...
>
> Thanks,
> Niklas
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

marathon 0.8.1 packages

[craig w]

Will the 0.8.1 packages (rpm / deb) be posted to github?

Re: Deploying containers to every mesos slave node

[craig w]

Perhaps you could query the Mesos API to see how many slaves there are,
then use that in the request to Marathon.

On Thu, Mar 12, 2015 at 5:09 AM, Michael Neale 
wrote:

> It would be idea if there was as a way to specify to marathon the number
> of instances to be a variable representing the mesos slaves (I don't think
> possible right now).
> On Thu, 12 Mar 2015 at 8:07 pm craig w  wrote:
>
>> If you know when the scaling occurs (perhaps there's an API you can query
>> or maybe it can notify you), then you can update the configuration for the
>> application (deployed using marathon) to change the number of instances
>> (via the Marathon REST API).
>>
>> On Thu, Mar 12, 2015 at 5:03 AM, Aaron Carey  wrote:
>>
>>>  Hi Craig,
>>>
>>> I'd looked into that, but I was thinking this may cause issues when our
>>> cluster auto scales up or down, as instances would no longer equal slaves?
>>>
>>> Thanks,
>>> Aaron
>>>
>>>  --
>>> *From:* craig w [codecr...@gmail.com]
>>> *Sent:* 12 March 2015 08:57
>>> *To:* user@mesos.apache.org
>>> *Subject:* Re: Deploying containers to every mesos slave node
>>>
>>>Aaron,
>>>
>>>  You could use Marathon (a Mesos framework) to deploy a container to
>>> each host by using constraints [1] and setting the number of instances of
>>> the container to equal the number of slaves.
>>>
>>>  [1] constraints -
>>> https://mesosphere.github.io/marathon/docs/constraints.html
>>>
>>> On Thu, Mar 12, 2015 at 4:54 AM, Aaron Carey  wrote:
>>>
>>>>  Hi All,
>>>>
>>>> In setting up our cluster, we require things like consul to be running
>>>> on all of our nodes. I was just wondering if there was any sort of best
>>>> practice (or a scheduler perhaps) that people could share for this sort of
>>>> thing?
>>>>
>>>> Currently the approach is to use salt to provision each node and add
>>>> consul/mesos slave process and so on to it, but it'd be nice to remove the
>>>> dependency on salt.
>>>>
>>>> Thanks,
>>>> Aaron
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> https://github.com/mindscratch
>>> https://www.google.com/+CraigWickesser
>>> https://twitter.com/mind_scratch
>>> https://twitter.com/craig_links
>>>
>>>
>>
>>
>> --
>>
>> https://github.com/mindscratch
>> https://www.google.com/+CraigWickesser
>> https://twitter.com/mind_scratch
>> https://twitter.com/craig_links
>>
>>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Deploying containers to every mesos slave node

[craig w]

If you know when the scaling occurs (perhaps there's an API you can query
or maybe it can notify you), then you can update the configuration for the
application (deployed using marathon) to change the number of instances
(via the Marathon REST API).

On Thu, Mar 12, 2015 at 5:03 AM, Aaron Carey  wrote:

>  Hi Craig,
>
> I'd looked into that, but I was thinking this may cause issues when our
> cluster auto scales up or down, as instances would no longer equal slaves?
>
> Thanks,
> Aaron
>
>  ------
> *From:* craig w [codecr...@gmail.com]
> *Sent:* 12 March 2015 08:57
> *To:* user@mesos.apache.org
> *Subject:* Re: Deploying containers to every mesos slave node
>
>Aaron,
>
>  You could use Marathon (a Mesos framework) to deploy a container to each
> host by using constraints [1] and setting the number of instances of the
> container to equal the number of slaves.
>
>  [1] constraints -
> https://mesosphere.github.io/marathon/docs/constraints.html
>
> On Thu, Mar 12, 2015 at 4:54 AM, Aaron Carey  wrote:
>
>>  Hi All,
>>
>> In setting up our cluster, we require things like consul to be running on
>> all of our nodes. I was just wondering if there was any sort of best
>> practice (or a scheduler perhaps) that people could share for this sort of
>> thing?
>>
>> Currently the approach is to use salt to provision each node and add
>> consul/mesos slave process and so on to it, but it'd be nice to remove the
>> dependency on salt.
>>
>> Thanks,
>> Aaron
>>
>
>
>
> --
>
> https://github.com/mindscratch
> https://www.google.com/+CraigWickesser
> https://twitter.com/mind_scratch
> https://twitter.com/craig_links
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Deploying containers to every mesos slave node

[craig w]

Aaron,

You could use Marathon (a Mesos framework) to deploy a container to each
host by using constraints [1] and setting the number of instances of the
container to equal the number of slaves.

[1] constraints -
https://mesosphere.github.io/marathon/docs/constraints.html

On Thu, Mar 12, 2015 at 4:54 AM, Aaron Carey  wrote:

>  Hi All,
>
> In setting up our cluster, we require things like consul to be running on
> all of our nodes. I was just wondering if there was any sort of best
> practice (or a scheduler perhaps) that people could share for this sort of
> thing?
>
> Currently the approach is to use salt to provision each node and add
> consul/mesos slave process and so on to it, but it'd be nice to remove the
> dependency on salt.
>
> Thanks,
> Aaron
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: mesos on coreos

[craig w]

Is there any documentation describing what's necessary to run mesos master
and slaves in Docker containers? You already mentioned a few things
(mounting work dir, /sys/fs, etc).

Thanks

On Tue, Mar 10, 2015 at 6:41 PM, Tim Chen  wrote:

> Hi all,
>
> As Alex said you can run Mesos in CoreOS without Docker if you put in the
> dependencies in.
>
> It is a common ask though to run Mesos-slave in a Docker container in
> general, either on CoreOS or not. It's definitely a bit involved as you
> need to mount in a directory for persisting work dir and also mounting in
> /sys/fs for cgroups, also you should use the --pid=host flag since Docker
> 1.5 so it shares the host pid namespace.
>
> Although you get a lot less isolation, there are still motivations to run
> slave in Docker regardless.
>
> One thing that's missing from the mesos docker containerizer is that it
> won't be able to recover tasks on restart, and I have a series of patches
> pending review to fix that.
>
> Tim
>
> On Tue, Mar 10, 2015 at 3:16 PM, Alex Rukletsov 
> wrote:
>
>> My 2¢.
>>
>>
>>> First of all, it doesn’t look like a great idea to package resource
>>> manager into Docker putting one more abstraction layer between a resource
>>> itself and resource manager.
>>>
>>
>> You can run mesos-slave on CoreOS node without putting it into a Docker
>> container.
>>
>> —Alex
>>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: multiple frameworks or one big one

[craig w]

Diego,

I'm also interested in hearing feedback to your qusestion. One minor thing
I'd point out is that a Framework is made up of a Scheduler and
Executor(s), so I think it's more correct to say you've created a Scheduler
(instead of "one big framework") and an Executor.

Anyhow, for what it's worth, the Aurora framework has multiple executors (
https://github.com/apache/incubator-aurora/blob/master/examples/vagrant/aurorabuild.sh#L61).
You might pop into the #aurora IRC chat room and ask, usually a few Aurora
contributors are in there answering questions when they can.

On Wed, Feb 25, 2015 at 9:02 PM, John Pampuch  wrote:

> Diego-
>
> You might want to look at this project for some insights:
>
> https://github.com/mesosphere/RENDLER
>
>
> -John
>
>
> On Wed, Feb 25, 2015 at 5:27 PM, Diego Medina  wrote:
>
>> Hi,
>>
>>
>> Short: Is it better to have one big framework and executor with if
>> statements to select what to do or several smaller framework <-> executors
>> when writing a Mesos app?
>>
>> Longer question:
>>
>> Last week I started a side project based on mesos (using Go),
>>
>> http://blog.fmpwizard.com/blog/web-crawler-using-mesos-and-golang
>> https://github.com/fmpwizard/owlcrawler
>>
>> It's a web crawler written on top of Mesos, The very first version of it
>> had a framework that sent a task to an executor and that single executor
>> would fetch the page, extract links from the html and then send them to a
>> message queue.
>>
>> Then the framework reads that queue and starts again, run the executor,
>> etc, etc.
>>
>> Now I'm splitting fetching the html and extracting links into two
>> separate tasks, and putting those two tasks in the same executor doesn't
>> feel right, so I'm thinking that I need at least two diff executors and one
>> framework, but then I wonder if people more experienced with mesos would
>> normally write several pairs of framework <-> executors to keep the design
>> cleaner.
>>
>> On this particular case, I can see the project growing into even more
>> tasks that can be decoupled.
>>
>> Any feedback on the design would be great and let me know if I should
>> explain this better.
>>
>> Thanks
>>
>>
>> Diego
>>
>>
>>
>>
>>
>> --
>> Diego Medina
>> Lift/Scala consultant
>> di...@fmpwizard.com
>> http://fmpwizard.telegr.am
>>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Re: Total resources available versus actually available

[craig w]

Is this the issue? https://issues.apache.org/jira/browse/MESOS-354

On Mon, Feb 9, 2015 at 12:30 PM, Niklas Nielsen 
wrote:

> The slave picks up total available resources (not free) on startup, but
> this can be overridden by the --resources="" flag.
> That way, you can leave resources for your out of bound processes.
>
> If you want to compute the slack (difference between allocated and
> actually used), you can compute that from the slave's
> /monitor/statistics.json endpoint.
> You should be able to pick up some documentation on that endpoint from
> /help/monitor/statistics.json
>
> If you are looking for something more dynamic, you should follow the
> discussion on oversubscription :)
>
> Cheers,
> Niklas
>
> On 9 February 2015 at 02:34, craig w  wrote:
>
>> Mesos slaves report the total resources they have available, such as 2
>> CPU, 8GB Memory and 100GB disk.
>>
>> Does Mesos keep track of the amount of resources that are actually
>> available to best schedule tasks?
>>
>> For example, imagine a slave has some other processes running on it (that
>> are not mesos tasks) that are taking up 6GB of RAM (out of the total 8GB).
>> A new task is created in Mesos that needs 4GB of RAM, would Mesos still try
>> to put the new task on the slave?
>>
>
>


-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

Total resources available versus actually available

[craig w]

Mesos slaves report the total resources they have available, such as 2 CPU,
8GB Memory and 100GB disk.

Does Mesos keep track of the amount of resources that are actually
available to best schedule tasks?

For example, imagine a slave has some other processes running on it (that
are not mesos tasks) that are taking up 6GB of RAM (out of the total 8GB).
A new task is created in Mesos that needs 4GB of RAM, would Mesos still try
to put the new task on the slave?