Re: [VOTE] Release Apache Mesos 0.26.0 (rc3)

[Bernd Mathiske]

We found an issue with the v1 API, which is not completely on par with the 
legacy mesos.proto. Spinning RC4 shortly…

> On Dec 7, 2015, at 10:48 AM, Bernd Mathiske  wrote:
> 
> +1 (binding)
> 
> Ubuntu 14 (clean without SSL, a few known flaky tests with SSL, all analyzed 
> and deemed non-blockers)
> CentOS 6.6 (a few known flaky tests with SSL, all analyzed and deemed 
> non-blockers)
> 
>> On Dec 4, 2015, at 4:52 AM, Benjamin Mahler  
>> wrote:
>> 
>> +1 (binding) tests pass on OS X 10.11.1 with both SSL and non-SSL
>> configurations.
>> 
>> Some feedback from framework developers would be great here.
>> 
>> Agreed that MESOS-3973  is
>> not a blocker, given it also occurs on 0.26.0 all the way back to 0.21.0.
>> 
>> 
>> On Wed, Dec 2, 2015 at 9:01 AM, Bernd Mathiske  wrote:
>> 
>>> We are still working on that, but we do not regard "make distcheck" on Mac
>>> as blocker. Other opinions?
>>> 
>>> On Dec 2, 2015, at 2:27 PM, Alex Rukletsov  wrote:
>>> 
>>> `make check -j7` — OK
>>> `make distcheck -j7` — fails, probably MESOS-3973
>>> , see hints below.
>>> 
>>> Both on Mac OS 10.10.4
>>> 
>>> I see the following lines in the log:
>>> ...
>>> libtool: warning: 'libmesos.la' has not been installed in
>>> '/Users/alex/Projects/mesos/build/default/mesos-0.26.0/_inst/lib'
>>> libtool: warning: 'libmesos.la' has not been installed in
>>> '/Users/alex/Projects/mesos/build/default/mesos-0.26.0/_inst/lib'
>>> ...
>>> libtool: warning: 'libmesos.la' has not been installed in
>>> '/Users/alex/Projects/mesos/build/default/mesos-0.26.0/_inst/lib'
>>> libtool: warning: 'libmesos.la' has not been installed in
>>> '/Users/alex/Projects/mesos/build/default/mesos-0.26.0/_inst/lib'
>>> ...
>>> Cannot uninstall requirement mesos, not installed
>>> Cannot uninstall requirement mesos.cli, not installed
>>> Cannot uninstall requirement mesos.interface, not installed
>>> Cannot uninstall requirement mesos.native, not installed
>>> ERROR: files left after uninstall:
>>> ...
>>> 
>>> On Tue, Dec 1, 2015 at 8:49 PM, Till Toenshoff  wrote:
>>> 
 Hi friends,
 
 Please vote on releasing the following candidate as Apache Mesos 0.26.0.
 
 The CHANGELOG for the release is available at:
 
 https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.26.0-rc3
 
 
 
 The candidate for Mesos 0.26.0 release is available at:
 
 https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc3/mesos-0.26.0.tar.gz
 
 The tag to be voted on is 0.26.0-rc3:
 https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.26.0-rc3
 
 The MD5 checksum of the tarball can be found at:
 
 https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc3/mesos-0.26.0.tar.gz.md5
 
 The signature of the tarball can be found at:
 
 https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc3/mesos-0.26.0.tar.gz.asc
 
 The PGP key used to sign the release is here:
 https://dist.apache.org/repos/dist/release/mesos/KEYS
 
 The JAR is up in Maven in a staging repository here:
 https://repository.apache.org/content/repositories/orgapachemesos-1091
 
 Please vote on releasing this package as Apache Mesos 0.26.0!
 
 The vote is open until Fri Dec  4 19:00:35 CET 2015 and passes if a
 majority of at least 3 +1 PMC votes are cast.
 
 [ ] +1 Release this package as Apache Mesos 0.26.0
 [ ] -1 Do not release this package because …
 
 Thanks,
 Bernd & Till
 
 
>>> 
>>> 
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: [VOTE] Release Apache Mesos 0.26.0 (rc3)

[Bernd Mathiske]

+1 (binding)

Ubuntu 14 (clean without SSL, a few known flaky tests with SSL, all analyzed 
and deemed non-blockers)
CentOS 6.6 (a few known flaky tests with SSL, all analyzed and deemed 
non-blockers)

> On Dec 4, 2015, at 4:52 AM, Benjamin Mahler  wrote:
> 
> +1 (binding) tests pass on OS X 10.11.1 with both SSL and non-SSL
> configurations.
> 
> Some feedback from framework developers would be great here.
> 
> Agreed that MESOS-3973  is
> not a blocker, given it also occurs on 0.26.0 all the way back to 0.21.0.
> 
> 
> On Wed, Dec 2, 2015 at 9:01 AM, Bernd Mathiske  wrote:
> 
>> We are still working on that, but we do not regard "make distcheck" on Mac
>> as blocker. Other opinions?
>> 
>> On Dec 2, 2015, at 2:27 PM, Alex Rukletsov  wrote:
>> 
>> `make check -j7` — OK
>> `make distcheck -j7` — fails, probably MESOS-3973
>> , see hints below.
>> 
>> Both on Mac OS 10.10.4
>> 
>> I see the following lines in the log:
>> ...
>> libtool: warning: 'libmesos.la' has not been installed in
>> '/Users/alex/Projects/mesos/build/default/mesos-0.26.0/_inst/lib'
>> libtool: warning: 'libmesos.la' has not been installed in
>> '/Users/alex/Projects/mesos/build/default/mesos-0.26.0/_inst/lib'
>> ...
>> libtool: warning: 'libmesos.la' has not been installed in
>> '/Users/alex/Projects/mesos/build/default/mesos-0.26.0/_inst/lib'
>> libtool: warning: 'libmesos.la' has not been installed in
>> '/Users/alex/Projects/mesos/build/default/mesos-0.26.0/_inst/lib'
>> ...
>> Cannot uninstall requirement mesos, not installed
>> Cannot uninstall requirement mesos.cli, not installed
>> Cannot uninstall requirement mesos.interface, not installed
>> Cannot uninstall requirement mesos.native, not installed
>> ERROR: files left after uninstall:
>> ...
>> 
>> On Tue, Dec 1, 2015 at 8:49 PM, Till Toenshoff  wrote:
>> 
>>> Hi friends,
>>> 
>>> Please vote on releasing the following candidate as Apache Mesos 0.26.0.
>>> 
>>> The CHANGELOG for the release is available at:
>>> 
>>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.26.0-rc3
>>> 
>>> 
>>> 
>>> The candidate for Mesos 0.26.0 release is available at:
>>> 
>>> https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc3/mesos-0.26.0.tar.gz
>>> 
>>> The tag to be voted on is 0.26.0-rc3:
>>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.26.0-rc3
>>> 
>>> The MD5 checksum of the tarball can be found at:
>>> 
>>> https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc3/mesos-0.26.0.tar.gz.md5
>>> 
>>> The signature of the tarball can be found at:
>>> 
>>> https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc3/mesos-0.26.0.tar.gz.asc
>>> 
>>> The PGP key used to sign the release is here:
>>> https://dist.apache.org/repos/dist/release/mesos/KEYS
>>> 
>>> The JAR is up in Maven in a staging repository here:
>>> https://repository.apache.org/content/repositories/orgapachemesos-1091
>>> 
>>> Please vote on releasing this package as Apache Mesos 0.26.0!
>>> 
>>> The vote is open until Fri Dec  4 19:00:35 CET 2015 and passes if a
>>> majority of at least 3 +1 PMC votes are cast.
>>> 
>>> [ ] +1 Release this package as Apache Mesos 0.26.0
>>> [ ] -1 Do not release this package because …
>>> 
>>> Thanks,
>>> Bernd & Till
>>> 
>>> 
>> 
>> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Mesos at Moz

[Jeff Schroeder]

>From a quick skim, this looks excellent. Are there any plans to try getting
the bamboo changes back upstream, or is it going to be a permanent fork?

On Monday, December 7, 2015, Arunabha Ghosh  wrote:

> Hi Folks,
>   We, at Moz have been working for a while on RogerOS, our
> next gen application platform built on top of Mesos. We've reached a point
> in the project where we feel it's ready to share with the world :-)
>
> The blog posts introducing RogerOS can be found at
>
> https://moz.com/devblog/introducing-rogeros-part-1/
> https://moz.com/devblog/introducing-rogeros-part-2/
>
> I can safely say that without Mesos, it would not have been possible for
> us to have built the system within the constraints of time and resources
> that we had. As we note in the blog
>
> " We are very glad that we chose Mesos though. It has delivered on all of
> its promises and more. We’ve had no issues with stability, extensibility,
> and performance of the system and it has allowed us to achieve our goals
> with a fraction of the development resources that would have been required
> otherwise. "
>
> We would also like to thank the wonderful Mesos community for all the help
> and support we've received. Along the way we've tried to contribute back to
> the community through talks at Mesoscon and now through open sourcing our
> efforts.
>
> Your feedback and thoughts are always welcome !
>
> Thanks,
> Arunabha
>
>
>


-- 
Text by Jeff, typos by iPhone

Re: Verifying Zero Downtime Upgrade Process For Existing Mesos Cluster

[Alex Rukletsov]

Hi Abishek,

I would strongly advise not to skip 6 versions. It's hard to say whether
there were any changes that will prevent 0.25 masters to talk to 0.19
slaves (my intuition says there were some breaking changes to protobufs).
We do *not* support upgrade by skipping version, so please upgrade to 0.20,
wait for stabilization, and repeat the procedure 5 more times.

In the future we may move to another deprecation cycle, but currently we
have a 2-version one.

Mind reporting your experience to the list once you're done? Thanks!

On Thu, Dec 3, 2015 at 10:28 PM, Abishek Ravi  wrote:

> Would the following process enable zero downtime upgrade of Mesos (0.19 to
> 0.25) in an existing Mesos cluster?
>
> 0. From [1] it doesn't seem like there are any incompatible changes
> introduced between 0.19 and 0.25.
> 1. Deploy Mesos(0.25) binaries to unelected master nodes
> 2. Deploy Mesos(0.25) binaries to leading master. This should potentially
> result in master re-election and elect a master which already has
> Mesos(0.25) installed from Step (1).
> 3. Deploy Mesos(0.25) binaries to mesos slave nodes. Existing tasks should
> continue to execute and report to the master after mesos process launch
> (with 0.25 binaries) on the slave node.
>
> Known gotchas:
> 1. Any monitoring built around state.json and stats.json should be updated
> accordingly as endpoints have changed [1].
> 2. Checkpointing should be enabled (It is not automatically enabled in
> 0.19) [2] .
> 3. recovery_timeout for slave nodes should be set to an appropriate value
> depending on how long it takes to install Mesos(0.25) on the slave nodes.
>
> Is any step missing in the upgrade process? Are there other gotchas that
> one needs to be aware of?
>
> [1] http://mesos.apache.org/documentation/latest/upgrades/
> [2] http://mesos.apache.org/documentation/latest/slave-recovery/
>
> Thanks,
> Abishek
>

Re: Task still 'active' after TASK_FINISHED status

[Vinod Kone]

The fact that the slave is retrying means that the TASK_FAILED status
hasn't reached the master or the scheduler or the acknowledgement hasn't
reached the slave.  Since the master hasn't released the resources for the
task (from what you say), I imagine it's the former.

What do master logs say?

On Wed, Nov 25, 2015 at 2:01 AM, James Vanns  wrote:

> Er, I could. At the moment it's pretty huge so maybe I'll just try and
> trim it down a bit. I've noticed that Chronos does the same, actually.
> There is a task that is 'active' and still holding onto resources yet it
> has already completed unsuccessfully with TASK_FAILED (16hrs ago!) state.
> Attached is a log of the events from the mesos slave that executed this
> particular Chronos task (before it continues to forward the same state over
> and over). Note that the last pair of lines is repeated ad-infinitum. I can
> confirm that this Chronos framework with the same ID is still running.
>
> Sorry to switch frameworks suddenly - this was simpler because it was one
> task instead of 100s.
>
> Jim
>
> On 24 November 2015 at 17:57, Vinod Kone  wrote:
>
>> Can you paste the logs?
>>
>> On Tue, Nov 24, 2015 at 2:16 AM, James Vanns 
>> wrote:
>>
>>> Hi again list.
>>>
>>> Mesos 0.24
>>> C++ Framework (still using the Protobufs based comms, not REST)
>>>
>>> My framework appears to be holding onto offers (somehow) from tasks that
>>> are finished!? I don't understand why. The task comprises of a shell
>>> command that executes within a docker container.
>>> The return code to the OS from the shell command is indeed zero for
>>> success, which Mesos honours and transitions to TASK_FINISHED state.
>>> However, using the UI these still register as 'active' (though acknowledged
>>> as FINISHED) and thus the resources are not yet freed.
>>>
>>> Any pointers appreciated!
>>>
>>> Cheers,
>>>
>>> Jim
>>>
>>> --
>>> Senior Code Pig
>>> Industrial Light & Magic
>>>
>>
>>
>
>
> --
> --
> Senior Code Pig
> Industrial Light & Magic
>

Mesos at Moz

[Arunabha Ghosh]

Hi Folks,
  We, at Moz have been working for a while on RogerOS, our next
gen application platform built on top of Mesos. We've reached a point in
the project where we feel it's ready to share with the world :-)

The blog posts introducing RogerOS can be found at

https://moz.com/devblog/introducing-rogeros-part-1/
https://moz.com/devblog/introducing-rogeros-part-2/

I can safely say that without Mesos, it would not have been possible for us
to have built the system within the constraints of time and resources that
we had. As we note in the blog

" We are very glad that we chose Mesos though. It has delivered on all of
its promises and more. We’ve had no issues with stability, extensibility,
and performance of the system and it has allowed us to achieve our goals
with a fraction of the development resources that would have been required
otherwise. "

We would also like to thank the wonderful Mesos community for all the help
and support we've received. Along the way we've tried to contribute back to
the community through talks at Mesoscon and now through open sourcing our
efforts.

Your feedback and thoughts are always welcome !

Thanks,
Arunabha

Re: Mesos at Moz

[Arunabha Ghosh]

We're definitely open to merging the changes to Bamboo back upstream if the
changes we made prove useful to Bamboo. The changes we made were pretty
specific to our needs and we anticipate making quite a few changes as we
evolve the platform, so a separate repo seemed to be a better idea till the
codebase stabilized somewhat and we could offer a concrete view of the
changes we wanted.

Arunabha

On Mon, Dec 7, 2015 at 7:03 PM, Jeff Schroeder 
wrote:

> From a quick skim, this looks excellent. Are there any plans to try
> getting the bamboo changes back upstream, or is it going to be a permanent
> fork?
>
>
> On Monday, December 7, 2015, Arunabha Ghosh  wrote:
>
>> Hi Folks,
>>   We, at Moz have been working for a while on RogerOS, our
>> next gen application platform built on top of Mesos. We've reached a point
>> in the project where we feel it's ready to share with the world :-)
>>
>> The blog posts introducing RogerOS can be found at
>>
>> https://moz.com/devblog/introducing-rogeros-part-1/
>> https://moz.com/devblog/introducing-rogeros-part-2/
>>
>> I can safely say that without Mesos, it would not have been possible for
>> us to have built the system within the constraints of time and resources
>> that we had. As we note in the blog
>>
>> " We are very glad that we chose Mesos though. It has delivered on all
>> of its promises and more. We’ve had no issues with stability,
>> extensibility, and performance of the system and it has allowed us to
>> achieve our goals with a fraction of the development resources that would
>> have been required otherwise. "
>>
>> We would also like to thank the wonderful Mesos community for all the
>> help and support we've received. Along the way we've tried to contribute
>> back to the community through talks at Mesoscon and now through open
>> sourcing our efforts.
>>
>> Your feedback and thoughts are always welcome !
>>
>> Thanks,
>> Arunabha
>>
>>
>>
>
>
> --
> Text by Jeff, typos by iPhone
>

Re: Mesos at Moz

[Arunabha Ghosh]

Thanks, Vinod.

I'm not worried too much about scaling the core, you guys have done the
hard work on that end :-). The other parts of the system can definitely run
into issues when running at larger scales. We've some ideas about what
these are and what the mitigation plan is. Others, of course we'll discover
in time. Still not worried too much though, the community has been
fantastic and we hope to leverage it's collective wisdom in helping us
improve the system.

Arunabha

On Mon, Dec 7, 2015 at 7:37 PM, Vinod Kone  wrote:

> Really cool to see a multi-framework compute platform. Let us know how
> things work for you guys as you scale!
>
> On Mon, Dec 7, 2015 at 7:24 PM, Arunabha Ghosh 
> wrote:
>
>> We're definitely open to merging the changes to Bamboo back upstream if
>> the changes we made prove useful to Bamboo. The changes we made were pretty
>> specific to our needs and we anticipate making quite a few changes as we
>> evolve the platform, so a separate repo seemed to be a better idea till the
>> codebase stabilized somewhat and we could offer a concrete view of the
>> changes we wanted.
>>
>> Arunabha
>>
>> On Mon, Dec 7, 2015 at 7:03 PM, Jeff Schroeder <
>> jeffschroe...@computer.org> wrote:
>>
>>> From a quick skim, this looks excellent. Are there any plans to try
>>> getting the bamboo changes back upstream, or is it going to be a permanent
>>> fork?
>>>
>>>
>>> On Monday, December 7, 2015, Arunabha Ghosh 
>>> wrote:
>>>
 Hi Folks,
   We, at Moz have been working for a while on RogerOS, our
 next gen application platform built on top of Mesos. We've reached a point
 in the project where we feel it's ready to share with the world :-)

 The blog posts introducing RogerOS can be found at

 https://moz.com/devblog/introducing-rogeros-part-1/
 https://moz.com/devblog/introducing-rogeros-part-2/

 I can safely say that without Mesos, it would not have been possible
 for us to have built the system within the constraints of time and
 resources that we had. As we note in the blog

 " We are very glad that we chose Mesos though. It has delivered on all
 of its promises and more. We’ve had no issues with stability,
 extensibility, and performance of the system and it has allowed us to
 achieve our goals with a fraction of the development resources that would
 have been required otherwise. "

 We would also like to thank the wonderful Mesos community for all the
 help and support we've received. Along the way we've tried to contribute
 back to the community through talks at Mesoscon and now through open
 sourcing our efforts.

 Your feedback and thoughts are always welcome !

 Thanks,
 Arunabha



>>>
>>>
>>> --
>>> Text by Jeff, typos by iPhone
>>>
>>
>>
>

Re: Mesos at Moz

[Vinod Kone]

Really cool to see a multi-framework compute platform. Let us know how
things work for you guys as you scale!

On Mon, Dec 7, 2015 at 7:24 PM, Arunabha Ghosh 
wrote:

> We're definitely open to merging the changes to Bamboo back upstream if
> the changes we made prove useful to Bamboo. The changes we made were pretty
> specific to our needs and we anticipate making quite a few changes as we
> evolve the platform, so a separate repo seemed to be a better idea till the
> codebase stabilized somewhat and we could offer a concrete view of the
> changes we wanted.
>
> Arunabha
>
> On Mon, Dec 7, 2015 at 7:03 PM, Jeff Schroeder  > wrote:
>
>> From a quick skim, this looks excellent. Are there any plans to try
>> getting the bamboo changes back upstream, or is it going to be a permanent
>> fork?
>>
>>
>> On Monday, December 7, 2015, Arunabha Ghosh 
>> wrote:
>>
>>> Hi Folks,
>>>   We, at Moz have been working for a while on RogerOS, our
>>> next gen application platform built on top of Mesos. We've reached a point
>>> in the project where we feel it's ready to share with the world :-)
>>>
>>> The blog posts introducing RogerOS can be found at
>>>
>>> https://moz.com/devblog/introducing-rogeros-part-1/
>>> https://moz.com/devblog/introducing-rogeros-part-2/
>>>
>>> I can safely say that without Mesos, it would not have been possible for
>>> us to have built the system within the constraints of time and resources
>>> that we had. As we note in the blog
>>>
>>> " We are very glad that we chose Mesos though. It has delivered on all
>>> of its promises and more. We’ve had no issues with stability,
>>> extensibility, and performance of the system and it has allowed us to
>>> achieve our goals with a fraction of the development resources that would
>>> have been required otherwise. "
>>>
>>> We would also like to thank the wonderful Mesos community for all the
>>> help and support we've received. Along the way we've tried to contribute
>>> back to the community through talks at Mesoscon and now through open
>>> sourcing our efforts.
>>>
>>> Your feedback and thoughts are always welcome !
>>>
>>> Thanks,
>>> Arunabha
>>>
>>>
>>>
>>
>>
>> --
>> Text by Jeff, typos by iPhone
>>
>
>

[VOTE] Release Apache Mesos 0.26.0 (rc4)

[Till Toenshoff]

Hi friends,

we had noticed some discrepancies between the V0 API and the V1 API,
hence we had to create a new release candidate even after the voting of
0.26.0-rc3 had officially ended. Sorry for that!

Please vote on releasing the following candidate as Apache Mesos 0.26.0.

The CHANGELOG for the release is available at:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.26.0-rc4


The candidate for Mesos 0.26.0 release is available at:
https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc4/mesos-0.26.0.tar.gz

The tag to be voted on is 0.26.0-rc4:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.26.0-rc4

The MD5 checksum of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc4/mesos-0.26.0.tar.gz.md5

The signature of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc4/mesos-0.26.0.tar.gz.asc

The PGP key used to sign the release is here:
https://dist.apache.org/repos/dist/release/mesos/KEYS

The JAR is up in Maven in a staging repository here:
https://repository.apache.org/content/repositories/orgapachemesos-1093

Please vote on releasing this package as Apache Mesos 0.26.0!

The vote is open until Fri Dec 11 04:50:51 CET 2015 and passes if a majority of 
at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Mesos 0.26.0
[ ] -1 Do not release this package because ...

Thanks,
Bernd & Till