Re: Unable to start mesos slave 0.25+ in docker

[Guangya Liu]

I think that you can take a look at
https://issues.apache.org/jira/browse/MESOS-3793 to see if it helps.

On Fri, Jan 29, 2016 at 4:41 PM, Giulio Eulisse 
wrote:

> Ciao,
>
> I filed:
>
> https://issues.apache.org/jira/browse/MESOS-4543
>
> about not being able to start a mesos slave in docker since 0.25
> (including 0.26, did not test master). Has anyone seen the same problem?
>
> --
> Ciao,
> Giulio
>

Unable to start mesos slave 0.25+ in docker

[Giulio Eulisse]

Ciao,

I filed:

https://issues.apache.org/jira/browse/MESOS-4543

about not being able to start a mesos slave in docker since 0.25 (including
0.26, did not test master). Has anyone seen the same problem?

-- 
Ciao,
Giulio

Re: [VOTE] Release Apache Mesos 0.27.0 (rc2)

[Vinod Kone]

Git fetch --tags

@vinodkone

> On Jan 29, 2016, at 7:00 PM, Marco Massenzio  wrote:
> 
> Is there a 0.27.0-rc2 branch cut?
> 
> $ git fetch --all
> Fetching origin
> 
> $ git co 0.27.0-rc2
> error: pathspec '0.27.0-rc2' did not match any file(s) known to git.
> 
> 
> -- 
> Marco Massenzio
> http://codetrips.com
> 
>> On Wed, Jan 27, 2016 at 11:12 PM, Michael Park  wrote:
>> Hi all,
>> 
>> Please vote on releasing the following candidate as Apache Mesos 0.27.0.
>> 
>> 0.27.0 includes the following:
>> 
>> We added major features such as Implicit Roles, Quota, Multiple Disks and 
>> more.
>> 
>> We also added major bug fixes such as performance improvements to
>> state.json requests and GLOG.
>> 
>> The CHANGELOG for the release is available at:
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.0-rc2
>> 
>> 
>> The candidate for Mesos 0.27.0 release is available at:
>> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz
>> 
>> The tag to be voted on is 0.27.0-rc2:
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.27.0-rc2
>> 
>> The MD5 checksum of the tarball can be found at:
>> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz.md5
>> 
>> The signature of the tarball can be found at:
>> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz.asc
>> 
>> The PGP key used to sign the release is here:
>> https://dist.apache.org/repos/dist/release/mesos/KEYS
>> 
>> The JAR is up in Maven in a staging repository here:
>> https://repository.apache.org/content/repositories/orgapachemesos-1100
>> 
>> Please vote on releasing this package as Apache Mesos 0.27.0!
>> 
>> The vote is open until Sat Jan 30 23:59:59 PST 2016 and passes if a majority 
>> of at least 3 +1 PMC votes are cast.
>> 
>> [ ] +1 Release this package as Apache Mesos 0.27.0
>> [ ] -1 Do not release this package because ...
>> 
>> Thanks,
>> 
>> Tim, Kapil, MPark
> 

Re: [VOTE] Release Apache Mesos 0.27.0 (rc2)

[Marco Massenzio]

On Fri, Jan 29, 2016 at 7:00 PM, Marco Massenzio 
wrote:

> Is there a 0.27.0-rc2 branch cut?
>
> $ git fetch --all
> Fetching origin
>
> $ git co 0.27.0-rc2
> error: pathspec '0.27.0-rc2' did not match any file(s) known to git.
>
> ​well, or a tag, for that matter...

$ git tag | grep 27
0.27.0-rc1
​


>
> --
> *Marco Massenzio*
> http://codetrips.com
>
> On Wed, Jan 27, 2016 at 11:12 PM, Michael Park  wrote:
>
>> Hi all,
>>
>> Please vote on releasing the following candidate as Apache Mesos 0.27.0.
>>
>> 0.27.0 includes the following:
>>
>> 
>> We added major features such as Implicit Roles, Quota, Multiple Disks and
>> more.
>>
>> We also added major bug fixes such as performance improvements to
>> state.json requests and GLOG.
>>
>> The CHANGELOG for the release is available at:
>>
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.0-rc2
>>
>> 
>>
>> The candidate for Mesos 0.27.0 release is available at:
>>
>> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz
>>
>> The tag to be voted on is 0.27.0-rc2:
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.27.0-rc2
>>
>> The MD5 checksum of the tarball can be found at:
>>
>> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz.md5
>>
>> The signature of the tarball can be found at:
>>
>> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz.asc
>>
>> The PGP key used to sign the release is here:
>> https://dist.apache.org/repos/dist/release/mesos/KEYS
>>
>> The JAR is up in Maven in a staging repository here:
>> https://repository.apache.org/content/repositories/orgapachemesos-1100
>>
>> Please vote on releasing this package as Apache Mesos 0.27.0!
>>
>> The vote is open until Sat Jan 30 23:59:59 PST 2016 and passes if a
>> majority of at least 3 +1 PMC votes are cast.
>>
>> [ ] +1 Release this package as Apache Mesos 0.27.0
>> [ ] -1 Do not release this package because ...
>>
>> Thanks,
>>
>> Tim, Kapil, MPark
>>
>
>

Re: [VOTE] Release Apache Mesos 0.27.0 (rc2)

[Marco Massenzio]

Is there a 0.27.0-rc2 branch cut?

$ git fetch --all
Fetching origin

$ git co 0.27.0-rc2
error: pathspec '0.27.0-rc2' did not match any file(s) known to git.


-- 
*Marco Massenzio*
http://codetrips.com

On Wed, Jan 27, 2016 at 11:12 PM, Michael Park  wrote:

> Hi all,
>
> Please vote on releasing the following candidate as Apache Mesos 0.27.0.
>
> 0.27.0 includes the following:
>
> 
> We added major features such as Implicit Roles, Quota, Multiple Disks and
> more.
>
> We also added major bug fixes such as performance improvements to
> state.json requests and GLOG.
>
> The CHANGELOG for the release is available at:
>
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.0-rc2
>
> 
>
> The candidate for Mesos 0.27.0 release is available at:
> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz
>
> The tag to be voted on is 0.27.0-rc2:
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.27.0-rc2
>
> The MD5 checksum of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz.md5
>
> The signature of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz.asc
>
> The PGP key used to sign the release is here:
> https://dist.apache.org/repos/dist/release/mesos/KEYS
>
> The JAR is up in Maven in a staging repository here:
> https://repository.apache.org/content/repositories/orgapachemesos-1100
>
> Please vote on releasing this package as Apache Mesos 0.27.0!
>
> The vote is open until Sat Jan 30 23:59:59 PST 2016 and passes if a
> majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Mesos 0.27.0
> [ ] -1 Do not release this package because ...
>
> Thanks,
>
> Tim, Kapil, MPark
>

Re: [VOTE] Release Apache Mesos 0.27.0 (rc2)

[Marco Massenzio]

Thanks, buddy - I keep forgetting that one!
(one assumes --all would, well, take care of that too :)

Have a great weekend!

-- 
*Marco Massenzio*
http://codetrips.com

On Fri, Jan 29, 2016 at 7:06 PM, Vinod Kone  wrote:

> Git fetch --tags
>
> @vinodkone
>
> On Jan 29, 2016, at 7:00 PM, Marco Massenzio 
> wrote:
>
> Is there a 0.27.0-rc2 branch cut?
>
> $ git fetch --all
> Fetching origin
>
> $ git co 0.27.0-rc2
> error: pathspec '0.27.0-rc2' did not match any file(s) known to git.
>
>
> --
> *Marco Massenzio*
> http://codetrips.com
>
> On Wed, Jan 27, 2016 at 11:12 PM, Michael Park  wrote:
>
>> Hi all,
>>
>> Please vote on releasing the following candidate as Apache Mesos 0.27.0.
>>
>> 0.27.0 includes the following:
>>
>> 
>> We added major features such as Implicit Roles, Quota, Multiple Disks and
>> more.
>>
>> We also added major bug fixes such as performance improvements to
>> state.json requests and GLOG.
>>
>> The CHANGELOG for the release is available at:
>>
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.0-rc2
>>
>> 
>>
>> The candidate for Mesos 0.27.0 release is available at:
>>
>> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz
>>
>> The tag to be voted on is 0.27.0-rc2:
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.27.0-rc2
>>
>> The MD5 checksum of the tarball can be found at:
>>
>> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz.md5
>>
>> The signature of the tarball can be found at:
>>
>> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz.asc
>>
>> The PGP key used to sign the release is here:
>> https://dist.apache.org/repos/dist/release/mesos/KEYS
>>
>> The JAR is up in Maven in a staging repository here:
>> https://repository.apache.org/content/repositories/orgapachemesos-1100
>>
>> Please vote on releasing this package as Apache Mesos 0.27.0!
>>
>> The vote is open until Sat Jan 30 23:59:59 PST 2016 and passes if a
>> majority of at least 3 +1 PMC votes are cast.
>>
>> [ ] +1 Release this package as Apache Mesos 0.27.0
>> [ ] -1 Do not release this package because ...
>>
>> Thanks,
>>
>> Tim, Kapil, MPark
>>
>
>

Re: [VOTE] Release Apache Mesos 0.27.0 (rc2)

[Vinod Kone]

+1 (binding)

Tested (non-root tests with ./support/docker_build.sh script) on ASF CI
with
the following configurations:

NOTE: clang is intentionally disabled on centos and "clang + ubuntu +
--verbose" build failed because of a docker image building issue (not mesos
issue).

Configuration Matrix gcc clang
centos:7 --verbose --enable-libevent --enable-ssl
[image: Success]

[image: Not run]
--verbose
[image: Success]

[image: Not run]
ubuntu:14.04 --verbose --enable-libevent --enable-ssl
[image: Success]

[image: Success]

--verbose
[image: Success]

[image: Failed]


On Fri, Jan 29, 2016 at 10:43 AM, Jörg Schad  wrote:

> [+1] (non-binding)
>
> I tested on Ubuntu 12.04 and OSX 10.10.5 without SSL.
>
> *make check* works on both platforms with no issues.
>
> With *sudo make check *some tests fail.
> The ubuntu failure also occurs with other tests and seems to be related to
> the cgroups configuration on that system. Currently looking into the
> details here (thanks to Avinash for his support!) and would open a Jira if
> necessary.
> The 3 failures on OSX are known issues (see below for the respective Jira
> tickets).
>
> *Ubuntu 12.04*
>
> [ RUN  ] AuthenticationTest.UnauthenticatedSlave
>
> F0129 15:55:48.241842 11049 cluster.cpp:369] CHECK_SOME(containerizer):
> Could not create MesosContainerizer: Failed to create launcher: Failed to
> create Linux launcher: Failed to mount cgroups hierarchy at
> '/sys/fs/cgroup/freezer': Failed to create directory
> '/sys/fs/cgroup/freezer': No such file or directory
>
> *** Check failure stack trace: ***
>
> @ 0x2b969832bd0e  google::LogMessage::Fail()
>
> @ 0x2b969832bc48  google::LogMessage::SendToLog()
>
> @ 0x2b969832b64a  google::LogMessage::Flush()
>
> @ 0x2b969832e562  google::LogMessageFatal::~LogMessageFatal()
>
> @   0x97fe2a  _CheckFatal::~_CheckFatal()
>
> @   0xa5376f  mesos::internal::tests::Cluster::Slaves::start()
>
> @   0xffc286  mesos::internal::tests::MesosTest::StartSlave()
>
> @   0x9bd542
> mesos::internal::tests::AuthenticationTest_UnauthenticatedSlave_Test::TestBody()
>
> @  0x164b8fc
> testing::internal::HandleSehExceptionsInMethodIfSupported<>()
>
> @  0x164678a
> testing::internal::HandleExceptionsInMethodIfSupported<>()
>
> @  0x1627919  testing::Test::Run()
>
> @  0x162809e  testing::TestInfo::Run()
>
> @  0x16286ee  testing::TestCase::Run()
>
> @  0x162ef36  testing::internal::UnitTestImpl::RunAllTests()
>
> @  0x164c521
> testing::internal::HandleSehExceptionsInMethodIfSupported<>()
>
> @  0x164732a
> testing::internal::HandleExceptionsInMethodIfSupported<>()
>
> @  0x162dc6c  testing::UnitTest::Run()
>
> @   0xde89b9  RUN_ALL_TESTS()
>
> @   0xde85d6  main
>
> @ 0x2b969a2f376d  (unknown)
>
> @   0x97e409  (unknown)
>
> make[3]: *** [check-local] Aborted (core dumped)
>
>
> *OSX:*
>
> *[  PASSED  ] 955 tests.*
>
> *[  FAILED  ] 3 tests, listed below:*
>
> *[  FAILED  ] ExamplesTest.JavaFramework **Seems to be
> known via *MESOS-3582 
>
> *[  FAILED  ] FetcherCacheTest.LocalUncachedExtract (Problem mentioned
> via MESOS-3579)*
>
> *[  FAILED  ] FetcherCacheHttpTest.HttpMixed **(Problem mentioned
> via MESOS-3579)*
>
>
>
> On Thu, Jan 28, 2016 at 8:12 AM, Michael Park  wrote:
>
>> Hi all,
>>
>> Please vote on releasing the following candidate as Apache Mesos 0.27.0.
>>
>> 0.27.0 includes the following:
>>
>> 
>> We added major features such as Implicit Roles, Quota, Multiple Disks and
>> more.
>>
>> We also added major bug fixes such as performance improvements to

Re: [VOTE] Release Apache Mesos 0.27.0 (rc2)

[Jörg Schad]

[+1] (non-binding)

I tested on Ubuntu 12.04 and OSX 10.10.5 without SSL.

*make check* works on both platforms with no issues.

With *sudo make check *some tests fail.
The ubuntu failure also occurs with other tests and seems to be related to
the cgroups configuration on that system. Currently looking into the
details here (thanks to Avinash for his support!) and would open a Jira if
necessary.
The 3 failures on OSX are known issues (see below for the respective Jira
tickets).

*Ubuntu 12.04*

[ RUN  ] AuthenticationTest.UnauthenticatedSlave

F0129 15:55:48.241842 11049 cluster.cpp:369] CHECK_SOME(containerizer):
Could not create MesosContainerizer: Failed to create launcher: Failed to
create Linux launcher: Failed to mount cgroups hierarchy at
'/sys/fs/cgroup/freezer': Failed to create directory
'/sys/fs/cgroup/freezer': No such file or directory

*** Check failure stack trace: ***

@ 0x2b969832bd0e  google::LogMessage::Fail()

@ 0x2b969832bc48  google::LogMessage::SendToLog()

@ 0x2b969832b64a  google::LogMessage::Flush()

@ 0x2b969832e562  google::LogMessageFatal::~LogMessageFatal()

@   0x97fe2a  _CheckFatal::~_CheckFatal()

@   0xa5376f  mesos::internal::tests::Cluster::Slaves::start()

@   0xffc286  mesos::internal::tests::MesosTest::StartSlave()

@   0x9bd542
mesos::internal::tests::AuthenticationTest_UnauthenticatedSlave_Test::TestBody()

@  0x164b8fc
testing::internal::HandleSehExceptionsInMethodIfSupported<>()

@  0x164678a
testing::internal::HandleExceptionsInMethodIfSupported<>()

@  0x1627919  testing::Test::Run()

@  0x162809e  testing::TestInfo::Run()

@  0x16286ee  testing::TestCase::Run()

@  0x162ef36  testing::internal::UnitTestImpl::RunAllTests()

@  0x164c521
testing::internal::HandleSehExceptionsInMethodIfSupported<>()

@  0x164732a
testing::internal::HandleExceptionsInMethodIfSupported<>()

@  0x162dc6c  testing::UnitTest::Run()

@   0xde89b9  RUN_ALL_TESTS()

@   0xde85d6  main

@ 0x2b969a2f376d  (unknown)

@   0x97e409  (unknown)

make[3]: *** [check-local] Aborted (core dumped)


*OSX:*

*[  PASSED  ] 955 tests.*

*[  FAILED  ] 3 tests, listed below:*

*[  FAILED  ] ExamplesTest.JavaFramework **Seems to be
known via *MESOS-3582 

*[  FAILED  ] FetcherCacheTest.LocalUncachedExtract (Problem mentioned
via MESOS-3579)*

*[  FAILED  ] FetcherCacheHttpTest.HttpMixed **(Problem mentioned
via MESOS-3579)*



On Thu, Jan 28, 2016 at 8:12 AM, Michael Park  wrote:

> Hi all,
>
> Please vote on releasing the following candidate as Apache Mesos 0.27.0.
>
> 0.27.0 includes the following:
>
> 
> We added major features such as Implicit Roles, Quota, Multiple Disks and
> more.
>
> We also added major bug fixes such as performance improvements to
> state.json requests and GLOG.
>
> The CHANGELOG for the release is available at:
>
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.0-rc2
>
> 
>
> The candidate for Mesos 0.27.0 release is available at:
> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz
>
> The tag to be voted on is 0.27.0-rc2:
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.27.0-rc2
>
> The MD5 checksum of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz.md5
>
> The signature of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.27.0-rc2/mesos-0.27.0.tar.gz.asc
>
> The PGP key used to sign the release is here:
> https://dist.apache.org/repos/dist/release/mesos/KEYS
>
> The JAR is up in Maven in a staging repository here:
> https://repository.apache.org/content/repositories/orgapachemesos-1100
>
> Please vote on releasing this package as Apache Mesos 0.27.0!
>
> The vote is open until Sat Jan 30 23:59:59 PST 2016 and passes if a
> majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Mesos 0.27.0
> [ ] -1 Do not release this package because ...
>
> Thanks,
>
> Tim, Kapil, MPark
>

Re: Mesos sometimes not allocating the entire cluster

[Benjamin Mahler]

Hi Tom,

I suspect you may be tripping the following issue:
https://issues.apache.org/jira/browse/MESOS-4302

Please have a read through this and see if it applies here. You may also be
able to apply the fix to your cluster to see if that helps things.

Ben

On Wed, Jan 20, 2016 at 10:19 AM, Tom Arnfeld  wrote:

> Hey,
>
> I've noticed some interesting behaviour recently when we have lots of
> different frameworks connected to our Mesos cluster at once, all using a
> variety of different shares. Some of the frameworks don't get offered more
> resources (for long periods of time, hours even) leaving the cluster under
> utilised.
>
> Here's an example state where we see this happen..
>
> Framework 1 - 13% (user A)
> Framework 2 - 22% (user B)
> Framework 3 - 4% (user C)
> Framework 4 - 0.5% (user C)
> Framework 5 - 1% (user C)
> Framework 6 - 1% (user C)
> Framework 7 - 1% (user C)
> Framework 8 - 0.8% (user C)
> Framework 9 - 11% (user D)
> Framework 10 - 7% (user C)
> Framework 11 - 1% (user C)
> Framework 12 - 1% (user C)
> Framework 13 - 6% (user E)
>
> In this example, there's another ~30% of the cluster that is unallocated,
> and it stays like this for a significant amount of time until something
> changes, perhaps another user joins and allocates the rest chunks of
> this spare resource is offered to some of the frameworks, but not all of
> them.
>
> I had always assumed that when lots of frameworks were involved,
> eventually the frameworks that would keep accepting resources indefinitely
> would consume the remaining resource, as every other framework had rejected
> the offers.
>
> Could someone elaborate a little on how the DRF allocator / sorter handles
> this situation, is this likely to be related to the different users being
> used? Is there a way to mitigate this?
>
> We're running version 0.23.1.
>
> Cheers,
>
> Tom.
>

Re: Get Task's labels on reconciliation

[Benjamin Mahler]

I see that the following was filed:
https://issues.apache.org/jira/browse/MESOS-4477

But this sounds like a bug: if the master knows about the task during
reconciliation, labels should be sent. For example we had this same bug for
health information:
https://issues.apache.org/jira/browse/MESOS-2206

On Fri, Jan 22, 2016 at 2:10 PM, Andrii Biletskyi <
andrii.bilets...@stealth.ly> wrote:

> Okay, thanks for your answer. I will rework my algorithm so it doesn't
> rely on task labels/data.
>
> Thanks,
> Andrii
>
> On Fri, Jan 22, 2016 at 11:49 PM, Vinod Kone  wrote:
>
>>
>> On Fri, Jan 22, 2016 at 1:07 PM, Andrii Biletskyi <
>> andrii.bilets...@stealth.ly> wrote:
>>
>>> 8. Scheduler.statusUpdate callback is called for
>>> taskId=t1;state=Running. But here I don't receive as part of TaskStatus
>>> task labels and taskData, they are just empty.
>>>
>>
>> Yea, this is expected behavior. Currently master *generates* new status
>> updates (with only a few fields set) when it gets a reconciliation request.
>>
>
>

Re: Completely rude alternative to HierarchicalDRF?

[Charles Allen]

Thanks Benjamin, that's quite insightful and also a bummer.

On Fri, Jan 29, 2016 at 11:50 AM Benjamin Mahler  wrote:

> Hey Charles,
>
> From this question it seems you're aware that the default allocator can be
> swapped out for alternative implementations. However, the allocator module
> interface is wide currently and so you'll be hard-pressed to find
> implementations. (No refactoring took place during modularization).
> Implementations will have had to keep up with a number of additional
> responsibilities over time (generating inverse offers, handling quota,
> etc). I'm thinking we'll need to figure out which pieces of functionality
> *within* the allocator should be made modularizable when we do some of the
> upcoming allocation work. For now, the expectation is that allocator module
> implementations are non-trivial to maintain.
>
> Hope that helps!
>
> On Mon, Jan 25, 2016 at 3:18 PM, Charles Allen <
> charles.al...@metamarkets.com> wrote:
>
>> Is there an allocator out there that will prioritize frameworks to favor
>> the oldest first?
>>
>> I'm using spark in coarse mode, and I actually want the earliest
>> registered course mode drivers to get as many resources as they want.
>>
>
>