Re: [VOTE] Release Apache Mesos 0.27.2 (rc1)

[Greg Mann]

+1 (non-binding)

`sudo make check` on Ubuntu 14.04 using gcc, with libevent and SSL enabled.

All tests pass except MemoryPressureMesosTest.CGROUPS_ROOT_Statistics,
which seems to be due to the issue found here:
https://issues.apache.org/jira/browse/MESOS-4053


On Mon, Feb 29, 2016 at 2:17 PM, Michael Park  wrote:

> Vinod, we've only committed the CHANGELOGs to the specific tags. I didn't
> realize that I should commit those to master as well, but it makes total
> sense to do so. I'll do that. Thanks.
>
> On 29 February 2016 at 13:50, Vinod Kone  wrote:
>
>> I don't see CHANGELOGs for these versions on the master branch?
>>
>> On Mon, Feb 29, 2016 at 1:39 PM, Neil Conway 
>> wrote:
>>
>> > As described (briefly) in the release emails, 0.27.2, 0.26.1, 0.25.1,
>> > and 0.24.2 contains a new feature: "reliable floating point for scalar
>> > resources" (MESOS-4687).
>> >
>> > To elaborate on that slightly, Mesos now only supports scalar resource
>> > values with three decimal digits of precision (e.g., reserving "5.001
>> > CPUs" for a task). As a result of this change, frameworks that do
>> > their own resource math may see slightly different results;
>> > furthermore, if any frameworks were trying to manage extremely
>> > fine-grained resource values (> 3 decimal digits of precision), that
>> > will no longer be supported.
>> >
>> > For more information, please see:
>> >
>> >
>> >
>> https://mail-archives.apache.org/mod_mbox/mesos-user/201602.mbox/%3CCAOW5sYZJn5caBOwZyPV008JgL1F2FYFxL_bM5CtYA2PF2OG7Bw%40mail.gmail.com%3E
>> >
>> >
>> https://docs.google.com/document/d/14qLxjZsfIpfynbx0USLJR0GELSq8hdZJUWw6kaY_DXc/edit?usp=sharing
>> > https://issues.apache.org/jira/browse/MESOS-4687
>> >
>> > Neil
>> >
>> >
>> > On Fri, Feb 26, 2016 at 8:54 PM, Michael Park 
>> wrote:
>> > > Hi all,
>> > >
>> > > Please vote on releasing the following candidate as Apache Mesos
>> 0.27.2.
>> > >
>> > >
>> > > 0.27.2 includes the following:
>> > >
>> >
>> 
>> > >
>> > > MESOS-4693 - Variable shadowing in
>> HookManager::slavePreLaunchDockerHook.
>> > > MESOS-4711 - Race condition in libevent poll implementation causes
>> crash.
>> > > MESOS-4754 - The "executors" field is exposed under a backwards
>> > incompatible
>> > > schema.
>> > > MESOS-4687 - Implement reliable floating point for scalar resources.
>> > >
>> > >
>> > > The CHANGELOG for the release is available at:
>> > >
>> >
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.2-rc1
>> > >
>> >
>> 
>> > >
>> > > The candidate for Mesos 0.27.2 release is available at:
>> > >
>> >
>> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz
>> > >
>> > > The tag to be voted on is 0.27.2-rc1:
>> > >
>> >
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.27.2-rc1
>> > >
>> > > The MD5 checksum of the tarball can be found at:
>> > >
>> >
>> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.md5
>> > >
>> > > The signature of the tarball can be found at:
>> > >
>> >
>> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.asc
>> > >
>> > > The PGP key used to sign the release is here:
>> > > https://dist.apache.org/repos/dist/release/mesos/KEYS
>> > >
>> > > The JAR is up in Maven in a staging repository here:
>> > >
>> https://repository.apache.org/content/repositories/orgapachemesos-1104
>> > >
>> > > Please vote on releasing this package as Apache Mesos 0.27.2!
>> > >
>> > > The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a
>> > majority
>> > > of at least 3 +1 PMC votes are cast.
>> > >
>> > > [ ] +1 Release this package as Apache Mesos 0.27.2
>> > > [ ] -1 Do not release this package because ...
>> > >
>> > > Thanks,
>> > >
>> > > MPark, Joris, Kapil
>> >
>>
>
>

Re: [VOTE] Release Apache Mesos 0.26.1 (rc1)

[Greg Mann]

+1 (non-binding)

`sudo make check` on Ubuntu 14.04 using gcc, with libevent and SSL enabled.

All tests pass except:

PerfEventIsolatorTest.ROOT_CGROUPS_Sample, which is covered here:
https://issues.apache.org/jira/browse/MESOS-4655

CgroupsAnyHierarchyWithPerfEventTest.ROOT_CGROUPS_Perf,
MemoryPressureMesosTest.CGROUPS_ROOT_Statistics, and
MemoryPressureMesosTest.CGROUPS_ROOT_SlaveRecovery, due to the issue here:
https://issues.apache.org/jira/browse/MESOS-3215

On Mon, Feb 29, 2016 at 2:27 PM, Vinod Kone  wrote:

> Looks like the ASF CI builds for CentOS7 are failing because they are
> unable to find JAVA_HOME. Couldn't tell if it's an issue with the docker
> build script or something in the configure script.
>
>
> checking for svn_txdelta in -lsvn_delta-1... yes
> checking for sasl_done in -lsasl2... yes
> checking SASL CRAM-MD5 support... yes
> checking for javac... /usr/bin/javac
> checking for java... /usr/bin/java
> checking value of Java system property 'java.home'... 
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.71-2.b15.el7_2.x86_64/jre
> configure: error: could not guess JAVA_HOME
>
>
>
> *Revision*: a05261dbed1c2577676b11235380de95d586aeeb
>
>- refs/tags/0.26.1-rc1
>
> Configuration Matrix gcc clang
> centos:7 --verbose --enable-libevent --enable-ssl
> [image: Failed]
> 
> [image: Not run]
> --verbose
> [image: Failed]
> 
> [image: Not run]
> ubuntu:14.04 --verbose --enable-libevent --enable-ssl
> [image: Success]
> 
> [image: Success]
> 
> --verbose
> [image: Success]
> 
> [image: Success]
> 
>
> On Mon, Feb 29, 2016 at 11:21 AM, Kapil Arya  wrote:
>
>> +1 (binding)
>>
>> Successful CI builds for the following distros:
>>
>> amd64/centos/6
>> amd64/centos/7
>> amd64/debian/jessie
>> amd64/ubuntu/precise
>> amd64/ubuntu/trusty
>> amd64/ubuntu/vivid
>>
>> Kapil
>>
>> On Sat, Feb 27, 2016 at 12:26 AM, Michael Park  wrote:
>>
>> > Hi all,
>> >
>> > Please vote on releasing the following candidate as Apache Mesos 0.26.1.
>> >
>> >
>> > 0.26.1 includes the following:
>> >
>> >
>> 
>> >
>> >- Improvements
>> >   - `/state` endpoint performance
>> >   - systemd integration
>> >   - GLOG performance
>> >   - Configurable task/framework history
>> >   - Offer filter timeout fix for backlogged allocator
>> >
>> >
>> >- Bugs
>> >- SSL
>> >   - Libevent
>> >   - Fixed point resources math
>> >- HDFS
>> >   - Agent upgrade compatibility
>> >
>> > The CHANGELOG for the release is available at:
>> >
>> >
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.26.1-rc1
>> >
>> >
>> 
>> >
>> > The candidate for Mesos 0.26.1 release is available at:
>> >
>> https://dist.apache.org/repos/dist/dev/mesos/0.26.1-rc1/mesos-0.26.1.tar.gz
>> >
>> > The tag to be voted on is 0.26.1-rc1:
>> >
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.26.1-rc1
>> >
>> > The MD5 checksum of the tarball can be found at:
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/mesos/0.26.1-rc1/mesos-0.26.1.tar.gz.md5
>> >
>> > The signature of the tarball can be found at:
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/mesos/0.26.1-rc1/mesos-0.26.1.tar.gz.asc
>> >
>> > The PGP key used to sign the release is here:
>> > https://dist.apache.org/repos/dist/release/mesos/KEYS
>> >
>> > The JAR is up in Maven in a staging repository here:
>> > https://repository.apache.org/content/repositories/orgapachemesos-1106
>> >
>> > Please vote on releasing this package as Apache Mesos 0.26.1!
>> >
>> > The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a
>> majority
>> > of at least 3 +1 PMC votes are cast.
>> >
>> > [ ] +1 

Asking for Help: Destroy docker container from marathon kills mesos slave

[shizhz]

Hi all,

Is this the correct place to ask for help? If it is could anybody help me on 
the problem I posted on SOF: 
http://stackoverflow.com/questions/35713985/destroy-docker-container-from-marathon-kills-mesos-slave
 
.

Sorry for bothering and wish you all have a nice day.

BR, Shizhz

Re: [VOTE] Release Apache Mesos 0.26.1 (rc1)

[Vinod Kone]

Looks like the ASF CI builds for CentOS7 are failing because they are
unable to find JAVA_HOME. Couldn't tell if it's an issue with the docker
build script or something in the configure script.


checking for svn_txdelta in -lsvn_delta-1... yes
checking for sasl_done in -lsasl2... yes
checking SASL CRAM-MD5 support... yes
checking for javac... /usr/bin/javac
checking for java... /usr/bin/java
checking value of Java system property 'java.home'...
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.71-2.b15.el7_2.x86_64/jre
configure: error: could not guess JAVA_HOME



*Revision*: a05261dbed1c2577676b11235380de95d586aeeb

   - refs/tags/0.26.1-rc1

Configuration Matrix gcc clang
centos:7 --verbose --enable-libevent --enable-ssl
[image: Failed]

[image: Not run]
--verbose
[image: Failed]

[image: Not run]
ubuntu:14.04 --verbose --enable-libevent --enable-ssl
[image: Success]

[image: Success]

--verbose
[image: Success]

[image: Success]


On Mon, Feb 29, 2016 at 11:21 AM, Kapil Arya  wrote:

> +1 (binding)
>
> Successful CI builds for the following distros:
>
> amd64/centos/6
> amd64/centos/7
> amd64/debian/jessie
> amd64/ubuntu/precise
> amd64/ubuntu/trusty
> amd64/ubuntu/vivid
>
> Kapil
>
> On Sat, Feb 27, 2016 at 12:26 AM, Michael Park  wrote:
>
> > Hi all,
> >
> > Please vote on releasing the following candidate as Apache Mesos 0.26.1.
> >
> >
> > 0.26.1 includes the following:
> >
> >
> 
> >
> >- Improvements
> >   - `/state` endpoint performance
> >   - systemd integration
> >   - GLOG performance
> >   - Configurable task/framework history
> >   - Offer filter timeout fix for backlogged allocator
> >
> >
> >- Bugs
> >- SSL
> >   - Libevent
> >   - Fixed point resources math
> >- HDFS
> >   - Agent upgrade compatibility
> >
> > The CHANGELOG for the release is available at:
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.26.1-rc1
> >
> >
> 
> >
> > The candidate for Mesos 0.26.1 release is available at:
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.26.1-rc1/mesos-0.26.1.tar.gz
> >
> > The tag to be voted on is 0.26.1-rc1:
> >
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.26.1-rc1
> >
> > The MD5 checksum of the tarball can be found at:
> >
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.26.1-rc1/mesos-0.26.1.tar.gz.md5
> >
> > The signature of the tarball can be found at:
> >
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.26.1-rc1/mesos-0.26.1.tar.gz.asc
> >
> > The PGP key used to sign the release is here:
> > https://dist.apache.org/repos/dist/release/mesos/KEYS
> >
> > The JAR is up in Maven in a staging repository here:
> > https://repository.apache.org/content/repositories/orgapachemesos-1106
> >
> > Please vote on releasing this package as Apache Mesos 0.26.1!
> >
> > The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a
> majority
> > of at least 3 +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Mesos 0.26.1
> > [ ] -1 Do not release this package because ...
> >
> > Thanks,
> >
> > Joris, Kapil, MPark
> >
>

Re: [VOTE] Release Apache Mesos 0.27.2 (rc1)

[Michael Park]

Vinod, we've only committed the CHANGELOGs to the specific tags. I didn't
realize that I should commit those to master as well, but it makes total
sense to do so. I'll do that. Thanks.

On 29 February 2016 at 13:50, Vinod Kone  wrote:

> I don't see CHANGELOGs for these versions on the master branch?
>
> On Mon, Feb 29, 2016 at 1:39 PM, Neil Conway 
> wrote:
>
> > As described (briefly) in the release emails, 0.27.2, 0.26.1, 0.25.1,
> > and 0.24.2 contains a new feature: "reliable floating point for scalar
> > resources" (MESOS-4687).
> >
> > To elaborate on that slightly, Mesos now only supports scalar resource
> > values with three decimal digits of precision (e.g., reserving "5.001
> > CPUs" for a task). As a result of this change, frameworks that do
> > their own resource math may see slightly different results;
> > furthermore, if any frameworks were trying to manage extremely
> > fine-grained resource values (> 3 decimal digits of precision), that
> > will no longer be supported.
> >
> > For more information, please see:
> >
> >
> >
> https://mail-archives.apache.org/mod_mbox/mesos-user/201602.mbox/%3CCAOW5sYZJn5caBOwZyPV008JgL1F2FYFxL_bM5CtYA2PF2OG7Bw%40mail.gmail.com%3E
> >
> >
> https://docs.google.com/document/d/14qLxjZsfIpfynbx0USLJR0GELSq8hdZJUWw6kaY_DXc/edit?usp=sharing
> > https://issues.apache.org/jira/browse/MESOS-4687
> >
> > Neil
> >
> >
> > On Fri, Feb 26, 2016 at 8:54 PM, Michael Park  wrote:
> > > Hi all,
> > >
> > > Please vote on releasing the following candidate as Apache Mesos
> 0.27.2.
> > >
> > >
> > > 0.27.2 includes the following:
> > >
> >
> 
> > >
> > > MESOS-4693 - Variable shadowing in
> HookManager::slavePreLaunchDockerHook.
> > > MESOS-4711 - Race condition in libevent poll implementation causes
> crash.
> > > MESOS-4754 - The "executors" field is exposed under a backwards
> > incompatible
> > > schema.
> > > MESOS-4687 - Implement reliable floating point for scalar resources.
> > >
> > >
> > > The CHANGELOG for the release is available at:
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.2-rc1
> > >
> >
> 
> > >
> > > The candidate for Mesos 0.27.2 release is available at:
> > >
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz
> > >
> > > The tag to be voted on is 0.27.2-rc1:
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.27.2-rc1
> > >
> > > The MD5 checksum of the tarball can be found at:
> > >
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.md5
> > >
> > > The signature of the tarball can be found at:
> > >
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.asc
> > >
> > > The PGP key used to sign the release is here:
> > > https://dist.apache.org/repos/dist/release/mesos/KEYS
> > >
> > > The JAR is up in Maven in a staging repository here:
> > > https://repository.apache.org/content/repositories/orgapachemesos-1104
> > >
> > > Please vote on releasing this package as Apache Mesos 0.27.2!
> > >
> > > The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a
> > majority
> > > of at least 3 +1 PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Mesos 0.27.2
> > > [ ] -1 Do not release this package because ...
> > >
> > > Thanks,
> > >
> > > MPark, Joris, Kapil
> >
>

Re: [VOTE] Release Apache Mesos 0.27.2 (rc1)

[Vinod Kone]

I don't see CHANGELOGs for these versions on the master branch?

On Mon, Feb 29, 2016 at 1:39 PM, Neil Conway  wrote:

> As described (briefly) in the release emails, 0.27.2, 0.26.1, 0.25.1,
> and 0.24.2 contains a new feature: "reliable floating point for scalar
> resources" (MESOS-4687).
>
> To elaborate on that slightly, Mesos now only supports scalar resource
> values with three decimal digits of precision (e.g., reserving "5.001
> CPUs" for a task). As a result of this change, frameworks that do
> their own resource math may see slightly different results;
> furthermore, if any frameworks were trying to manage extremely
> fine-grained resource values (> 3 decimal digits of precision), that
> will no longer be supported.
>
> For more information, please see:
>
>
> https://mail-archives.apache.org/mod_mbox/mesos-user/201602.mbox/%3CCAOW5sYZJn5caBOwZyPV008JgL1F2FYFxL_bM5CtYA2PF2OG7Bw%40mail.gmail.com%3E
>
> https://docs.google.com/document/d/14qLxjZsfIpfynbx0USLJR0GELSq8hdZJUWw6kaY_DXc/edit?usp=sharing
> https://issues.apache.org/jira/browse/MESOS-4687
>
> Neil
>
>
> On Fri, Feb 26, 2016 at 8:54 PM, Michael Park  wrote:
> > Hi all,
> >
> > Please vote on releasing the following candidate as Apache Mesos 0.27.2.
> >
> >
> > 0.27.2 includes the following:
> >
> 
> >
> > MESOS-4693 - Variable shadowing in HookManager::slavePreLaunchDockerHook.
> > MESOS-4711 - Race condition in libevent poll implementation causes crash.
> > MESOS-4754 - The "executors" field is exposed under a backwards
> incompatible
> > schema.
> > MESOS-4687 - Implement reliable floating point for scalar resources.
> >
> >
> > The CHANGELOG for the release is available at:
> >
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.2-rc1
> >
> 
> >
> > The candidate for Mesos 0.27.2 release is available at:
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz
> >
> > The tag to be voted on is 0.27.2-rc1:
> >
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.27.2-rc1
> >
> > The MD5 checksum of the tarball can be found at:
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.md5
> >
> > The signature of the tarball can be found at:
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.asc
> >
> > The PGP key used to sign the release is here:
> > https://dist.apache.org/repos/dist/release/mesos/KEYS
> >
> > The JAR is up in Maven in a staging repository here:
> > https://repository.apache.org/content/repositories/orgapachemesos-1104
> >
> > Please vote on releasing this package as Apache Mesos 0.27.2!
> >
> > The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a
> majority
> > of at least 3 +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Mesos 0.27.2
> > [ ] -1 Do not release this package because ...
> >
> > Thanks,
> >
> > MPark, Joris, Kapil
>

Re: [VOTE] Release Apache Mesos 0.27.2 (rc1)

[Neil Conway]

As described (briefly) in the release emails, 0.27.2, 0.26.1, 0.25.1,
and 0.24.2 contains a new feature: "reliable floating point for scalar
resources" (MESOS-4687).

To elaborate on that slightly, Mesos now only supports scalar resource
values with three decimal digits of precision (e.g., reserving "5.001
CPUs" for a task). As a result of this change, frameworks that do
their own resource math may see slightly different results;
furthermore, if any frameworks were trying to manage extremely
fine-grained resource values (> 3 decimal digits of precision), that
will no longer be supported.

For more information, please see:

https://mail-archives.apache.org/mod_mbox/mesos-user/201602.mbox/%3CCAOW5sYZJn5caBOwZyPV008JgL1F2FYFxL_bM5CtYA2PF2OG7Bw%40mail.gmail.com%3E
https://docs.google.com/document/d/14qLxjZsfIpfynbx0USLJR0GELSq8hdZJUWw6kaY_DXc/edit?usp=sharing
https://issues.apache.org/jira/browse/MESOS-4687

Neil


On Fri, Feb 26, 2016 at 8:54 PM, Michael Park  wrote:
> Hi all,
>
> Please vote on releasing the following candidate as Apache Mesos 0.27.2.
>
>
> 0.27.2 includes the following:
> 
>
> MESOS-4693 - Variable shadowing in HookManager::slavePreLaunchDockerHook.
> MESOS-4711 - Race condition in libevent poll implementation causes crash.
> MESOS-4754 - The "executors" field is exposed under a backwards incompatible
> schema.
> MESOS-4687 - Implement reliable floating point for scalar resources.
>
>
> The CHANGELOG for the release is available at:
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.2-rc1
> 
>
> The candidate for Mesos 0.27.2 release is available at:
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz
>
> The tag to be voted on is 0.27.2-rc1:
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.27.2-rc1
>
> The MD5 checksum of the tarball can be found at:
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.md5
>
> The signature of the tarball can be found at:
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.asc
>
> The PGP key used to sign the release is here:
> https://dist.apache.org/repos/dist/release/mesos/KEYS
>
> The JAR is up in Maven in a staging repository here:
> https://repository.apache.org/content/repositories/orgapachemesos-1104
>
> Please vote on releasing this package as Apache Mesos 0.27.2!
>
> The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a majority
> of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Mesos 0.27.2
> [ ] -1 Do not release this package because ...
>
> Thanks,
>
> MPark, Joris, Kapil

0.28.0 release

[Vinod Kone]

Hi folks,

I'm volunteering to be the Release Manager for 0.28.0. Joris and Kevin
Klues have kindly agreed to help me out. The plan is cut an RC tomorrow
03/01.

The dashboard for the release is here:
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12327751

*If you have a ticket marked with "Fix Version 028.0" and is not in
"Resolved" state, verify if it's a blocker for 0.28.0. If not, please unset
the Fix Version.*


Thanks,
Vinod

Re: [VOTE] Release Apache Mesos 0.27.2 (rc1)

[Vinod Kone]

+1 (binding)

Tested on ASF CI.

*Revision*: 3c9ec4a0f34420b7803848af597de00fedefe0e2

   - refs/tags/0.27.2-rc1

Configuration Matrix gcc clang
centos:7 --verbose --enable-libevent --enable-ssl
[image: Success]

[image: Not run]
--verbose
[image: Success]

[image: Not run]
ubuntu:14.04 --verbose --enable-libevent --enable-ssl
[image: Success]

[image: Success]

--verbose
[image: Success]

[image: Success]


On Mon, Feb 29, 2016 at 11:21 AM, Kapil Arya  wrote:

> +1 (binding)
>
> Successful CI builds for the following distros:
>
> amd64/centos/6
> amd64/centos/7
> amd64/debian/jessie
> amd64/ubuntu/precise
> amd64/ubuntu/trusty
> amd64/ubuntu/vivid
>
> Kapil
>
>
> On Fri, Feb 26, 2016 at 11:54 PM, Michael Park  wrote:
>
> > Hi all,
> >
> > Please vote on releasing the following candidate as Apache Mesos 0.27.2.
> >
> >
> > 0.27.2 includes the following:
> >
> >
> 
> >
> >- MESOS-4693  -
> >Variable shadowing in HookManager::slavePreLaunchDockerHook.
> >- MESOS-4711  -
> Race
> >condition in libevent poll implementation causes crash.
> >- MESOS-4754  - The
> >"executors" field is exposed under a backwards incompatible schema.
> >- MESOS-4687  -
> >Implement reliable floating point for scalar resources.
> >
> >
> > The CHANGELOG for the release is available at:
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.2-rc1
> >
> >
> 
> >
> > The candidate for Mesos 0.27.2 release is available at:
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz
> >
> > The tag to be voted on is 0.27.2-rc1:
> >
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.27.2-rc1
> >
> > The MD5 checksum of the tarball can be found at:
> >
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.md5
> >
> > The signature of the tarball can be found at:
> >
> >
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.asc
> >
> > The PGP key used to sign the release is here:
> > https://dist.apache.org/repos/dist/release/mesos/KEYS
> >
> > The JAR is up in Maven in a staging repository here:
> > https://repository.apache.org/content/repositories/orgapachemesos-1104
> >
> > Please vote on releasing this package as Apache Mesos 0.27.2!
> >
> > The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a
> > majority of at least 3 +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Mesos 0.27.2
> > [ ] -1 Do not release this package because ...
> >
> > Thanks,
> >
> > MPark, Joris, Kapil
> >
>

Re: [VOTE] Release Apache Mesos 0.24.2 (rc1)

[Kapil Arya]

+1 (binding)

Successful CI builds for the following distros:

amd64/centos/6
amd64/centos/7
amd64/debian/jessie
amd64/ubuntu/precise
amd64/ubuntu/trusty
amd64/ubuntu/vivid

Kapil

On Sat, Feb 27, 2016 at 1:12 AM, Michael Park  wrote:

> Hi all,
>
> Please vote on releasing the following candidate as Apache Mesos 0.24.2.
>
>
> 0.24.2 includes the following:
>
> 
>
>- Improvements
>   - Allocator filter performance
>   - Port Ranges performance
>   - UUID performance
>   - `/state` endpoint performance
>   - GLOG performance
>   - Configurable task/framework history
>   - Offer filter timeout fix for backlogged allocator
>
>
>- Bugs
>- SSL
>   - Libevent
>   - Fixed point resources math
>- HDFS
>   - Agent upgrade compatibility
>   - Health checks
>
> The CHANGELOG for the release is available at:
>
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.24.2-rc1
>
> 
>
> The candidate for Mesos 0.24.2 release is available at:
> https://dist.apache.org/repos/dist/dev/mesos/0.24.2-rc1/mesos-0.24.2.tar.gz
>
> The tag to be voted on is 0.24.2-rc1:
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.24.2-rc1
>
> The MD5 checksum of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.24.2-rc1/mesos-0.24.2.tar.gz.md5
>
> The signature of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.24.2-rc1/mesos-0.24.2.tar.gz.asc
>
> The PGP key used to sign the release is here:
> https://dist.apache.org/repos/dist/release/mesos/KEYS
>
> The JAR is up in Maven in a staging repository here:
> https://repository.apache.org/content/repositories/orgapachemesos-1110
>
> Please vote on releasing this package as Apache Mesos 0.24.2!
>
> The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a
> majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Mesos 0.24.2
> [ ] -1 Do not release this package because ...
>
> Thanks,
>
> Joris, Kapil, MPark
>

Re: [VOTE] Release Apache Mesos 0.26.1 (rc1)

[Kapil Arya]

+1 (binding)

Successful CI builds for the following distros:

amd64/centos/6
amd64/centos/7
amd64/debian/jessie
amd64/ubuntu/precise
amd64/ubuntu/trusty
amd64/ubuntu/vivid

Kapil

On Sat, Feb 27, 2016 at 12:26 AM, Michael Park  wrote:

> Hi all,
>
> Please vote on releasing the following candidate as Apache Mesos 0.26.1.
>
>
> 0.26.1 includes the following:
>
> 
>
>- Improvements
>   - `/state` endpoint performance
>   - systemd integration
>   - GLOG performance
>   - Configurable task/framework history
>   - Offer filter timeout fix for backlogged allocator
>
>
>- Bugs
>- SSL
>   - Libevent
>   - Fixed point resources math
>- HDFS
>   - Agent upgrade compatibility
>
> The CHANGELOG for the release is available at:
>
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.26.1-rc1
>
> 
>
> The candidate for Mesos 0.26.1 release is available at:
> https://dist.apache.org/repos/dist/dev/mesos/0.26.1-rc1/mesos-0.26.1.tar.gz
>
> The tag to be voted on is 0.26.1-rc1:
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.26.1-rc1
>
> The MD5 checksum of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.26.1-rc1/mesos-0.26.1.tar.gz.md5
>
> The signature of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.26.1-rc1/mesos-0.26.1.tar.gz.asc
>
> The PGP key used to sign the release is here:
> https://dist.apache.org/repos/dist/release/mesos/KEYS
>
> The JAR is up in Maven in a staging repository here:
> https://repository.apache.org/content/repositories/orgapachemesos-1106
>
> Please vote on releasing this package as Apache Mesos 0.26.1!
>
> The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a majority
> of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Mesos 0.26.1
> [ ] -1 Do not release this package because ...
>
> Thanks,
>
> Joris, Kapil, MPark
>

Re: [VOTE] Release Apache Mesos 0.27.2 (rc1)

[Kapil Arya]

+1 (binding)

Successful CI builds for the following distros:

amd64/centos/6
amd64/centos/7
amd64/debian/jessie
amd64/ubuntu/precise
amd64/ubuntu/trusty
amd64/ubuntu/vivid

Kapil


On Fri, Feb 26, 2016 at 11:54 PM, Michael Park  wrote:

> Hi all,
>
> Please vote on releasing the following candidate as Apache Mesos 0.27.2.
>
>
> 0.27.2 includes the following:
>
> 
>
>- MESOS-4693  -
>Variable shadowing in HookManager::slavePreLaunchDockerHook.
>- MESOS-4711  - Race
>condition in libevent poll implementation causes crash.
>- MESOS-4754  - The
>"executors" field is exposed under a backwards incompatible schema.
>- MESOS-4687  -
>Implement reliable floating point for scalar resources.
>
>
> The CHANGELOG for the release is available at:
>
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.2-rc1
>
> 
>
> The candidate for Mesos 0.27.2 release is available at:
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz
>
> The tag to be voted on is 0.27.2-rc1:
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.27.2-rc1
>
> The MD5 checksum of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.md5
>
> The signature of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.asc
>
> The PGP key used to sign the release is here:
> https://dist.apache.org/repos/dist/release/mesos/KEYS
>
> The JAR is up in Maven in a staging repository here:
> https://repository.apache.org/content/repositories/orgapachemesos-1104
>
> Please vote on releasing this package as Apache Mesos 0.27.2!
>
> The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a
> majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Mesos 0.27.2
> [ ] -1 Do not release this package because ...
>
> Thanks,
>
> MPark, Joris, Kapil
>

Re: [VOTE] Release Apache Mesos 0.25.1 (rc1)

[Kapil Arya]

+1 (binding)

Successful CI builds for the following distros:

amd64/centos/6
amd64/centos/7
amd64/debian/jessie
amd64/ubuntu/precise
amd64/ubuntu/trusty
amd64/ubuntu/vivid

Kapil

On Sat, Feb 27, 2016 at 12:53 AM, Michael Park  wrote:

> Hi all,
>
> Please vote on releasing the following candidate as Apache Mesos 0.25.1.
>
>
> 0.25.1 includes the following:
>
> 
>
>- Improvements
>   - `/state` endpoint performance
>   - systemd integration
>   - GLOG performance
>   - Configurable task/framework history
>   - Offer filter timeout fix for backlogged allocator
>
>
>- Bugs
>- SSL
>   - Libevent
>   - Fixed point resources math
>- HDFS
>   - Agent upgrade compatibility
>   - Health checks
>
> The CHANGELOG for the release is available at:
>
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.25.1-rc1
>
> 
>
> The candidate for Mesos 0.25.1 release is available at:
> https://dist.apache.org/repos/dist/dev/mesos/0.25.1-rc1/mesos-0.25.1.tar.gz
>
> The tag to be voted on is 0.25.1-rc1:
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.25.1-rc1
>
> The MD5 checksum of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.25.1-rc1/mesos-0.25.1.tar.gz.md5
>
> The signature of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/0.25.1-rc1/mesos-0.25.1.tar.gz.asc
>
> The PGP key used to sign the release is here:
> https://dist.apache.org/repos/dist/release/mesos/KEYS
>
> The JAR is up in Maven in a staging repository here:
> https://repository.apache.org/content/repositories/orgapachemesos-1108
>
> Please vote on releasing this package as Apache Mesos 0.25.1!
>
> The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a majority
> of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Mesos 0.25.1
> [ ] -1 Do not release this package because ...
>
> Thanks,
>
> Joris, Kapil, MPark
>

RE: How did the mesos master detect the disconnect of a framework (scheduler)

[Chong Chen]

Thanks, it is clear and helpful!

From: haosdent [mailto:haosd...@gmail.com]
Sent: Saturday, February 27, 2016 2:28 AM
To: user
Subject: Re: How did the mesos master detect the disconnect of a framework 
(scheduler)


Joseph's  explanation quite detail.
On Feb 27, 2016 3:33 AM, "Joseph Wu" 
> wrote:
Here's a brief(?) run-down:

  1.  
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/src/master/master.cpp#L5739-L5748
When a new framework is added, the master opens a socket connection with the 
framework.

 *   If this is a scheduler-driver-based framework, this is a plain socket 
connection.
 *   If this is a new HTTP API framework, the master uses the streaming 
HTTP connection instead.

  1.  The HTTP API framework's exit logic is simpler to explain.  When the 
streaming connection closes, the master considers the framework to have exited. 
 In the above code, see this chunk of code:
http.closed()
  .onAny(defer(self(), ::exited, framework->id(), http));
  2.  The scheduler-driver-based framework exit is a bit more involved:

 *   
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/3rdparty/libprocess/src/process.cpp#L1326
Libprocess has a SocketManager which, as the name suggests, managed sockets.  
Linking the master <-> framework spawns a socket here.
 *   
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/3rdparty/libprocess/src/process.cpp#L1394-L1400
Linking will install a dispatch loop, which continually reads the data from the 
socket until the socket closes.
 *   
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/3rdparty/libprocess/src/process.cpp#L1300-L1312
The dispatch loop calls "ignore_recv_data".  This detects when the socket 
closes and calls "SocketManager->close(s)".
 *   
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/3rdparty/libprocess/src/process.cpp#L1928
"SocketManager->close" will generate a libprocess "ExitedEvent".
 *   
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/src/master/master.cpp#L1352
Master has a listener for "ExitedEvent" which rate-limits these events.
 *   
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/src/master/master.cpp#L1161
The "ExitedEvent" eventually gets propagated to that ^ method (through a 
libprocess event visitor).
 *   
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/src/master/master.cpp#L1165
Finally, the framework gets removed.

Hope that helps,

~Joseph

On Fri, Feb 26, 2016 at 10:45 AM, Chong Chen 
> wrote:
Hi,
When a running framework was disconnected (manually terminated), the Mesos 
master will detect it immediately.  The master::exited() function will be 
invoked with log info “framework disconnected”.
I just wondering, how this disconnect detection was implemented in Mesos? I 
can’t find any place in mesos src directory where the Master::exit() function 
was called.

Thanks!

Best Regards,
Chong

URL for viewing persistent volumes

[Zhitao Li]

Hi,

Is there a HTTP url to list and view existing persistent volume created so
far? I'm running 0.27.1 and couldn't find how to obtain such info.

Thanks!

-- 
Cheers,

Zhitao Li

[MESOS][MARATHON][MESOS_DNS][DOCKER]

[Manoj Gunawardena]

Hi All,

In my environment running mesos-slave, mesos-master marathon and mesos-dns
in standalone mode.
I deployed mysql app to marathon to run as docker container.

MySql app configurations as follows.

{
"id": "mysql",
"cpus": 0.5,
"mem": 512,
"instances": 1,
"container": {
"type": "DOCKER",
"docker": {
"image": "mysql:5.6.27",
"network": "BRIDGE",
  "portMappings": [
{
  "containerPort": 3306,
  "hostPort": 32000,
  "protocol": "tcp"
}
]
}
},
"constraints": [
[
  "hostname",
  "UNIQUE"
]],
"env": {
"MYSQL_ROOT_PASSWORD": "password"
},
  "minimumHealthCapacity" :0,
  "maximumOverCapacity" : 0.0
}

Then I deploy app called mysql client. Mysql client app needs to connect to
mysql app.

mysql app config as follows.

{
"id": "mysqlclient",
"cpus": 0.3,
"mem": 512.0,
"cmd": "/scripts/create_mysql_dbs.sh",
"instances": 1,
"container": {
"type": "DOCKER",
"docker": {
"image": "mysqlclient:latest",
"network": "BRIDGE",
"portMappings": [{
"containerPort": 3306,
"hostPort": 0,
"protocol": "tcp"
}]
}
},
"env": {
"MYSQL_ENV_MYSQL_ROOT_PASSWORD": "password",
"MYSQL_PORT_3306_TCP_ADDR": "mysql.marathon.slave.mesos.",
"MYSQL_PORT_3306_TCP_PORT": "32000"
},
  "minimumHealthCapacity" :0,
  "maximumOverCapacity" : 0.0
}


>From the host machines I can ping to mysql.marathon.slave.mesos. But from
the mysql client docker conatiner I couldnt reach host
"mysql.marathon.slave.mesos."

My mess-dns configurations as follows.

{
  "zk": "zk://127.0.0.1:2181/mesos",
  "masters": ["127.0.0.1:5050"],
  "refreshSeconds": 60,
  "ttl": 60,
  "domain": "mesos",
  "port": 53,
  "resolvers": ["127.0.0.1"],
  "timeout": 5,
  "httpon": true,
  "dnson": true,
  "httpport": 8123,
  "externalon": true,
  "listener": "127.0.0.1",
  "SOAMname": "ns1.mesos",
  "SOARname": "root.ns1.mesos",
  "SOARefresh": 60,
  "SOARetry":   600,
  "SOAExpire":  86400,
  "SOAMinttl": 60,
  "IPSources": ["mesos", "host"]
}
*I tried with giving dns as parameter to mysql client app. But that also
not work. What is the reason for this?*

Thanks

Re: ansible modules?

[Ludovic Claude]

Hello,

I have created a module to control Marathon from Ansible. There is a
pending pull request to the Ansible project and it needs a little bit of
help from the community to get accepted.
https://github.com/ansible/ansible-modules-extras/pull/1258

Otherwise, you can use today the module by adding this role as a dependency
to your Ansible project:
https://github.com/ludovicc/ansible-marathon_app

You can see this plugin in action here:
https://github.com/LREN-CHUV/mip-microservices-infrastructure/blob/master/roles/docker-registry/tasks/main.yml

Ludovic

RE: Downloading s3 uris

[Aaron Carey]

Ah I think I've solved my own problem:

We are using the dind mesos slave container and hadn't mounted the host's 
/var/lib/mesos folder within the container so it wasn't showing up in the other 
containers.

Oops!

Thanks for your help,

Aaron


From: Aaron Carey [aca...@ilm.com]
Sent: 29 February 2016 08:53
To: user@mesos.apache.org
Subject: RE: Downloading s3 uris

Sorry apparently the inline image didn't work:

http://i.imgur.com/x1cPXvW.png


From: Aaron Carey [aca...@ilm.com]
Sent: 29 February 2016 08:50
To: user@mesos.apache.org
Subject: RE: Downloading s3 uris

To illustrate:

[X]

/var/lib/mesos/slaves/20160212-131720-1510021036-5050-1-S0/frameworks/20160212-131720-1510021036-5050-1-/executors/test.3d01cbd4-dcb4-11e5-868c-02420a4d969a/runs/25d29966-0515-4aa6-8d99-63a377aa68e8$
 ls -alh
total 8.0K
drwxr-xr-x 2 root root 4.0K Feb 26 18:10 .
drwxr-xr-x 3 root root 4.0K Feb 26 18:10 ..





From: Aaron Carey [aca...@ilm.com]
Sent: 29 February 2016 08:45
To: user@mesos.apache.org
Subject: RE: Downloading s3 uris

Yeah, I've managed to find the sandbox itself on disk, but it's empty, even 
though the file shows up in the web UI...

My task is a docker container and it doesn't show up in the container either

Any ideas?

Thanks!
Aaron


From: Joseph Wu [jos...@mesosphere.io]
Sent: 26 February 2016 18:27
To: user@mesos.apache.org
Subject: Re: Downloading s3 uris

The sandbox directory structure is a bit deep...  See the "Where is the 
sandbox?" section here: http://mesos.apache.org/documentation/latest/sandbox/


On Fri, Feb 26, 2016 at 10:15 AM, Aaron Carey 
> wrote:
A second question for you all..

I'm testing http uri downloads, and all the logs say that the file has 
downloaded (it even shows up in the mesos UI in the sandbox) but I can't find 
the file on disk anywhere. It doesn't appear in the docker container I'm 
running either (shouldn't it be in /mnt/mesos/sandbox?)

Am I missing something here?

Thanks for your help,

Aaron



From: Radoslaw Gruchalski [ra...@gruchalski.com]
Sent: 26 February 2016 17:41

To: user@mesos.apache.org; 
user@mesos.apache.org
Subject: Re: Downloading s3 uris

Just keep in mind that every execution of such command starts a jvm and is, 
generally, heavyweight. Use WebHDFS if you can.

Sent from Outlook Mobile




On Fri, Feb 26, 2016 at 9:13 AM -0800, "Shuai Lin" 
> wrote:

If you don't want to configure hadoop on your mesos slaves, the only workaround 
I see is to write a "hadoop" script and put it in your PATH. It need to support 
the following usage patterns:

- hadoop version
- hadoop fs -copyToLocal s3n://path /target/directory/

On Sat, Feb 27, 2016 at 12:31 AM, Aaron Carey 
> wrote:
I was trying to avoid generating urls for everything as this will complicate 
things a lot.

Is there a straight forward way to get the fetcher to do it directly?


From: haosdent [haosd...@gmail.com]
Sent: 26 February 2016 16:27
To: user
Subject: Re: Downloading s3 uris

I think still could pass AWSAccessKeyId if it is private? 
http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html

On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar 
> wrote:
In that case do we need to keep bucket/files public?

-Abhishek

From: Zhitao Li >
Reply-To: "user@mesos.apache.org" 
>
Date: Friday, 26 February 2016 at 8:23 AM
To: "user@mesos.apache.org" 
>
Subject: Re: Downloading s3 uris

Haven't directly used s3 download, but I think a workaround (if you don't care 
ACL about the files) is to use 
http
 url instead.
On Feb 26, 2016, at 8:17 AM, Aaron Carey 
> wrote:

I'm attempting to fetch files from s3 uris in mesos, but we're not using hdfs 
in our cluster... however I believe I need the client installed.

Is it possible to just have the client running without a full hdfs setup?

I haven't been able to find much information in the docs, could someone point 
me in the right direction?

Thanks!

Aaron




--
Best Regards,
Haosdent Huang

RE: Downloading s3 uris

[Aaron Carey]

Sorry apparently the inline image didn't work:

http://i.imgur.com/x1cPXvW.png


From: Aaron Carey [aca...@ilm.com]
Sent: 29 February 2016 08:50
To: user@mesos.apache.org
Subject: RE: Downloading s3 uris

To illustrate:

[X]

/var/lib/mesos/slaves/20160212-131720-1510021036-5050-1-S0/frameworks/20160212-131720-1510021036-5050-1-/executors/test.3d01cbd4-dcb4-11e5-868c-02420a4d969a/runs/25d29966-0515-4aa6-8d99-63a377aa68e8$
 ls -alh
total 8.0K
drwxr-xr-x 2 root root 4.0K Feb 26 18:10 .
drwxr-xr-x 3 root root 4.0K Feb 26 18:10 ..





From: Aaron Carey [aca...@ilm.com]
Sent: 29 February 2016 08:45
To: user@mesos.apache.org
Subject: RE: Downloading s3 uris

Yeah, I've managed to find the sandbox itself on disk, but it's empty, even 
though the file shows up in the web UI...

My task is a docker container and it doesn't show up in the container either

Any ideas?

Thanks!
Aaron


From: Joseph Wu [jos...@mesosphere.io]
Sent: 26 February 2016 18:27
To: user@mesos.apache.org
Subject: Re: Downloading s3 uris

The sandbox directory structure is a bit deep...  See the "Where is the 
sandbox?" section here: http://mesos.apache.org/documentation/latest/sandbox/


On Fri, Feb 26, 2016 at 10:15 AM, Aaron Carey 
> wrote:
A second question for you all..

I'm testing http uri downloads, and all the logs say that the file has 
downloaded (it even shows up in the mesos UI in the sandbox) but I can't find 
the file on disk anywhere. It doesn't appear in the docker container I'm 
running either (shouldn't it be in /mnt/mesos/sandbox?)

Am I missing something here?

Thanks for your help,

Aaron



From: Radoslaw Gruchalski [ra...@gruchalski.com]
Sent: 26 February 2016 17:41

To: user@mesos.apache.org; 
user@mesos.apache.org
Subject: Re: Downloading s3 uris

Just keep in mind that every execution of such command starts a jvm and is, 
generally, heavyweight. Use WebHDFS if you can.

Sent from Outlook Mobile




On Fri, Feb 26, 2016 at 9:13 AM -0800, "Shuai Lin" 
> wrote:

If you don't want to configure hadoop on your mesos slaves, the only workaround 
I see is to write a "hadoop" script and put it in your PATH. It need to support 
the following usage patterns:

- hadoop version
- hadoop fs -copyToLocal s3n://path /target/directory/

On Sat, Feb 27, 2016 at 12:31 AM, Aaron Carey 
> wrote:
I was trying to avoid generating urls for everything as this will complicate 
things a lot.

Is there a straight forward way to get the fetcher to do it directly?


From: haosdent [haosd...@gmail.com]
Sent: 26 February 2016 16:27
To: user
Subject: Re: Downloading s3 uris

I think still could pass AWSAccessKeyId if it is private? 
http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html

On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar 
> wrote:
In that case do we need to keep bucket/files public?

-Abhishek

From: Zhitao Li >
Reply-To: "user@mesos.apache.org" 
>
Date: Friday, 26 February 2016 at 8:23 AM
To: "user@mesos.apache.org" 
>
Subject: Re: Downloading s3 uris

Haven't directly used s3 download, but I think a workaround (if you don't care 
ACL about the files) is to use 
http
 url instead.
On Feb 26, 2016, at 8:17 AM, Aaron Carey 
> wrote:

I'm attempting to fetch files from s3 uris in mesos, but we're not using hdfs 
in our cluster... however I believe I need the client installed.

Is it possible to just have the client running without a full hdfs setup?

I haven't been able to find much information in the docs, could someone point 
me in the right direction?

Thanks!

Aaron




--
Best Regards,
Haosdent Huang

RE: Downloading s3 uris

[Aaron Carey]

To illustrate:

[X]

/var/lib/mesos/slaves/20160212-131720-1510021036-5050-1-S0/frameworks/20160212-131720-1510021036-5050-1-/executors/test.3d01cbd4-dcb4-11e5-868c-02420a4d969a/runs/25d29966-0515-4aa6-8d99-63a377aa68e8$
 ls -alh
total 8.0K
drwxr-xr-x 2 root root 4.0K Feb 26 18:10 .
drwxr-xr-x 3 root root 4.0K Feb 26 18:10 ..





From: Aaron Carey [aca...@ilm.com]
Sent: 29 February 2016 08:45
To: user@mesos.apache.org
Subject: RE: Downloading s3 uris

Yeah, I've managed to find the sandbox itself on disk, but it's empty, even 
though the file shows up in the web UI...

My task is a docker container and it doesn't show up in the container either

Any ideas?

Thanks!
Aaron


From: Joseph Wu [jos...@mesosphere.io]
Sent: 26 February 2016 18:27
To: user@mesos.apache.org
Subject: Re: Downloading s3 uris

The sandbox directory structure is a bit deep...  See the "Where is the 
sandbox?" section here: http://mesos.apache.org/documentation/latest/sandbox/


On Fri, Feb 26, 2016 at 10:15 AM, Aaron Carey 
> wrote:
A second question for you all..

I'm testing http uri downloads, and all the logs say that the file has 
downloaded (it even shows up in the mesos UI in the sandbox) but I can't find 
the file on disk anywhere. It doesn't appear in the docker container I'm 
running either (shouldn't it be in /mnt/mesos/sandbox?)

Am I missing something here?

Thanks for your help,

Aaron



From: Radoslaw Gruchalski [ra...@gruchalski.com]
Sent: 26 February 2016 17:41

To: user@mesos.apache.org; 
user@mesos.apache.org
Subject: Re: Downloading s3 uris

Just keep in mind that every execution of such command starts a jvm and is, 
generally, heavyweight. Use WebHDFS if you can.

Sent from Outlook Mobile




On Fri, Feb 26, 2016 at 9:13 AM -0800, "Shuai Lin" 
> wrote:

If you don't want to configure hadoop on your mesos slaves, the only workaround 
I see is to write a "hadoop" script and put it in your PATH. It need to support 
the following usage patterns:

- hadoop version
- hadoop fs -copyToLocal s3n://path /target/directory/

On Sat, Feb 27, 2016 at 12:31 AM, Aaron Carey 
> wrote:
I was trying to avoid generating urls for everything as this will complicate 
things a lot.

Is there a straight forward way to get the fetcher to do it directly?


From: haosdent [haosd...@gmail.com]
Sent: 26 February 2016 16:27
To: user
Subject: Re: Downloading s3 uris

I think still could pass AWSAccessKeyId if it is private? 
http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html

On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar 
> wrote:
In that case do we need to keep bucket/files public?

-Abhishek

From: Zhitao Li >
Reply-To: "user@mesos.apache.org" 
>
Date: Friday, 26 February 2016 at 8:23 AM
To: "user@mesos.apache.org" 
>
Subject: Re: Downloading s3 uris

Haven't directly used s3 download, but I think a workaround (if you don't care 
ACL about the files) is to use 
http
 url instead.
On Feb 26, 2016, at 8:17 AM, Aaron Carey 
> wrote:

I'm attempting to fetch files from s3 uris in mesos, but we're not using hdfs 
in our cluster... however I believe I need the client installed.

Is it possible to just have the client running without a full hdfs setup?

I haven't been able to find much information in the docs, could someone point 
me in the right direction?

Thanks!

Aaron




--
Best Regards,
Haosdent Huang