[VOTE] Release Apache Mesos 1.3.1 (rc1)

[Michael Park]

Hi all,

Please vote on releasing the following candidate as Apache Mesos 1.3.1.

The CHANGELOG for the release is available at:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.3.1-rc1


The candidate for Mesos 1.3.1 release is available at:
https://dist.apache.org/repos/dist/dev/mesos/1.3.1-rc1/mesos-1.3.1.tar.gz

The tag to be voted on is 1.3.1-rc1:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=1.3.1-rc1

The MD5 checksum of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/1.3.1-rc1/mesos-1.3.1.tar.gz.md5

The signature of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/1.3.1-rc1/mesos-1.3.1.tar.gz.asc

The PGP key used to sign the release is here:
https://dist.apache.org/repos/dist/release/mesos/KEYS

The JAR is up in Maven in a staging repository here:
https://repository.apache.org/content/repositories/orgapachemesos-1200

Please vote on releasing this package as Apache Mesos 1.3.1!

The vote is open until *Aug 2, 2017* and passes if a majority of at least 3
+1 PMC votes are cast.

[ ] +1 Release this package as Apache Mesos 1.3.1
[ ] -1 Do not release this package because ...

Thanks,

MPark

Re: Mesos Python Daemon Launch

[Benjamin Mahler]

This is generally not something we want users to do (i.e. leak something
outside of their container).

Mesos will kill all tasks in the cgroup if you're using cgroup isolation,
so you would have to ensure the daemon escapes the cgroup. If you're using
the posix isolation, you also need to be sure the process is not
"reachable" via the process tree rooted at the executor (i.e. you'll need
to double fork).

On Fri, Jul 21, 2017 at 1:59 AM, Chawla,Sumit 
wrote:

> No i am not using docker container.  the process is being launched
> directly on host
>
> Regards
> Sumit Chawla
>
>
> On Thu, Jul 20, 2017 at 10:56 PM, Timothy Chen  wrote:
>
>> Are you using Docker containerizer or?
>>
>> Tim
>>
>> On Thu, Jul 20, 2017 at 10:50 PM, Chawla,Sumit 
>> wrote:
>> > Any clue on this one?
>> >
>> > The python daemon is getting launched in different session and process
>> > group.  Not sure why its getting killed when the mesos slave is
>> terminating
>> > the framework.
>> >
>> > Regards
>> > Sumit Chawla
>> >
>> >
>> > On Wed, Jul 19, 2017 at 4:24 PM, Chawla,Sumit 
>> > wrote:
>> >
>> >> I am using Mesos 0.27.  I am launching a Python Daemon from spark task.
>> >> Idea is that this Daemon should keep running even when the mesos
>> framework
>> >> shuts dowm. However, I am facing issues in keeping this Python Daeamon
>> >> process alive. The process is getting killed as soon as Mesos
>> framework is
>> >> dying.
>> >>
>> >>
>> >>
>> >> Regards
>> >> Sumit Chawla
>> >>
>> >>
>>
>
>

Re: How to deploy Hadoop on Mesos

[Traiano Welcome]

Hadoop definitely seems to be on the list of frameworks for mesos:

http://mesos.apache.org/documentation/latest/frameworks/

Has anyone recently tested getting it to work?




On Thu, Jul 27, 2017 at 5:39 PM, Stephen Gran 
wrote:

> Hi,
>
> On 27/07/17 13:54, Traiano Welcome wrote:
> > Hi Stephen
> >
> >
> > On Thu, Jul 27, 2017 at 12:19 PM, Stephen Gran 
> wrote:
> > Both spark and flink integrate natively with mesos, so no need for an
> > intermediate yarn layer.  For batch work, we're looking at the aurora
> > project for job scheduling.
> >
> >
> >
> > I haven't looked at Aurora before - would you consider it a drop in
> > replacement for hadoop for distributed batch workloads?
>
> It's definitely not a drop in replacement - they have very different
> APIs and capabilities.  What aurora gives us is a DSL to build the DAG
> of an execution, and with a little work, some primitives to run those
> executions.  So, the functionality ends up being similar for 'just
> batch', but the language, the bindings, etc are all very different.
>
> Cheers,
> --
> Stephen Gran
> Senior Technical Architect
>
> picture the possibilities | piksel.com
>

Re: dynamic resource reservations

[Vinod Kone]

Typically a framework with no role cannot use resources reserved for
another role. So, it would be interesting to see what happened.

Also, please be aware that directly upgrading from 0.28.0 to 1.3.0 is not
supported. You need to go from 0.28.0 to 1.0.0 and then jump from 1.0.0 to
1.3.0.

On Fri, Jul 28, 2017 at 4:01 AM, Hendrik Haddorp 
wrote:

> Hi,
>
> we did a migration from Mesos 0.28 to 1.3.0 and somehow it looks like one
> framework "stole" resources another framework had reserved earlier.
> Unfortunately I do not have any logs for the time frame so I'm not certain
> what exactly happened. Currently we have one framework running with a role
> and principal while the others are running with roles * and no principal.
> Would a framework running with no role be able to use a resource that
> another framework reserved for a specific role?
>
> regards,
> Hendrik
>

Latest Mesos (1.4.0) Fails to Build on Ubuntu 14.04.5

[Traiano Welcome]

Hi All

The latest version of mesos fails to build on Ubuntu 14.04.5. make fails
with the following error:

-
.
.
.
cp: cannot stat '../../src/python/cli/src/mesos/__init__.py': No such file
or directory
make[2]: *** [python/cli/src/mesos/__init__.py] Error 1
make[2]: Leaving directory `/opt/mesos/build/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/opt/mesos/build/src'
make: *** [all-recursive] Error 1
-

Installation steps leading up to the make failure:


sudo apt-get update
sudo apt-get install -y tar wget git

git clone https://git-wip-us.apache.org/repos/asf/mesos.git

sudo apt-get install -y openjdk-7-jdk
sudo apt-get install -y autoconf libtool
sudo apt-get -y install build-essential python-dev python-six
python-virtualenv libcurl4-nss-dev libsasl2-dev libsasl2-modules maven
libapr1-dev libsvn-dev
cd mesos

mkdir build
../configure
make



Does anyone know how to fix this?

Thanks,
Traiano

dynamic resource reservations

[Hendrik Haddorp]

Hi,

we did a migration from Mesos 0.28 to 1.3.0 and somehow it looks like 
one framework "stole" resources another framework had reserved earlier. 
Unfortunately I do not have any logs for the time frame so I'm not 
certain what exactly happened. Currently we have one framework running 
with a role and principal while the others are running with roles * and 
no principal. Would a framework running with no role be able to use a 
resource that another framework reserved for a specific role?


regards,
Hendrik

Containerizers & Executors

[Thodoris Zois]

Hello, 

I am trying to understand some basic components of Mesos and i have made some 
notes corresponding to my conclusion. If somebody could check this out and 
review them, or can provide me some more information about the specific topic i 
would be thankful. Below you will find my notes.


- Containerizers: 
1. Mesos Containerizer
- posix isolators 
- cgroups isolators

2. Docker containerizer
- docker isolators

3. Custom containerizer
- my isolators


- Executors:
Generally: Each executor has the minimum resources assigned by default 
(0.01 CPU & 32MB MEM)
   Executor expands its resources when a task is assigned (executor 
default resources + task resources)

1. Mesos commandExecutor
- run shell commands or docker
- Each executor is a container that can have only one task to execute, 
you can't specify group of tasks
- Isolation between executors/containers so isolation between tasks 
because each task runs in one container

2. Mesos defaultExecutor
- can run shell commands or a custom executor file e.g 
TestExecutor.java (from tests)
- can execute one task per executor/container or multiple tasks (1 
group).
- No resource isolation between tasks of the same container

3. Custom Executor
- ?


So, i guess i can use one offer to run some tasks on the same agent with 
commandExecutor or with defaultExecutor….
But how would somebody specify if the offer corresponds to one agent or 
multiple agents? 

Thank you,
Thodoris