[VOTE] Release Apache Mesos 1.3.1 (rc1)
Hi all, Please vote on releasing the following candidate as Apache Mesos 1.3.1. The CHANGELOG for the release is available at: https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.3.1-rc1 The candidate for Mesos 1.3.1 release is available at: https://dist.apache.org/repos/dist/dev/mesos/1.3.1-rc1/mesos-1.3.1.tar.gz The tag to be voted on is 1.3.1-rc1: https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=1.3.1-rc1 The MD5 checksum of the tarball can be found at: https://dist.apache.org/repos/dist/dev/mesos/1.3.1-rc1/mesos-1.3.1.tar.gz.md5 The signature of the tarball can be found at: https://dist.apache.org/repos/dist/dev/mesos/1.3.1-rc1/mesos-1.3.1.tar.gz.asc The PGP key used to sign the release is here: https://dist.apache.org/repos/dist/release/mesos/KEYS The JAR is up in Maven in a staging repository here: https://repository.apache.org/content/repositories/orgapachemesos-1200 Please vote on releasing this package as Apache Mesos 1.3.1! The vote is open until *Aug 2, 2017* and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Mesos 1.3.1 [ ] -1 Do not release this package because ... Thanks, MPark
Re: Mesos Python Daemon Launch
This is generally not something we want users to do (i.e. leak something outside of their container). Mesos will kill all tasks in the cgroup if you're using cgroup isolation, so you would have to ensure the daemon escapes the cgroup. If you're using the posix isolation, you also need to be sure the process is not "reachable" via the process tree rooted at the executor (i.e. you'll need to double fork). On Fri, Jul 21, 2017 at 1:59 AM, Chawla,Sumitwrote: > No i am not using docker container. the process is being launched > directly on host > > Regards > Sumit Chawla > > > On Thu, Jul 20, 2017 at 10:56 PM, Timothy Chen wrote: > >> Are you using Docker containerizer or? >> >> Tim >> >> On Thu, Jul 20, 2017 at 10:50 PM, Chawla,Sumit >> wrote: >> > Any clue on this one? >> > >> > The python daemon is getting launched in different session and process >> > group. Not sure why its getting killed when the mesos slave is >> terminating >> > the framework. >> > >> > Regards >> > Sumit Chawla >> > >> > >> > On Wed, Jul 19, 2017 at 4:24 PM, Chawla,Sumit >> > wrote: >> > >> >> I am using Mesos 0.27. I am launching a Python Daemon from spark task. >> >> Idea is that this Daemon should keep running even when the mesos >> framework >> >> shuts dowm. However, I am facing issues in keeping this Python Daeamon >> >> process alive. The process is getting killed as soon as Mesos >> framework is >> >> dying. >> >> >> >> >> >> >> >> Regards >> >> Sumit Chawla >> >> >> >> >> > >
Re: How to deploy Hadoop on Mesos
Hadoop definitely seems to be on the list of frameworks for mesos: http://mesos.apache.org/documentation/latest/frameworks/ Has anyone recently tested getting it to work? On Thu, Jul 27, 2017 at 5:39 PM, Stephen Granwrote: > Hi, > > On 27/07/17 13:54, Traiano Welcome wrote: > > Hi Stephen > > > > > > On Thu, Jul 27, 2017 at 12:19 PM, Stephen Gran > wrote: > > Both spark and flink integrate natively with mesos, so no need for an > > intermediate yarn layer. For batch work, we're looking at the aurora > > project for job scheduling. > > > > > > > > I haven't looked at Aurora before - would you consider it a drop in > > replacement for hadoop for distributed batch workloads? > > It's definitely not a drop in replacement - they have very different > APIs and capabilities. What aurora gives us is a DSL to build the DAG > of an execution, and with a little work, some primitives to run those > executions. So, the functionality ends up being similar for 'just > batch', but the language, the bindings, etc are all very different. > > Cheers, > -- > Stephen Gran > Senior Technical Architect > > picture the possibilities | piksel.com >
Re: dynamic resource reservations
Typically a framework with no role cannot use resources reserved for another role. So, it would be interesting to see what happened. Also, please be aware that directly upgrading from 0.28.0 to 1.3.0 is not supported. You need to go from 0.28.0 to 1.0.0 and then jump from 1.0.0 to 1.3.0. On Fri, Jul 28, 2017 at 4:01 AM, Hendrik Haddorpwrote: > Hi, > > we did a migration from Mesos 0.28 to 1.3.0 and somehow it looks like one > framework "stole" resources another framework had reserved earlier. > Unfortunately I do not have any logs for the time frame so I'm not certain > what exactly happened. Currently we have one framework running with a role > and principal while the others are running with roles * and no principal. > Would a framework running with no role be able to use a resource that > another framework reserved for a specific role? > > regards, > Hendrik >
Latest Mesos (1.4.0) Fails to Build on Ubuntu 14.04.5
Hi All The latest version of mesos fails to build on Ubuntu 14.04.5. make fails with the following error: - . . . cp: cannot stat '../../src/python/cli/src/mesos/__init__.py': No such file or directory make[2]: *** [python/cli/src/mesos/__init__.py] Error 1 make[2]: Leaving directory `/opt/mesos/build/src' make[1]: *** [all] Error 2 make[1]: Leaving directory `/opt/mesos/build/src' make: *** [all-recursive] Error 1 - Installation steps leading up to the make failure: sudo apt-get update sudo apt-get install -y tar wget git git clone https://git-wip-us.apache.org/repos/asf/mesos.git sudo apt-get install -y openjdk-7-jdk sudo apt-get install -y autoconf libtool sudo apt-get -y install build-essential python-dev python-six python-virtualenv libcurl4-nss-dev libsasl2-dev libsasl2-modules maven libapr1-dev libsvn-dev cd mesos mkdir build ../configure make Does anyone know how to fix this? Thanks, Traiano
dynamic resource reservations
Hi, we did a migration from Mesos 0.28 to 1.3.0 and somehow it looks like one framework "stole" resources another framework had reserved earlier. Unfortunately I do not have any logs for the time frame so I'm not certain what exactly happened. Currently we have one framework running with a role and principal while the others are running with roles * and no principal. Would a framework running with no role be able to use a resource that another framework reserved for a specific role? regards, Hendrik
Containerizers & Executors
Hello, I am trying to understand some basic components of Mesos and i have made some notes corresponding to my conclusion. If somebody could check this out and review them, or can provide me some more information about the specific topic i would be thankful. Below you will find my notes. - Containerizers: 1. Mesos Containerizer - posix isolators - cgroups isolators 2. Docker containerizer - docker isolators 3. Custom containerizer - my isolators - Executors: Generally: Each executor has the minimum resources assigned by default (0.01 CPU & 32MB MEM) Executor expands its resources when a task is assigned (executor default resources + task resources) 1. Mesos commandExecutor - run shell commands or docker - Each executor is a container that can have only one task to execute, you can't specify group of tasks - Isolation between executors/containers so isolation between tasks because each task runs in one container 2. Mesos defaultExecutor - can run shell commands or a custom executor file e.g TestExecutor.java (from tests) - can execute one task per executor/container or multiple tasks (1 group). - No resource isolation between tasks of the same container 3. Custom Executor - ? So, i guess i can use one offer to run some tasks on the same agent with commandExecutor or with defaultExecutor…. But how would somebody specify if the offer corresponds to one agent or multiple agents? Thank you, Thodoris