Re: Welcome Greg Mann as a new committer and PMC member!

[Artem Harutyunyan]

Awesome!!! Congrats Greg, very well deserved.

Artem.

On Tue, Jun 13, 2017 at 2:42 PM Vinod Kone  wrote:

> Hi folks,
>
> Please welcome Greg Mann as the newest committer and PMC member of the
> Apache Mesos project.
>
> Greg has been an active contributor to the Mesos project for close to 2
> years now and has made many solid contributions. His biggest source code
> contribution to the project has been around adding authentication support
> for default executor. This was a major new feature that involved quite a
> few moving parts. Additionally, he also worked on improving the scheduler
> and executor APIs.
>
> Here is his more formal checklist for your perusal.
>
>
> https://docs.google.com/document/d/1S6U5OFVrl7ySmpJsfD4fJ3_R8JYRRc5spV0yKrpsGBw/edit
>
> Thanks,
> Vinod
>
>

Re: Using mesos' cfs limits on a docker container?

[Artem Harutyunyan]

Hi Mark,

Good to hear you figured it out. Can you please post curl errors that you
were observing and describe your image repository setup? I'd like to make
sure that we have instructions on how to mitigate those.

Artem.

On Sunday, August 14, 2016, Mark Hammons 
wrote:

> In specific, I wanted the process control capabilities of a mesos
> framework with custom schedulers and executors, but wanted to run my tasks
> in a framework definable environment (like running my tasks on a copy of
> Ubuntu 14 with certain libs installed). Using mixed-mode containerization
> worked with some fiddling, but it was painful in certain ways. The sandbox
> mounted in a mixed-mode container wasn't accessible from within the
> container thanks to selinux unless I ran the container in privileged mode
> and the cou limits per executor were no longer enforced unlike a mesos task
> with cfs isolation enabled. Further, setting up the default working
> directory and user was a pain.
>
> Unified mode (also called mesos containerizer for some reason) solves a
> lot of these issues, though using it with private image repositories was
> not as straightforward as the docker containerizer. I eventually had to use
> an image directory to get that working, cause curl kept throwing vague ssl
> errors(I'm fairly certain this is due to my private image repository not
> having https set up since it's a test environment).
>
> Once I get things set up and cleaned up I'll post a more involved guide on
> how to get this particular use case setup and running, especially a part on
> preparing your container image for use with mesos.
>
> Mark Edgar Hammons II - Research Engineer at BioEmergences
> 0603695656
>
> On 14 Aug 2016, at 18:11, Erik Weathers  > wrote:
>
> What was the problem and how did you overcome it?  (i.e. This would be a
> sad resolution to this thread for someone faced with this same problem in
> the future.)
>
> On Sunday, August 14, 2016, Mark Hammons  > wrote:
>
>> I finally got this working after fiddling with it all night. It works
>> great so far!
>>
>> Mark Edgar Hammons II - Research Engineer at BioEmergences
>> 0603695656
>>
>> On 14 Aug 2016, at 04:50, Joseph Wu  wrote:
>>
>> If you're not against running Docker containers without the Docker
>> daemon, try using the Unified containerizer.
>> See the latter half of this document: http://mesos.apache.org/docume
>> ntation/latest/mesos-containerizer/
>> 
>>
>> On Sat, Aug 13, 2016 at 7:02 PM, Mark Hammons <
>> mark.hamm...@inaf.cnrs-gif.fr> wrote:
>>
>>> Hi All,
>>>
>>>
>>>
>>> I was having a lot of success having mesos force sandboxed programs to
>>> work within cpu and memory constraints, but when I added docker into the
>>> mix, the cpu limitations go out the window (not sure about the memory
>>> limitations. Is there any way to mix these two methods of isolation? I'd
>>> like my executor/algorithm to run inside a docker container, but have that
>>> container's memory and cpu usage controlled by systemd/mesos.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Mark
>>> --
>>>
>>> Mark Hammons - +33 06 03 69 56 56
>>>
>>> Research Engineer @ BioEmergences
>>> 
>>>
>>> Lab Phone: 01 69 82 34 19
>>>
>>
>>

Re: Windows Build

[Artem Harutyunyan]

Hi Rinaldo,

It is possible to build and run the Agent on Windows from the 1.0.0-rc2
branch. The CMake files, as well as the ported Agent code were committed a
couple of weeks ago. Joseph is currently in process of setting up a Windows
build on our Jenkins. Folks in #windows channel in the community slack
should be able to help you in case you encounter problems (register here
https://mesos-slackin.herokuapp.com/).

Please keep in mind that the Windows support is still in an early alpha
phase.

Artem.

On Sat, Jul 9, 2016 at 6:19 AM, Rinaldo Digiorgio 
wrote:

> Hi,
> Would someone be able to suggest how to get started with building
> mesos on windows. I am under the assumption that the windows branch is not
> integrated into the current 1.* RC.
>
> Rinaldo

Re: Documentation for ACCEPT HTTP API

[Artem Harutyunyan]

- the list

Hi Giulio,

You probably remember me, I used to work with Predrag and Jakob and we met
when BenH was at CERN.

Please let me know how it goes with using HTTP API. The documentation is
indeed rough on edges so I am not surprised you're having questions. I'll
be happy to setup a google hangout with one of the engineers who worked on
the API in case you want some help.

Please let me know.

Cheers,
Artem.

On Monday, July 4, 2016, Giulio Eulisse  wrote:

> Dear all,
>
> I've started writing a simple framework using node.js and the HTTP
> Scheduler API. I've managed to subscribe to the event stream, parse
> messages and decline offers quite easily, however I'm having a bit of
> trouble accepting the offers and launching tasks, since I cannot find any
> complete example for the JSON format the various operations should have. I
> assume I can reverse engine er mesos.proto and do a bit of trial and error,
> but I was wondering if I was simply missing some proper documentation. Any
> suggestions?
>
> --
> Ciao,
> Giulio
>

Re: On a whim I sync'd my local master branch

[Artem Harutyunyan]

You're welcome! Kudos to Gilbert and Jie for landing this :).

On Wednesday, June 29, 2016, Charles Allen 
wrote:

> And ran the unit tests (through jenkins)
>
> [--] Global test environment tear-down
> [==] 1209 tests from 140 test cases ran. (360352 ms total)
> [  PASSED  ] 1209 tests.
>
>   YOU HAVE 6 DISABLED TESTS
>
> Finished: *SUCCESS*
>
>
> That's a result I hadn't seen in a few days. Good to see the progress,
> thanks mesos team
>

Re: Unstability on Mesos 0.27

[Artem Harutyunyan]

Hi Guillermo,

We would really like to help you, and to understand what the issues are.
Could you please send us all the logs you have so we can inspect them and
figure out what happened?

Artem.

On Thursday, March 17, 2016, Guillermo Rodriguez 
wrote:

> Update to 0.27.2 or wait for 0.28.0.
>
> I experienced many crashes as well with 0.27.1 due to crashes in the
> frameworks bringing down the whole cluster (swarm specially). Also problems
> in the resource precision that also crashed the servers and crashes when
> nodes disconnected.
>
> I really found 0.27 very unstable.
>
> Many of this problems were solved for 0.27.2 and my latest environment has
> proven way more stable. It is still not fully stable as the cluster crashed
> yesterday due to a crash in marathon, but way better overall and quick to
> recover.
>
> Luck!
> Guimo
>
>
> --
> *From*: "Klaus Ma"  >
> *Sent*: Thursday, March 17, 2016 1:36 PM
> *To*: user@mesos.apache.org
> 
> *Cc*: "Gabriel Menegatti"  >
> *Subject*: Re: Unstability on Mesos 0.27
>
> If Mesos daemon crashed, I'd suggest to log a JIRA and append more detail,
> e.g. steps, master/agent log.
>
> 
> Da (Klaus), Ma (??) | PMP® | Advisory Software Engineer
> Platform OpenSource Technology, STG, IBM GCG
> +86-10-8245 4084 | klaus1982...@gmail.com
>  | http://k82.me
>
> On Thu, Mar 17, 2016 at 8:26 AM, Vinod Kone  > wrote:
>>
>> Hey Gabriel,
>>
>> Could you share more details on what the crashes are and what your setup
>> is (docker containerizer?). Any logs (master, agent, application) that can
>> shed light would be useful to diagnose.
>>
>> On Wed, Mar 16, 2016 at 5:12 PM, Alfredo Carneiro <
>> alfr...@simbioseventures.com
>> > wrote:
>>>
>>> Hello guys,
>>>
>>> I am using Mesos 0.27 with different kinds of applications, such as,
>>> crawlers, databases and websites. However, I have faced many crashes and I
>>> couldn't find what it is the matter.
>>>
>>> We have 14 machines with 8Gb of ram and 4 cpu each. Usually, we run
>>> about 40 instance of our crawler, which they start stopping of nowhere (but
>>> the containers keep running). The day before yesterday we decided try to
>>> test our entire infrastrcuture and we scaled our crawler up to 110
>>> instances. Unfortunately, today we've faced a big crash that affected
>>> mainly our crawler and our databases.
>>>
>>> So, I am wondering if anyone else have the same problem, such as apps
>>> which crashes of nowhere or something else which could be related to some
>>> unstability on Mesos.
>>>
>>> --
>>> Alfredo Miranda
>>>
>>>
>>

Re: Official RPMs

[Artem Harutyunyan]

We use https://github.com/mesosphere/mesos-deb-packaging in conjunction
with some internal tooling (that Marco mentioned) to distribute package
building. So you should be able to find all the actual package building
code in that repo. Please let us know if you have questions.

Cheers,
Artem.

On Saturday, September 26, 2015, CCAAT <cc...@tampabay.rr.com> wrote:

> On 09/25/2015 07:36 PM, Marco Massenzio wrote:
>
>> Yes, the plan is definitely to make the tooling available to the
>> project: there is nothing "secret" about it - at the moment,
>> unfortunately, it relies on a bit of internal infrastructure and, well,
>> yesss, it's a bit too crafty to be ready for "external consumption"
>> but we're working on it!
>>
>
> Hello Marco,
>
> I' packaging up for gentoo linux. Just the itemized list of
> what/where/when you setup config files and such would be a keen help to my
> efforts. I can substitute in gentoo-centric tools, if you can provide a
> brief description on what your infra/tools are doing.
> A skeletal spec, if you like.
>
>
> James
>
>
>
>
>
>> /Marco Massenzio/
>> /Distributed Systems Engineer
>> http://codetrips.com/
>>
>> On Fri, Sep 25, 2015 at 11:33 AM, Zameer Manji <zma...@apache.org
>> <mailto:zma...@apache.org>> wrote:
>>
>> Could mesosphere donate their tooling for packaging mesos to the
>> project? This way any project member or contributor can build
>> packages and it can be apart of the release process.
>>
>> On Fri, Sep 25, 2015 at 10:53 AM, Artem Harutyunyan
>> <ar...@mesosphere.io <mailto:ar...@mesosphere.io>> wrote:
>>
>> The repositories have been updated yesterday, and the downloads
>> page
>> was updated today. Mesos 0.24 packages are now available at
>> https://mesosphere.com/downloads/. Thank you very much for your
>> patience!
>>
>> Cheers,
>> Artem.
>>
>> On Tue, Sep 22, 2015 at 11:02 AM, Marco Massenzio
>> <ma...@mesosphere.io <mailto:ma...@mesosphere.io>> wrote:
>>  > Hi guys,
>>  >
>>  > just wanted to let you all know that we (Mesosphere) fully
>> intend to
>>  > continue supporting distributing binary packages for the
>> current set of
>>  > supported OSes (namely, Ubuntu / Debian / RedHat / CentOS as
>> listed in [0]).
>>  >
>>  > Sorry that 0.24 slipped through the cracks, the person who
>> actually takes
>>  > care of that and knows the magic incantations has been
>> unwell, and a number
>>  > of other competing priorities got in the way - we will
>> eventually be
>>  > automating the process, so that downloadable binary packages
>> are created out
>>  > of each release/RC build (and, possibly, even more often)
>> without pesky
>>  > humans getting in the way :) but this may take some time.
>>  > We're building the 0.24 ones as we speak, so please bear with
>> us while this
>>  > gets done.
>>  >
>>  > Any questions / suggestions, we'd love to hear those too!
>>  >
>>  > [0] https://mesosphere.com/downloads/
>>  >
>>  > Marco Massenzio
>>  > Distributed Systems Engineer
>>  > http://codetrips.com
>>  >
>>  > On Tue, Sep 22, 2015 at 10:54 AM, CCAAT
>> <cc...@tampabay.rr.com <mailto:cc...@tampabay.rr.com>> wrote:
>>  >>
>>  >> On 09/21/2015 03:01 PM, Vinod Kone wrote:
>>  >>>
>>  >>> +Jake Farrell
>>  >>>
>>  >>> The mesos project doesn't publish platform dependent
>> artifacts.  We
>>  >>> currently only publish platform independent artifacts like
>> JAR (to
>>  >>> apache maven) and interface EGG (to PyPI).
>>  >>>
>>  >>> Recently we made the decision
>>  >>>
>> <http://www.mail-archive.com/dev%40mesos.apache.org/msg33148.html
>> >
>> for
>>  >>> the project to not officially support different language
>> (java, python)
>>  >>> framework libraries going forward (l

Re: Official RPMs

[Artem Harutyunyan]

The repositories have been updated yesterday, and the downloads page
was updated today. Mesos 0.24 packages are now available at
https://mesosphere.com/downloads/. Thank you very much for your
patience!

Cheers,
Artem.

On Tue, Sep 22, 2015 at 11:02 AM, Marco Massenzio  wrote:
> Hi guys,
>
> just wanted to let you all know that we (Mesosphere) fully intend to
> continue supporting distributing binary packages for the current set of
> supported OSes (namely, Ubuntu / Debian / RedHat / CentOS as listed in [0]).
>
> Sorry that 0.24 slipped through the cracks, the person who actually takes
> care of that and knows the magic incantations has been unwell, and a number
> of other competing priorities got in the way - we will eventually be
> automating the process, so that downloadable binary packages are created out
> of each release/RC build (and, possibly, even more often) without pesky
> humans getting in the way :) but this may take some time.
> We're building the 0.24 ones as we speak, so please bear with us while this
> gets done.
>
> Any questions / suggestions, we'd love to hear those too!
>
> [0] https://mesosphere.com/downloads/
>
> Marco Massenzio
> Distributed Systems Engineer
> http://codetrips.com
>
> On Tue, Sep 22, 2015 at 10:54 AM, CCAAT  wrote:
>>
>> On 09/21/2015 03:01 PM, Vinod Kone wrote:
>>>
>>> +Jake Farrell
>>>
>>> The mesos project doesn't publish platform dependent artifacts.  We
>>> currently only publish platform independent artifacts like JAR (to
>>> apache maven) and interface EGG (to PyPI).
>>>
>>> Recently we made the decision
>>>  for
>>> the project to not officially support different language (java, python)
>>> framework libraries going forward (likely after 1.0). The project will
>>> only support C++ libraries which will live in the repo and link to other
>>> language libraries from our website.
>>>
>>> The main reason was that the PMC lacks the expertise to support various
>>> language bindings and hence we wanted to remove the support burden.
>>>
>>> Option #1) It looks like we could do a similar thing with RPMs/DEBs,
>>> i.e., link to 3rd party artifacts from the project website. Similar to
>>> the client library authors, we could hold package maintainers
>>> accountable by providing guidelines.
>>>
>>> Option #2) Since the project officially supports certain platforms
>>> (Ubuntu, CentOS, OSX) and continuously tests this via CI, we could've
>>> the CI continuously build and upload the packages. Not sure what's ASF
>>> stance on this is. I filed a ticket
>>>  a while ago with
>>> INFRA regarding something similar, but never received any response.
>>>
>>> Personally, with the direction the project is headed towards, I prefer
>>> #1.
>>
>>
>> +1 (Option #1)
>>
>> This 'Option #1' approach will require the core dev team to clearly convey
>> what is needed for any OS supported, not the chosen OSes for support. Right
>> now, I'm having to parse many documents to figure out how to extend the
>> gentoo ebuild for mesos. And where to cut off what I do in the ebuilds and
>> what to put into the configuration documents for gentoo. Naturally the
>> minimial is only what should be in the the gentoo ebuild; with other items,
>> such as HDFS as a compiler option. Once I get the btrfs/ceph work
>> stabilized, there will be a compile time option for btrfs/ceph with the
>> gentoo ebuild. Other distros that are not going that
>> way should have other Distributed File System options 'baked into' their
>> installation on that OS.
>>
>>
>>
>> 'Option #1' sets the stage for many OSes to be supported and the core dev
>> team only has to support  a single document to clarify what any distro needs
>> to robustly support mesos for their user community. This will facilitate a
>> wider variety of experimentation, at the companion repos too. This  Option
>> #1 approach will further accelerate adoption of Mesos on a very wide variety
>> of platforms and architectures, imho. It sets the stage for valid benchmark
>> performance comparison between distros; something that the gentoo community
>> will no doubt win
>>
>> ;-)
>>
>> James
>>
>>
>>
>>
>>>
>>> On Sat, Sep 19, 2015 at 3:39 AM, Carlos Sanchez >> > wrote:
>>>
>>> I'm using the same repo with some changes to build SSL enabled
>>> packages
>>>
>>>
>>> https://github.com/carlossg/mesos-deb-packaging/compare/master...carlossg:ssl
>>>
>>>
>>> On Sat, Sep 19, 2015 at 4:22 AM, Rad Gruchalski
>>> > wrote:
>>>  > Should be rather easy to package it with this little tool from
>>> Mesosphere:
>>>  > https://github.com/mesosphere/mesos-deb-packaging. I’ve done it
>>> myself for
>>>  > ubuntu 12.04 and 14.04.
>>>  > The only thing that needs to be changed are the 

Re: API client libraries

[Artem Harutyunyan]

Thanks for bringing this up, Vinod!

We have to make sure that there are reference library implementations for
at least Python, Java, and Go. They may end up being owned and maintained
by the community, but I feel that Mesos developers should at least
kickstart the process and incubate those libraries. Once the initial
implementations of those libraries are in place we should also make sure to
have reference usage examples for them (like we do right now with Rendler).

In any case, this is a very important topic so I will go ahead and add it
to tomorrow's community sync agenda.

Cheers,
Artem.

On Wed, Sep 2, 2015 at 11:49 AM, Vinod Kone  wrote:

> Hi folks,
>
> Now that the v1 scheduler HTTP API (beta) is on the verge of being
> released, I wanted to open up the discussion about client libraries for the
> API. Mainly around support and home for the libs.
>
> One idea is that, going forward, the only supported client library would be
> C++ library which will live in the mesos repo. All other client libraries
> (java, python, go etc) will not be officially supported but linked to from
> our webpage/docs.
>
> Pros:
> --> The PMC/committers won't have the burden to maintain client libraries
> in languages we don't have expertise in.
> --> Gives more control (reviews, releases) and attribution (could live in
> the author's org's or personal repo) to 3rd party client library authors
>
> Cons:
> --> Might be a step backward because we would be officially dropping
> support for Java and Python. This is probably a good thing?
> --> No quality control of the libraries by the PMC. Need co-ordination with
> library authors to incorporate API changes. Could lead to bad user
> experience.
>
> I've taken a quick look at what other major projects do and it looks like
> most of them officially support a few api libs and then link to 3rdparty
> libs.
>
> Docker
> <
> https://docs.docker.com/reference/api/remote_api_client_libraries/#docker-remote-api-client-libraries
> >:
> No official library? Links to 3rd party libs.
>
> GitHub : Official support for
> Ruby, .Net, Obj-C. Links to 3rd party libs.
>
> Google : All
> official libraries? No links to 3rd party libs?
>
> K8S : Official
> support for Go. Links to 3rd party libs.
>
> Twitter : Official
> support for Java. Links to 3rd party libs.
>
>
> Is this the way we want to go? This does mean we won't need a mesos/commons
> repo because the project would be not be officially supporting 3rd party
> libs. The supported C++ libs will live in the mesos repo.
>
> Thoughts?
>