Re: Welcome Zhitao Li as Mesos Committer and PMC Member

2018-03-12 Thread Avinash Sridharan
Congrats Zhitao !! Thanks for all the hard work and contributions !!

On Mon, Mar 12, 2018 at 3:28 PM, Chun-Hung Hsiao <chhs...@mesosphere.io>
wrote:

> Congrats Zhitao!
>
> On Mon, Mar 12, 2018 at 2:51 PM, Benjamin Mahler <bmah...@apache.org>
> wrote:
>
>> Welcome Zhitao! Thanks for your contributions so far
>>
>> On Mon, Mar 12, 2018 at 2:02 PM, Gilbert Song <gilb...@apache.org> wrote:
>>
>> > Hi,
>> >
>> > I am excited to announce that the PMC has voted Zhitao Li as a new
>> > committer and member of PMC for the Apache Mesos project. Please join
>> me to
>> > congratulate Zhitao!
>> >
>> > Zhitao has been an active contributor to Mesos for one and a half years.
>> > His main contributions include:
>> >
>> >- Designed and implemented Container Image Garbage Collection (
>> >MESOS-4945 <https://issues.apache.org/jira/browse/MESOS-4945>);
>> >- Designed and implemented part of the HTTP Operator API (MESOS-6007
>> ><https://issues.apache.org/jira/browse/MESOS-6007>);
>> >- Reported and fixed a lot of bugs
>> ><https://issues.apache.org/jira/issues/?jql=type%20%3D%20Bug
>> %20AND%20(assignee%20%3D%20zhitao%20OR%20reporter%20%3D%
>> 20zhitao%20)%20ORDER%20BY%20priority%20>
>> >.
>> >
>> > Zhitao spares no effort to improve the project quality and to propose
>> > ideas. Thank you Zhitao for all contributions!
>> >
>> > Here is his committer candidate checklist for your perusal:
>> > https://docs.google.com/document/d/1HGz7iBdo1Q9z9c8fNRgNNLnj0XQ_
>> > PhDhjXLAfOx139s/
>> >
>> > Congrats Zhitao!
>> >
>> > Cheers,
>> > Gilbert
>> >
>>
>
>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Welcome Chun-Hung Hsiao as Mesos Committer and PMC Member

2018-03-11 Thread Avinash Sridharan
 Very well deserved Chun !!
On Sun, Mar 11, 2018 at 4:32 PM Qian Zhang  wrote:

> Congratulations Chun!
>
>
> Regards,
> Qian Zhang
>
> On Sun, Mar 11, 2018 at 2:08 PM, Sam  wrote:
>
>> Congrats Chun
>>
>>
>> Regards,
>> Sam Chen | APJ Country Director | DC/OS Evangelist
>>
>> [image: Watch the Video]
>> 
>>  [image:
>> .]Build and run modern apps
>> at scale using DC/OS
>> 
>>
>>
>> On Mar 11, 2018, at 1:14 PM, Jie Yu  wrote:
>>
>> Chun
>>
>>
>


Re: Dedicated ip to task

2017-12-11 Thread Avinash Sridharan
Hi Marc,
 In that demo since we new that those were the only containers that were
getting launched, and the containers are getting launched on a bridge
network using a `host-local` IPAM the ip-addresses were predictable.

However, in a real setup you would probably need to rely on something like
`mesos-dns` to peg your backend to a DNS name rather than an IP address.
The DNS name will be predictable and DNS resolution would end up resolving
to the right backend.

If you are looking at Marathon + Mesos, the same can be achieved by
`marathon-lb` (https://github.com/mesosphere/marathon-lb). The limitation
here being that `marathon-lb` will peg against Marathon tasks rather than
Mesos tasks. So if you want use your own Mesos frameworks to launch tasks
that need to be load-balanced you won't be able to use `marathon-lb`.

On Mon, Dec 11, 2017 at 3:37 PM, Jie Yu <yujie@gmail.com> wrote:

> + Avinash
>
> On Mon, Dec 11, 2017 at 2:21 PM, Marc Roos <m.r...@f1-outsourcing.eu>
> wrote:
>
>>
>>
>> In this https://youtu.be/0UMCoojACOs?t=1737 cni video of Avinash
>> Sridharan, he has a haproxy setup with two webservers on different
>> networks. But how does he know what these ip adresses will be, so he can
>> configure them in the proxy?
>>
>>
>>
>>
>>
>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: cni macvlan config

2017-12-11 Thread Avinash Sridharan
On Mon, Dec 11, 2017 at 1:31 PM, Marc Roos <m.r...@f1-outsourcing.eu> wrote:

>
> I think I have maybe a syntax error or so. On the other hand I also
> don’t see anything significant in the slave WARN, INFO and ERROR log.
>
>
> This one is not working. I can create a manually a macvlan interface at
> the host on eth0. And dhcp is running on this network.
> [@m01 ~]# cat  /etc/mesos-cni/15-cni-storagelan.conf
> {
>   "name": "cni-storagelan",
>   "type": "macvlan",
>   "master": "eth0",
>   "ipam": {
> "type": "host-local",
> "subnet": "192.168.10.0/24"
>   }
> }
>
> You need to provide communication between your container and the agent.
The executor within the container and the agent need to be able to register
with each other. This is done over IP. With a MacVLAN interface if you run
the agent on the master and the container on a slave, a master can never
talk to the slave ( the kernel won't allow it) hence the container launch
will fail due to a lack of registration. To use Macvlan make sure that the
agent is also listening on a slave interface having the same master as the
containers launched on that node.



> This one is working and giving the app an ip
> [@m01 ~]# cat  /etc/mesos-cni/30-cni-bridge-test.conf
> {
> "name": "cni-bridge-test",
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipam": {
> "type": "host-local",
> "subnet": "192.168.0.0/16",
> "routes": [
> { "dst":
>   "0.0.0.0/0" }
> ]
>   }
> }
>
> {
>   "id": "hello-play",
>   "cmd": "./Hello-*/bin/hello -Dhttp.port=80",
>   "cpus": 0.1,
>   "mem": 32,
>   "disk": 0,
>   "instances": 1,
>   "acceptedResourceRoles": [
> "*"
>   ],
>   "ipAddress": {"networkName": "cni-bridge-test"},
>   "uris": [
> "http://192.168.10.2/cobbler/PlayHello.zip;
>   ]
> }
>
>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Marathon 1.5 necessary for testing with cni

2017-10-17 Thread Avinash Sridharan
Hi Marc,
The Marathon app def:
""networks": [ { "mode": "container", "name": "mynet" } ],"

is the new API that go introduced in 1.5.

Prior to 1.5 you would do:
"ipAddress": [{"networkName":"mynet"}]

@jdef ^^?


On Tue, Oct 17, 2017 at 1:24 PM, Marc Roos <m.r...@f1-outsourcing.eu> wrote:

>
> I want to test a bit with mesos, docker images on the mesos
> containerizer and cni. Just give a container an ip.
>
> This is not possible with marathon <1.5? I am using the mesosphere repo
> for el7, is there some repo that has 1.5?
>
>
> I tried to add this to the container configuration
>
> "networks": [ { "mode": "container", "name": "mynet" } ],
>
>
> [@m03 ~]# cat /etc/mesos-cni/10-mynet.conf
> {
>  "cniVersion": "0.2.0",
>  "name": "mynet",
>  "type": "bridge",
>  "bridge": "cni0",
>  "isGateway": true,
>  "ipMasq": true,
>  "ipam": {
>"type": "host-local",
>"subnet": "10.22.0.0/16",
>"routes": [
>  { "dst": "0.0.0.0/0" }
>]
>  }
> }
>
>
> On the github page of marathon
> (https://github.com/mesosphere/marathon/blob/master/docs/docs/networking
> .md) they are writing 1.5 is necessary for this?
>
> More info I got from this demo https://youtu.be/0UMCoojACOs?t=1411
>
>
> CentOS7 3.10.0-693.2.2.el7.x86_64
> mesos-1.4.0-2.0.1.x86_64
> marathon-1.4.8-1.0.660.el7.x86_64
> containernetworking-cni-0.5.1-1.el7.x86_64
> mesosphere-zookeeper-3.4.6-0.1.20141204175332.centos7.x86_6
>
>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Welcome James Peach as a new committer and PMC memeber!

2017-09-06 Thread Avinash Sridharan
Congrats James !! Very well deserved.
On Wed, Sep 6, 2017 at 5:47 PM Benjamin Mahler  wrote:

> Thanks for all that you've done so far for the project James!
>
> On Wed, Sep 6, 2017 at 2:08 PM, Yan Xu  wrote:
>
>> Hi Mesos devs and users,
>>
>> Please welcome James Peach as a new Apache Mesos committer and PMC member.
>>
>> James has been an active contributor to Mesos for over two years now. He
>> has made many great contributions to the project which include XFS disk
>> isolator, improvement to Linux capabilities support and IPC namespace
>> isolator. He's super active on the mailing lists and slack channels, always
>> eager to help folks in the community and he has been helping with a lot of
>> Mesos reviews as well.
>>
>> Here is his formal committer candidate checklist:
>>
>>
>> https://docs.google.com/document/d/19G5zSxhrRBdS6GXn9KjCznjX3cp0mUbck6Jy1Hgn3RY/edit?usp=sharing
>> 
>>
>> Congrats James!
>>
>> Yan
>>
>>


Re: Welcome Greg Mann as a new committer and PMC member!

2017-06-13 Thread Avinash Sridharan
Awesome !! Congrats Greg !!

On Tue, Jun 13, 2017 at 2:44 PM, Neil Conway <neil.con...@gmail.com> wrote:

> Congratulations, Greg!! Very well-deserved. Looking forward to
> continuing to work with you on the project.
>
> Neil
>
>
> On Tue, Jun 13, 2017 at 2:42 PM, Vinod Kone <vinodk...@apache.org> wrote:
> > Hi folks,
> >
> > Please welcome Greg Mann as the newest committer and PMC member of the
> > Apache Mesos project.
> >
> > Greg has been an active contributor to the Mesos project for close to 2
> > years now and has made many solid contributions. His biggest source code
> > contribution to the project has been around adding authentication support
> > for default executor. This was a major new feature that involved quite a
> few
> > moving parts. Additionally, he also worked on improving the scheduler and
> > executor APIs.
> >
> > Here is his more formal checklist for your perusal.
> >
> > https://docs.google.com/document/d/1S6U5OFVrl7ySmpJsfD4fJ3_
> R8JYRRc5spV0yKrpsGBw/edit
> >
> > Thanks,
> > Vinod
> >
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Welcome Gilbert Song as a new committer and PMC member!

2017-05-24 Thread Avinash Sridharan
Congrats Gilbert !! Very well deserved !!

On Wed, May 24, 2017 at 11:56 AM, Timothy Chen <tnac...@gmail.com> wrote:

> Congrats! Rocking the containerizer world!
>
> Tim
>
> On Wed, May 24, 2017 at 11:23 AM, Zhitao Li <zhitaoli...@gmail.com> wrote:
> > Congrats Gilbert!
> >
> > On Wed, May 24, 2017 at 11:08 AM, Yan Xu <y...@jxu.me> wrote:
> >
> >> Congrats! Well deserved!
> >>
> >> ---
> >> Jiang Yan Xu <y...@jxu.me> | @xujyan <https://twitter.com/xujyan>
> >>
> >> On Wed, May 24, 2017 at 10:54 AM, Vinod Kone <vinodk...@apache.org>
> wrote:
> >>
> >>> Congrats Gilbert!
> >>>
> >>> On Wed, May 24, 2017 at 1:32 PM, Neil Conway <neil.con...@gmail.com>
> >>> wrote:
> >>>
> >>> > Congratulations Gilbert! Well-deserved!
> >>> >
> >>> > Neil
> >>> >
> >>> > On Wed, May 24, 2017 at 10:32 AM, Jie Yu <yujie@gmail.com>
> wrote:
> >>> > > Hi folks,
> >>> > >
> >>> > > I' happy to announce that the PMC has voted Gilbert Song as a new
> >>> > committer
> >>> > > and member of PMC for the Apache Mesos project. Please join me to
> >>> > > congratulate him!
> >>> > >
> >>> > > Gilbert has been working on Mesos project for 1.5 years now. His
> main
> >>> > > contribution is his work on unified containerizer, nested container
> >>> (aka
> >>> > > Pod) support. He also helped a lot of folks in the community
> regarding
> >>> > their
> >>> > > patches, questions and etc. He also played an important role
> >>> organizing
> >>> > > MesosCon Asia last year and this year!
> >>> > >
> >>> > > His formal committer checklist can be found here:
> >>> > > https://docs.google.com/document/d/1iSiqmtdX_0CU-YgpViA6r6PU_
> >>> > aMCVuxuNUZ458FR7Qw/edit?usp=sharing
> >>> > >
> >>> > > Welcome, Gilbert!
> >>> > >
> >>> > > - Jie
> >>> >
> >>>
> >>
> >>
> >
> >
> > --
> > Cheers,
> >
> > Zhitao Li
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Mesos (and Marathon) port mapping

2017-04-07 Thread Avinash Sridharan
On Fri, Apr 7, 2017 at 6:36 AM, Thomas HUMMEL <thomas.hum...@pasteur.fr>
wrote:

> On 03/31/2017 07:51 PM, Jie Yu wrote:
>
>> Tomek and Olivier,
>>
>> The bridge network support (with port mapping) has been added to Mesos
>> 1.2. See this doc for more details how to use it:
>> https://github.com/apache/mesos/blob/master/docs/cni.md#a-
>> port-mapper-plugin-for-cni-networks
>>
>> TL;DR: we developed a CNI port mapper plugin (DNAT) in Mesos repo, and
>> uses a delegation model in CNI. For the bridge CNI plugin, you can simply
>> use the default bridge plugin in CNI repo (https://github.com/containern
>> etworking/cni). @avinash can explain more here.
>>
>
> Am I right thinking we can achieve the same as
> MesosContainerizer/CNI/Bridge network plugin wrapped by the port mapper
> plugin with the DockerContainerizer ?
>
> Yes, MesosContainerizer/CNI/Bridge network plugin + CNI port-mapper plugin
is identical to the docker's BRIDGE mode networking.

> Thanks.
>
> --
> TH
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Welcome Kevin Klues as a Mesos Committer and PMC member!

2017-03-01 Thread Avinash Sridharan
Awesome !! Congrats Kevin !!

On Wed, Mar 1, 2017 at 2:07 PM, Jie Yu <yujie@gmail.com> wrote:

> Congrats! Kevin! Well deserved!
>
> On Wed, Mar 1, 2017 at 2:05 PM, Benjamin Mahler <bmah...@apache.org>
> wrote:
>
> > Hi all,
> >
> > Please welcome Kevin Klues as the newest committer and PMC member of the
> > Apache Mesos project.
> >
> > Kevin has been an active contributor in the project for over a year, and
> in
> > this time he made a number of contributions to the project: Nvidia GPU
> > support [1], the containerization side of POD support (new container init
> > process), and support for "attach" and "exec" of commands within running
> > containers [2].
> >
> > Also, Kevin took on an effort with Haris Choudhary to revive the CLI [3]
> > via a better structured python implementation (to be more accessible to
> > contributors) and a more extensible architecture to better support adding
> > new or custom subcommands. The work also adds a unit test framework for
> the
> > CLI functionality (we had no tests previously!). I think it's great that
> > Kevin took on this much needed improvement with Haris, and I'm very much
> > looking forward to seeing this land in the project.
> >
> > Here is his committer eligibility document for perusal:
> > https://docs.google.com/document/d/1mlO1yyLCoCSd85XeDKIxTYyboK_
> > uiOJ4Uwr6ruKTlFM/edit
> >
> > Thanks!
> > Ben
> >
> > [1] http://mesos.apache.org/documentation/latest/gpu-support/
> > [2]
> > https://docs.google.com/document/d/1nAVr0sSSpbDLrgUlAEB5hKzCl482N
> > SVk8V0D56sFMzU
> > [3]
> > https://docs.google.com/document/d/1r6Iv4Efu8v8IBrcUTjgYkvZ32WVsc
> > gYqrD07OyIglsA/
> >
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: [VOTE] Release Apache Mesos 1.0.3 (rc1)

2017-01-24 Thread Avinash Sridharan
-1 (Non-binding)

I realized that we missed the following commit in 1.0.3:
https://github.com/apache/mesos/commit/3e52a107c4073778de9c14bf5fcdeb6e342821aa

This commit fixes a bug seen in CoreOS because of which containers cannot
be launched on a CNI network. While specific to CoreOS, the bug can
manifest itself in any distro that does not setup `/etc/hosts`/ and
`/etc/hostname` by default. Hence, wanted this commit to be backported to
1.0.3.

Thanks,
Avinash


On Thu, Jan 19, 2017 at 8:43 AM, Vinod Kone <vinodk...@apache.org> wrote:

> Hi all,
>
>
> Please vote on releasing the following candidate as Apache Mesos 1.0.3.
>
>
> 1.0.3 includes the following:
>
> 
> 
>
> * [MESOS-6142] - Frameworks may RESERVE for an arbitrary role.
>
>
> * [MESOS-6621] - SSL downgrade path will CHECK-fail when using both
> temporary and persistent sockets
>
> * [MESOS-6676] - Always re-link with scheduler during re-registration.
>
>
> * [MESOS-6917] - Segfault when the executor sets an invalid UUID when
> sending a status update.
>
>
> The CHANGELOG for the release is available at:
>
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_
> plain;f=CHANGELOG;hb=1.0.3-rc1
>
> 
> 
>
>
> The candidate for Mesos 1.0.3 release is available at:
>
> https://dist.apache.org/repos/dist/dev/mesos/1.0.3-rc1/mesos-1.0.3.tar.gz
>
>
> The tag to be voted on is 1.0.3-rc1:
>
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=1.0.3-rc1
>
>
> The MD5 checksum of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/1.0.3-rc1/
> mesos-1.0.3.tar.gz.md5
>
>
> The signature of the tarball can be found at:
>
> https://dist.apache.org/repos/dist/dev/mesos/1.0.3-rc1/
> mesos-1.0.3.tar.gz.asc
>
>
> The PGP key used to sign the release is here:
>
> https://dist.apache.org/repos/dist/release/mesos/KEYS
>
>
> The JAR is up in Maven in a staging repository here:
>
> https://repository.apache.org/content/repositories/orgapachemesos-1172
>
>
> Please vote on releasing this package as Apache Mesos 1.0.3!
>
>
> The vote is open until Mon Jan 23rd 17:00:00 PST 2017 and passes if a
> majority of at least 3 +1 PMC votes are cast.
>
>
> [ ] +1 Release this package as Apache Mesos 1.0.3
>
> [ ] -1 Do not release this package because ...
>
>
> Thanks,
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Welcome Neil Conway as Mesos Committer and PMC member!

2017-01-21 Thread Avinash Sridharan
Congrats Neil !! Very well deserved !!

On Sat, Jan 21, 2017 at 2:07 AM, Jay Guo <guojiannan1...@gmail.com> wrote:

> Congrats Neil!!! Well deserved!!
>
> /J
>
> On Sat, Jan 21, 2017 at 4:46 PM, DhilipKumar Sankaranarayanan
> <s.dhilipku...@gmail.com> wrote:
> > Congratulations Neil.
> >
> > On 21 Jan 2017 14:13, "Guangya Liu" <gyliu...@gmail.com> wrote:
> >>
> >> Congrats Neil!!
> >>
> >> On Sat, 21 Jan 2017 at 12:34 Vinod Kone <vinodk...@apache.org> wrote:
> >>>
> >>> Hi folks,
> >>>
> >>> Please welcome Neil Conway as the newest committer and PMC member of
> the
> >>> Apache Mesos project.
> >>>
> >>> Neil has been an active contributor to Mesos for more than a year now.
> As
> >>> part of his work, he has contributed some major features (Partition
> aware
> >>> frameworks, floating point operations for resources). Neil also took
> the
> >>> initiative to improve the documentation of our project and shepherded
> >>> several improvements over time. Doing that even without being a
> committer,
> >>> shows that he takes ownership of the project seriously.
> >>>
> >>> Here is his more formal checklist for your perusal.
> >>>
> >>>
> >>> https://docs.google.com/document/d/137MYwxEw9QCZRH09CXfn1544p1LuM
> uoj9LxS-sk2_F4/edit
> >>>
> >>> Thanks,
> >>> Vinod
> >>>
> >>>
> >
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Welcome Haosdent Huang as Mesos Committer and PMC member!

2016-12-16 Thread Avinash Sridharan
Congrats Haosdent !!

Very well deserved !!

On Fri, Dec 16, 2016 at 10:59 AM, Vinod Kone <vinodk...@apache.org> wrote:

> Hi folks,
>
> Please join me in formally welcoming Haosdent Huang as Mesos Committer and
> PMC member.
>
> Haosdent has been an active contributor to the project for more than a year
> now. He has contributed a number of patches and features to the Mesos code
> base, most notably the unified cgroups isolator and health check
> improvements. The most impressive thing about him is that he always
> volunteers to help out people in the community, be it on slack/IRC or
> mailing lists. The fact that he does all this even though working on Mesos
> is not part of his day job is even more impressive.
>
> Here is his more formal checklist
> <https://docs.google.com/document/d/1wq-M4KoMOJWZTNTN-
> hvy-H8ZGLXG6CF9VP2IY_UU5_0/edit?ts=57e0029d>
> for your perusal.
>
> Thanks,
> Vinod
>
> P.S: Sorry for the delay in sending the welcome email.
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Problems with shared libraries with mesos containerization....

2016-10-17 Thread Avinash Sridharan
Maybe just attach the docker file and make sure layers are available in
docker hub?
On Mon, Oct 17, 2016 at 9:07 AM Mark Hammons <mark.hamm...@inaf.cnrs-gif.fr>
wrote:

> Oh, the container image itself is very large. Do you have some place I can
> upload it to?
>
> On Monday, October 17, 2016 8:43:10 AM CEST Avinash Sridharan wrote:
> > Also would be good to see the env variables (LD_LIBRARY_PATH) being seen
> by
> > the container when you start it, just to make sure the env are getting
> set
> > correctly.
> >
> > On Mon, Oct 17, 2016 at 8:42 AM, Avinash Sridharan <
> avin...@mesosphere.io>
> >
> > wrote:
> > > Does look like the symlink exists:
> > > /usr/lib/libtiff.so.5
> > >
> > > I am assuming you have checked the realpath exists as well (that's why
> was
> > > asking for `ls -al`) ?
> > >
> > > Don't see that you have volume mounts that might obfuscate the path.
> Could
> > > you create a JIRA and if possible point give access to your docker
> image
> > > for us to try? Do mention the exact version of Mesos and the Distro you
> > > are
> > > trying to run this on.
> > >
> > > -Avinash
> > >
> > > On Mon, Oct 17, 2016 at 8:35 AM, Mark Hammons <
> > >
> > > mark.hamm...@inaf.cnrs-gif.fr> wrote:
> > >> No, it's a regular linux log. I've reattached it.
> > >>
> > >> On Monday, October 17, 2016 8:32:07 AM CEST Avinash Sridharan wrote:
> > >> > can't seem to open the attached logs, is it gzip?
> > >> >
> > >> > On Mon, Oct 17, 2016 at 8:14 AM, Mark Hammons <
> > >>
> > >> mark.hamm...@inaf.cnrs-gif.fr
> > >>
> > >> > > wrote:
> > >> > >
> > >> > > Adding LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu doesn't fix the
> > >>
> > >> error.
> > >>
> > >> > > On Monday, October 17, 2016 5:06:34 PM CEST Mark Hammons wrote:
> > >> > > > As I've shown in the original logs, the symbolic link
> libtiff.so.5
> > >>
> > >> is in
> > >>
> > >> > > > /usr/ lib. I used LD_PRELOAD just to try to force
> > >>
> > >> /usr/lib/libtiff.so.5
> > >>
> > >> > > to
> > >> > >
> > >> > > > be loaded since it wasn't for some reason.
> > >> > > >
> > >> > > > On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan
> wrote:
> > >> > > > > Can you prepend your command with `ls -al /usr/lib` and check
> in
> > >> > >
> > >> > > stdout if
> > >> > >
> > >> > > > > you are seeing the shared object? By the way why are you using
> > >> > >
> > >> > > LD_PRELOAD
> > >> > >
> > >> > > > > instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker
> to
> > >>
> > >> load
> > >>
> > >> > > your
> > >> > >
> > >> > > > > library before any other libraries. It's usually used to
> override
> > >> > >
> > >> > > system
> > >> > >
> > >> > > > > libraries so found a bit odd that you are using it here?
> > >> > > > >
> > >> > > > > On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons
> > >> > > > > <mark.hamm...@inaf.cnrs-gif.fr>
> > >> > > > >
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for
> > >>
> > >> the
> > >>
> > >> > > process
> > >> > >
> > >> > > > > > environment it says it can't load /usr/lib/libtiff.so.5.
> > >> > > > > >
> > >> > > > > > On Monday, October 17, 2016 7:52:16 AM CEST Avinash
> Sridharan
> > >>
> > >> wrote:
> > >> > > > > > > Are you setting the env in the dockerfile?
> > >> > > > > > >
> > >> > > > > > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
> > >> > > > > >
> > >> > > > > > mark.hamm...@inaf.cnrs-gif.fr
&g

Re: Problems with shared libraries with mesos containerization....

2016-10-17 Thread Avinash Sridharan
Also would be good to see the env variables (LD_LIBRARY_PATH) being seen by
the container when you start it, just to make sure the env are getting set
correctly.

On Mon, Oct 17, 2016 at 8:42 AM, Avinash Sridharan <avin...@mesosphere.io>
wrote:

> Does look like the symlink exists:
> /usr/lib/libtiff.so.5
>
> I am assuming you have checked the realpath exists as well (that's why was
> asking for `ls -al`) ?
>
> Don't see that you have volume mounts that might obfuscate the path. Could
> you create a JIRA and if possible point give access to your docker image
> for us to try? Do mention the exact version of Mesos and the Distro you are
> trying to run this on.
>
> -Avinash
>
> On Mon, Oct 17, 2016 at 8:35 AM, Mark Hammons <
> mark.hamm...@inaf.cnrs-gif.fr> wrote:
>
>> No, it's a regular linux log. I've reattached it.
>>
>> On Monday, October 17, 2016 8:32:07 AM CEST Avinash Sridharan wrote:
>> > can't seem to open the attached logs, is it gzip?
>> >
>> > On Mon, Oct 17, 2016 at 8:14 AM, Mark Hammons <
>> mark.hamm...@inaf.cnrs-gif.fr
>> > > wrote:
>> > >
>> > > Adding LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu doesn't fix the
>> error.
>> > >
>> > > On Monday, October 17, 2016 5:06:34 PM CEST Mark Hammons wrote:
>> > > > As I've shown in the original logs, the symbolic link libtiff.so.5
>> is in
>> > > > /usr/ lib. I used LD_PRELOAD just to try to force
>> /usr/lib/libtiff.so.5
>> > >
>> > > to
>> > >
>> > > > be loaded since it wasn't for some reason.
>> > > >
>> > > > On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan wrote:
>> > > > > Can you prepend your command with `ls -al /usr/lib` and check in
>> > >
>> > > stdout if
>> > >
>> > > > > you are seeing the shared object? By the way why are you using
>> > >
>> > > LD_PRELOAD
>> > >
>> > > > > instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to
>> load
>> > >
>> > > your
>> > >
>> > > > > library before any other libraries. It's usually used to override
>> > >
>> > > system
>> > >
>> > > > > libraries so found a bit odd that you are using it here?
>> > > > >
>> > > > > On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons
>> > > > > <mark.hamm...@inaf.cnrs-gif.fr>
>> > > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for
>> the
>> > >
>> > > process
>> > >
>> > > > > > environment it says it can't load /usr/lib/libtiff.so.5.
>> > > > > >
>> > > > > > On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan
>> wrote:
>> > > > > > > Are you setting the env in the dockerfile?
>> > > > > > >
>> > > > > > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
>> > > > > >
>> > > > > > mark.hamm...@inaf.cnrs-gif.fr
>> > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > Here's the code I define my executor to mesos with:
>> > > > > > > > val iuwtURI = CommandInfo.URI.newBuilder().
>> > >
>> > > setValue("http://
>> > >
>> > > > > > ***/
>> > > > > >
>> > > > > > > > IUWT.tar.gz").setExtract(true).setCache(false).build()
>> > > > > > > >
>> > > > > > > > val iuwtjURI =
>> > > > > > > > CommandInfo.URI.newBuilder().setValue("http://
>> > > > > >
>> > > > > > ***/
>> > > > > >
>> > > > > > > > iuwtExecutor-assembly-0.1-
>> > > > > > > > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
>> > > > > > > >
>> > > > > > > > val iuwtExec = "java -jar
>> > > > > > > > iuwtExecutor-assembly-0.1-SNAPSHOT.jar
>> > > > > >
>> > > > > > -
>> >

Re: Problems with shared libraries with mesos containerization....

2016-10-17 Thread Avinash Sridharan
Does look like the symlink exists:
/usr/lib/libtiff.so.5

I am assuming you have checked the realpath exists as well (that's why was
asking for `ls -al`) ?

Don't see that you have volume mounts that might obfuscate the path. Could
you create a JIRA and if possible point give access to your docker image
for us to try? Do mention the exact version of Mesos and the Distro you are
trying to run this on.

-Avinash

On Mon, Oct 17, 2016 at 8:35 AM, Mark Hammons <mark.hamm...@inaf.cnrs-gif.fr
> wrote:

> No, it's a regular linux log. I've reattached it.
>
> On Monday, October 17, 2016 8:32:07 AM CEST Avinash Sridharan wrote:
> > can't seem to open the attached logs, is it gzip?
> >
> > On Mon, Oct 17, 2016 at 8:14 AM, Mark Hammons <
> mark.hamm...@inaf.cnrs-gif.fr
> > > wrote:
> > >
> > > Adding LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu doesn't fix the
> error.
> > >
> > > On Monday, October 17, 2016 5:06:34 PM CEST Mark Hammons wrote:
> > > > As I've shown in the original logs, the symbolic link libtiff.so.5
> is in
> > > > /usr/ lib. I used LD_PRELOAD just to try to force
> /usr/lib/libtiff.so.5
> > >
> > > to
> > >
> > > > be loaded since it wasn't for some reason.
> > > >
> > > > On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan wrote:
> > > > > Can you prepend your command with `ls -al /usr/lib` and check in
> > >
> > > stdout if
> > >
> > > > > you are seeing the shared object? By the way why are you using
> > >
> > > LD_PRELOAD
> > >
> > > > > instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to
> load
> > >
> > > your
> > >
> > > > > library before any other libraries. It's usually used to override
> > >
> > > system
> > >
> > > > > libraries so found a bit odd that you are using it here?
> > > > >
> > > > > On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons
> > > > > <mark.hamm...@inaf.cnrs-gif.fr>
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for the
> > >
> > > process
> > >
> > > > > > environment it says it can't load /usr/lib/libtiff.so.5.
> > > > > >
> > > > > > On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan
> wrote:
> > > > > > > Are you setting the env in the dockerfile?
> > > > > > >
> > > > > > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
> > > > > >
> > > > > > mark.hamm...@inaf.cnrs-gif.fr
> > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > Here's the code I define my executor to mesos with:
> > > > > > > > val iuwtURI = CommandInfo.URI.newBuilder().
> > >
> > > setValue("http://
> > >
> > > > > > ***/
> > > > > >
> > > > > > > > IUWT.tar.gz").setExtract(true).setCache(false).build()
> > > > > > > >
> > > > > > > > val iuwtjURI =
> > > > > > > > CommandInfo.URI.newBuilder().setValue("http://
> > > > > >
> > > > > > ***/
> > > > > >
> > > > > > > > iuwtExecutor-assembly-0.1-
> > > > > > > > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
> > > > > > > >
> > > > > > > > val iuwtExec = "java -jar
> > > > > > > > iuwtExecutor-assembly-0.1-SNAPSHOT.jar
> > > > > >
> > > > > > -
> > > > > >
> > > > > > > > Xmx1024M -Xmx128M"
> > > > > > > >
> > > > > > > > val iuwtCommand =
> > > > > > > >
> > > > > > > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(
> > >
> > > List(iuwtjURI,
> > >
> > > > > > > > iuwtURI).asJava).setShell(true).build()
> > > > > > > >
> > > > > > > > val iuwtImageInfo =
> > > > > > > >
> > > > > > > > Image.newBuilder().setType(Image.

Re: Problems with shared libraries with mesos containerization....

2016-10-17 Thread Avinash Sridharan
Can you prepend your command with `ls -al /usr/lib` and check in stdout if
you are seeing the shared object? By the way why are you using LD_PRELOAD
instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to load your
library before any other libraries. It's usually used to override system
libraries so found a bit odd that you are using it here?

On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons <mark.hamm...@inaf.cnrs-gif.fr
> wrote:

> No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for the process
> environment it says it can't load /usr/lib/libtiff.so.5.
>
> On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan wrote:
> > Are you setting the env in the dockerfile?
> >
> > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
> mark.hamm...@inaf.cnrs-gif.fr
> > > wrote:
> > >
> > > Here's the code I define my executor to mesos with:
> > > val iuwtURI = CommandInfo.URI.newBuilder().setValue("http://
> ***/
> > >
> > > IUWT.tar.gz").setExtract(true).setCache(false).build()
> > >
> > > val iuwtjURI = CommandInfo.URI.newBuilder().setValue("http://
> ***/
> > >
> > > iuwtExecutor-assembly-0.1-
> > > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
> > >
> > > val iuwtExec = "java -jar iuwtExecutor-assembly-0.1-SNAPSHOT.jar
> -
> > >
> > > Xmx1024M -Xmx128M"
> > >
> > > val iuwtCommand =
> > >
> > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(List(iuwtjURI,
> > > iuwtURI).asJava).setShell(true).build()
> > >
> > > val iuwtImageInfo =
> > >
> > > Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> > > Image.Docker.newBuilder.setName("ubuntu-
> > > mesos:0.11-17102016-IUWT").build()).build()
> > >
> > > val iuwtContInfo =
> > >
> > > ContainerInfo.MesosInfo.newBuilder().setImage(iuwtImageInfo).build()
> > >
> > > val iuwtContainer = ContainerInfo.newBuilder()
> > >
> > > .setMesos(iuwtContInfo)
> > >
> > >   .setType(ContainerInfo.Type.MESOS)
> > >   .build()
> > >
> > > val iuwtExecutor = ExecutorInfo.newBuilder()
> > >
> > > .setCommand(iuwtCommand)
> > > .setContainer(iuwtContainer)
> > > .setExecutorId(ExecutorID.newBuilder().setValue("iuwt-
> > >
> > > executor"))
> > >
> > > .setName("iuwt-executor").build()
> > >
> > > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted docker image.
> > > IUWT is
> > > the program I'm trying to run, and it runs perfectly fine in the
> > > aforementioned docker container when running on docker. The
> libtiff.so.5
> > > problem only manifests when I'm using mesos' containerizer.
> > >
> > > On Monday, October 17, 2016 6:52:17 AM CEST Avinash Sridharan wrote:
> > > > You are running a container with its own image right? So is
> > >
> > > /usr/lib/x86_64
> > >
> > > > in the container's file system or the host file system?
> > > >
> > > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> > >
> > > mark.hamm...@inaf.cnrs-gif.fr
> > >
> > > > > wrote:
> > > > >
> > > > > Yes, it's installed under /usr/lib/x86_64 or whatever the multilib
> > >
> > > path is
> > >
> > > > > in
> > > > > debian. It seems files under this path are not accessible.
> > > > >
> > > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the
> symlink
> > >
> > > to
> > >
> > > > > load
> > > > > but it refused to load it. I think the mesos containerizer is
> > >
> > > preventing
> > >
> > > > > the
> > > > > program from accessing anything in a directory under /usr/lib/ for
> > > > > some
> > > > > reason, as the same program runs fine in the same container running
> > >
> > > under
> > >
> > > > > docker.
> > > > >
> > > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash Sridharan
> wrote:
> > > > > > Is the library part of the image that you are running? Also you
> > > > > > might
> > > >

Re: Problems with shared libraries with mesos containerization....

2016-10-17 Thread Avinash Sridharan
Are you setting the env in the dockerfile?

On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <mark.hamm...@inaf.cnrs-gif.fr
> wrote:

> Here's the code I define my executor to mesos with:
>
> val iuwtURI = CommandInfo.URI.newBuilder().setValue("http://***/
> IUWT.tar.gz").setExtract(true).setCache(false).build()
>
> val iuwtjURI = CommandInfo.URI.newBuilder().setValue("http://***/
> iuwtExecutor-assembly-0.1-
> SNAPSHOT.jar").setExecutable(false).setCache(false).build()
>
> val iuwtExec = "java -jar iuwtExecutor-assembly-0.1-SNAPSHOT.jar -
> Xmx1024M -Xmx128M"
>
> val iuwtCommand =
> CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(List(iuwtjURI,
> iuwtURI).asJava).setShell(true).build()
>
> val iuwtImageInfo =
> Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> Image.Docker.newBuilder.setName("ubuntu-
> mesos:0.11-17102016-IUWT").build()).build()
>
> val iuwtContInfo =
> ContainerInfo.MesosInfo.newBuilder().setImage(iuwtImageInfo).build()
>
>
> val iuwtContainer = ContainerInfo.newBuilder()
> .setMesos(iuwtContInfo)
>   .setType(ContainerInfo.Type.MESOS)
>   .build()
>
> val iuwtExecutor = ExecutorInfo.newBuilder()
> .setCommand(iuwtCommand)
> .setContainer(iuwtContainer)
> .setExecutorId(ExecutorID.newBuilder().setValue("iuwt-
> executor"))
> .setName("iuwt-executor").build()
>
> The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted docker image.
> IUWT is
> the program I'm trying to run, and it runs perfectly fine in the
> aforementioned docker container when running on docker. The libtiff.so.5
> problem only manifests when I'm using mesos' containerizer.
>
> On Monday, October 17, 2016 6:52:17 AM CEST Avinash Sridharan wrote:
> > You are running a container with its own image right? So is
> /usr/lib/x86_64
> > in the container's file system or the host file system?
> >
> > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> mark.hamm...@inaf.cnrs-gif.fr
> > > wrote:
> > >
> > > Yes, it's installed under /usr/lib/x86_64 or whatever the multilib
> path is
> > > in
> > > debian. It seems files under this path are not accessible.
> > >
> > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the symlink
> to
> > > load
> > > but it refused to load it. I think the mesos containerizer is
> preventing
> > > the
> > > program from accessing anything in a directory under /usr/lib/ for some
> > > reason, as the same program runs fine in the same container running
> under
> > > docker.
> > >
> > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash Sridharan wrote:
> > > > Is the library part of the image that you are running? Also you might
> > >
> > > need
> > >
> > > > to setup LD_LIBRARY_PATH in your env while launching the image so
> that
> > >
> > > the
> > >
> > > > container process knows where to look for the shared object.
> > > >
> > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > >
> > > mark.hamm...@inaf.cnrs-gif.fr
> > >
> > > > > wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > I've been working with the mesos containerizer to handle my docker
> > > > > containers
> > > > > recently. I created a docker container that requires libtiff.so.5,
> and
> > >
> > > the
> > >
> > > > > application within it runs well. But when I try to run it within
> the
> > >
> > > mesos
> > >
> > > > > containerizer I get an error saying libtiff.so.5 doesn't exist.
> > > > >
> > > > > The application is being launched via java's process mechanism from
> > >
> > > inside
> > >
> > > > > a
> > > > > java thread in a custom java executor if that makes a difference.
> > > > >
> > > > > Any idea what could be causing this change in behavior? As you can
> see
> > >
> > > in
> > >
> > > > > the
> > > > > attached log file, I check /usr/lib, and a symbolic link to
> > > > > /usr/lib/x86/
> > > > > libtiff.so.5 exists in /usr/lib so the program should be able to
> find
> > >
> > > and
> > >
> > > > > load
> > > > > that
> > > > > 
> > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > >
> > > --
> > > 
> > > Mark Edgar Hammons II | +33 06 03 69 56 56
>
>
> --
> 
> Mark Edgar Hammons II | +33 06 03 69 56 56
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Unified cgroups isolator

2016-09-13 Thread Avinash Sridharan
Awesome !!


On Tue, Sep 13, 2016 at 10:59 AM, Jie Yu <yujie@gmail.com> wrote:

> Hi,
>
> We just merged the unified cgroups isolator. Huge shout out to @haosdent
> and @qianzhang to make this happen!
> https://issues.apache.org/jira/browse/MESOS-4697
>
> Just to give you some context. Previously, it's a huge pain to add a new
> cgroups subsystem to Mesos because it requires creating a new isolator (a
> lot of code duplication). Now, we merge all the subsystems into one single
> isolator, that makes adding a new subsystem very easy.
>
> More importantly, the new cgroups isolator supports cgroups v2!
>
> - Jie
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: what is the status on this?

2016-09-06 Thread Avinash Sridharan
Also, I think, the replicated log itself uses Zookeeper for leader election.

On Tue, Sep 6, 2016 at 12:15 PM, Zameer Manji <zma...@apache.org> wrote:

> If we use the replicated log for leader election, how will frameworks
> detect the leading master? Right now the scheduler driver uses the
> MasterInfo in ZK to discover the leader and detect leadership changes.
>
> On Mon, Sep 5, 2016 at 10:18 AM, Dario Rexin <dre...@apple.com> wrote:
>
>> If we go and change this, why not simply remove any dependencies to
>> external systems and simply use the replicated log for leader election?
>>
>> On Sep 5, 2016, at 9:02 AM, Alex Rukletsov <a...@mesosphere.com> wrote:
>>
>> Kant—
>>
>> thanks a lot for the feedback! Are you interested in helping out with
>> Consul module once Jay and Joseph are done with modularizing patches?
>>
>> On Mon, Sep 5, 2016 at 8:50 AM, Jay JN Guo <guojian...@cn.ibm.com> wrote:
>>
>>> Patches are currently under review by @Joseph and can be found at the
>>> links provided by @haosdent.
>>>
>>> I took a quick look at Consul key/value HTTP APIs and they look very
>>> similar to Etcd APIs. You could actually reuse our Etcd module
>>> implementation once we manage to push the module into Mesos community.
>>>
>>> The only technical problem I could see for now is that Consul does not
>>> support `POST` with incremental key index. We may need to leverage
>>> `?cas=` operation in Consul to emulate the behaviour of joining a
>>> key group.
>>>
>>> We could have a discussion on how to implement Consul HA module.
>>>
>>> cheers,
>>> /J
>>>
>>>
>>> - Original message -
>>> From: haosdent <haosd...@gmail.com>
>>> To: user <user@mesos.apache.org>
>>> Cc: Jay JN Guo/China/IBM@IBMCN
>>> Subject: Re: what is the status on this?
>>> Date: Sun, Sep 4, 2016 6:10 PM
>>>
>>> Jay has some patches for de-couple Mesos with Zookeeper
>>>
>>> https://issues.apache.org/jira/browse/MESOS-5828
>>> https://issues.apache.org/jira/browse/MESOS-5829
>>>
>>> I think it should be possible to support consul by custom modules after
>>> jay's work done.
>>>
>>> On Sun, Sep 4, 2016 at 6:02 PM, kant kodali <kanth...@gmail.com> wrote:
>>>
>>> Hi Alex,
>>>
>>> We have some experienced devops people here and they all had one thing
>>> in common which is Zookeeper is a pain to maintain. In fact we refused to
>>> bring in new tech stacks that require Zookeeper such as Kafka for example.
>>> so we desperately in search for alternative preferably using consul. I just
>>> hear lot of positive response when comes it consul. It will be great to see
>>> mesos and consul working together in which we would be ready to jump at it
>>> and make a switch for YARN to Mesos.
>>>
>>> Thanks,
>>> Kant
>>>
>>>
>>>
>>>
>>> On Wed, Aug 31, 2016 1:03 AM, Alex Rukletsov a...@mesosphere.com wrote:
>>>
>>> Kant—
>>>
>>> mind telling us what is your use case and why this ticket is important
>>> for you? It will help us prioritize work.
>>>
>>> On Fri, Aug 26, 2016 at 2:46 AM, tommy xiao <xia...@gmail.com> wrote:
>>>
>>> Hi guys, i always focus on t his case. but good news is etcd always have
>>> patchs. so the coming consul is very easy, just need some time to do coding
>>> on it. if you have interesting it? let us collaborate it.
>>>
>>> 2016-08-26 8:11 GMT+08:00 Joseph Wu <jos...@mesosphere.io>:
>>>
>>> There is no timeline as no one has done any work on the issue.
>>>
>>>
>>> On Thu, Aug 25, 2016 at 4:54 PM, kant kodali <kanth...@gmail.com>
>>> wrote:
>>>
>>> Hi Guys,
>>>
>>> I see this ticket and other related tickets should be part of sprints in
>>> 2015 and it is still not resolved yet. can we have a timeline on this? This
>>> would be really helpful
>>>
>>> https://issues.apache.org/jira/browse/MESOS-3797
>>>
>>> Thanks!
>>>
>>>
>>>
>>> --
>>> Deshi Xiao
>>> Twitter: xds2000
>>> E-mail: xiaods(AT)gmail.com
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>>
>>>
>>>
>>>
>>
>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Using mesos' cfs limits on a docker container?

2016-08-14 Thread Avinash Sridharan
Hi Mark,
 That would be awesome (user facing documentation forthe
`UnifiedContainerizer`). We have bits and pieces of the unified
containerizer (and what it actually is), but would be great to land a more
comprehensive documentation into its motivation and usability. May be have
a separate `unified-containerized.md` and update the `
https://github.com/apache/mesos/blob/master/docs/mesos-containerizer.md`
with a link to the new 'unified-containerized.md'. Jie ??

Would also be very useful if you file some JIRAs related to the problems
you faced in setting up docker private registries to be used with the
unified-containerizer.

Thanks,
Avinash

On Sun, Aug 14, 2016 at 10:36 AM, Mark Hammons <
mark.hamm...@inaf.cnrs-gif.fr> wrote:

> In specific, I wanted the process control capabilities of a mesos
> framework with custom schedulers and executors, but wanted to run my tasks
> in a framework definable environment (like running my tasks on a copy of
> Ubuntu 14 with certain libs installed). Using mixed-mode containerization
> worked with some fiddling, but it was painful in certain ways. The sandbox
> mounted in a mixed-mode container wasn't accessible from within the
> container thanks to selinux unless I ran the container in privileged mode
> and the cou limits per executor were no longer enforced unlike a mesos task
> with cfs isolation enabled. Further, setting up the default working
> directory and user was a pain.
>
> Unified mode (also called mesos containerizer for some reason) solves a
> lot of these issues, though using it with private image repositories was
> not as straightforward as the docker containerizer. I eventually had to use
> an image directory to get that working, cause curl kept throwing vague ssl
> errors(I'm fairly certain this is due to my private image repository not
> having https set up since it's a test environment).
>
> Once I get things set up and cleaned up I'll post a more involved guide on
> how to get this particular use case setup and running, especially a part on
> preparing your container image for use with mesos.
>
> Mark Edgar Hammons II - Research Engineer at BioEmergences
> 0603695656
>
> On 14 Aug 2016, at 18:11, Erik Weathers <eweath...@groupon.com> wrote:
>
> What was the problem and how did you overcome it?  (i.e. This would be a
> sad resolution to this thread for someone faced with this same problem in
> the future.)
>
> On Sunday, August 14, 2016, Mark Hammons <mark.hamm...@inaf.cnrs-gif.fr>
> wrote:
>
>> I finally got this working after fiddling with it all night. It works
>> great so far!
>>
>> Mark Edgar Hammons II - Research Engineer at BioEmergences
>> 0603695656
>>
>> On 14 Aug 2016, at 04:50, Joseph Wu <jos...@mesosphere.io> wrote:
>>
>> If you're not against running Docker containers without the Docker
>> daemon, try using the Unified containerizer.
>> See the latter half of this document: http://mesos.apache.org/docume
>> ntation/latest/mesos-containerizer/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__mesos.apache.org_documentation_latest_mesos-2Dcontainerizer_=DQMFaQ=LNdz7nrxyGFUIUTz2qIULQ=cPg4mUupZEtURFK34GyDCtRjHoUmKrI7oHRZqAh3hZY=p3yjpxMelmcew1dQtqJniCFVDpbSbJQBXaW-mA1QVHU=6sjCv4C-sSI7jwRLgPi2uCrQR8G0D_Kvtde-tRjBybc=>
>>
>> On Sat, Aug 13, 2016 at 7:02 PM, Mark Hammons <
>> mark.hamm...@inaf.cnrs-gif.fr> wrote:
>>
>>> Hi All,
>>>
>>>
>>>
>>> I was having a lot of success having mesos force sandboxed programs to
>>> work within cpu and memory constraints, but when I added docker into the
>>> mix, the cpu limitations go out the window (not sure about the memory
>>> limitations. Is there any way to mix these two methods of isolation? I'd
>>> like my executor/algorithm to run inside a docker container, but have that
>>> container's memory and cpu usage controlled by systemd/mesos.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Mark
>>> --
>>>
>>> Mark Hammons - +33 06 03 69 56 56
>>>
>>> Research Engineer @ BioEmergences
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__bioemergences.iscpif.fr=DQMFaQ=LNdz7nrxyGFUIUTz2qIULQ=cPg4mUupZEtURFK34GyDCtRjHoUmKrI7oHRZqAh3hZY=p3yjpxMelmcew1dQtqJniCFVDpbSbJQBXaW-mA1QVHU=hlyM8jpFaEkcQ5X8UJs0BTG53J2X6F-zEs0JIKxCFEQ=>
>>>
>>> Lab Phone: 01 69 82 34 19
>>>
>>
>>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Using mesos' cfs limits on a docker container?

2016-08-13 Thread Avinash Sridharan
+1 . Was just about to suggest the same !!

On Sat, Aug 13, 2016 at 7:50 PM, Joseph Wu <jos...@mesosphere.io> wrote:

> If you're not against running Docker containers without the Docker daemon,
> try using the Unified containerizer.
> See the latter half of this document: http://mesos.apache.org/
> documentation/latest/mesos-containerizer/
>
> On Sat, Aug 13, 2016 at 7:02 PM, Mark Hammons <
> mark.hamm...@inaf.cnrs-gif.fr> wrote:
>
>> Hi All,
>>
>>
>>
>> I was having a lot of success having mesos force sandboxed programs to
>> work within cpu and memory constraints, but when I added docker into the
>> mix, the cpu limitations go out the window (not sure about the memory
>> limitations. Is there any way to mix these two methods of isolation? I'd
>> like my executor/algorithm to run inside a docker container, but have that
>> container's memory and cpu usage controlled by systemd/mesos.
>>
>>
>>
>> Thanks,
>>
>> Mark
>> --
>>
>> Mark Hammons - +33 06 03 69 56 56
>>
>> Research Engineer @ BioEmergences <http://bioemergences.iscpif.fr>
>>
>> Lab Phone: 01 69 82 34 19
>>
>
>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


A port mapping plugin for CNI

2016-08-05 Thread Avinash Sridharan
Hi All,
 One of the most used networking mode when users run containers using the
`DockerContainerizer` is docker bridge mode networking. In this mode
containers run in an isolator private address space, and services on the
container are exposed to the outside world using DNAT.

As we move towards the unified containerizer becoming the de-facto
container runtime, and users start running their docker containers on the
`MesosContainerizer`, the expectation of exposing services running on
isolator bridges with DNAT starts becoming a must have.

With the introduction of the `network/cni` isolator we can use CNI plugins
to start attaching containers on the `MesosContainerizer` to different
types of IP networks. Corresponding to docker bridge network, CNI has its
own bridge plugin, however unlike docker bridge networking the CNI bridge
plugin does not provide DNAT services to expose containers on a bridge.
None of the core CNI plugins provide a port mapping functionality, and it
is only recently that there seems to be a push for having port mapping
functionality in a CNI plugin.

We are therefore proposing implementing a CNI plugin that can setup port
mapping rules for different CNI plugins for Mesos. This CNI plugin is
generic enough that it can be used in conjunction with any other CNI
plugin, such as the bridge plugin.

The motivation, design and operational aspects of the plugin have been
captured in this document:
https://docs.google.com/document/d/1ZwXZ_utpxmy9vccYiL0q86efgpWpjmmKLQ0S4Mmz9N4/edit?usp=sharing

Would be great if the community can share their feedback on the proposed
port mapping CNI plugin.

Thanks,
-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: A Redis Framework for Apache Mesos

2016-07-04 Thread Avinash Sridharan
On Mon, Jul 4, 2016 at 12:11 AM, DhilipKumar Sankaranarayanan <
s.dhilipku...@gmail.com> wrote:

> *"*
> *Would be nice to get on the frameworks page:*
>
> *http://mesos.apache.org/documentation/latest/frameworks/
> <http://mesos.apache.org/documentation/latest/frameworks/> *
> *?*
> *"*
> Sure i would love to do that, where should i raise a PR?
>
I think you can raise a PR on github. The preferred route though is to
generate an RB request on Apache review board (https://reviews.apache.org/r/)
. One of the Shepherds could help with this.
+Vinod ^^

> *"Also is it already part of the DC/OS universe ?"*
>
> Yes we have reasonable documentation
> <https://github.com/mesos/mr-redis#dcos>with regards to DCOS integration,
> Please give it a try and it will be awesome to get your feedback on that.
>
> Regards,
> Dhilip
>
> On Sun, Jul 3, 2016 at 6:09 PM, Avinash Sridharan <avin...@mesosphere.io>
> wrote:
>
>> Would be nice to get on the frameworks page:
>> http://mesos.apache.org/documentation/latest/frameworks/
>> ?
>>
>> Also is it already part of the DC/OS universe ?
>>
>> On Sun, Jul 3, 2016 at 2:29 PM, Christoph Heer <christ...@thelabmill.de>
>> wrote:
>>
>>> Hi,
>>>
>>> it looks really cool. Can you maybe explain why do you use etcd for
>>> leader election instead of the normally already existing Zookeeper cluster?
>>> Do you use some special etcd features?
>>>
>>> Best regards
>>> Christoph
>>>
>>> > On 03 Jul 2016, at 17:49, tommy xiao <xia...@gmail.com> wrote:
>>> >
>>> > Cool. thanks for your sharing.
>>> >
>>> > 2016-07-03 23:44 GMT+08:00 DhilipKumar Sankaranarayanan <
>>> s.dhilipku...@gmail.com>:
>>> > Hello All,
>>> >
>>> >
>>> >
>>> > We have built a framework for provisioning redis-servers in Apache
>>> Mesos enabled infrastructure. It would be awesome to get communities
>>> feedback. While mesos ecosystem is strengthening its capability in storage
>>> layer, redis could be an addition to the lineup.  This is primarily
>>> intended for providers who would like to host redis as a service in their
>>> infrastructure.
>>> >
>>> >
>>> >
>>> > There is an elaborate README about the project which should help in
>>> setting it up: https://github.com/mesos/mr-redis. Please let us know if
>>> there is anything missing in the documentation by raising a PR or opening
>>> an issue or even writing to us.
>>> >
>>> >
>>> >
>>> > This project is also packaged as  mr-redis with DCOS so should be
>>> pretty straight forward to install it via DCOS CLI or DCOS GUI.
>>> >
>>> >
>>> >
>>> > (A Step by Step guide is provided in the README for your convenience)
>>> >
>>> >
>>> >
>>> > Salient Features of this project include:
>>> >
>>> > 1)  Create multiple redis clusters (Master-Slave Cluster) with ease
>>> >
>>> > 2)  Redis instances recover in seconds and not in minutes
>>> >
>>> > 3)  If a Master fails a slaves is automatically promoted as the
>>> New master, all the old slaves now replicate from the newly master plus a
>>> new slave is added to the cluster.  All this without using redis-sentinel
>>> in your datacenter and all these happens in a couple of seconds.
>>> >
>>> > 4)  A CLI to perform basic operations such as create / status /
>>> delete redis instances on the fly.  CLI is cross compiled for Windows and
>>> Darwin users too.
>>> >
>>> > 5)  Scheduler is a HTTP REST server which also can respond to
>>> simple Angular UI we have built to get started with.  Instructions on how
>>> to setup the UI is here.
>>> https://github.com/mesos/mr-redis/tree/master/ui/app
>>> >
>>> >
>>> >
>>> > We also had an opportunity to talk about this during the recent
>>> MesosCon 2016:
>>> https://www.youtube.com/watch?v=xe-Gom5tOl0=32=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC
>>> >
>>> >
>>> >
>>> > Future Work:
>>> >
>>> > · Implement a proxy technique to expose one single endpoint
>>> for the redis instance.  (Work in progress)
>>> >
>>> > · Implement Memory Cgroups per redis PROCS
>>> >
>>> > · Add support for Redis 3.0 cluster instances (Adding shards
>>> to a running redis instance)
>>> >
>>> > · Implement integration test suite and benchmarking suite to
>>> the framework.
>>> >
>>> >
>>> >
>>> > Special thanks to Adobe.io team who expressed interest in
>>> collaborating in the development of this product. I’m sure it’s going to be
>>> great working with all of you folks.
>>> >
>>> >
>>> >
>>> > Advanced happy Independence day America and happy week ahead rest of
>>> the world.
>>> >
>>> >
>>> >
>>> > Looking forward to hear from you all,
>>> >
>>> > Dhilip
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Deshi Xiao
>>> > Twitter: xds2000
>>> > E-mail: xiaods(AT)gmail.com
>>>
>>>
>>
>>
>> --
>> Avinash Sridharan, Mesosphere
>> +1 (323) 702 5245
>>
>
>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: A Redis Framework for Apache Mesos

2016-07-03 Thread Avinash Sridharan
Would be nice to get on the frameworks page:
http://mesos.apache.org/documentation/latest/frameworks/
?

Also is it already part of the DC/OS universe ?

On Sun, Jul 3, 2016 at 2:29 PM, Christoph Heer <christ...@thelabmill.de>
wrote:

> Hi,
>
> it looks really cool. Can you maybe explain why do you use etcd for leader
> election instead of the normally already existing Zookeeper cluster? Do you
> use some special etcd features?
>
> Best regards
> Christoph
>
> > On 03 Jul 2016, at 17:49, tommy xiao <xia...@gmail.com> wrote:
> >
> > Cool. thanks for your sharing.
> >
> > 2016-07-03 23:44 GMT+08:00 DhilipKumar Sankaranarayanan <
> s.dhilipku...@gmail.com>:
> > Hello All,
> >
> >
> >
> > We have built a framework for provisioning redis-servers in Apache Mesos
> enabled infrastructure. It would be awesome to get communities feedback.
> While mesos ecosystem is strengthening its capability in storage layer,
> redis could be an addition to the lineup.  This is primarily intended for
> providers who would like to host redis as a service in their infrastructure.
> >
> >
> >
> > There is an elaborate README about the project which should help in
> setting it up: https://github.com/mesos/mr-redis. Please let us know if
> there is anything missing in the documentation by raising a PR or opening
> an issue or even writing to us.
> >
> >
> >
> > This project is also packaged as  mr-redis with DCOS so should be pretty
> straight forward to install it via DCOS CLI or DCOS GUI.
> >
> >
> >
> > (A Step by Step guide is provided in the README for your convenience)
> >
> >
> >
> > Salient Features of this project include:
> >
> > 1)  Create multiple redis clusters (Master-Slave Cluster) with ease
> >
> > 2)  Redis instances recover in seconds and not in minutes
> >
> > 3)  If a Master fails a slaves is automatically promoted as the New
> master, all the old slaves now replicate from the newly master plus a new
> slave is added to the cluster.  All this without using redis-sentinel in
> your datacenter and all these happens in a couple of seconds.
> >
> > 4)  A CLI to perform basic operations such as create / status /
> delete redis instances on the fly.  CLI is cross compiled for Windows and
> Darwin users too.
> >
> > 5)  Scheduler is a HTTP REST server which also can respond to simple
> Angular UI we have built to get started with.  Instructions on how to setup
> the UI is here. https://github.com/mesos/mr-redis/tree/master/ui/app
> >
> >
> >
> > We also had an opportunity to talk about this during the recent MesosCon
> 2016:
> https://www.youtube.com/watch?v=xe-Gom5tOl0=32=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC
> >
> >
> >
> > Future Work:
> >
> > · Implement a proxy technique to expose one single endpoint for
> the redis instance.  (Work in progress)
> >
> > · Implement Memory Cgroups per redis PROCS
> >
> > · Add support for Redis 3.0 cluster instances (Adding shards to
> a running redis instance)
> >
> > · Implement integration test suite and benchmarking suite to the
> framework.
> >
> >
> >
> > Special thanks to Adobe.io team who expressed interest in collaborating
> in the development of this product. I’m sure it’s going to be great working
> with all of you folks.
> >
> >
> >
> > Advanced happy Independence day America and happy week ahead rest of the
> world.
> >
> >
> >
> > Looking forward to hear from you all,
> >
> > Dhilip
> >
> >
> >
> >
> >
> > --
> > Deshi Xiao
> > Twitter: xds2000
> > E-mail: xiaods(AT)gmail.com
>
>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Proposal: move content in Wiki to docs in code repo

2016-06-06 Thread Avinash Sridharan
+1

I think moving the wiki into docs would make information about working
groups, release tracking first class citizens and help disseminate
information better.

On Mon, Jun 6, 2016 at 11:29 AM, Jie Yu <yujie@gmail.com> wrote:

> Hi folks,
>
> I am proposing moving our content in Wiki (e.g., working groups, release
> tracking, etc.) to our docs in the code repo. I personally found that wiki
> is hard to use and there's no reviewing process for changes in the Wiki.
> The content in Wiki historically received less attention than that in the
> docs.
>
> What do you think?
>
> - Jie
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Mesos Calico CNI

2016-05-17 Thread Avinash Sridharan
Thanks Daniel,
 Would love to have some instructions on running Calico with Mesos CNI. We
can link the instructions to the RB patch I am working on:
https://github.com/asridharan/mesos/blob/MESOS-4771/docs/cni.md

On Tue, May 17, 2016 at 8:57 AM, Daniel Osborne <t...@tigera.io> wrote:

> Frank,
>
> I’ve done some work to get Calico-CNI working in Mesos. The work is based
> on the net-modules repo (even though CNI doesn’t have much to do with
> netmdoules) only because it already had a pretty slick docker demo set up.
> Here’s the branch: https://github.com/djosborne/net-modules/tree/cni
>
> I’d be happy to assist you in getting it working on your repo.  Let me
> know if you need any assistance.
>
> We haven't published much information on this as of yet, since Mesos CNI
> support is only just rolling out now.
>
> -Dan
>
> On Tue, May 17, 2016 at 8:16 AM, Avinash Sridharan <avin...@mesosphere.io>
> wrote:
>
>> Hi Frank,
>>  I am in the process of putting up the documentation for CNI support in
>> Mesos. You can find the RB patch for the documentation here:
>> https://reviews.apache.org/r/47463/
>>
>> You can find a rendering of the markdown on my github over here:
>> https://github.com/asridharan/mesos/blob/MESOS-4771/docs/cni.md
>>
>>
>> I have put up one example of using the `network/cni` isolator with a
>> "bridge" plugin. Working on adding some more examples, but given that
>> people have already started showing some interest thought would be a good
>> dry run for the documentation if someone could test out the instructions.
>>
>> Would be great if you can try following the instructions and leave any
>> feedback on the review board.
>>
>>
>> Thanks,
>> Avinash
>>
>> On Tue, May 17, 2016 at 6:51 AM, Frank Scholten <fr...@frankscholten.nl>
>> wrote:
>>
>>> In the meantime I am looking at an alternative route trying to figure
>>> out how an ipAddress value on a Marathon app get propagated into Mesos
>>> CNI.
>>>
>>> Marathon reads the ipAddress value from the AppDefinition and then
>>> publishes on the eventbus. I don't see what happens to it from that
>>> point.
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Tue, May 17, 2016 at 2:58 PM, Jay JN Guo <guojian...@cn.ibm.com>
>>> wrote:
>>> > - net::links() -> stout/net.hpp
>>> > - Personally, I'm not very familiar with CLion build. Maybe somebody
>>> else
>>> > could answer that.
>>> > - I think this is very much related to dev mailing list, so +dev
>>> >
>>> > /J
>>> >
>>> > Frank Scholten <fr...@frankscholten.nl> wrote on 05/17/2016 20:47:12:
>>> >
>>> >> From: Frank Scholten <fr...@frankscholten.nl>
>>> >> To: user@mesos.apache.org
>>> >> Cc: Qian AZ Zhang/China/IBM@IBMCN, avin...@mesosphere.io
>>> >> Date: 05/17/2016 20:48
>>> >> Subject: Re: Mesos Calico CNI
>>> >>
>>> >> Thanks. I now like to run that unit test via the debugger from the
>>> >> Mesos source try in CLion. Is there a doc on how to build Mesos in
>>> >> CLion and debug a unit test?
>>> >>
>>> >> Also, where does net::links() come from? Can't find it in the sources.
>>> >>
>>> >>
>>> >> On Tue, May 17, 2016 at 11:00 AM, haosdent <haosd...@gmail.com>
>>> wrote:
>>> >> >>Is there some user documentation for this feature?
>>> >> > Unfortunately, the document is not ready.
>>> >> > https://issues.apache.org/jira/browse/MESOS-4771
>>> >> >
>>> >> >>but I am not sure what I have to do to create an IP for a task.
>>> >> > Qian Zhang show an example of configuration in his test cases, I
>>> think
>>> > you
>>> >> > may take a look first.
>>> >> > https://reviews.apache.org/r/46097/diff/7#index_header
>>> >> >
>>> >> > On Tue, May 17, 2016 at 4:20 PM, Frank Scholten
>>> > <fr...@frankscholten.nl>
>>> >> > wrote:
>>> >> >>
>>> >> >> Hi all,
>>> >> >>
>>> >> >> I tried out CNI support with Calico but I am not sure what I have
>>> to
>>> >> >> do to create an IP for a task.
>>> >> >>
>>> >> >> See this sandbox repository on Github
>>> >> >>
>>> >> >> https://github.com/ContainerSolutions/mesos-calico-cni-sandbox
>>> >> >>
>>> >> >> In this repo I build the Mesos master branch and
>>> >> >> https://github.com/projectcalico/calico-cni, create a local
>>> cluster
>>> >> >> and deploy an application. I don't see anything in the logs about
>>> cni,
>>> >> >> except that it loads the cni isolator.
>>> >> >>
>>> >> >> Is there some user documentation for this feature? If not I am
>>> happy
>>> >> >> to write documentation once I figure out how this feature works.
>>> >> >>
>>> >> >> Cheers,
>>> >> >>
>>> >> >> Frank
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Best Regards,
>>> >> > Haosdent Huang
>>> >>
>>>
>>
>>
>>
>> --
>> Avinash Sridharan, Mesosphere
>> +1 (323) 702 5245
>>
>
>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Mesos Calico CNI

2016-05-17 Thread Avinash Sridharan
Hi Frank,
 I am in the process of putting up the documentation for CNI support in
Mesos. You can find the RB patch for the documentation here:
https://reviews.apache.org/r/47463/

You can find a rendering of the markdown on my github over here:
https://github.com/asridharan/mesos/blob/MESOS-4771/docs/cni.md


I have put up one example of using the `network/cni` isolator with a
"bridge" plugin. Working on adding some more examples, but given that
people have already started showing some interest thought would be a good
dry run for the documentation if someone could test out the instructions.

Would be great if you can try following the instructions and leave any
feedback on the review board.


Thanks,
Avinash

On Tue, May 17, 2016 at 6:51 AM, Frank Scholten <fr...@frankscholten.nl>
wrote:

> In the meantime I am looking at an alternative route trying to figure
> out how an ipAddress value on a Marathon app get propagated into Mesos
> CNI.
>
> Marathon reads the ipAddress value from the AppDefinition and then
> publishes on the eventbus. I don't see what happens to it from that
> point.
>
>
>
>
>
>
> On Tue, May 17, 2016 at 2:58 PM, Jay JN Guo <guojian...@cn.ibm.com> wrote:
> > - net::links() -> stout/net.hpp
> > - Personally, I'm not very familiar with CLion build. Maybe somebody else
> > could answer that.
> > - I think this is very much related to dev mailing list, so +dev
> >
> > /J
> >
> > Frank Scholten <fr...@frankscholten.nl> wrote on 05/17/2016 20:47:12:
> >
> >> From: Frank Scholten <fr...@frankscholten.nl>
> >> To: user@mesos.apache.org
> >> Cc: Qian AZ Zhang/China/IBM@IBMCN, avin...@mesosphere.io
> >> Date: 05/17/2016 20:48
> >> Subject: Re: Mesos Calico CNI
> >>
> >> Thanks. I now like to run that unit test via the debugger from the
> >> Mesos source try in CLion. Is there a doc on how to build Mesos in
> >> CLion and debug a unit test?
> >>
> >> Also, where does net::links() come from? Can't find it in the sources.
> >>
> >>
> >> On Tue, May 17, 2016 at 11:00 AM, haosdent <haosd...@gmail.com> wrote:
> >> >>Is there some user documentation for this feature?
> >> > Unfortunately, the document is not ready.
> >> > https://issues.apache.org/jira/browse/MESOS-4771
> >> >
> >> >>but I am not sure what I have to do to create an IP for a task.
> >> > Qian Zhang show an example of configuration in his test cases, I think
> > you
> >> > may take a look first.
> >> > https://reviews.apache.org/r/46097/diff/7#index_header
> >> >
> >> > On Tue, May 17, 2016 at 4:20 PM, Frank Scholten
> > <fr...@frankscholten.nl>
> >> > wrote:
> >> >>
> >> >> Hi all,
> >> >>
> >> >> I tried out CNI support with Calico but I am not sure what I have to
> >> >> do to create an IP for a task.
> >> >>
> >> >> See this sandbox repository on Github
> >> >>
> >> >> https://github.com/ContainerSolutions/mesos-calico-cni-sandbox
> >> >>
> >> >> In this repo I build the Mesos master branch and
> >> >> https://github.com/projectcalico/calico-cni, create a local cluster
> >> >> and deploy an application. I don't see anything in the logs about
> cni,
> >> >> except that it loads the cni isolator.
> >> >>
> >> >> Is there some user documentation for this feature? If not I am happy
> >> >> to write documentation once I figure out how this feature works.
> >> >>
> >> >> Cheers,
> >> >>
> >> >> Frank
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Best Regards,
> >> > Haosdent Huang
> >>
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: mesos docker vs native container

2016-04-26 Thread Avinash Sridharan
Docker does use bridged networking by default, but it uses linux bridges to
perform the bridging, there is no docker-proxy process. The problem with
docker bridge network is that the address space of the container spawned on
the docker bridge are different than the host network, so you need to
perform DNAT to get to any docker container on that bridge. The performance
hit is because of the DNAT. If you want container to container
communication on a single docker bridge, or if you configure the address
space of the docker bridge to be that of the host network (which in most
cases is not possible) you can get close to line speed performance.

Docker host networking is effectively attaching containers to the linux
host network namespace. So the performance results will be the same as that
of running a process natively on the host.

On Tue, Apr 26, 2016 at 5:07 PM, Jeff Schroeder <jeffschroe...@computer.org>
wrote:

> I think you might be a bit confused now this all works. Docker by default
> uses bridged networking, which by default spins up a little crappy
> docker-proxy process for every port. You can disable docker-proxy and
> instead use hairpin routing mode if you have a modern kernel. However, I'm
> almost certain that any task you run via docker on mesos default to host
> networking. Docker, LXC, mesos containers all use the Linux kernel network
> namespace + perhaps some iptables/libnl magic for the networking bits.
> Docker in host networking mode will do networking at mostly native speed. I
> suggest you run iperf on mesos in the various configurations. It should be
> pretty straightforward to test the overhead, but I suspect docker + host
> networking will more than work. Give it a go and let us know!
>
>
> On Tuesday, April 26, 2016, vincent gromakowski <
> vincent.gromakow...@gmail.com> wrote:
>
>> Question is more related  to mesos.  I am thinking of using docker
>> instead of native (LXC?) containers but I suspect network performance
>> decrease which is important on big data workloads.
>> Can you explain why its not secured In host mode ?
>> Le 26 avr. 2016 7:51 PM, "Avinash Sridharan" <avin...@mesosphere.io> a
>> écrit :
>>
>>> Hi Vincent,
>>>  What do you mean by native container through Docker? Can you clarify
>>> your question a bit. Also if it's a DC/OS specific question you might want
>>> to post at us...@dcos.io .
>>>
>>> Thanks,
>>> Avinash
>>>
>>> On Tue, Apr 26, 2016 at 10:41 AM, vincent gromakowski <
>>> vincent.gromakow...@gmail.com> wrote:
>>>
>>>> Nobody experienced docker vs native container performance ?
>>>> Le 25 avr. 2016 9:37 AM, "vincent gromakowski" <
>>>> vincent.gromakow...@gmail.com> a écrit :
>>>>
>>>>> I am very interesting in getting some feedback of people who has moved
>>>>> from native container through Docker specially from network performance
>>>>> perspective.
>>>>> DCOS has been open sourced and I like all automation it brings with
>>>>> frameworks but it seems everything is running in docker ?
>>>>> I am looking for the smack stack for which network perf is important.
>>>>> Tx
>>>>>
>>>>
>>>
>>>
>>> --
>>> Avinash Sridharan, Mesosphere
>>> +1 (323) 702 5245
>>>
>>
>
> --
> Text by Jeff, typos by iPhone
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: mesos docker vs native container

2016-04-26 Thread Avinash Sridharan
Hi Vincent,
 What do you mean by native container through Docker? Can you clarify your
question a bit. Also if it's a DC/OS specific question you might want to
post at us...@dcos.io .

Thanks,
Avinash

On Tue, Apr 26, 2016 at 10:41 AM, vincent gromakowski <
vincent.gromakow...@gmail.com> wrote:

> Nobody experienced docker vs native container performance ?
> Le 25 avr. 2016 9:37 AM, "vincent gromakowski" <
> vincent.gromakow...@gmail.com> a écrit :
>
>> I am very interesting in getting some feedback of people who has moved
>> from native container through Docker specially from network performance
>> perspective.
>> DCOS has been open sourced and I like all automation it brings with
>> frameworks but it seems everything is running in docker ?
>> I am looking for the smack stack for which network perf is important.
>> Tx
>>
>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Custom IPTables rules

2016-04-13 Thread Avinash Sridharan
You need a docker chain in the NAT table as well. The output you are
showing is in the default table.

Try "iptable -t nat -L" to list all rules and chain in the NAT table. You
can add the docker chain in the NAT table
"iptable -t nat -N Docker" to create a docker Chain in the NAT table.

As Rad suggested restarting the docker daemon would allow Docker to
recreate all the iptable chains and rules it needs. That might be a cleaner
approach, than trying to insert rules on your own.

On Wed, Apr 13, 2016 at 12:53 PM, Alfredo Carneiro <
alfr...@simbioseventures.com> wrote:

> Hey Rad,
>
> Thanks for your answer! I have added theses lines and now looks very
> similar before.
>
> *iptables -N DOCKER*
> *iptables -A FORWARD -o docker0 -j DOCKER*
> *iptables -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED
> -j ACCEPT*
> *iptables -A FORWARD -i docker0 ! -o docker0 -j ACCEPT*
> *iptables -A FORWARD -i docker0 -o docker0 -j ACCEPT*
>
> However, I am still getting errors.
>
> *docker: Error response from daemon: failed to create endpoint
> cranky_kilby on network bridge: iptables failed: iptables --wait -t nat -A
> DOCKER -p tcp -d 0/0 --dport 8080 -j DNAT --to-destination 172.17.0.2:8080
> <http://172.17.0.2:8080> ! -i docker0: iptables: No chain/target/match by
> that name.*
> * (exit status 1).*
>
> This is my iptables -L output:
>
> *Chain FORWARD (policy DROP)*
> *target prot opt source   destination *
> *DOCKER all  --  anywhere anywhere*
> *ACCEPT all  --  anywhere anywhere ctstate
> RELATED,ESTABLISHED*
> *ACCEPT all  --  anywhere anywhere*
> *ACCEPT all  --  anywhere anywhere*
>
> *Chain OUTPUT (policy ACCEPT)*
> *target prot opt source   destination *
> *ACCEPT all  --  anywhere anywhere*
>
> *Chain DOCKER (1 references)*
> *target prot opt source   destination*
>
> I hid the INPUT chain because is very big!
>
> Best Regards,
>
> On Wed, Apr 13, 2016 at 4:29 PM, Rad Gruchalski <ra...@gruchalski.com>
> wrote:
>
>> Hi Alfredo,
>>
>> The only thing you need is:
>>
>> -A FORWARD -o docker0 -j DOCKER
>> -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
>> -A FORWARD -i docker0 ! -o docker0 -j ACCEPT
>> -A FORWARD -i docker0 -o docker0 -j ACCEPT
>>
>> Best regards,
>> Radek Gruchalski
>> ra...@gruchalski.com <ra...@gruchalski.com>
>> de.linkedin.com/in/radgruchalski/
>>
>>
>> *Confidentiality:*This communication is intended for the above-named
>> person and may be confidential and/or legally privileged.
>> If it has come to you in error you must take no action based on it, nor
>> must you copy or show it to anyone; please delete/destroy and inform the
>> sender immediately.
>>
>> On Wednesday, 13 April 2016 at 21:27, Alfredo Carneiro wrote:
>>
>> Hello guys,
>>
>> I don't know if that is the right place to ask. So, since we use public
>> cloud, we are trying to hardening our servers allowing traffic just from
>> our subnetworks. However, when I tried to implement some iptables rules I
>> got problems with Docker, which couldn't find its chain anymore.
>>
>> Then, I am wondering if anyone has ever implemented any iptables rule in
>> this scenario.
>>
>> I've seen this[1] "tip", however, I think that it is not apply to this
>> case, because it is very "static".
>>
>> [1] - https://fralef.me/docker-and-iptables.html
>>
>> Best Regards,
>>
>> --
>> Alfredo Miranda
>>
>>
>>
>
>
> --
> Alfredo Miranda
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: [Proposal] Remove the default value for agent work_dir

2016-04-12 Thread Avinash Sridharan
+1

On Tue, Apr 12, 2016 at 9:31 PM, Jie Yu <yujie@gmail.com> wrote:

> +1
>
> On Tue, Apr 12, 2016 at 9:29 PM, James Peach <jor...@gmail.com> wrote:
>
> >
> > > On Apr 12, 2016, at 3:58 PM, Greg Mann <g...@mesosphere.io> wrote:
> > >
> > > Hey folks!
> > > A number of situations have arisen in which the default value of the
> > Mesos agent `--work_dir` flag (/tmp/mesos) has caused problems on systems
> > in which the automatic cleanup of '/tmp' deletes agent metadata. To
> resolve
> > this, we would like to eliminate the default value of the agent
> > `--work_dir` flag. You can find the relevant JIRA here.
> > >
> > > We considered simply changing the default value to a more appropriate
> > location, but decided against this because the expected filesystem
> > structure varies from platform to platform, and because it isn't
> guaranteed
> > that the Mesos agent would have access to the default path on a
> particular
> > platform.
> > >
> > > Eliminating the default `--work_dir` value means that the agent would
> > exit immediately if the flag is not provided, whereas currently it
> launches
> > successfully in this case. This will break existing infrastructure which
> > relies on launching the Mesos agent without specifying the work
> directory.
> > I believe this is an acceptable change because '/tmp/mesos' is not a
> > suitable location for the agent work directory except for short-term
> local
> > testing, and any production scenario that is currently using this
> location
> > should be altered immediately.
> >
> > +1 from me too. Defaulting to /tmp just helps people shoot themselves in
> > the foot.
> >
> > J
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Deploying mesos software

2016-03-15 Thread Avinash Sridharan
There are Mesos packages available for various distributions at the
Mesosphere download page.

Would this work ?
https://mesosphere.com/downloads/



On Tue, Mar 15, 2016 at 2:37 PM, Peter Steele <pste...@peaxy.net> wrote:

> I've just downloaded and built mesos for the first time. Once we figure
> things out, we'll want to install the mesos software on hardware different
> than where it is built. Ordinarily we'd have binary only tarballs for the
> software we're installing on our servers but there doesn't appear to be a
> binary only distribution available yet. What's the recommended way to build
> in one place and install in another?
>
> Peter
>
>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: [VOTE] Release Apache Mesos 0.28.0 (rc1)

2016-03-06 Thread Avinash Sridharan
Looks like this issue is not a regression. I can see the same crash on
Debian 8 running on AWS with 0.27.2-rc1 .

The root cause is that the cgroups memory subsystem was disabled. Enabled
it at boot up and all root test cases passed.

BOOT_IMAGE=/boot/vmlinuz-3.16.0-4-amd64
root=UUID=5cfc0e59-b149-4ddc-8741-74bd7e71f91c ro init=/bin/systemd
console=hvc0 console=ttyS0 cgroup_enable=memory

I think the test filters should have caught the absence of the cgroup
memory subsystem. If I can't find a JIRA for this, will go ahead and file
one.



On Fri, Mar 4, 2016 at 6:52 PM, Avinash Sridharan <avin...@mesosphere.io>
wrote:

> Sorry,
> Forgot to mention the platform details. Seeing the crash on Debian 8
> dmin@ip-172-31-26-151:~/mesos/build$ cat /etc/issue
> Debian GNU/Linux 8 \n \l
>
> admin@ip-172-31-26-151:~/mesos/build$ uname -a
> Linux ip-172-31-26-151 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3
> (2016-01-17) x86_64 GNU/Linux
>
> On Fri, Mar 4, 2016 at 6:49 PM, Avinash Sridharan <avin...@mesosphere.io>
> wrote:
>
>> I am seeing the following crash when I run mesos build of 0.28.0-rc1
>> `sudo bin/mesos-tests.sh` :
>> [ RUN  ] MemIsolatorTest/1.MemUsage
>> F0305 02:43:01.791003 32494 isolator_tests.cpp:905] CHECK_SOME(isolator):
>> Failed to create memory cgroup: Failed to mount cg
>> roups hierarchy at '/sys/fs/cgroup/memory': 'memory' is not enabled by
>> the kernel
>> *** Check failure stack trace: ***
>> @ 0x7fc4630a025d  google::LogMessage::Fail()
>> @ 0x7fc46309f63e  google::LogMessage::SendToLog()
>> @ 0x7fc46309ff1d  google::LogMessage::Flush()
>> @ 0x7fc4630a3388  google::LogMessageFatal::~LogMessageFatal()
>> @   0x8bed57  _CheckFatal::~_CheckFatal()
>> @  0x17c0ab7
>>  mesos::internal::tests::MemIsolatorTest_MemUsage_Test<>::TestBody()
>> @  0x1985243
>>  testing::internal::HandleSehExceptionsInMethodIfSupported<>()
>> @  0x1971d31
>>  testing::internal::HandleExceptionsInMethodIfSupported<>()
>> @  0x1952a55  testing::Test::Run()
>> @  0x19536cb  testing::TestInfo::Run()
>> @  0x1953dd7  testing::TestCase::Run()
>> @  0x195b4ce  testing::internal::UnitTestImpl::RunAllTests()
>> @  0x19825d3
>>  testing::internal::HandleSehExceptionsInMethodIfSupported<>()
>> @  0x1973d01
>>  testing::internal::HandleExceptionsInMethodIfSupported<>()
>> @  0x195b18b  testing::UnitTest::Run()
>> @   0xe43611  RUN_ALL_TESTS()
>> @   0xe424dc  main
>> @ 0x7fc45e147b45  (unknown)
>> @   0x8a0bbc  (unknown)
>>
>>
>> Looks like the memory cgroup memory is disabled:
>>
>> admin@ip-172-31-26-151:~/mesos/build$ cat /proc/cgroups
>> #subsys_namehierarchy   num_cgroups enabled
>> cpuset  2   1   1
>> cpu 3   2   1
>> cpuacct 3   2   1
>> memory  0   1   0
>> devices 4   1   1
>> freezer 5   2   1
>> net_cls 6   1   1
>>
>> But I thought the test filters should have caught this ?
>>
>> On Fri, Mar 4, 2016 at 1:52 PM, Kapil Arya <ka...@mesosphere.io> wrote:
>>
>>> +1 (Binding)
>>>
>>> Successful CI builds for the following distros:
>>>
>>> amd64/centos/6
>>> amd64/centos/7
>>> amd64/debian/jessie
>>> amd64/ubuntu/precise
>>> amd64/ubuntu/trusty
>>> amd64/ubuntu/vivid
>>>
>>>
>>> On Fri, Mar 4, 2016 at 1:06 PM, Vinod Kone <vinodk...@apache.org> wrote:
>>>
>>>> Bad copy paste into the email, sorry. It looks fine in the CHANGELOG.
>>>>
>>>> On Fri, Mar 4, 2016 at 12:52 AM, Shuai Lin <linshuai2...@gmail.com>
>>>> wrote:
>>>>
>>>> >   * [MESOS-4712] - Remove 'force' field from the Subscribe Call in v1
>>>> >> Scheduler API.
>>>> >>   * [MESOS-4591] - Change the object of ReserveResources and
>>>> CreateVolume
>>>> >> ACLs to `roles`.
>>>> >>   * [MESOS-4712] - Remove 'force' field from the Subscribe Call in v1
>>>> >> Scheduler API.
>>>> >
>>>> >
>>>> > MESOS-4712 is included twice.
>>>> >
>>>> > On Fri, Mar 4, 2016 at 1:25 PM, Vinod Kone <vinodk...@apache.org>
>>>> wrote:
>>>> >
>>>> >> On Thu, Mar 3, 2016 at 5:43 PM, Vinod Kone <vinodk...@apache.org>
>>>> wrote:
>>>> >>
>>>> >> > Tue Mar  10 17:00:00 PST 2016
>>>> >>
>>>> >>
>>>> >> Sorry. This should be Mar 8th not 10th.
>>>> >>
>>>> >
>>>> >
>>>>
>>>
>>>
>>
>>
>> --
>> Avinash Sridharan, Mesosphere
>> +1 (323) 702 5245
>>
>
>
>
> --
> Avinash Sridharan, Mesosphere
> +1 (323) 702 5245
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: [VOTE] Release Apache Mesos 0.28.0 (rc1)

2016-03-04 Thread Avinash Sridharan
Sorry,
Forgot to mention the platform details. Seeing the crash on Debian 8
dmin@ip-172-31-26-151:~/mesos/build$ cat /etc/issue
Debian GNU/Linux 8 \n \l

admin@ip-172-31-26-151:~/mesos/build$ uname -a
Linux ip-172-31-26-151 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3
(2016-01-17) x86_64 GNU/Linux

On Fri, Mar 4, 2016 at 6:49 PM, Avinash Sridharan <avin...@mesosphere.io>
wrote:

> I am seeing the following crash when I run mesos build of 0.28.0-rc1 `sudo
> bin/mesos-tests.sh` :
> [ RUN  ] MemIsolatorTest/1.MemUsage
> F0305 02:43:01.791003 32494 isolator_tests.cpp:905] CHECK_SOME(isolator):
> Failed to create memory cgroup: Failed to mount cg
> roups hierarchy at '/sys/fs/cgroup/memory': 'memory' is not enabled by the
> kernel
> *** Check failure stack trace: ***
> @ 0x7fc4630a025d  google::LogMessage::Fail()
> @ 0x7fc46309f63e  google::LogMessage::SendToLog()
> @ 0x7fc46309ff1d  google::LogMessage::Flush()
> @ 0x7fc4630a3388  google::LogMessageFatal::~LogMessageFatal()
> @   0x8bed57  _CheckFatal::~_CheckFatal()
> @  0x17c0ab7
>  mesos::internal::tests::MemIsolatorTest_MemUsage_Test<>::TestBody()
> @  0x1985243
>  testing::internal::HandleSehExceptionsInMethodIfSupported<>()
> @  0x1971d31
>  testing::internal::HandleExceptionsInMethodIfSupported<>()
> @  0x1952a55  testing::Test::Run()
> @  0x19536cb  testing::TestInfo::Run()
> @  0x1953dd7  testing::TestCase::Run()
> @  0x195b4ce  testing::internal::UnitTestImpl::RunAllTests()
> @  0x19825d3
>  testing::internal::HandleSehExceptionsInMethodIfSupported<>()
> @  0x1973d01
>  testing::internal::HandleExceptionsInMethodIfSupported<>()
> @  0x195b18b  testing::UnitTest::Run()
> @   0xe43611  RUN_ALL_TESTS()
> @   0xe424dc  main
> @ 0x7fc45e147b45  (unknown)
> @   0x8a0bbc  (unknown)
>
>
> Looks like the memory cgroup memory is disabled:
>
> admin@ip-172-31-26-151:~/mesos/build$ cat /proc/cgroups
> #subsys_namehierarchy   num_cgroups enabled
> cpuset  2   1   1
> cpu 3   2   1
> cpuacct 3   2   1
> memory  0   1   0
> devices 4   1   1
> freezer 5   2   1
> net_cls 6   1   1
>
> But I thought the test filters should have caught this ?
>
> On Fri, Mar 4, 2016 at 1:52 PM, Kapil Arya <ka...@mesosphere.io> wrote:
>
>> +1 (Binding)
>>
>> Successful CI builds for the following distros:
>>
>> amd64/centos/6
>> amd64/centos/7
>> amd64/debian/jessie
>> amd64/ubuntu/precise
>> amd64/ubuntu/trusty
>> amd64/ubuntu/vivid
>>
>>
>> On Fri, Mar 4, 2016 at 1:06 PM, Vinod Kone <vinodk...@apache.org> wrote:
>>
>>> Bad copy paste into the email, sorry. It looks fine in the CHANGELOG.
>>>
>>> On Fri, Mar 4, 2016 at 12:52 AM, Shuai Lin <linshuai2...@gmail.com>
>>> wrote:
>>>
>>> >   * [MESOS-4712] - Remove 'force' field from the Subscribe Call in v1
>>> >> Scheduler API.
>>> >>   * [MESOS-4591] - Change the object of ReserveResources and
>>> CreateVolume
>>> >> ACLs to `roles`.
>>> >>   * [MESOS-4712] - Remove 'force' field from the Subscribe Call in v1
>>> >> Scheduler API.
>>> >
>>> >
>>> > MESOS-4712 is included twice.
>>> >
>>> > On Fri, Mar 4, 2016 at 1:25 PM, Vinod Kone <vinodk...@apache.org>
>>> wrote:
>>> >
>>> >> On Thu, Mar 3, 2016 at 5:43 PM, Vinod Kone <vinodk...@apache.org>
>>> wrote:
>>> >>
>>> >> > Tue Mar  10 17:00:00 PST 2016
>>> >>
>>> >>
>>> >> Sorry. This should be Mar 8th not 10th.
>>> >>
>>> >
>>> >
>>>
>>
>>
>
>
> --
> Avinash Sridharan, Mesosphere
> +1 (323) 702 5245
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: [VOTE] Release Apache Mesos 0.28.0 (rc1)

2016-03-04 Thread Avinash Sridharan
I am seeing the following crash when I run mesos build of 0.28.0-rc1 `sudo
bin/mesos-tests.sh` :
[ RUN  ] MemIsolatorTest/1.MemUsage
F0305 02:43:01.791003 32494 isolator_tests.cpp:905] CHECK_SOME(isolator):
Failed to create memory cgroup: Failed to mount cg
roups hierarchy at '/sys/fs/cgroup/memory': 'memory' is not enabled by the
kernel
*** Check failure stack trace: ***
@ 0x7fc4630a025d  google::LogMessage::Fail()
@ 0x7fc46309f63e  google::LogMessage::SendToLog()
@ 0x7fc46309ff1d  google::LogMessage::Flush()
@ 0x7fc4630a3388  google::LogMessageFatal::~LogMessageFatal()
@   0x8bed57  _CheckFatal::~_CheckFatal()
@  0x17c0ab7
 mesos::internal::tests::MemIsolatorTest_MemUsage_Test<>::TestBody()
@  0x1985243
 testing::internal::HandleSehExceptionsInMethodIfSupported<>()
@  0x1971d31
 testing::internal::HandleExceptionsInMethodIfSupported<>()
@  0x1952a55  testing::Test::Run()
@  0x19536cb  testing::TestInfo::Run()
@  0x1953dd7  testing::TestCase::Run()
@  0x195b4ce  testing::internal::UnitTestImpl::RunAllTests()
@  0x19825d3
 testing::internal::HandleSehExceptionsInMethodIfSupported<>()
@  0x1973d01
 testing::internal::HandleExceptionsInMethodIfSupported<>()
@  0x195b18b  testing::UnitTest::Run()
@   0xe43611  RUN_ALL_TESTS()
@   0xe424dc  main
@ 0x7fc45e147b45  (unknown)
@   0x8a0bbc  (unknown)


Looks like the memory cgroup memory is disabled:

admin@ip-172-31-26-151:~/mesos/build$ cat /proc/cgroups
#subsys_namehierarchy   num_cgroups enabled
cpuset  2   1   1
cpu 3   2   1
cpuacct 3   2   1
memory  0   1   0
devices 4   1   1
freezer 5   2   1
net_cls 6   1   1

But I thought the test filters should have caught this ?

On Fri, Mar 4, 2016 at 1:52 PM, Kapil Arya <ka...@mesosphere.io> wrote:

> +1 (Binding)
>
> Successful CI builds for the following distros:
>
> amd64/centos/6
> amd64/centos/7
> amd64/debian/jessie
> amd64/ubuntu/precise
> amd64/ubuntu/trusty
> amd64/ubuntu/vivid
>
>
> On Fri, Mar 4, 2016 at 1:06 PM, Vinod Kone <vinodk...@apache.org> wrote:
>
>> Bad copy paste into the email, sorry. It looks fine in the CHANGELOG.
>>
>> On Fri, Mar 4, 2016 at 12:52 AM, Shuai Lin <linshuai2...@gmail.com>
>> wrote:
>>
>> >   * [MESOS-4712] - Remove 'force' field from the Subscribe Call in v1
>> >> Scheduler API.
>> >>   * [MESOS-4591] - Change the object of ReserveResources and
>> CreateVolume
>> >> ACLs to `roles`.
>> >>   * [MESOS-4712] - Remove 'force' field from the Subscribe Call in v1
>> >> Scheduler API.
>> >
>> >
>> > MESOS-4712 is included twice.
>> >
>> > On Fri, Mar 4, 2016 at 1:25 PM, Vinod Kone <vinodk...@apache.org>
>> wrote:
>> >
>> >> On Thu, Mar 3, 2016 at 5:43 PM, Vinod Kone <vinodk...@apache.org>
>> wrote:
>> >>
>> >> > Tue Mar  10 17:00:00 PST 2016
>> >>
>> >>
>> >> Sorry. This should be Mar 8th not 10th.
>> >>
>> >
>> >
>>
>
>


-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Need CHANGELOG updates

2016-03-03 Thread Avinash Sridharan
net_cls isolator update under review:
https://reviews.apache.org/r/44350/

On Thu, Mar 3, 2016 at 11:11 AM, Avinash Sridharan <avin...@mesosphere.io>
wrote:

> Hi Vinod,
>  Will file a ticket and update the change log for the net_cls isolator.
>
> -Avinash
>
> On Thu, Mar 3, 2016 at 11:08 AM, Neil Conway <neil.con...@gmail.com>
> wrote:
>
>> I sent https://reviews.apache.org/r/44348/ for the floating point math
>> changes; if you'd prefer a different format or more/less details, just
>> let me know.
>>
>> Thanks,
>> Neil
>>
>> On Thu, Mar 3, 2016 at 10:57 AM, Vinod Kone <vi...@mesosphere.io> wrote:
>> > Hi guys,
>> >
>> > The 0.28.0 release is currently blocked on the updates to CHANGELOG.
>> > Basically I'm looking for shepherds/owners of feature tickets to add a
>> > blurb in the CHANGELOG for their tickets.
>> >
>> > The big ticket items that went into 0.28.0 that I know of
>> >
>> > --> net_cls_isolator
>> > --> floating point math for resources
>> > --> unified containerizer
>> > --> executor api v1
>> >
>> > If there are other things that need to be called out specifically in the
>> > CHANGELOG, please reach out to me.
>> >
>> > Thanks,
>>
>
>
>
> --
> Avinash Sridharan, Mesosphere
> +1 (323) 702 5245
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Need CHANGELOG updates

2016-03-03 Thread Avinash Sridharan
Hi Vinod,
 Will file a ticket and update the change log for the net_cls isolator.

-Avinash

On Thu, Mar 3, 2016 at 11:08 AM, Neil Conway <neil.con...@gmail.com> wrote:

> I sent https://reviews.apache.org/r/44348/ for the floating point math
> changes; if you'd prefer a different format or more/less details, just
> let me know.
>
> Thanks,
> Neil
>
> On Thu, Mar 3, 2016 at 10:57 AM, Vinod Kone <vi...@mesosphere.io> wrote:
> > Hi guys,
> >
> > The 0.28.0 release is currently blocked on the updates to CHANGELOG.
> > Basically I'm looking for shepherds/owners of feature tickets to add a
> > blurb in the CHANGELOG for their tickets.
> >
> > The big ticket items that went into 0.28.0 that I know of
> >
> > --> net_cls_isolator
> > --> floating point math for resources
> > --> unified containerizer
> > --> executor api v1
> >
> > If there are other things that need to be called out specifically in the
> > CHANGELOG, please reach out to me.
> >
> > Thanks,
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: Multicast blocked?

2016-03-01 Thread Avinash Sridharan
Hi David,
 Is your discovery service part of the Executor ? Some more context of the
interaction of the Executor and the discovery service would be helpful. If
the discovery service is relying specifically on multicast, and you no the
packet payload for the discovery messages, might help to run tcpdump on the
NIC to see if the messages are going out when you start your executor.

Thanks,
Avinash

On Tue, Mar 1, 2016 at 1:40 PM, David Wood <daw...@us.ibm.com> wrote:

> I'm building an Executor that uses a separate discovery service that
> relies on multicast.  If I run the executor without calling
> MesosExecutorDriver.run() the executor discovers other (non-mesos) nodes in
> my distributed set up just fine.  But if I call run(), then it does not
> discover the other nodes.  Interestingly, the other  nodes can discover my
> executor regardless of whether run() is called or not.   My executor is a
> Java app started using a command line task.  Any help is much appreciated.
>
>
> David Wood
> Computing Systems for Wireless Networks
> IBM TJ Watson Research Center
> daw...@us.ibm.com
> 914-945-4923 (office), 914-396-6515 (mobile)
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245


Re: The issue of "Failed to shutdown socket with fd xx: Transport endpoint is not connected" on Mesos master

2015-12-30 Thread Avinash Sridharan
Thanks for the update Nan. k8s enabling firewall rules that would block
traffic to the master seems a bit odd. Looks like a bug to me, in the head
of the branch. If you are able to reproduce it consistently, could you file
an issue against kubernetes mesos.

regards,
Avinash

On Tue, Dec 29, 2015 at 11:04 PM, Nan Xiao <xiaonan830...@gmail.com> wrote:

> Hi Avinash,
>
> Thanks very much for your reply!
>
> The root cause has been found: the k8s server has enabled the iptables
> which blocks connection from
> Mesos master; after disable it, it works!
>
> Best Regards
> Nan Xiao
>
>
> On Wed, Dec 30, 2015 at 3:22 AM, Avinash Sridharan
> <avin...@mesosphere.io> wrote:
> > lsof command will show only actively opened file descriptors. So if you
> ran
> > the command after seeing the error logs in the master, most probably the
> > master had already closed this fd. Just throwing a few other things to
> look
> > at, that might give some more insights.
> >
> > * Run the "netstat -na" and netstat -nt" commands on the master and the
> > kubernetes master node to make sure that the master is listening to the
> > right port, and the k8s scheduler is trying to connect to the right port.
> > From the logs it does look like the master is receiving the registration
> > request, so there shouldn't be a network configuration issue here.
> > * Make sure there are no firewall rules getting turned on in your cluster
> > since it looks like the k8s scheduler is not able to connect to the
> master
> > (though it was able to register the first time).
> >
> > On Tue, Dec 29, 2015 at 1:37 AM, Nan Xiao <xiaonan830...@gmail.com>
> wrote:
> >>
> >> BTW, using "lsof" command finds there are only 16 file descriptors. I
> >> don't know why Mesos
> >> master try to close "fd 17".
> >> Best Regards
> >> Nan Xiao
> >>
> >>
> >> On Tue, Dec 29, 2015 at 11:32 AM, Nan Xiao <xiaonan830...@gmail.com>
> >> wrote:
> >> > Hi Klaus,
> >> >
> >> > Firstly, thanks very much for your answer!
> >> >
> >> > The km processes are all live:
> >> > root 129474 128024  2 22:26 pts/000:00:00 km apiserver
> >> > --address=15.242.100.60 --etcd-servers=http://15.242.100.60:4001
> >> > --service-cluster-ip-range=10.10.10.0/24 --port=
> >> > --cloud-provider=mesos --cloud-config=mesos-cloud.conf --secure-port=0
> >> > --v=1
> >> > root 129509 128024  2 22:26 pts/000:00:00 km
> >> > controller-manager --master=15.242.100.60: --cloud-provider=mesos
> >> > --cloud-config=./mesos-cloud.conf --v=1
> >> > root 129538 128024  0 22:26 pts/000:00:00 km scheduler
> >> > --address=15.242.100.60 --mesos-master=15.242.100.56:5050
> >> > --etcd-servers=http://15.242.100.60:4001 --mesos-user=root
> >> > --api-servers=15.242.100.60: --cluster-dns=10.10.10.10
> >> > --cluster-domain=cluster.local --v=2
> >> >
> >> > All the logs are also seem OK, except the logs from scheduler.log:
> >> > ..
> >> > I1228 22:26:37.883092  129538 messenger.go:381] Receiving message
> >> > mesos.internal.InternalMasterChangeDetected from
> >> > scheduler(1)@15.242.100.60:33077
> >> > I1228 22:26:37.883225  129538 scheduler.go:374] New master
> >> > master@15.242.100.56:5050 detected
> >> > I1228 22:26:37.883268  129538 scheduler.go:435] No credentials were
> >> > provided. Attempting to register scheduler without authentication.
> >> > I1228 22:26:37.883356  129538 scheduler.go:928] Registering with
> >> > master: master@15.242.100.56:5050
> >> > I1228 22:26:37.883460  129538 messenger.go:187] Sending message
> >> > mesos.internal.RegisterFrameworkMessage to master@15.242.100.56:5050
> >> > I1228 22:26:37.883504  129538 scheduler.go:881] will retry
> >> > registration in 1.209320575s if necessary
> >> > I1228 22:26:37.883758  129538 http_transporter.go:193] Sending message
> >> > to master@15.242.100.56:5050 via http
> >> > I1228 22:26:37.883873  129538 http_transporter.go:587] libproc target
> >> > URL
> >> >
> http://15.242.100.56:5050/master/mesos.internal.RegisterFrameworkMessage
> >> > I1228 22:26:39.093560  129538 scheduler.go:928] Registering with
> >> > master: master@15.242.100.56:5050
> >> > I1228 22:26:39.093659  129538 messenger.go:187] Sending message
> &g

Re: The issue of "Failed to shutdown socket with fd xx: Transport endpoint is not connected" on Mesos master

2015-12-29 Thread Avinash Sridharan
;>> Hi all,
> >>>
> >>> Greetings from me!
> >>>
> >>> I am trying to follow this tutorial
> >>>
> >>> (
> https://github.com/kubernetes/kubernetes/blob/master/docs/getting-started-guides/mesos.md
> )
> >>> to deploy "k8s on Mesos" on local machines: The k8s is the newest
> >>> master branch, and Mesos is the 0.26 edition.
> >>>
> >>> After running Mesos master(IP:15.242.100.56), Mesos
> >>> slave(IP:15.242.100.16),, and the k8s(IP:15.242.100.60), I can see the
> >>> following logs from Mesos master:
> >>>
> >>> ..
> >>> I1227 22:52:34.494478  8069 master.cpp:4269] Received update of slave
> >>> 9c3c6c78-0b62-4eaa-b27a-498f172e7fe6-S0 at slave(1)@15.242.100.16:5051
> >>> (pqsfc016.ftc.rdlabs.hpecorp.net) with total oversubscribed resources
> >>> I1227 22:52:34.494940  8065 hierarchical.cpp:400] Slave
> >>> 9c3c6c78-0b62-4eaa-b27a-498f172e7fe6-S0
> >>> (pqsfc016.ftc.rdlabs.hpecorp.net) updated with oversubscribed
> >>> resources  (total: cpus(*):32; mem(*):127878; disk(*):4336;
> >>> ports(*):[31000-32000], allocated: )
> >>> I1227 22:53:06.740757 8053 http.cpp:334] HTTP GET for
> >>> /master/state.json from 15.242.100.60:56219 with
> >>> User-Agent='Go-http-client/1.1'
> >>> I1227 22:53:07.736419 8065 http.cpp:334] HTTP GET for
> >>> /master/state.json from 15.242.100.60:56241 with
> >>> User-Agent='Go-http-client/1.1'
> >>> I1227 22:53:07.767196  8070 http.cpp:334] HTTP GET for
> >>> /master/state.json from 15.242.100.60:56252 with
> >>> User-Agent='Go-http-client/1.1'
> >>> I1227 22:53:08.808171  8053 http.cpp:334] HTTP GET for
> >>> /master/state.json from 15.242.100.60:56272 with
> >>> User-Agent='Go-http-client/1.1'
> >>> I1227 22:53:08.815811 8060 master.cpp:2176] Received SUBSCRIBE call
> >>> for framework 'Kubernetes' at scheduler(1)@15.242.100.60:59488
> >>> I1227 22:53:08.816182 8060 master.cpp:2247] Subscribing framework
> >>> Kubernetes with checkpointing enabled and capabilities [  ]
> >>> I1227 22:53:08.817294  8052 hierarchical.cpp:195] Added framework
> >>> 9c3c6c78-0b62-4eaa-b27a-498f172e7fe6-
> >>> I1227 22:53:08.817464  8050 master.cpp:1122] Framework
> >>> 9c3c6c78-0b62-4eaa-b27a-498f172e7fe6- (Kubernetes) at
> >>> scheduler(1)@15.242.100.60:59488 disconnected
> >>> E1227 22:53:08.817497 8073 process.cpp:1911] Failed to shutdown
> >>> socket with fd 17: Transport endpoint is not connected
> >>> I1227 22:53:08.817533  8050 master.cpp:2472] Disconnecting framework
> >>> 9c3c6c78-0b62-4eaa-b27a-498f172e7fe6- (Kubernetes) at
> >>> scheduler(1)@15.242.100.60:59488
> >>> I1227 22:53:08.817595 8050 master.cpp:2496] Deactivating framework
> >>> 9c3c6c78-0b62-4eaa-b27a-498f172e7fe6- (Kubernetes) at
> >>> scheduler(1)@15.242.100.60:59488
> >>> I1227 22:53:08.817797 8050 master.cpp:1146] Giving framework
> >>> 9c3c6c78-0b62-4eaa-b27a-498f172e7fe6- (Kubernetes) at
> >>> scheduler(1)@15.242.100.60:59488 7625.14222623576weeks to failover
> >>> W1227 22:53:08.818389 8062 master.cpp:4840] Master returning
> >>> resources offered to framework
> >>> 9c3c6c78-0b62-4eaa-b27a-498f172e7fe6- because the framework has
> >>> terminated or is inactive
> >>> I1227 22:53:08.818397  8052 hierarchical.cpp:273] Deactivated
> >>> framework 9c3c6c78-0b62-4eaa-b27a-498f172e7fe6-
> >>> I1227 22:53:08.819046  8066 hierarchical.cpp:744] Recovered
> >>> cpus(*):32; mem(*):127878; disk(*):4336; ports(*):[31000-32000]
> >>> (total: cpus(*):32; mem(*):127878; disk(*):4336;
> >>> ports(*):[31000-32000], allocated: ) on slave
> >>> 9c3c6c78-0b62-4eaa-b27a-498f172e7fe6-S0 from framework
> >>> 9c3c6c78-0b62-4eaa-b27a-498f172e7fe6-
> >>> ..
> >>>
> >>> I can't figure out why Mesos master complains "Failed to shutdown
> >>> socket with fd 17: Transport endpoint is not connected".
> >>> Could someone give some clues on this issue?
> >>>
> >>> Thanks very much in advance!
> >>>
> >>> Best Regards
> >>> Nan Xiao
> >>
> >>
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245