Re: Is it possible to monitor resource usage per-task for the same executor?

[Alex Rukletsov]

In mesos, resources are isolated and accounted per container. A task is
basically a description, it is up to an executor how to interpret it. In
some cases, for example if an executor *just* creates a message in its
internal queue for incoming tasks, it is almost impossible to track
resource usage per task.
On 2 Nov 2015 2:00 pm, "sujz" <43183...@qq.com> wrote:

> Hi, all:
> If we submit a job to framework like Spark, slave node runs our job
> concurrently with launching multiple tasks within the same container, I am
> not sure these tasks are run in per-process or per-thread? If they are in
> thread, can we  monitor resource usage for each task in mesos?
>
> Thank you!

Re: mess slave can't register to master via master ip:port

[haosdent]

I think it is not correct.

On Tue, Nov 3, 2015 at 12:44 PM, Xiaodong Zhang  wrote:

> If that so. I think this document should be modified.
>
> http://mesos.apache.org/documentation/latest/configuration/#SlaveOptions
>
>
> Right?
>
>
> 发件人: Guangya Liu 
> 答复: "user@mesos.apache.org" 
> 日期: 2015年11月3日 星期二 下午12:39
> 至: "user@mesos.apache.org" 
> 主题: Re: mess slave can't register to master via master ip:port
>
> Seems mesos does not support such mode, please refer to
> https://github.com/apache/mesos/blob/master/src/slave/main.cpp#L105-L111
> for the format of "--master". Thanks!
>
> On Tue, Nov 3, 2015 at 12:28 PM, haosdent  wrote:
>
>> After checking code, seems Mesos only support --master=IP1:5050
>> or --master=zk://xx or --master=file:///.
>>
>> On Tue, Nov 3, 2015 at 12:15 PM, haosdent  wrote:
>>
>>> Do your masters have already managed by zookeeper? And what is your
>>> master start command?
>>>
>>> On Tue, Nov 3, 2015 at 12:06 PM, Xiaodong Zhang 
>>> wrote:
>>>
 Hi all:

 My slave command like this:

 /usr/sbin/mesos-slave --master=IP1:5050,IP2:5050,IP3:5050 ….
 --credential …

 Only if IP1 is the leader, the slave can register to master
 successfully, Or it will register fail.

 Slave log like this:

 Creating new client SASL connection
 Authentication timed out
 Failed to authenticate with master master@172.31.43.77:5050:
 Authentication discarded
 Authenticating with master master@172.31.43.77:5050
 Using default CRAM-MD5 authenticatee

 Is this a bug?Or it is designed like this.

 BTW: --master:zk://xxx work well.

>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>>
>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>
>


-- 
Best Regards,
Haosdent Huang

Re: mess slave can't register to master via master ip:port

[Guangya Liu]

I filed a jira ticket https://issues.apache.org/jira/browse/MESOS-3822 to
trace this. Thanks.

On Tue, Nov 3, 2015 at 2:02 PM, haosdent  wrote:

> I think it is not correct.
>
> On Tue, Nov 3, 2015 at 12:44 PM, Xiaodong Zhang  wrote:
>
>> If that so. I think this document should be modified.
>>
>> http://mesos.apache.org/documentation/latest/configuration/#SlaveOptions
>>
>>
>> Right?
>>
>>
>> 发件人: Guangya Liu 
>> 答复: "user@mesos.apache.org" 
>> 日期: 2015年11月3日 星期二 下午12:39
>> 至: "user@mesos.apache.org" 
>> 主题: Re: mess slave can't register to master via master ip:port
>>
>> Seems mesos does not support such mode, please refer to
>> https://github.com/apache/mesos/blob/master/src/slave/main.cpp#L105-L111
>> for the format of "--master". Thanks!
>>
>> On Tue, Nov 3, 2015 at 12:28 PM, haosdent  wrote:
>>
>>> After checking code, seems Mesos only support --master=IP1:5050
>>> or --master=zk://xx or --master=file:///.
>>>
>>> On Tue, Nov 3, 2015 at 12:15 PM, haosdent  wrote:
>>>
 Do your masters have already managed by zookeeper? And what is your
 master start command?

 On Tue, Nov 3, 2015 at 12:06 PM, Xiaodong Zhang 
 wrote:

> Hi all:
>
> My slave command like this:
>
> /usr/sbin/mesos-slave --master=IP1:5050,IP2:5050,IP3:5050 ….
> --credential …
>
> Only if IP1 is the leader, the slave can register to master
> successfully, Or it will register fail.
>
> Slave log like this:
>
> Creating new client SASL connection
> Authentication timed out
> Failed to authenticate with master master@172.31.43.77:5050:
> Authentication discarded
> Authenticating with master master@172.31.43.77:5050
> Using default CRAM-MD5 authenticatee
>
> Is this a bug?Or it is designed like this.
>
> BTW: --master:zk://xxx work well.
>



 --
 Best Regards,
 Haosdent Huang

>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>>
>>
>>
>
>
> --
> Best Regards,
> Haosdent Huang
>

Re: Can't start docker container when SSL_ENABLED is on.

[Xiaodong Zhang]

Hi, haosdent.

1、command line arguments works not well.

Command:

/usr/sbin/mesos-slave --master=zk://xxx/mesos --log_dir=/var/log/mesos 
--containerizers=docker,mesos --credential=/etc/mesos-slave-auth 
--docker=/usr/bin/docker --executor_environment_variables={"SSL_KEY_FILE": 
"/home/ubuntu/cert/xxx.pem", "SSL_CERT_FILE": "/home/ubuntu/cert/xxx.key", 
"SSL_ENABLED": "true"} --executor_registration_timeout=60mins

env without ssl

Error info:

stderr:
Could not load cert file

Stdout:
--container="mesos-20151102-085117-3565115700-5050-25211-S1.2b784e8d-0bdd-4ffa-a7db-b6dcf35f0a03"
 --docker="/usr/bin/docker" --help="false" --initialize_driver_logging="true" 
--logbufsecs="0" --logging_level="INFO" --mapped_directory="/mnt/mesos/sandbox" 
--quiet="false" 
--sandbox_directory="/tmp/mesos/slaves/20151102-085117-3565115700-5050-25211-S1/frameworks/20151102-085117-3565115700-5050-25211-/executors/c310fa88-af8e-4fdd-92b6-eabf372bd187.85ff0237-8140-11e5-a875-021121f8fdf7/runs/2b784e8d-0bdd-4ffa-a7db-b6dcf35f0a03"
 --stop_timeout=“0ns"


2、the patch works well.(thanks again)

1 and 2 read the same cert file.

The format of the cert file like this:

-BEGIN CERTIFICATE-
Xx
-END CERTIFICATE——

发件人: Xiaodong Zhang <xdzh...@alauda.io<mailto:xdzh...@alauda.io>>
日期: 2015年11月2日 星期一 上午11:22
至: "user@mesos.apache.org<mailto:user@mesos.apache.org>" 
<user@mesos.apache.org<mailto:user@mesos.apache.org>>
主题: Re: Can't start docker container when SSL_ENABLED is on.

Thanks@haosdent

I will test the command line arguments and then test patch.

Have a nice day!~~

发件人: haosdent <haosd...@gmail.com<mailto:haosd...@gmail.com>>
答复: "user@mesos.apache.org<mailto:user@mesos.apache.org>" 
<user@mesos.apache.org<mailto:user@mesos.apache.org>>
日期: 2015年11月1日 星期日 下午5:40
至: user <user@mesos.apache.org<mailto:user@mesos.apache.org>>
主题: Re: Can't start docker container when SSL_ENABLED is on.

@Xiaodong I create a ticket to trace this 
https://issues.apache.org/jira/browse/MESOS-3815 and post a patch in it. Feel 
free to review and test it together. Thank you!

On Sun, Nov 1, 2015 at 4:54 PM, haosdent 
<haosd...@gmail.com<mailto:haosd...@gmail.com>> wrote:
Hi, @Xiaodong I could reproduce your problem in my testing today. A quickly 
workaround is adding environment variables when you launch slave.

```
./bin/mesos-slave.sh  --containerizers=docker,mesos 
--executor_environment_variables='{"SSL_KEY_FILE": "/tmp/server.key", 
"SSL_CERT_FILE": "/tmp/ssl.chain.crt", "SSL_ENABLED": "true"}''
```

As you see above, pass the ssl env to docker-executor through specifying 
--executor_environment_variables when starting. So far it works well for me. 
Anyway I would submit a patch later to fix the docker environment variables 
passing. After that, you could launch slave without 
executor_environment_variables flag.

On Sat, Oct 31, 2015 at 2:56 PM, Tim Chen 
<t...@mesosphere.io<mailto:t...@mesosphere.io>> wrote:
Hi Xiaodong,

If you follow the reviewboard you'll see that the fix is not correct, I believe 
Jojy will be posting a new patch.

Tim

On Fri, Oct 30, 2015 at 6:58 PM, Xiaodong Zhang 
<xdzh...@alauda.io<mailto:xdzh...@alauda.io>> wrote:
it is still not working!

Only if I remove SSL_ENABLED from envs before I start the slave it works well.

I applied the patch in version 0.24.1. And rebuild it with `--enable-libevent 
--enable-ssl` 。

发件人: Xiaodong Zhang <xdzh...@alauda.io<mailto:xdzh...@alauda.io>>
日期: 2015年10月31日 星期六 上午7:45

至: "user@mesos.apache.org<mailto:user@mesos.apache.org>" 
<user@mesos.apache.org<mailto:user@mesos.apache.org>>
主题: Re: Can't start docker container when SSL_ENABLED is on.

Thanks Jojy.

I will patch this in version 0.24.1, and rebuild it. I will let you know if it 
work well after I finish testing.

发件人: Jojy Varghese <j...@mesosphere.io<mailto:j...@mesosphere.io>>
答复: "user@mesos.apache.org<mailto:user@mesos.apache.org>" 
<user@mesos.apache.org<mailto:user@mesos.apache.org>>
日期: 2015年10月31日 星期六 上午12:45
至: "user@mesos.apache.org<mailto:user@mesos.apache.org>" 
<user@mesos.apache.org<mailto:user@mesos.apache.org>>
主题: Re: Can't start docker container when SSL_ENABLED is on.

Thanks Xiaodong.

Based on the hypothesis that the container process launched with SSL_ENABLED in 
environment is the problem, I have created a patch 
https://reviews.apache.org/r/39818/.  This might be a quick and dirty was to 
test the hypothesis. Would it be possible for you to test again after applying 
the patch?

-Jojy



On Oct 30, 2015, at 8:29 AM, Xiaodong Zhang 

Re: Marathon 0.11.1 - Mesos 0.25 - Mesos-DNS 0.4.0

[James DeFelice]

What settings worked for you? We did aim for least surprise. Sounds like we
missed a bit. We're happy to accept suggestions for improvement via gh
issues filed against the mesos-dns repo.
On Oct 29, 2015 7:39 AM, "John Omernik"  wrote:

> That is good to know, however, I would challenge the group on something
> like this not being bug based on the documentation.  When a change in
> mesos-dns, and what fields it looks at is not affected by the mesos-dns
> component, but instead other components in a way that could have serious
> negative impacts on folks who are running this, there should be some
> fanfare there about changes.  Also, I would advocate that in mesos-dns the
> default should have been the same as previous releases (which I would
> assume was host ip) as default, then allow people who are aware of the
> underpinnings to make the change.
>
> On Wed, Oct 28, 2015 at 3:02 PM, Grzegorz Graczyk 
> wrote:
>
>> It's not a bug, it's a feature -
>> http://mesosphere.github.io/mesos-dns/docs/configuration-parameters.html look
>> at IPSources config
>>
>> śr., 28.10.2015 o 15:59 użytkownik John Omernik 
>> napisał:
>>
>>> If I rolled back mesos-dns to v0.2.0 (on the releases page) then it
>>> pulls the right IP address..   (Mesos-dns version is the easiest of the
>>> three to change)
>>>
>>> John
>>>
>>> On Wed, Oct 28, 2015 at 9:52 AM, John Omernik  wrote:
>>>
 So, the issues that are listed appear to be resolved with marathon
 0.11.1, and the mesos-dns issue is not listed at all.

 Note, I tried mesos-dns 0.3.0 and that has the same problem as 0.4.0.

 On Wed, Oct 28, 2015 at 9:46 AM, John Omernik  wrote:

> I will check out those issues and report back.
>
> On Wed, Oct 28, 2015 at 9:42 AM, craig w  wrote:
>
>> I've had no issue with the following combination:
>>
>> MesosDNS 0.4.0
>> Marathon 0.11.0
>> Mesos 0.24.1
>>
>> I've been waiting to upgrade to Mesos 0.25.0 because of issues
>> mentioned in the mesos mailing list regarding Marathon 0.11.x and Mesos
>> 0.25.0
>>
>> On Wed, Oct 28, 2015 at 10:38 AM, John Omernik 
>> wrote:
>>
>>> Hey all -
>>>
>>> I am cross posting this because it's a number of moving parts that
>>> could be at issue here (Mesos, Mesos-dns, and/or Marathon).
>>>
>>> Basically: At the version combination in Subject, the IP that is
>>> registered in mesos-dns for Docker containers running in Marathon is the
>>> internal (container) IP address of the docker (in bridged mode) not the
>>> nodes. This obviously causes issues.  Note this doesn't happen when the
>>> Marathon application is non-Docker.
>>>
>>> I was running Mesos-dns 0.4.0 on a cluster running Mesos 0.24.0 and
>>> Marathon 0.10.0 and I upgraded to Mesos 0.25.0 and Marathon 0.11.1 and
>>> noticed this behavior happening.
>>>
>>> I thought that was odd because I have another cluster that was
>>> running Mesos 0.25.0 and Marathon 0.11.1 and it wasn't happening, until 
>>> I
>>> realized that I hadn't upgraded Mesos-dns lately, I upgraded to 
>>> Mesos-dns
>>> 0.4.0 and the problem started occurring.
>>>
>>> Is there a setting that I need to use the external IP of the
>>> container? Is this issue known? Is there a workaround? This is pretty 
>>> major
>>> for Docker running on Marathon and using Mesos-dns for service 
>>> discovery.
>>>
>>> John Omernik
>>>
>>>
>>>
>>
>>
>> --
>>
>> https://github.com/mindscratch
>> https://www.google.com/+CraigWickesser
>> https://twitter.com/mind_scratch
>> https://twitter.com/craig_links
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "marathon-framework" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to marathon-framework+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

>>> --
>> You received this message because you are subscribed to the Google Groups
>> "marathon-framework" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to marathon-framework+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

mess slave can't register to master via master ip:port

[Xiaodong Zhang]

Hi all:

My slave command like this:

/usr/sbin/mesos-slave --master=IP1:5050,IP2:5050,IP3:5050 …. --credential …

Only if IP1 is the leader, the slave can register to master successfully, Or it 
will register fail.

Slave log like this:

Creating new client SASL connection
Authentication timed out
Failed to authenticate with master master@172.31.43.77:5050: Authentication 
discarded
Authenticating with master master@172.31.43.77:5050
Using default CRAM-MD5 authenticatee

Is this a bug?Or it is designed like this.

BTW: --master:zk://xxx work well.

Re: mess slave can't register to master via master ip:port

[haosdent]

Do your masters have already managed by zookeeper? And what is your master
start command?

On Tue, Nov 3, 2015 at 12:06 PM, Xiaodong Zhang  wrote:

> Hi all:
>
> My slave command like this:
>
> /usr/sbin/mesos-slave --master=IP1:5050,IP2:5050,IP3:5050 …. --credential …
>
> Only if IP1 is the leader, the slave can register to master successfully,
> Or it will register fail.
>
> Slave log like this:
>
> Creating new client SASL connection
> Authentication timed out
> Failed to authenticate with master master@172.31.43.77:5050:
> Authentication discarded
> Authenticating with master master@172.31.43.77:5050
> Using default CRAM-MD5 authenticatee
>
> Is this a bug?Or it is designed like this.
>
> BTW: --master:zk://xxx work well.
>



-- 
Best Regards,
Haosdent Huang

Re: mess slave can't register to master via master ip:port

[haosdent]

After checking code, seems Mesos only support --master=IP1:5050
or --master=zk://xx or --master=file:///.

On Tue, Nov 3, 2015 at 12:15 PM, haosdent  wrote:

> Do your masters have already managed by zookeeper? And what is your master
> start command?
>
> On Tue, Nov 3, 2015 at 12:06 PM, Xiaodong Zhang  wrote:
>
>> Hi all:
>>
>> My slave command like this:
>>
>> /usr/sbin/mesos-slave --master=IP1:5050,IP2:5050,IP3:5050 …. --credential
>> …
>>
>> Only if IP1 is the leader, the slave can register to master successfully,
>> Or it will register fail.
>>
>> Slave log like this:
>>
>> Creating new client SASL connection
>> Authentication timed out
>> Failed to authenticate with master master@172.31.43.77:5050:
>> Authentication discarded
>> Authenticating with master master@172.31.43.77:5050
>> Using default CRAM-MD5 authenticatee
>>
>> Is this a bug?Or it is designed like this.
>>
>> BTW: --master:zk://xxx work well.
>>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>



-- 
Best Regards,
Haosdent Huang

Re: mess slave can't register to master via master ip:port

[Guangya Liu]

Seems mesos does not support such mode, please refer to
https://github.com/apache/mesos/blob/master/src/slave/main.cpp#L105-L111
for the format of "--master". Thanks!

On Tue, Nov 3, 2015 at 12:28 PM, haosdent  wrote:

> After checking code, seems Mesos only support --master=IP1:5050
> or --master=zk://xx or --master=file:///.
>
> On Tue, Nov 3, 2015 at 12:15 PM, haosdent  wrote:
>
>> Do your masters have already managed by zookeeper? And what is your
>> master start command?
>>
>> On Tue, Nov 3, 2015 at 12:06 PM, Xiaodong Zhang 
>> wrote:
>>
>>> Hi all:
>>>
>>> My slave command like this:
>>>
>>> /usr/sbin/mesos-slave --master=IP1:5050,IP2:5050,IP3:5050 ….
>>> --credential …
>>>
>>> Only if IP1 is the leader, the slave can register to master
>>> successfully, Or it will register fail.
>>>
>>> Slave log like this:
>>>
>>> Creating new client SASL connection
>>> Authentication timed out
>>> Failed to authenticate with master master@172.31.43.77:5050:
>>> Authentication discarded
>>> Authenticating with master master@172.31.43.77:5050
>>> Using default CRAM-MD5 authenticatee
>>>
>>> Is this a bug?Or it is designed like this.
>>>
>>> BTW: --master:zk://xxx work well.
>>>
>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>

Re: mess slave can't register to master via master ip:port

[Xiaodong Zhang]

If that so. I think this document should be modified.

http://mesos.apache.org/documentation/latest/configuration/#SlaveOptions

[cid:30B6D8D5-217E-42F9-925E-BBBEBCF636A3]

Right?


发件人: Guangya Liu >
答复: "user@mesos.apache.org" 
>
日期: 2015年11月3日 星期二 下午12:39
至: "user@mesos.apache.org" 
>
主题: Re: mess slave can't register to master via master ip:port

Seems mesos does not support such mode, please refer to 
https://github.com/apache/mesos/blob/master/src/slave/main.cpp#L105-L111 for 
the format of "--master". Thanks!

On Tue, Nov 3, 2015 at 12:28 PM, haosdent 
> wrote:
After checking code, seems Mesos only support --master=IP1:5050 or 
--master=zk://xx or --master=file:///.

On Tue, Nov 3, 2015 at 12:15 PM, haosdent 
> wrote:
Do your masters have already managed by zookeeper? And what is your master 
start command?

On Tue, Nov 3, 2015 at 12:06 PM, Xiaodong Zhang 
> wrote:
Hi all:

My slave command like this:

/usr/sbin/mesos-slave --master=IP1:5050,IP2:5050,IP3:5050 …. --credential …

Only if IP1 is the leader, the slave can register to master successfully, Or it 
will register fail.

Slave log like this:

Creating new client SASL connection
Authentication timed out
Failed to authenticate with master 
master@172.31.43.77:5050: Authentication 
discarded
Authenticating with master 
master@172.31.43.77:5050
Using default CRAM-MD5 authenticatee

Is this a bug?Or it is designed like this.

BTW: --master:zk://xxx work well.



--
Best Regards,
Haosdent Huang



--
Best Regards,
Haosdent Huang