date:20151101

Re: Can't start docker container when SSL_ENABLED is on.

2015-11-01 Thread haosdent

Hi, @Xiaodong I could reproduce your problem in my testing today. A quickly
workaround is adding environment variables when you launch slave.

```
./bin/mesos-slave.sh  --containerizers=docker,mesos
--executor_environment_variables='{"SSL_KEY_FILE": "/tmp/server.key",
"SSL_CERT_FILE": "/tmp/ssl.chain.crt", "SSL_ENABLED": "true"}''
```

As you see above, pass the ssl env to docker-executor through specifying
--executor_environment_variables when starting. So far it works well for
me. Anyway I would submit a patch later to fix the docker environment
variables passing. After that, you could launch slave without
executor_environment_variables flag.

On Sat, Oct 31, 2015 at 2:56 PM, Tim Chen  wrote:

> Hi Xiaodong,
>
> If you follow the reviewboard you'll see that the fix is not correct, I
> believe Jojy will be posting a new patch.
>
> Tim
>
> On Fri, Oct 30, 2015 at 6:58 PM, Xiaodong Zhang  wrote:
>
>> it is still not working!
>>
>> Only if I remove SSL_ENABLED from envs before I start the slave it works
>> well.
>>
>> I applied the patch in version 0.24.1. And rebuild it with `--enable-libevent
>> --enable-ssl` 。
>>
>> 发件人: Xiaodong Zhang 
>> 日期: 2015年10月31日 星期六 上午7:45
>>
>> 至: "user@mesos.apache.org" 
>> 主题: Re: Can't start docker container when SSL_ENABLED is on.
>>
>> Thanks Jojy.
>>
>> I will patch this in version 0.24.1, and rebuild it. I will let you know
>> if it work well after I finish testing.
>>
>> 发件人: Jojy Varghese 
>> 答复: "user@mesos.apache.org" 
>> 日期: 2015年10月31日 星期六 上午12:45
>> 至: "user@mesos.apache.org" 
>> 主题: Re: Can't start docker container when SSL_ENABLED is on.
>>
>> Thanks Xiaodong.
>>
>> Based on the hypothesis that the container process launched with
>> SSL_ENABLED in environment is the problem, I have created a patch
>> https://reviews.apache.org/r/39818/.  This might be a quick and dirty
>> was to test the hypothesis. Would it be possible for you to test again
>> after applying the patch?
>>
>> -Jojy
>>
>>
>>
>> On Oct 30, 2015, at 8:29 AM, Xiaodong Zhang  wrote:
>>
>> Thanks @Jojy
>>
>>
>>
>> Flags at startup: --appc_store_dir="/tmp/mesos/store/appc"
>> --authenticatee="crammd5" --cgroups_cpu_enable_pids_and_tids_count="false"
>> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
>> --cgroups_limit_swap="false" --cgroups_root="mesos"
>> --container_disk_watch_interval="15secs" --containerizers="docker,mesos"
>> --credential="/etc/mesos-slave-auth" --default_role="*"
>> --disk_watch_interval="1mins" --docker="/usr/bin/docker"
>> --docker_kill_orphans="true" --docker_remove_delay="6hrs"
>> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
>> --enforce_container_disk_quota="false"
>> --executor_registration_timeout="1hrs"
>> --executor_shutdown_grace_period="5secs"
>> --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB"
>> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
>> --hadoop_home="" --help="false" --initialize_driver_logging="true"
>> --isolation="posix/cpu,posix/mem" --launcher_dir="/usr/libexec/mesos"
>> --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO"
>> --master="zk://172.31.43.77:2181,172.31.44.2:2181,172.31.36.91:2181/mesos"
>> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
>> --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns"
>> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
>> --registration_backoff_factor="1secs"
>> --resource_monitoring_interval="1secs" --revocable_cpu_low_priority="true"
>> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
>> --switch_user="true" --version="false" --work_dir="/tmp/mesos"
>>
>> 发件人: Jojy Varghese 
>> 答复: "user@mesos.apache.org" 
>> 日期: 2015年10月30日 星期五 下午11:17
>> 至: "user@mesos.apache.org" 
>> 主题: Re: Can't start docker container when SSL_ENABLED is on.
>>
>> Hi Xiaodong
>>   This might be because the executor inherits the SSL environment
>> variables of slave and thus expects SSL key password to launch. Could you
>> please add the part of the slave logs that says "Flags at startup” so that
>> we can have more information?
>>
>> thanks
>> Jojy
>>
>>
>> On Oct 29, 2015, at 8:55 PM, Xiaodong Zhang  wrote:
>>
>> Thanks a lot !~ @haosent
>>
>> 发件人: haosdent 
>> 答复: "user@mesos.apache.org" 
>> 日期: 2015年10月30日 星期五 上午11:45
>> 至: user 
>> 主题: Re: Can't start docker container when SSL_ENABLED is on.
>>
>> Hi, @Xiaodong I interested in your problem. But recently days I don't
>> have enough time to try reproduce your problem. I think I could try to dig
>> your problem at this Sunday and give you feedback.
>>
>> On Fri, Oct 30, 2015 at 11:30 AM, Xiaodong Zhang

Re: How to tell master which ip to connect.

2015-11-01 Thread haosdent

Hi, @Xiaodong I think ssl + register auth is enough, I don't think you need
worried about that. Let me also attach some other mesos documents except
ssl relate to security.

authentication:
https://github.com/apache/mesos/blob/master/docs/authorization.md
authentication:
https://github.com/apache/mesos/blob/master/docs/authentication.md
firewall_rules:
https://github.com/apache/mesos/blob/master/docs/configuration.md#master-and-slave-options

And if you want to use external firewall of iptables to limit access, it is
also OK when you using mesos.

On Fri, Oct 30, 2015 at 7:50 AM, Xiaodong Zhang  wrote:

> oh！connect via ssl and register with auth is not safety enough？
>
> 发自我的 iPhone
>
> 在 2015年10月30日，上午12:55，tommy xiao  写道：
>
> public ip is very dangerous for mesos cluster, you need a firewall on your
> solution.
>
> 2015-10-28 10:16 GMT+08:00 Xiaodong Zhang :
>
>> Hi teams:
>>
>> My scenarios is like this:
>>
>> My master nodes were deployed in AWS. My slaves were in AZURE.So they
>> communicate via public ip.
>> I got trouble when slaves try to register to master.
>> Now slaves can get master’s public ip address,and can send register
>> request.But they can only send there private ip to master.(Because they
>> don’t know there public ip,thus they can’t not bind a public ip via —ip
>> flag), thus  masters can’t connect slaves.How can the slave to tell master
>> which ip master should connect(I can’t find any flags like —advertise_ip
>> in master).
>>
>
>
>
> --
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com
>
>


-- 
Best Regards,
Haosdent Huang

Re: Can't start docker container when SSL_ENABLED is on.

2015-11-01 Thread haosdent

@Xiaodong I create a ticket to trace this
https://issues.apache.org/jira/browse/MESOS-3815 and post a patch in it.
Feel free to review and test it together. Thank you!

On Sun, Nov 1, 2015 at 4:54 PM, haosdent  wrote:

> Hi, @Xiaodong I could reproduce your problem in my testing today. A
> quickly workaround is adding environment variables when you launch slave.
>
> ```
> ./bin/mesos-slave.sh  --containerizers=docker,mesos
> --executor_environment_variables='{"SSL_KEY_FILE": "/tmp/server.key",
> "SSL_CERT_FILE": "/tmp/ssl.chain.crt", "SSL_ENABLED": "true"}''
> ```
>
> As you see above, pass the ssl env to docker-executor through specifying
> --executor_environment_variables when starting. So far it works well for
> me. Anyway I would submit a patch later to fix the docker environment
> variables passing. After that, you could launch slave without
> executor_environment_variables flag.
>
> On Sat, Oct 31, 2015 at 2:56 PM, Tim Chen  wrote:
>
>> Hi Xiaodong,
>>
>> If you follow the reviewboard you'll see that the fix is not correct, I
>> believe Jojy will be posting a new patch.
>>
>> Tim
>>
>> On Fri, Oct 30, 2015 at 6:58 PM, Xiaodong Zhang 
>> wrote:
>>
>>> it is still not working!
>>>
>>> Only if I remove SSL_ENABLED from envs before I start the slave it works
>>> well.
>>>
>>> I applied the patch in version 0.24.1. And rebuild it with 
>>> `--enable-libevent
>>> --enable-ssl` 。
>>>
>>> 发件人: Xiaodong Zhang 
>>> 日期: 2015年10月31日 星期六 上午7:45
>>>
>>> 至: "user@mesos.apache.org" 
>>> 主题: Re: Can't start docker container when SSL_ENABLED is on.
>>>
>>> Thanks Jojy.
>>>
>>> I will patch this in version 0.24.1, and rebuild it. I will let you know
>>> if it work well after I finish testing.
>>>
>>> 发件人: Jojy Varghese 
>>> 答复: "user@mesos.apache.org" 
>>> 日期: 2015年10月31日 星期六 上午12:45
>>> 至: "user@mesos.apache.org" 
>>> 主题: Re: Can't start docker container when SSL_ENABLED is on.
>>>
>>> Thanks Xiaodong.
>>>
>>> Based on the hypothesis that the container process launched with
>>> SSL_ENABLED in environment is the problem, I have created a patch
>>> https://reviews.apache.org/r/39818/.  This might be a quick and dirty
>>> was to test the hypothesis. Would it be possible for you to test again
>>> after applying the patch?
>>>
>>> -Jojy
>>>
>>>
>>>
>>> On Oct 30, 2015, at 8:29 AM, Xiaodong Zhang  wrote:
>>>
>>> Thanks @Jojy
>>>
>>>
>>>
>>> Flags at startup: --appc_store_dir="/tmp/mesos/store/appc"
>>> --authenticatee="crammd5" --cgroups_cpu_enable_pids_and_tids_count="false"
>>> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
>>> --cgroups_limit_swap="false" --cgroups_root="mesos"
>>> --container_disk_watch_interval="15secs" --containerizers="docker,mesos"
>>> --credential="/etc/mesos-slave-auth" --default_role="*"
>>> --disk_watch_interval="1mins" --docker="/usr/bin/docker"
>>> --docker_kill_orphans="true" --docker_remove_delay="6hrs"
>>> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
>>> --enforce_container_disk_quota="false"
>>> --executor_registration_timeout="1hrs"
>>> --executor_shutdown_grace_period="5secs"
>>> --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB"
>>> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
>>> --hadoop_home="" --help="false" --initialize_driver_logging="true"
>>> --isolation="posix/cpu,posix/mem" --launcher_dir="/usr/libexec/mesos"
>>> --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO"
>>> --master="
>>> zk://172.31.43.77:2181,172.31.44.2:2181,172.31.36.91:2181/mesos"
>>> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
>>> --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns"
>>> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
>>> --registration_backoff_factor="1secs"
>>> --resource_monitoring_interval="1secs" --revocable_cpu_low_priority="true"
>>> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
>>> --switch_user="true" --version="false" --work_dir="/tmp/mesos"
>>>
>>> 发件人: Jojy Varghese 
>>> 答复: "user@mesos.apache.org" 
>>> 日期: 2015年10月30日 星期五 下午11:17
>>> 至: "user@mesos.apache.org" 
>>> 主题: Re: Can't start docker container when SSL_ENABLED is on.
>>>
>>> Hi Xiaodong
>>>   This might be because the executor inherits the SSL environment
>>> variables of slave and thus expects SSL key password to launch. Could you
>>> please add the part of the slave logs that says "Flags at startup” so that
>>> we can have more information?
>>>
>>> thanks
>>> Jojy
>>>
>>>
>>> On Oct 29, 2015, at 8:55 PM, Xiaodong Zhang  wrote:
>>>
>>> Thanks a lot !~ @haosent
>>>
>>> 发件人: haosdent 
>>> 答复: "user@mesos.apache.org" 
>>> 日期: 2015年10月30日 星期五

Re: Can't start docker container when SSL_ENABLED is on.

2015-11-01 Thread Xiaodong Zhang

Thanks@haosdent

I will test the command line arguments and then test patch.

Have a nice day!~~

发件人: haosdent >
答复: "user@mesos.apache.org" 
>
日期: 2015年11月1日 星期日 下午5:40
至: user >
主题: Re: Can't start docker container when SSL_ENABLED is on.

@Xiaodong I create a ticket to trace this 
https://issues.apache.org/jira/browse/MESOS-3815 and post a patch in it. Feel 
free to review and test it together. Thank you!

On Sun, Nov 1, 2015 at 4:54 PM, haosdent 
> wrote:
Hi, @Xiaodong I could reproduce your problem in my testing today. A quickly 
workaround is adding environment variables when you launch slave.

```
./bin/mesos-slave.sh  --containerizers=docker,mesos 
--executor_environment_variables='{"SSL_KEY_FILE": "/tmp/server.key", 
"SSL_CERT_FILE": "/tmp/ssl.chain.crt", "SSL_ENABLED": "true"}''
```

As you see above, pass the ssl env to docker-executor through specifying 
--executor_environment_variables when starting. So far it works well for me. 
Anyway I would submit a patch later to fix the docker environment variables 
passing. After that, you could launch slave without 
executor_environment_variables flag.

On Sat, Oct 31, 2015 at 2:56 PM, Tim Chen 
> wrote:
Hi Xiaodong,

If you follow the reviewboard you'll see that the fix is not correct, I believe 
Jojy will be posting a new patch.

Tim

On Fri, Oct 30, 2015 at 6:58 PM, Xiaodong Zhang 
> wrote:
it is still not working!

Only if I remove SSL_ENABLED from envs before I start the slave it works well.

I applied the patch in version 0.24.1. And rebuild it with `--enable-libevent 
--enable-ssl` 。

发件人: Xiaodong Zhang >
日期: 2015年10月31日 星期六 上午7:45

至: "user@mesos.apache.org" 
>
主题: Re: Can't start docker container when SSL_ENABLED is on.

Thanks Jojy.

I will patch this in version 0.24.1, and rebuild it. I will let you know if it 
work well after I finish testing.

发件人: Jojy Varghese >
答复: "user@mesos.apache.org" 
>
日期: 2015年10月31日 星期六 上午12:45
至: "user@mesos.apache.org" 
>
主题: Re: Can't start docker container when SSL_ENABLED is on.

Thanks Xiaodong.

Based on the hypothesis that the container process launched with SSL_ENABLED in 
environment is the problem, I have created a patch 
https://reviews.apache.org/r/39818/.  This might be a quick and dirty was to 
test the hypothesis. Would it be possible for you to test again after applying 
the patch?

-Jojy



On Oct 30, 2015, at 8:29 AM, Xiaodong Zhang 
> wrote:

Thanks @Jojy



Flags at startup: --appc_store_dir="/tmp/mesos/store/appc" 
--authenticatee="crammd5" --cgroups_cpu_enable_pids_and_tids_count="false" 
--cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" 
--cgroups_limit_swap="false" --cgroups_root="mesos" 
--container_disk_watch_interval="15secs" --containerizers="docker,mesos" 
--credential="/etc/mesos-slave-auth" --default_role="*" 
--disk_watch_interval="1mins" --docker="/usr/bin/docker" 
--docker_kill_orphans="true" --docker_remove_delay="6hrs" 
--docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" 
--enforce_container_disk_quota="false" --executor_registration_timeout="1hrs" 
--executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/mesos/fetch" 
--fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" 
--gc_disk_headroom="0.1" --hadoop_home="" --help="false" 
--initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" 
--launcher_dir="/usr/libexec/mesos" --log_dir="/var/log/mesos" --logbufsecs="0" 
--logging_level="INFO" 
--master="zk://172.31.43.77:2181,172.31.44.2:2181,172.31.36.91:2181/mesos" 
--oversubscribed_resources_interval="15secs" --perf_duration="10secs" 
--perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" 
--quiet="false" --recover="reconnect" --recovery_timeout="15mins" 
--registration_backoff_factor="1secs" --resource_monitoring_interval="1secs" 
--revocable_cpu_low_priority="true" --sandbox_directory="/mnt/mesos/sandbox" 
--strict="true" --switch_user="true" --version="false" --work_dir="/tmp/mesos"

发件人: Jojy Varghese >
答复: "user@mesos.apache.org" 
>
日期: 2015年10月30日 星期五 下午11:17
至: "user@mesos.apache.org" 
>

how to use agent config: docker_mesos_image better? or is it deprecated?

2015-11-01 Thread zhou weitao

Hi, there,

I got this config docker_mesos_image=VALUE with the following description:

The docker image used to launch this mesos slave instance. If an image is
specified, the docker containerizer assumes the slave is running in a
docker container, and launches executors with docker containers in order to
recover them when the slave restarts and recovers.

while more guys are containerizing slave by docker mountPoints
AFAIK. So which one is the best practice or I am misunderstanding
docker_mesos_image usage.


Any comment is welcomed.

Re: Can't start docker container when SSL_ENABLED is on.

Re: How to tell master which ip to connect.

Re: Can't start docker container when SSL_ENABLED is on.

Re: Can't start docker container when SSL_ENABLED is on.

how to use agent config: docker_mesos_image better? or is it deprecated?

5 matches

Site Navigation

Mail list logo

Footer information