@Xiaodong I create a ticket to trace this https://issues.apache.org/jira/browse/MESOS-3815 and post a patch in it. Feel free to review and test it together. Thank you!
On Sun, Nov 1, 2015 at 4:54 PM, haosdent <[email protected]> wrote: > Hi, @Xiaodong I could reproduce your problem in my testing today. A > quickly workaround is adding environment variables when you launch slave. > > ``` > ./bin/mesos-slave.sh xxxx --containerizers=docker,mesos > --executor_environment_variables='{"SSL_KEY_FILE": "/tmp/server.key", > "SSL_CERT_FILE": "/tmp/ssl.chain.crt", "SSL_ENABLED": "true"}'' > ``` > > As you see above, pass the ssl env to docker-executor through specifying > --executor_environment_variables when starting. So far it works well for > me. Anyway I would submit a patch later to fix the docker environment > variables passing. After that, you could launch slave without > executor_environment_variables flag. > > On Sat, Oct 31, 2015 at 2:56 PM, Tim Chen <[email protected]> wrote: > >> Hi Xiaodong, >> >> If you follow the reviewboard you'll see that the fix is not correct, I >> believe Jojy will be posting a new patch. >> >> Tim >> >> On Fri, Oct 30, 2015 at 6:58 PM, Xiaodong Zhang <[email protected]> >> wrote: >> >>> it is still not working! >>> >>> Only if I remove SSL_ENABLED from envs before I start the slave it works >>> well. >>> >>> I applied the patch in version 0.24.1. And rebuild it with >>> `--enable-libevent >>> --enable-ssl` 。 >>> >>> 发件人: Xiaodong Zhang <[email protected]> >>> 日期: 2015年10月31日 星期六 上午7:45 >>> >>> 至: "[email protected]" <[email protected]> >>> 主题: Re: Can't start docker container when SSL_ENABLED is on. >>> >>> Thanks Jojy. >>> >>> I will patch this in version 0.24.1, and rebuild it. I will let you know >>> if it work well after I finish testing. >>> >>> 发件人: Jojy Varghese <[email protected]> >>> 答复: "[email protected]" <[email protected]> >>> 日期: 2015年10月31日 星期六 上午12:45 >>> 至: "[email protected]" <[email protected]> >>> 主题: Re: Can't start docker container when SSL_ENABLED is on. >>> >>> Thanks Xiaodong. >>> >>> Based on the hypothesis that the container process launched with >>> SSL_ENABLED in environment is the problem, I have created a patch >>> https://reviews.apache.org/r/39818/. This might be a quick and dirty >>> was to test the hypothesis. Would it be possible for you to test again >>> after applying the patch? >>> >>> -Jojy >>> >>> >>> >>> On Oct 30, 2015, at 8:29 AM, Xiaodong Zhang <[email protected]> wrote: >>> >>> Thanks @Jojy >>> >>> >>> >>> Flags at startup: --appc_store_dir="/tmp/mesos/store/appc" >>> --authenticatee="crammd5" --cgroups_cpu_enable_pids_and_tids_count="false" >>> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" >>> --cgroups_limit_swap="false" --cgroups_root="mesos" >>> --container_disk_watch_interval="15secs" --containerizers="docker,mesos" >>> --credential="/etc/mesos-slave-auth" --default_role="*" >>> --disk_watch_interval="1mins" --docker="/usr/bin/docker" >>> --docker_kill_orphans="true" --docker_remove_delay="6hrs" >>> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" >>> --enforce_container_disk_quota="false" >>> --executor_registration_timeout="1hrs" >>> --executor_shutdown_grace_period="5secs" >>> --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB" >>> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" >>> --hadoop_home="" --help="false" --initialize_driver_logging="true" >>> --isolation="posix/cpu,posix/mem" --launcher_dir="/usr/libexec/mesos" >>> --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO" >>> --master=" >>> zk://172.31.43.77:2181,172.31.44.2:2181,172.31.36.91:2181/mesos" >>> --oversubscribed_resources_interval="15secs" --perf_duration="10secs" >>> --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" >>> --quiet="false" --recover="reconnect" --recovery_timeout="15mins" >>> --registration_backoff_factor="1secs" >>> --resource_monitoring_interval="1secs" --revocable_cpu_low_priority="true" >>> --sandbox_directory="/mnt/mesos/sandbox" --strict="true" >>> --switch_user="true" --version="false" --work_dir="/tmp/mesos" >>> >>> 发件人: Jojy Varghese <[email protected]> >>> 答复: "[email protected]" <[email protected]> >>> 日期: 2015年10月30日 星期五 下午11:17 >>> 至: "[email protected]" <[email protected]> >>> 主题: Re: Can't start docker container when SSL_ENABLED is on. >>> >>> Hi Xiaodong >>> This might be because the executor inherits the SSL environment >>> variables of slave and thus expects SSL key password to launch. Could you >>> please add the part of the slave logs that says "Flags at startup” so that >>> we can have more information? >>> >>> thanks >>> Jojy >>> >>> >>> On Oct 29, 2015, at 8:55 PM, Xiaodong Zhang <[email protected]> wrote: >>> >>> Thanks a lot !~ @haosent >>> >>> 发件人: haosdent <[email protected]> >>> 答复: "[email protected]" <[email protected]> >>> 日期: 2015年10月30日 星期五 上午11:45 >>> 至: user <[email protected]> >>> 主题: Re: Can't start docker container when SSL_ENABLED is on. >>> >>> Hi, @Xiaodong I interested in your problem. But recently days I don't >>> have enough time to try reproduce your problem. I think I could try to dig >>> your problem at this Sunday and give you feedback. >>> >>> On Fri, Oct 30, 2015 at 11:30 AM, Xiaodong Zhang <[email protected]> >>> wrote: >>> >>>> Anybody know about this? >>>> >>>> 发件人: Xiaodong Zhang <[email protected]> >>>> 答复: "[email protected]" <[email protected]> >>>> 日期: 2015年10月29日 星期四 下午7:38 >>>> >>>> 至: "[email protected]" <[email protected]> >>>> 主题: Re: Can't start docker container when SSL_ENABLED is on. >>>> >>>> I think it is easy to reproduce this error. >>>> >>>> Start master with env: >>>> >>>> SSL_SUPPORT_DOWNGRADE >>>> SSL_ENABLED >>>> SSL_KEY_FILE >>>> SSL_CERT_FILE >>>> >>>> Start slave with env: >>>> >>>> SSL_ENABLED >>>> SSL_KEY_FILE >>>> SSL_CERT_FILE >>>> LIBPROCESS_ADVERTISE_IP >>>> >>>> >>>> Then run a docker task via marathon. >>>> >>>> 发件人: Xiaodong Zhang <[email protected]> >>>> 日期: 2015年10月29日 星期四 下午3:09 >>>> 至: "[email protected]" <[email protected]> >>>> 主题: Re: Can't start docker container when SSL_ENABLED is on. >>>> >>>> So now, mesos task work well but docker task doesn’t. >>>> >>>> 发件人: Xiaodong Zhang <[email protected]> >>>> 答复: "[email protected]" <[email protected]> >>>> 日期: 2015年10月29日 星期四 下午2:08 >>>> 至: "[email protected]" <[email protected]> >>>> 主题: Re: Can't start docker container when SSL_ENABLED is on. >>>> >>>> I run a task by marathon: >>>> >>>> { >>>> "id": "basic-0", >>>> "cmd": "while [ true ] ; do echo 'Hello Marathon' ; sleep 5 ; done", >>>> "cpus": 0.1, >>>> "mem": 10.0, >>>> "instances": 1} >>>> >>>> >>>> It works well. >>>> >>>> <742629F2-78E8-43F2-9015-F3D22720826B.png> >>>> >>>> Docker task can pull image but can’t run as I mentioned. >>>> >>>> My docker version 1.5.0 >>>> >>>> 发件人: Tim Chen <[email protected]> >>>> 答复: "[email protected]" <[email protected]> >>>> 日期: 2015年10月29日 星期四 下午1:48 >>>> 至: "[email protected]" <[email protected]> >>>> 主题: Re: Can't start docker container when SSL_ENABLED is on. >>>> >>>> Does running a task without docker container (Mesos containerizer) >>>> works with ssl in your environment? >>>> >>>> Tim >>>> >>>> On Wed, Oct 28, 2015 at 10:19 PM, Xiaodong Zhang <[email protected]> >>>> wrote: >>>> >>>>> Thanks a lot. I find the log file in slave. >>>>> >>>>> One of the task: >>>>> >>>>> Stdout: >>>>> >>>>> --container="mesos-20151029-043755-3549436724-5050-5674-S0.e2c2580f-8082-4f17-b0cc-4e32e040d444" >>>>> --docker="/home/ubuntu/luna/bin/docker" --help="false" >>>>> --initialize_driver_logging="true" --logbufsecs="0" --logging_level="INFO" >>>>> --mapped_directory="/mnt/mesos/sandbox" --quiet="false" >>>>> --sandbox_directory="/tmp/mesos/slaves/20151029-043755-3549436724-5050-5674-S0/frameworks/20151029-043755-3549436724-5050-5674-0000/executors/e4a3bed5-64e6-4970-8bb1-df6404656a48.e3a20f3b-7dfb-11e5-b57b-0247b493b22f/runs/e2c2580f-8082-4f17-b0cc-4e32e040d444" >>>>> --stop_timeout="0ns" >>>>> --container="mesos-20151029-043755-3549436724-5050-5674-S0.e2c2580f-8082-4f17-b0cc-4e32e040d444" >>>>> --docker="/home/ubuntu/luna/bin/docker" --help="false" >>>>> --initialize_driver_logging="true" --logbufsecs="0" --logging_level="INFO" >>>>> --mapped_directory="/mnt/mesos/sandbox" --quiet="false" >>>>> --sandbox_directory="/tmp/mesos/slaves/20151029-043755-3549436724-5050-5674-S0/frameworks/20151029-043755-3549436724-5050-5674-0000/executors/e4a3bed5-64e6-4970-8bb1-df6404656a48.e3a20f3b-7dfb-11e5-b57b-0247b493b22f/runs/e2c2580f-8082-4f17-b0cc-4e32e040d444" >>>>> --stop_timeout="0ns" >>>>> Shutting down >>>>> >>>>> Stderr: >>>>> >>>>> I1029 05:14:06.529364 27862 fetcher.cpp:414] Fetcher Info: >>>>> {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/20151029-043755-3549436724-5050-5674-S0","items":[{"action":"BYPASS_CACHE","uri":{"extract":false,"value":"file:\/\/\/etc\/.dockercfg"}}],"sandbox_directory":"\/tmp\/mesos\/slaves\/20151029-043755-3549436724-5050-5674-S0\/frameworks\/20151029-043755-3549436724-5050-5674-0000\/executors\/e4a3bed5-64e6-4970-8bb1-df6404656a48.e3a20f3b-7dfb-11e5-b57b-0247b493b22f\/runs\/e2c2580f-8082-4f17-b0cc-4e32e040d444"} >>>>> I1029 05:14:06.530562 27862 fetcher.cpp:369] Fetching URI ' >>>>> file:///etc/.dockercfg' >>>>> I1029 05:14:06.530580 27862 fetcher.cpp:243] Fetching directly into >>>>> the sandbox directory >>>>> I1029 05:14:06.530594 27862 fetcher.cpp:180] Fetching URI ' >>>>> file:///etc/.dockercfg' >>>>> I1029 05:14:06.530609 27862 fetcher.cpp:160] Copying resource with >>>>> command:cp '/etc/.dockercfg' >>>>> '/tmp/mesos/slaves/20151029-043755-3549436724-5050-5674-S0/frameworks/20151029-043755-3549436724-5050-5674-0000/executors/e4a3bed5-64e6-4970-8bb1-df6404656a48.e3a20f3b-7dfb-11e5-b57b-0247b493b22f/runs/e2c2580f-8082-4f17-b0cc-4e32e040d444/.dockercfg' >>>>> I1029 05:14:06.532165 27862 fetcher.cpp:446] Fetched ' >>>>> file:///etc/.dockercfg' to >>>>> '/tmp/mesos/slaves/20151029-043755-3549436724-5050-5674-S0/frameworks/20151029-043755-3549436724-5050-5674-0000/executors/e4a3bed5-64e6-4970-8bb1-df6404656a48.e3a20f3b-7dfb-11e5-b57b-0247b493b22f/runs/e2c2580f-8082-4f17-b0cc-4e32e040d444/.dockercfg' >>>>> I1029 05:14:07.782054 27955 exec.cpp:133] Version: 0.24.1 >>>>> I1029 05:14:07.785039 27963 exec.cpp:462] Slave exited ... shutting >>>>> down >>>>> E1029 05:14:07.785158 27964 socket.hpp:174] Shutdown failed on fd=7: >>>>> Transport endpoint is not connected [107] >>>>> >>>>> 发件人: haosdent <[email protected]> >>>>> 答复: "[email protected]" <[email protected]> >>>>> 日期: 2015年10月29日 星期四 下午1:13 >>>>> >>>>> 至: user <[email protected]> >>>>> 主题: Re: Can't start docker container when SSL_ENABLED is on. >>>>> >>>>> <5185_02_04.png> >>>>> <5185_02_07.png> >>>>> >>>>> I capture how I find tasks log in my local webui, could you find the >>>>> stderr and stdout for your tasks according above screenshots? >>>>> >>>>> >>>>> On Thu, Oct 29, 2015 at 1:07 PM, Xiaodong Zhang <[email protected]> >>>>> wrote: >>>>> >>>>>> I didn’t see some useful info. >>>>>> >>>>>> In mesos slave log, there is a line : >>>>>> I1029 03:29:53.160143 9292 slave.cpp:3399] Executor >>>>>> '279bcb34-f705-4857-96ad-d96843b848fb.4b3abdcd-7ded-11e5-a82d-0240afabf713' >>>>>> of framework 20151029-031549-1294671788-5050-4937-0000 terminated >>>>>> with signal Killed >>>>>> >>>>>> I check the normal log, it shows: >>>>>> >>>>>> I1014 15:22:21.276007 23163 slave.cpp:3326] Executor >>>>>> 'ffc08dce-997f-41f7-9b03-57c1b4bc1f85.47ed02aa-7285-11e5-80d7-000d3a8033de' >>>>>> of framework 20150814-115157-1677721866-5050-6185-0000 exited with >>>>>> status 0 >>>>>> >>>>>> Is this helpful? >>>>>> >>>>>> 发件人: Xiaodong Zhang <[email protected]> >>>>>> 答复: "[email protected]" <[email protected]> >>>>>> 日期: 2015年10月29日 星期四 下午12:59 >>>>>> 至: "[email protected]" <[email protected]> >>>>>> >>>>>> 主题: Re: Can't start docker container when SSL_ENABLED is on. >>>>>> >>>>>> <9D46724C-457C-4BE1-B0E4-F57B147F6DC8.png> >>>>>> >>>>>> The webui have a LOG link, when click it shows like this: >>>>>> >>>>>> I1029 04:44:32.293445 5697 http.cpp:321] HTTP GET for >>>>>> /master/state.json from 114.113.20.135:55682 with >>>>>> User-Agent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) >>>>>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36' >>>>>> I1029 04:44:34.533504 5704 master.cpp:4613] Sending 1 offers to >>>>>> framework 20151029-043755-3549436724-5050-5674-0000 (marathon) at >>>>>> [email protected]:53373 >>>>>> I1029 04:44:34.539579 5702 master.cpp:2739] Processing ACCEPT call >>>>>> for offers: [ 20151029-043755-3549436724-5050-5674-O2 ] on slave >>>>>> 20151029-043755-3549436724-5050-5674-S0 at slave(1)@ >>>>>> 50.112.136.148:5051 ( >>>>>> ec2-50-112-136-148.us-west-2.compute.amazonaws.com) for framework >>>>>> 20151029-043755-3549436724-5050-5674-0000 (marathon) at >>>>>> [email protected]:53373 >>>>>> I1029 04:44:34.539710 5702 hierarchical.hpp:814] Recovered >>>>>> cpus(*):1; mem(*):999; disk(*):3962; ports(*):[31000-32000] (total: >>>>>> cpus(*):1; mem(*):999; disk(*):3962; ports(*):[31000-32000], allocated: ) >>>>>> on slave 20151029-043755-3549436724-5050-5674-S0 from framework >>>>>> 20151029-043755-3549436724-5050-5674-0000 >>>>>> I1029 04:44:37.360901 5703 master.cpp:4294] Performing implicit task >>>>>> state reconciliation for framework >>>>>> 20151029-043755-3549436724-5050-5674-0000 (marathon) at >>>>>> [email protected]:53373 >>>>>> I1029 04:44:40.539989 5704 master.cpp:4613] Sending 1 offers to >>>>>> framework 20151029-043755-3549436724-5050-5674-0000 (marathon) at >>>>>> [email protected]:53373 >>>>>> I1029 04:44:40.610321 5702 master.cpp:2739] Processing ACCEPT call >>>>>> for offers: [ 20151029-043755-3549436724-5050-5674-O3 ] on slave >>>>>> 20151029-043755-3549436724-5050-5674-S0 at slave(1)@ >>>>>> 50.112.136.148:5051 ( >>>>>> ec2-50-112-136-148.us-west-2.compute.amazonaws.com) for framework >>>>>> 20151029-043755-3549436724-5050-5674-0000 (marathon) at >>>>>> [email protected]:53373 >>>>>> I1029 04:44:40.610846 5702 master.hpp:170] Adding task >>>>>> e4a3bed5-64e6-4970-8bb1-df6404656a48.c4239b84-7df7-11e5-b57b-0247b493b22f >>>>>> with resources cpus(*):0.0625; mem(*):256; ports(*):[31864-31864] on >>>>>> slave >>>>>> 20151029-043755-3549436724-5050-5674-S0 ( >>>>>> ec2-50-112-136-148.us-west-2.compute.amazonaws.com) >>>>>> I1029 04:44:40.610911 5702 master.cpp:3069] Launching task >>>>>> e4a3bed5-64e6-4970-8bb1-df6404656a48.c4239b84-7df7-11e5-b57b-0247b493b22f >>>>>> of framework 20151029-043755-3549436724-5050-5674-0000 (marathon) at >>>>>> [email protected]:53373 >>>>>> with resources cpus(*):0.0625; mem(*):256; ports(*):[31864-31864] on >>>>>> slave >>>>>> 20151029-043755-3549436724-5050-5674-S0 at slave(1)@ >>>>>> 50.112.136.148:5051 ( >>>>>> ec2-50-112-136-148.us-west-2.compute.amazonaws.com) >>>>>> I1029 04:44:40.611095 5702 hierarchical.hpp:814] Recovered >>>>>> cpus(*):0.9375; mem(*):743; disk(*):3962; ports(*):[31000-31863, >>>>>> 31865-32000] (total: cpus(*):1; mem(*):999; disk(*):3962; >>>>>> ports(*):[31000-32000], allocated: cpus(*):0.0625; mem(*):256; >>>>>> ports(*):[31864-31864]) on slave 20151029-043755-3549436724-5050-5674-S0 >>>>>> from framework 20151029-043755-3549436724-5050-5674-0000 >>>>>> I1029 04:44:43.324970 5698 http.cpp:321] HTTP GET for >>>>>> /master/state.json from 114.113.20.135:55682 with >>>>>> User-Agent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) >>>>>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36' >>>>>> I1029 04:44:46.546671 5703 master.cpp:4613] Sending 1 offers to >>>>>> framework 20151029-043755-3549436724-5050-5674-0000 (marathon) at >>>>>> [email protected]:53373 >>>>>> I1029 04:44:46.557266 5699 master.cpp:2739] Processing ACCEPT call >>>>>> for offers: [ 20151029-043755-3549436724-5050-5674-O4 ] on slave >>>>>> 20151029-043755-3549436724-5050-5674-S0 at slave(1)@ >>>>>> 50.112.136.148:5051 ( >>>>>> ec2-50-112-136-148.us-west-2.compute.amazonaws.com) for framework >>>>>> 20151029-043755-3549436724-5050-5674-0000 (marathon) at >>>>>> [email protected]:53373 >>>>>> I1029 04:44:46.557394 5699 hierarchical.hpp:814] Recovered >>>>>> cpus(*):0.9375; mem(*):743; disk(*):3962; ports(*):[31000-31863, >>>>>> 31865-32000] (total: cpus(*):1; mem(*):999; disk(*):3962; >>>>>> ports(*):[31000-32000], allocated: cpus(*):0.0625; mem(*):256; >>>>>> ports(*):[31864-31864]) on slave 20151029-043755-3549436724-5050-5674-S0 >>>>>> from framework 20151029-043755-3549436724-5050-5674-0000 >>>>>> I1029 04:44:47.267562 5700 master.cpp:4069] Status update >>>>>> TASK_FAILED (UUID: 0ea607fc-bf24-4bda-b107-55a54aba31cf) for task >>>>>> e4a3bed5-64e6-4970-8bb1-df6404656a48.c4239b84-7df7-11e5-b57b-0247b493b22f >>>>>> of framework 20151029-043755-3549436724-5050-5674-0000 from slave >>>>>> 20151029-043755-3549436724-5050-5674-S0 at slave(1)@ >>>>>> 50.112.136.148:5051 ( >>>>>> ec2-50-112-136-148.us-west-2.compute.amazonaws.com) >>>>>> I1029 04:44:47.267645 5700 master.cpp:4108] Forwarding status update >>>>>> TASK_FAILED (UUID: 0ea607fc-bf24-4bda-b107-55a54aba31cf) for task >>>>>> e4a3bed5-64e6-4970-8bb1-df6404656a48.c4239b84-7df7-11e5-b57b-0247b493b22f >>>>>> of framework 20151029-043755-3549436724-5050-5674-0000 >>>>>> I1029 04:44:47.267774 5700 master.cpp:5576] Updating the latest >>>>>> state of task >>>>>> e4a3bed5-64e6-4970-8bb1-df6404656a48.c4239b84-7df7-11e5-b57b-0247b493b22f >>>>>> of framework 20151029-043755-3549436724-5050-5674-0000 to TASK_FAILED >>>>>> I1029 04:44:47.267907 5700 hierarchical.hpp:814] Recovered >>>>>> cpus(*):0.0625; mem(*):256; ports(*):[31864-31864] (total: cpus(*):1; >>>>>> mem(*):999; disk(*):3962; ports(*):[31000-32000], allocated: ) on slave >>>>>> 20151029-043755-3549436724-5050-5674-S0 from framework >>>>>> 20151029-043755-3549436724-5050-5674-0000 >>>>>> I1029 04:44:47.289356 5698 master.cpp:5644] Removing task >>>>>> e4a3bed5-64e6-4970-8bb1-df6404656a48.c4239b84-7df7-11e5-b57b-0247b493b22f >>>>>> with resources cpus(*):0.0625; mem(*):256; ports(*):[31864-31864] of >>>>>> framework 20151029-043755-3549436724-5050-5674-0000 on slave >>>>>> 20151029-043755-3549436724-5050-5674-S0 at slave(1)@ >>>>>> 50.112.136.148:5051 ( >>>>>> ec2-50-112-136-148.us-west-2.compute.amazonaws.com) >>>>>> I1029 04:44:47.289459 5698 master.cpp:3398] Processing ACKNOWLEDGE >>>>>> call 0ea607fc-bf24-4bda-b107-55a54aba31cf for task >>>>>> e4a3bed5-64e6-4970-8bb1-df6404656a48.c4239b84-7df7-11e5-b57b-0247b493b22f >>>>>> of framework 20151029-043755-3549436724-5050-5674-0000 (marathon) at >>>>>> [email protected]:53373 on >>>>>> slave 20151029-043755-3549436724-5050-5674-S0 >>>>>> >>>>>> >>>>>> >>>>>> 发件人: haosdent <[email protected]> >>>>>> 答复: "[email protected]" <[email protected]> >>>>>> 日期: 2015年10月29日 星期四 下午12:02 >>>>>> 至: user <[email protected]> >>>>>> 主题: Re: Can't start docker container when SSL_ENABLED is on. >>>>>> >>>>>> Oh, I mean you task logs. They could be get from Mesos webui. >>>>>> >>>>>> On Thu, Oct 29, 2015 at 11:52 AM, Xiaodong Zhang <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Thanks for your reply. >>>>>>> >>>>>>> Yes I build mesos with `--enable-libevent --enable-ssl`. If I don’t >>>>>>> provide key and pem when start slave, it will register fail(That means >>>>>>> the >>>>>>> ssl work well right?) >>>>>>> >>>>>>> As I said the odd thing is the container nerver run(`docker ps –a >>>>>>> show nothing`). So it can’t have any stdout or stderr. >>>>>>> >>>>>>> 发件人: haosdent <[email protected]> >>>>>>> 答复: "[email protected]" <[email protected]> >>>>>>> 日期: 2015年10月29日 星期四 上午11:47 >>>>>>> 至: user <[email protected]> >>>>>>> 主题: Re: Can't start docker container when SSL_ENABLED is on. >>>>>>> >>>>>>> Do you compile mesos with ssl support? The default compile don't >>>>>>> contains ssl. And does docker container have stdour and stderr? >>>>>>> >>>>>>> On Thu, Oct 29, 2015 at 11:41 AM, Xiaodong Zhang <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> My scenarios is like previous email says, masters and slaves are in >>>>>>>> different IaaS. Now the slaves can register to the masters with >>>>>>>> SSL_ENABLED >>>>>>>> is on . >>>>>>>> >>>>>>>> But I meet another problem. Slaves can’t run container(the odd >>>>>>>> thing is they can pull image successfully,just can not run container, >>>>>>>> `docker ps –a ` list nothing) >>>>>>>> >>>>>>>> The logs like this: >>>>>>>> >>>>>>>> I1029 03:29:45.967741 9288 docker.cpp:758] Starting container >>>>>>>> 'd4f4e236-0d0a-492c-86df-eef48a414e23' for task >>>>>>>> '279bcb34-f705-4857-96ad-d96843b848fb.4b3abdcd-7ded-11e5-a82d-0240afabf713' >>>>>>>> (and executor >>>>>>>> '279bcb34-f705-4857-96ad-d96843b848fb.4b3abdcd-7ded-11e5-a82d-0240afabf713') >>>>>>>> of framework '20151029-031549-1294671788-5050-4937-0000' >>>>>>>> I1029 03:29:48.044148 9292 docker.cpp:382] Checkpointing pid 12062 >>>>>>>> to >>>>>>>> '/tmp/mesos/meta/slaves/20151029-031549-1294671788-5050-4937-S0/frameworks/20151029-031549-1294671788-5050-4937-0000/executors/279bcb34-f705-4857-96ad-d96843b848fb.4b3abdcd-7ded-11e5-a82d-0240afabf713/runs/d4f4e236-0d0a-492c-86df-eef48a414e23/pids/forked.pid' >>>>>>>> I1029 03:29:53.159361 9292 docker.cpp:1576] Executor for container >>>>>>>> 'd4f4e236-0d0a-492c-86df-eef48a414e23' has exited >>>>>>>> I1029 03:29:53.159572 9292 docker.cpp:1374] Destroying container >>>>>>>> 'd4f4e236-0d0a-492c-86df-eef48a414e23' >>>>>>>> I1029 03:29:53.159822 9292 docker.cpp:1478] Running docker stop on >>>>>>>> container 'd4f4e236-0d0a-492c-86df-eef48a414e23' >>>>>>>> I1029 03:29:53.160143 9292 slave.cpp:3399] Executor >>>>>>>> '279bcb34-f705-4857-96ad-d96843b848fb.4b3abdcd-7ded-11e5-a82d-0240afabf713' >>>>>>>> of framework 20151029-031549-1294671788-5050-4937-0000 terminated >>>>>>>> with signal Killed >>>>>>>> I1029 03:29:53.160884 9292 slave.cpp:2696] Handling status update >>>>>>>> TASK_FAILED (UUID: 27a2080a-8807-449e-9077-837ec45b4c51) for task >>>>>>>> 279bcb34-f705-4857-96ad-d96843b848fb.4b3abdcd-7ded-11e5-a82d-0240afabf713 >>>>>>>> of framework 20151029-031549-1294671788-5050-4937-0000 from @ >>>>>>>> 0.0.0.0:0 >>>>>>>> W1029 03:29:53.161247 9288 docker.cpp:986] Ignoring updating >>>>>>>> unknown container: d4f4e236-0d0a-492c-86df-eef48a414e23 >>>>>>>> I1029 03:29:53.161548 9293 status_update_manager.cpp:322] Received >>>>>>>> status update TASK_FAILED (UUID: 27a2080a-8807-449e-9077-837ec45b4c51) >>>>>>>> for >>>>>>>> task >>>>>>>> 279bcb34-f705-4857-96ad-d96843b848fb.4b3abdcd-7ded-11e5-a82d-0240afabf713 >>>>>>>> of framework 20151029-031549-1294671788-5050-4937-0000 >>>>>>>> >>>>>>>> I run master node with env: >>>>>>>> >>>>>>>> SSL_SUPPORT_DOWNGRADE=true >>>>>>>> SSL_ENABLED=true >>>>>>>> SSL_KEY_FILE=/home/ubuntu/xx.key >>>>>>>> SSL_CERT_FILE=/home/ubuntu/xx.pem >>>>>>>> >>>>>>>> Slave node with env: >>>>>>>> >>>>>>>> SSL_ENABLED=true >>>>>>>> SSL_KEY_FILE=/home/ubuntu/xx.key >>>>>>>> SSL_CERT_FILE=/home/ubuntu/xx.pem >>>>>>>> LIBPROCESS_ADVERTISE_IP=xxx.xxx.xxx.xxx >>>>>>>> >>>>>>>> When I remove all SSL envs. Slaves work well. >>>>>>>> >>>>>>>> Did I miss sth? >>>>>>>> >>>>>>>> Version: >>>>>>>> >>>>>>>> Mesos 0.24.1 >>>>>>>> Maraton 0.9.2 >>>>>>>> >>>>>>>> OS >>>>>>>> ubuntu 14.04 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 发件人: Anindya Sinha <[email protected]> >>>>>>>> 答复: "[email protected]" <[email protected]> >>>>>>>> 日期: 2015年10月28日 星期三 下午2:32 >>>>>>>> 至: "[email protected]" <[email protected]> >>>>>>>> 主题: Re: How to tell master which ip to connect. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Oct 27, 2015 at 7:43 PM, Xiaodong Zhang <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> It works! Thanks a lot. >>>>>>>>> >>>>>>>> >>>>>>>> Ok. So we should expose advertise_ip and advertise_port as command >>>>>>>> line options for mesos-slave as well (instead of using the environment >>>>>>>> variables)? Opened https://issues.apache.org/jira/browse/MESOS-3809 >>>>>>>> . >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Another question. Do masters and slaves communicate each other via >>>>>>>>> a safety way？Is the data encrypted? I want to make sure deploy >>>>>>>>> masters and >>>>>>>>> slaves into different IaaS is PROD-READY. >>>>>>>>> >>>>>>>>> 发件人: haosdent <[email protected]> >>>>>>>>> 答复: "[email protected]" <[email protected]> >>>>>>>>> 日期: 2015年10月28日 星期三 上午10:23 >>>>>>>>> 至: user <[email protected]> >>>>>>>>> 主题: Re: How to tell master which ip to connect. >>>>>>>>> >>>>>>>>> Do you try `export LIBPROCESS_ADVERTISE_IP=xxx` and >>>>>>>>> `LIBPROCESS_ADVERTISE_PORT` when start slave? >>>>>>>>> >>>>>>>>> On Wed, Oct 28, 2015 at 10:16 AM, Xiaodong Zhang < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi teams: >>>>>>>>>> >>>>>>>>>> My scenarios is like this: >>>>>>>>>> >>>>>>>>>> My master nodes were deployed in AWS. My slaves were in AZURE.So >>>>>>>>>> they communicate via public ip. >>>>>>>>>> I got trouble when slaves try to register to master. >>>>>>>>>> Now slaves can get master’s public ip address,and can send >>>>>>>>>> register request.But they can only send there private ip to >>>>>>>>>> master.(Because >>>>>>>>>> they don’t know there public ip,thus they can’t not bind a public ip >>>>>>>>>> via >>>>>>>>>> —ip flag), thus masters can’t connect slaves.How can the slave to >>>>>>>>>> tell >>>>>>>>>> master which ip master should connect(I can’t find any flags like >>>>>>>>>> —advertise_ip >>>>>>>>>> in master). >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Best Regards, >>>>>>>>> Haosdent Huang >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Best Regards, >>>>>>> Haosdent Huang >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best Regards, >>>>>> Haosdent Huang >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best Regards, >>>>> Haosdent Huang >>>>> >>>> >>>> >>> >>> >>> -- >>> Best Regards, >>> Haosdent Huang >>> <5185_02_07.png><9D46724C-457C-4BE1-B0E4-F57B147F6DC8.png> >>> <742629F2-78E8-43F2-9015-F3D22720826B.png><5185_02_04.png> >>> >>> >>> >>> >> > > > -- > Best Regards, > Haosdent Huang > -- Best Regards, Haosdent Huang
