[jira] [Updated] (MESOS-6057) docker isolator does not overwrite Dockerfile ENV

2016-08-19 Thread Jie Yu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MESOS-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jie Yu updated MESOS-6057:
--
Labels: mesosphere  (was: )

> docker isolator does not overwrite Dockerfile ENV
> -
>
> Key: MESOS-6057
> URL: https://issues.apache.org/jira/browse/MESOS-6057
> Project: Mesos
>  Issue Type: Bug
>  Components: containerization
>Affects Versions: 1.0.0, 1.0.1, 1.1.0
>Reporter: Stéphane Cottin
>Priority: Critical
>  Labels: mesosphere
> Fix For: 1.1.0
>
>
> The docker/runtime isolator does not overwrite env values when a default 
> value is present in the Dockerfile.
> Steps to reproduce : 
> {code}
> mesos-execute --master=leader.mesos:5050 --name=test 
> --docker_image=bashell/alpine-bash --env="{\"LC_ALL\": 
> \fr_FR.UTF-8\",\"LC_TEST\": \"fr_FR.UTF-8\"}" --command="env"
> {code}
> outputs in stdout :
> {code}
> [...]
> LC_ALL=en_US.UTF-8
> LC_TEST=fr_FR.UTF-8
> [...]
> {code}
>  
> {{en_US.UTF-8}} is the default value from the dockerfile, see 
> https://hub.docker.com/r/bashell/alpine-bash/~/dockerfile/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-6057) docker isolator does not overwrite Dockerfile ENV

2016-08-19 Thread Jie Yu (JIRA) 

[ 
https://issues.apache.org/jira/browse/MESOS-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429208#comment-15429208
 ] 

Jie Yu commented on MESOS-6057:
---

Will address this in 1.1.0.

> docker isolator does not overwrite Dockerfile ENV
> -
>
> Key: MESOS-6057
> URL: https://issues.apache.org/jira/browse/MESOS-6057
> Project: Mesos
>  Issue Type: Bug
>  Components: containerization
>Affects Versions: 1.0.0, 1.0.1, 1.1.0
>Reporter: Stéphane Cottin
>Priority: Critical
> Fix For: 1.1.0
>
>
> The docker/runtime isolator does not overwrite env values when a default 
> value is present in the Dockerfile.
> Steps to reproduce : 
> {code}
> mesos-execute --master=leader.mesos:5050 --name=test 
> --docker_image=bashell/alpine-bash --env="{\"LC_ALL\": 
> \fr_FR.UTF-8\",\"LC_TEST\": \"fr_FR.UTF-8\"}" --command="env"
> {code}
> outputs in stdout :
> {code}
> [...]
> LC_ALL=en_US.UTF-8
> LC_TEST=fr_FR.UTF-8
> [...]
> {code}
>  
> {{en_US.UTF-8}} is the default value from the dockerfile, see 
> https://hub.docker.com/r/bashell/alpine-bash/~/dockerfile/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-6057) docker isolator does not overwrite Dockerfile ENV

2016-08-19 Thread Jie Yu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MESOS-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jie Yu updated MESOS-6057:
--
Fix Version/s: (was: 1.0.1)

> docker isolator does not overwrite Dockerfile ENV
> -
>
> Key: MESOS-6057
> URL: https://issues.apache.org/jira/browse/MESOS-6057
> Project: Mesos
>  Issue Type: Bug
>  Components: containerization
>Affects Versions: 1.0.0, 1.0.1, 1.1.0
>Reporter: Stéphane Cottin
>Priority: Critical
> Fix For: 1.1.0
>
>
> The docker/runtime isolator does not overwrite env values when a default 
> value is present in the Dockerfile.
> Steps to reproduce : 
> {code}
> mesos-execute --master=leader.mesos:5050 --name=test 
> --docker_image=bashell/alpine-bash --env="{\"LC_ALL\": 
> \fr_FR.UTF-8\",\"LC_TEST\": \"fr_FR.UTF-8\"}" --command="env"
> {code}
> outputs in stdout :
> {code}
> [...]
> LC_ALL=en_US.UTF-8
> LC_TEST=fr_FR.UTF-8
> [...]
> {code}
>  
> {{en_US.UTF-8}} is the default value from the dockerfile, see 
> https://hub.docker.com/r/bashell/alpine-bash/~/dockerfile/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-6057) docker isolator does not overwrite Dockerfile ENV

2016-08-19 Thread Jie Yu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MESOS-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jie Yu updated MESOS-6057:
--
Fix Version/s: 1.1.0
   1.0.1

> docker isolator does not overwrite Dockerfile ENV
> -
>
> Key: MESOS-6057
> URL: https://issues.apache.org/jira/browse/MESOS-6057
> Project: Mesos
>  Issue Type: Bug
>  Components: containerization
>Affects Versions: 1.0.0, 1.0.1, 1.1.0
>Reporter: Stéphane Cottin
>Priority: Critical
> Fix For: 1.0.1, 1.1.0
>
>
> The docker/runtime isolator does not overwrite env values when a default 
> value is present in the Dockerfile.
> Steps to reproduce : 
> {code}
> mesos-execute --master=leader.mesos:5050 --name=test 
> --docker_image=bashell/alpine-bash --env="{\"LC_ALL\": 
> \fr_FR.UTF-8\",\"LC_TEST\": \"fr_FR.UTF-8\"}" --command="env"
> {code}
> outputs in stdout :
> {code}
> [...]
> LC_ALL=en_US.UTF-8
> LC_TEST=fr_FR.UTF-8
> [...]
> {code}
>  
> {{en_US.UTF-8}} is the default value from the dockerfile, see 
> https://hub.docker.com/r/bashell/alpine-bash/~/dockerfile/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-6057) docker isolator does not overwrite Dockerfile ENV

2016-08-19 Thread Jie Yu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MESOS-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jie Yu updated MESOS-6057:
--
Priority: Critical  (was: Major)

> docker isolator does not overwrite Dockerfile ENV
> -
>
> Key: MESOS-6057
> URL: https://issues.apache.org/jira/browse/MESOS-6057
> Project: Mesos
>  Issue Type: Bug
>  Components: containerization
>Affects Versions: 1.0.0, 1.0.1, 1.1.0
>Reporter: Stéphane Cottin
>Priority: Critical
>
> The docker/runtime isolator does not overwrite env values when a default 
> value is present in the Dockerfile.
> Steps to reproduce : 
> {code}
> mesos-execute --master=leader.mesos:5050 --name=test 
> --docker_image=bashell/alpine-bash --env="{\"LC_ALL\": 
> \fr_FR.UTF-8\",\"LC_TEST\": \"fr_FR.UTF-8\"}" --command="env"
> {code}
> outputs in stdout :
> {code}
> [...]
> LC_ALL=en_US.UTF-8
> LC_TEST=fr_FR.UTF-8
> [...]
> {code}
>  
> {{en_US.UTF-8}} is the default value from the dockerfile, see 
> https://hub.docker.com/r/bashell/alpine-bash/~/dockerfile/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MESOS-6057) docker isolator does not overwrite Dockerfile ENV

2016-08-19 Thread JIRA 
Stéphane Cottin created MESOS-6057:
--

 Summary: docker isolator does not overwrite Dockerfile ENV
 Key: MESOS-6057
 URL: https://issues.apache.org/jira/browse/MESOS-6057
 Project: Mesos
  Issue Type: Bug
  Components: containerization
Affects Versions: 1.0.0, 1.0.1, 1.1.0
Reporter: Stéphane Cottin


The docker/runtime isolator does not overwrite env values when a default value 
is present in the Dockerfile.

Steps to reproduce : 

{code}
mesos-execute --master=leader.mesos:5050 --name=test 
--docker_image=bashell/alpine-bash --env="{\"LC_ALL\": 
\fr_FR.UTF-8\",\"LC_TEST\": \"fr_FR.UTF-8\"}" --command="env"
{code}

outputs in stdout :

{code}
[...]
LC_ALL=en_US.UTF-8
LC_TEST=fr_FR.UTF-8
[...]
{code}
 
{{en_US.UTF-8}} is the default value from the dockerfile, see 
https://hub.docker.com/r/bashell/alpine-bash/~/dockerfile/





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-6056) add NOOP Container Logger for mesos

2016-08-19 Thread Joseph Wu (JIRA) 

[ 
https://issues.apache.org/jira/browse/MESOS-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15428539#comment-15428539
 ] 

Joseph Wu commented on MESOS-6056:
--

In my opinion, we don't want to silence logs as it makes tasks even harder to 
debug (it's already pretty hard).  Using the docker daemon for logs is not 
sufficient as this only covers a (rapidly shrinking) number of use cases.  The 
docker daemon does not manage logs from the Mesos containerizer, for example.

It would be perfectly acceptable to turn off logging in your own setup, with 
this module.  But as a project/community, we don't encourage it.

P.S. We don't add features in Github PRs.  See: 
http://mesos.apache.org/documentation/latest/submitting-a-patch/

> add NOOP Container Logger for mesos
> ---
>
> Key: MESOS-6056
> URL: https://issues.apache.org/jira/browse/MESOS-6056
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization, slave
>Affects Versions: 1.0.0
> Environment: mesos 1.0.0, docker
>Reporter: IvanJobs
>Priority: Trivial
>  Labels: easyfix, features
>   Original Estimate: 96h
>  Remaining Estimate: 96h
>
> mesos has two Container Loggers in its source files. 
> One is build into mesos-agent: sandbox Container Logger, it just redirects 
> stderr/stdout to sandbox, causing fill disk usage problem.
> The other is LogrotateContainerLogger module lib, it's good, we can make sure 
> stdout/stderr in sandbox be in a constant size.
> But there is a common need: don't write stdout/stderr into sandbox, pity, we 
> don't have any flags for turning it off. 
> This is a come around for this: developing a new module lib for 
> ContainerLogger for doing nothing(redirect stdout/stderr to /dev/null)
> yep, that's it. We need a NOOP ContainerLogger, BTW, FYI, we can also 
> retrieve stderr/stdout from docker daemon either.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-6056) add NOOP Container Logger for mesos

2016-08-19 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/MESOS-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427966#comment-15427966
 ] 

ASF GitHub Bot commented on MESOS-6056:
---

GitHub user IvanJobs opened a pull request:

https://github.com/apache/mesos/pull/159

MESOS-6056 add NOOP Container Logger for mesos.

mesos has two Container Loggers in its source files. 
One is build into mesos-agent: sandbox Container Logger, it just redirects 
stderr/stdout to sandbox, causing fill disk usage problem.
The other is LogrotateContainerLogger module lib, it's good, we can make 
sure stdout/stderr in sandbox be in a constant size.
But there is a common need: don't write stdout/stderr into sandbox, pity, 
we don't have any flags for turning it off. 

This is a come around for this: developing a new module lib for 
ContainerLogger for doing nothing(redirect stdout/stderr to /dev/null)

yep, that's it. We need a NOOP ContainerLogger,.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/IvanJobs/mesos master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/mesos/pull/159.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #159


commit 3afdacffee17b87af6677b9dc7687022b76dea15
Author: IvanJobs 
Date:   2016-08-19T10:21:23Z

MESOS-6056 add NOOP Container Logger for mesos.




> add NOOP Container Logger for mesos
> ---
>
> Key: MESOS-6056
> URL: https://issues.apache.org/jira/browse/MESOS-6056
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization, slave
>Affects Versions: 1.0.0
> Environment: mesos 1.0.0, docker
>Reporter: IvanJobs
>Priority: Trivial
>  Labels: easyfix, features
>   Original Estimate: 96h
>  Remaining Estimate: 96h
>
> mesos has two Container Loggers in its source files. 
> One is build into mesos-agent: sandbox Container Logger, it just redirects 
> stderr/stdout to sandbox, causing fill disk usage problem.
> The other is LogrotateContainerLogger module lib, it's good, we can make sure 
> stdout/stderr in sandbox be in a constant size.
> But there is a common need: don't write stdout/stderr into sandbox, pity, we 
> don't have any flags for turning it off. 
> This is a come around for this: developing a new module lib for 
> ContainerLogger for doing nothing(redirect stdout/stderr to /dev/null)
> yep, that's it. We need a NOOP ContainerLogger, BTW, FYI, we can also 
> retrieve stderr/stdout from docker daemon either.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-6047) Mesos Slave is unable to register with Master

2016-08-19 Thread Sankar Mittapally (JIRA) 

[ 
https://issues.apache.org/jira/browse/MESOS-6047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427940#comment-15427940
 ] 

Sankar Mittapally commented on MESOS-6047:
--

root@ip-10-0-0-24:/var/log/mesos# vi mesos-slave.ERROR
Log file created at: 2016/08/16 10:44:56
Running on machine: ip-10-0-0-24
Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
E0816 10:44:56.592535 18440 process.cpp:2105] Failed to shutdown socket with fd 
12: Transport endpoint is not connected
E0816 10:46:11.616282 18440 process.cpp:2105] Failed to shutdown socket with fd 
12: Transport endpoint is not connected
E0817 04:31:50.218361 18440 process.cpp:2105] Failed to shutdown socket with fd 
12: Transport endpoint is not connected
E0817 04:31:50.733708 18440 process.cpp:2105] Failed to shutdown socket with fd 
12: Transport endpoint is not connected
E0817 04:31:51.552975 18440 process.cpp:2105] Failed to shutdown socket with fd 
12: Transport endpoint is not connected
E0817 04:31:52.250030 18440 process.cpp:2105] Failed to shutdown socket with fd 
12: Transport endpoint is not connected



Log file created at: 2016/08/16 09:55:13
Running on machine: ip-10-0-0-24
Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
I0816 09:55:13.757421 18421 logging.cpp:194] INFO level logging started!
I0816 09:55:13.760455 18421 containerizer.cpp:196] Using isolation: 
posix/cpu,posix/mem,filesystem/posix,network/cni
I0816 09:55:13.762310 18421 linux_launcher.cpp:101] Using 
/sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
I0816 09:55:13.764400 18421 main.cpp:434] Starting Mesos agent
I0816 09:55:13.765594 18437 slave.cpp:198] Agent started on 1)@10.0.0.24:5051
I0816 09:55:13.765614 18437 slave.cpp:199] Flags at startup: 
--appc_simple_discovery_uri_prefix="http://; 
--appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false" 
--authenticate_http_readwrite="false" --authenticatee="crammd5" 
--authentication_backoff_factor="1secs" --authorizer="local" 
--cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" 
--cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" 
--cgroups_root="mesos" --container_disk_watch_interval="15secs" 
--containerizers="mesos" --default_role="*" --disk_watch_interval="1mins" 
--docker="docker" --docker_kill_orphans="true" 
--docker_registry="https://registry-1.docker.io; --docker_remove_delay="6hrs" 
--docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" 
--docker_store_dir="/tmp/mesos/store/docker" 
--docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" 
--enforce_container_disk_quota="false" --executor_registration_timeout="1mins" 
--executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/mesos/fetch" 
--fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" 
--gc_disk_headroom="0.1" --hadoop_home="" --help="false" 
--hostname_lookup="true" --http_authenticators="basic" 
--http_command_executor="false" --image_provisioner_backend="copy" 
--initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" 
--launcher_dir="/usr/libexec/mesos" --log_dir="/var/log/mesos" --logbufsecs="0" 
--logging_level="INFO" --master="zk://10.0.0.197:2181,10.0.0.203:2181/mesos" 
--oversubscribed_resources_interval="15secs" --perf_duration="10secs" 
--perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" 
--quiet="false" --recover="reconnect" --recovery_timeout="15mins" 
--registration_backoff_factor="1secs" --revocable_cpu_low_priority="true" 
--sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" 
--systemd_enable_support="true" 
--systemd_runtime_directory="/run/systemd/system" --version="false" 
--work_dir="/var/lib/mesos"
I0816 09:55:13.766063 18437 slave.cpp:519] Agent resources: cpus(*):1; 
mem(*):1000; disk(*):3962; ports(*):[31000-32000]
I0816 09:55:13.766104 18437 slave.cpp:527] Agent attributes: [  ]
I0816 09:55:13.766113 18437 slave.cpp:532] Agent hostname: server1
I0816 09:55:13.768714 18437 state.cpp:57] Recovering state from 
'/var/lib/mesos/meta'
I0816 09:55:13.768750 18437 state.cpp:697] No checkpointed resources found at 
'/var/lib/mesos/meta/resources/resources.info'
I0816 09:55:13.769193 18437 status_update_manager.cpp:200] Recovering status 
update manager
I0816 09:55:13.769718 18434 containerizer.cpp:522] Recovering containerizer
I0816 09:55:13.770333 18434 provisioner.cpp:253] Provisioner recovery complete
I0816 09:55:13.770530 18438 slave.cpp:4782] Finished recovery
I0816 09:55:13.894096 18435 group.cpp:349] Group process 
(group(1)@10.0.0.24:5051) connected to ZooKeeper
I0816 09:55:13.894140 18435 group.cpp:837] Syncing group operations: queue size 
(joins, cancels, datas) = (0, 0, 0)
I0816 09:55:13.894150 18435 group.cpp:427] Trying to create path '/mesos' in 
ZooKeeper
I0816 09:56:11.984453 18439 detector.cpp:152] Detected a new leader: (id='3')

[jira] [Created] (MESOS-6056) add NOOP Container Logger for mesos

2016-08-19 Thread IvanJobs (JIRA) 
IvanJobs created MESOS-6056:
---

 Summary: add NOOP Container Logger for mesos
 Key: MESOS-6056
 URL: https://issues.apache.org/jira/browse/MESOS-6056
 Project: Mesos
  Issue Type: Improvement
  Components: containerization, slave
Affects Versions: 1.0.0
 Environment: mesos 1.0.0, docker
Reporter: IvanJobs
Priority: Trivial


mesos has two Container Loggers in its source files. 
One is build into mesos-agent: sandbox Container Logger, it just redirects 
stderr/stdout to sandbox, causing fill disk usage problem.
The other is LogrotateContainerLogger module lib, it's good, we can make sure 
stdout/stderr in sandbox be in a constant size.
But there is a common need: don't write stdout/stderr into sandbox, pity, we 
don't have any flags for turning it off. 

This is a come around for this: developing a new module lib for ContainerLogger 
for doing nothing(redirect stdout/stderr to /dev/null)

yep, that's it. We need a NOOP ContainerLogger, BTW, FYI, we can also retrieve 
stderr/stdout from docker daemon either.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)