Re: Review Request 41294: Logger Module: Adds the ContainerLogger into the DockerContainerizer.

[Joseph Wu]

> On Dec. 14, 2015, 8:06 p.m., Timothy Chen wrote:
> > src/docker/docker.cpp, line 603
> > 
> >
> > Just a general question, what is the reasoning behind allowing the 
> > logger to modify the path and argv of the original subprocess?
> 
> Joseph Wu wrote:
> A couple of reasons:
> 
> * We need the thing-performing-the-loggging to be run as a process 
> separate from the logging module.  `Subprocess::IO::PIPE` will not be 
> sufficient since we can't read from the pipe when the agent process falls 
> over.
> * The logging process might need to do something more complicated than 
> redirecting stdout/stderr to a FD.
> * Some logging daemons (wrappers) are launched by prepending the to the 
> argc/argv array.  i.e. `logger-daemon   
> `

We're going to change this in the next iteration.  The logger will only have 
access to `out` and `err`.


- Joseph


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41294/#review110395
---


On Dec. 15, 2015, 12:41 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41294/
> ---
> 
> (Updated Dec. 15, 2015, 12:41 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman, haosdent huang, and Artem 
> Harutyunyan.
> 
> 
> Bugs: MESOS-4137
> https://issues.apache.org/jira/browse/MESOS-4137
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The `DockerContainerizer` creates and initializes its own `ContainerLogger` 
> and passes it into the `Docker` wrapper.
> 
> The `ContainerLogger` is used before launching executors, in two two call 
> sites depending on whether the agent is running in a container or not.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp 33d6fb3e82ff7328ad093648a45546a18ec7b8cb 
>   src/docker/docker.cpp 5dc4667d93b143b54841d85606affe3e4926757a 
>   src/slave/containerizer/docker.hpp 35712f599395b5f5fbc311a37c6e29b33bac0c8e 
>   src/slave/containerizer/docker.cpp 2d2dd4e0df36207c5f3cbb4fe2c50df51c0f3e9e 
> 
> Diff: https://reviews.apache.org/r/41294/diff/
> 
> 
> Testing
> ---
> 
> Tests will be modified and run later in the review chain.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>

Re: Review Request 41294: Logger Module: Adds the ContainerLogger into the DockerContainerizer.

[Joseph Wu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41294/
---

(Updated Dec. 21, 2015, 5:32 p.m.)


Review request for mesos, Benjamin Hindman, haosdent huang, and Artem 
Harutyunyan.


Changes
---

Remove `ContainerLogger` injection from the `Docker` class.


Bugs: MESOS-4137
https://issues.apache.org/jira/browse/MESOS-4137


Repository: mesos


Description
---

The `DockerContainerizer` creates and initializes its own `ContainerLogger` and 
passes it into the `Docker` wrapper.

The `ContainerLogger` is used before launching executors, in two two call sites 
depending on whether the agent is running in a container or not.


Diffs (updated)
-

  src/docker/docker.hpp c769372090513b702daa2def265aedf536565f96 
  src/slave/containerizer/docker.hpp 35712f599395b5f5fbc311a37c6e29b33bac0c8e 
  src/slave/containerizer/docker.cpp 2d2dd4e0df36207c5f3cbb4fe2c50df51c0f3e9e 

Diff: https://reviews.apache.org/r/41294/diff/


Testing
---

Tests will be modified and run later in the review chain.


Thanks,

Joseph Wu

Re: Review Request 41294: Logger Module: Adds the ContainerLogger into the DockerContainerizer.

[Joseph Wu]

> On Dec. 20, 2015, 3:17 p.m., Benjamin Hindman wrote:
> > We shouldn't pass `ContainerLogger` into `Docker` while also keeping it 
> > `Owned` inside `DockerContainerizer`. It's confusing to people, we're 
> > explicitly breaking the contract of `Owned`.
> > 
> > Instead, why not change the `Docker::run` to take `Subprocess::IO` instead 
> > of the stdout/stderr paths? Could we make that work?

Note: The updated diff no longer passes the `ContainerLogger` into the `Docker` 
class.


- Joseph


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41294/#review111430
---


On Dec. 21, 2015, 5:32 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41294/
> ---
> 
> (Updated Dec. 21, 2015, 5:32 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman, haosdent huang, and Artem 
> Harutyunyan.
> 
> 
> Bugs: MESOS-4137
> https://issues.apache.org/jira/browse/MESOS-4137
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The `DockerContainerizer` creates and initializes its own `ContainerLogger` 
> and passes it into the `Docker` wrapper.
> 
> The `ContainerLogger` is used before launching executors, in two two call 
> sites depending on whether the agent is running in a container or not.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp c769372090513b702daa2def265aedf536565f96 
>   src/slave/containerizer/docker.hpp 35712f599395b5f5fbc311a37c6e29b33bac0c8e 
>   src/slave/containerizer/docker.cpp 2d2dd4e0df36207c5f3cbb4fe2c50df51c0f3e9e 
> 
> Diff: https://reviews.apache.org/r/41294/diff/
> 
> 
> Testing
> ---
> 
> Tests will be modified and run later in the review chain.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>

Re: Review Request 41294: Logger Module: Adds the ContainerLogger into the DockerContainerizer.

[Joseph Wu]

> On Dec. 14, 2015, 8:06 p.m., Timothy Chen wrote:
> > src/docker/docker.cpp, line 603
> > 
> >
> > Just a general question, what is the reasoning behind allowing the 
> > logger to modify the path and argv of the original subprocess?

A couple of reasons:

* We need the thing-performing-the-loggging to be run as a process separate 
from the logging module.  `Subprocess::IO::PIPE` will not be sufficient since 
we can't read from the pipe when the agent process falls over.
* The logging process might need to do something more complicated than 
redirecting stdout/stderr to a FD.
* Some logging daemons (wrappers) are launched by prepending the to the 
argc/argv array.  i.e. `logger-daemon   
`


- Joseph


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41294/#review110395
---


On Dec. 14, 2015, 6:14 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41294/
> ---
> 
> (Updated Dec. 14, 2015, 6:14 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman, haosdent huang, and Artem 
> Harutyunyan.
> 
> 
> Bugs: MESOS-4137
> https://issues.apache.org/jira/browse/MESOS-4137
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The `DockerContainerizer` creates and initializes its own `ContainerLogger` 
> and passes it into the `Docker` wrapper.
> 
> The `ContainerLogger` is used before launching executors, in two two call 
> sites depending on whether the agent is running in a container or not.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp 33d6fb3e82ff7328ad093648a45546a18ec7b8cb 
>   src/docker/docker.cpp 5dc4667d93b143b54841d85606affe3e4926757a 
>   src/slave/containerizer/docker.hpp 35712f599395b5f5fbc311a37c6e29b33bac0c8e 
>   src/slave/containerizer/docker.cpp 2d2dd4e0df36207c5f3cbb4fe2c50df51c0f3e9e 
> 
> Diff: https://reviews.apache.org/r/41294/diff/
> 
> 
> Testing
> ---
> 
> Tests will be modified and run later in the review chain.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>

Re: Review Request 41294: Logger Module: Adds the ContainerLogger into the DockerContainerizer.

[Joseph Wu]

> On Dec. 14, 2015, 7:51 p.m., Timothy Chen wrote:
> > src/slave/containerizer/docker.cpp, line 139
> > 
> >
> > Why not let the create also initialize?

This was an artifact of one of the approaches (now discarded) I tried, 
regarding where to insert the module.

I agree that we can `init` in `create`.


- Joseph


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41294/#review110388
---


On Dec. 14, 2015, 6:14 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41294/
> ---
> 
> (Updated Dec. 14, 2015, 6:14 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman, haosdent huang, and Artem 
> Harutyunyan.
> 
> 
> Bugs: MESOS-4137
> https://issues.apache.org/jira/browse/MESOS-4137
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The `DockerContainerizer` creates and initializes its own `ContainerLogger` 
> and passes it into the `Docker` wrapper.
> 
> The `ContainerLogger` is used before launching executors, in two two call 
> sites depending on whether the agent is running in a container or not.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp 33d6fb3e82ff7328ad093648a45546a18ec7b8cb 
>   src/docker/docker.cpp 5dc4667d93b143b54841d85606affe3e4926757a 
>   src/slave/containerizer/docker.hpp 35712f599395b5f5fbc311a37c6e29b33bac0c8e 
>   src/slave/containerizer/docker.cpp 2d2dd4e0df36207c5f3cbb4fe2c50df51c0f3e9e 
> 
> Diff: https://reviews.apache.org/r/41294/diff/
> 
> 
> Testing
> ---
> 
> Tests will be modified and run later in the review chain.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>

Re: Review Request 41294: Logger Module: Adds the ContainerLogger into the DockerContainerizer.

[Joseph Wu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41294/
---

(Updated Dec. 15, 2015, 12:41 p.m.)


Review request for mesos, Benjamin Hindman, haosdent huang, and Artem 
Harutyunyan.


Changes
---

Moved `initialize` into `create`. Remove `options` in favor of module 
parameters.


Bugs: MESOS-4137
https://issues.apache.org/jira/browse/MESOS-4137


Repository: mesos


Description
---

The `DockerContainerizer` creates and initializes its own `ContainerLogger` and 
passes it into the `Docker` wrapper.

The `ContainerLogger` is used before launching executors, in two two call sites 
depending on whether the agent is running in a container or not.


Diffs (updated)
-

  src/docker/docker.hpp 33d6fb3e82ff7328ad093648a45546a18ec7b8cb 
  src/docker/docker.cpp 5dc4667d93b143b54841d85606affe3e4926757a 
  src/slave/containerizer/docker.hpp 35712f599395b5f5fbc311a37c6e29b33bac0c8e 
  src/slave/containerizer/docker.cpp 2d2dd4e0df36207c5f3cbb4fe2c50df51c0f3e9e 

Diff: https://reviews.apache.org/r/41294/diff/


Testing
---

Tests will be modified and run later in the review chain.


Thanks,

Joseph Wu

Re: Review Request 41294: Logger Module: Adds the ContainerLogger into the DockerContainerizer.

[Joseph Wu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41294/
---

(Updated Dec. 14, 2015, 6:14 p.m.)


Review request for mesos, Benjamin Hindman, haosdent huang, and Artem 
Harutyunyan.


Changes
---

Finish up the meat of the logic.  Migrated ContainerLogger into the 
containerizer itself.


Summary (updated)
-

Logger Module: Adds the ContainerLogger into the DockerContainerizer.


Bugs: MESOS-4137
https://issues.apache.org/jira/browse/MESOS-4137


Repository: mesos


Description (updated)
---

The `DockerContainerizer` creates and initializes its own `ContainerLogger` and 
passes it into the `Docker` wrapper.

The `ContainerLogger` is used before launching executors, in two two call sites 
depending on whether the agent is running in a container or not.


Diffs (updated)
-

  src/docker/docker.hpp 33d6fb3e82ff7328ad093648a45546a18ec7b8cb 
  src/docker/docker.cpp 5dc4667d93b143b54841d85606affe3e4926757a 
  src/slave/containerizer/docker.hpp 35712f599395b5f5fbc311a37c6e29b33bac0c8e 
  src/slave/containerizer/docker.cpp 2d2dd4e0df36207c5f3cbb4fe2c50df51c0f3e9e 

Diff: https://reviews.apache.org/r/41294/diff/


Testing (updated)
---

Tests will be modified and run later in the review chain.


Thanks,

Joseph Wu

Re: Review Request 41294: Logger Module: Adds the ContainerLogger into the DockerContainerizer.

[Timothy Chen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41294/#review110388
---



src/slave/containerizer/docker.cpp (line 139)


Why not let the create also initialize?


- Timothy Chen


On Dec. 15, 2015, 2:14 a.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41294/
> ---
> 
> (Updated Dec. 15, 2015, 2:14 a.m.)
> 
> 
> Review request for mesos, Benjamin Hindman, haosdent huang, and Artem 
> Harutyunyan.
> 
> 
> Bugs: MESOS-4137
> https://issues.apache.org/jira/browse/MESOS-4137
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The `DockerContainerizer` creates and initializes its own `ContainerLogger` 
> and passes it into the `Docker` wrapper.
> 
> The `ContainerLogger` is used before launching executors, in two two call 
> sites depending on whether the agent is running in a container or not.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp 33d6fb3e82ff7328ad093648a45546a18ec7b8cb 
>   src/docker/docker.cpp 5dc4667d93b143b54841d85606affe3e4926757a 
>   src/slave/containerizer/docker.hpp 35712f599395b5f5fbc311a37c6e29b33bac0c8e 
>   src/slave/containerizer/docker.cpp 2d2dd4e0df36207c5f3cbb4fe2c50df51c0f3e9e 
> 
> Diff: https://reviews.apache.org/r/41294/diff/
> 
> 
> Testing
> ---
> 
> Tests will be modified and run later in the review chain.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>

Re: Review Request 41294: Logger Module: Adds the ContainerLogger into the DockerContainerizer.

[Timothy Chen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41294/#review110395
---



src/docker/docker.cpp (line 593)


Just a general question, what is the reasoning behind allowing the logger 
to modify the path and argv of the original subprocess?


- Timothy Chen


On Dec. 15, 2015, 2:14 a.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41294/
> ---
> 
> (Updated Dec. 15, 2015, 2:14 a.m.)
> 
> 
> Review request for mesos, Benjamin Hindman, haosdent huang, and Artem 
> Harutyunyan.
> 
> 
> Bugs: MESOS-4137
> https://issues.apache.org/jira/browse/MESOS-4137
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The `DockerContainerizer` creates and initializes its own `ContainerLogger` 
> and passes it into the `Docker` wrapper.
> 
> The `ContainerLogger` is used before launching executors, in two two call 
> sites depending on whether the agent is running in a container or not.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp 33d6fb3e82ff7328ad093648a45546a18ec7b8cb 
>   src/docker/docker.cpp 5dc4667d93b143b54841d85606affe3e4926757a 
>   src/slave/containerizer/docker.hpp 35712f599395b5f5fbc311a37c6e29b33bac0c8e 
>   src/slave/containerizer/docker.cpp 2d2dd4e0df36207c5f3cbb4fe2c50df51c0f3e9e 
> 
> Diff: https://reviews.apache.org/r/41294/diff/
> 
> 
> Testing
> ---
> 
> Tests will be modified and run later in the review chain.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>