Re: Review Request 54147: Added a server side component for the IOSwitchboard.

[Kevin Klues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54147/
---

(Updated Dec. 1, 2016, 1:31 a.m.)


Review request for mesos and Jie Yu.


Changes
---

Updated test to work for non root user.


Bugs: MESOS-6639
https://issues.apache.org/jira/browse/MESOS-6639


Repository: mesos


Description
---

The 'IOSwitchboardServer' component encapsulates the server side logic
for redirecting the 'stdin/stdout/stderr' of a container to/from
multiple sources/targets. For now, we only redirect IO from a
container to the FDs supplied to us by the logger.  We also send the
stdout/stderr data to a simple HTTP server that we launch on a unix
domain socket set up by the agent.  Right now this server is just a
stub and doesn't do anything useful.

In future commits, we will expand this HTTP server to handle
'ATTACH_CONTAINER_INPUT' and 'ATTACH_CONTAINER_OUTPUT' calls on behalf
of a container. It will use the stdout/stderr messages passed to it to
and send that data over the response stream to any clients connected
with an 'ATTACH_CONTAINER_OUTPUT' call. Likewise, it will take any
input streamed in over a 'ATTACH_CONTAINER_INPUT' request and write it
to a container's stdin.

In 'local' mode, it will be run inside the agent itself. In
'non-local' mode, it will be run as an external process to survive
agent restarts.


Diffs (updated)
-

  src/Makefile.am 7750ed756d60aa61225667c129df35c6ec70f239 
  src/slave/containerizer/mesos/io/switchboard.hpp 
d3ab52d98ade067ccec63f3539073c152840099f 
  src/slave/containerizer/mesos/io/switchboard.cpp 
ae7a1fe3e24d5dc18269d9da951493700ee957d0 
  src/tests/containerizer/io_switchboard_tests.cpp PRE-CREATION 

Diff: https://reviews.apache.org/r/54147/diff/


Testing
---

GTEST_FILTER="" make -j check
sudo src/mesos-tests


Thanks,

Kevin Klues

Re: Review Request 54147: Added a server side component for the IOSwitchboard.

[Kevin Klues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54147/
---

(Updated Dec. 1, 2016, 12:53 a.m.)


Review request for mesos and Jie Yu.


Changes
---

Updated to address Jie's newest comments.


Bugs: MESOS-6639
https://issues.apache.org/jira/browse/MESOS-6639


Repository: mesos


Description
---

The 'IOSwitchboardServer' component encapsulates the server side logic
for redirecting the 'stdin/stdout/stderr' of a container to/from
multiple sources/targets. For now, we only redirect IO from a
container to the FDs supplied to us by the logger.  We also send the
stdout/stderr data to a simple HTTP server that we launch on a unix
domain socket set up by the agent.  Right now this server is just a
stub and doesn't do anything useful.

In future commits, we will expand this HTTP server to handle
'ATTACH_CONTAINER_INPUT' and 'ATTACH_CONTAINER_OUTPUT' calls on behalf
of a container. It will use the stdout/stderr messages passed to it to
and send that data over the response stream to any clients connected
with an 'ATTACH_CONTAINER_OUTPUT' call. Likewise, it will take any
input streamed in over a 'ATTACH_CONTAINER_INPUT' request and write it
to a container's stdin.

In 'local' mode, it will be run inside the agent itself. In
'non-local' mode, it will be run as an external process to survive
agent restarts.


Diffs (updated)
-

  src/Makefile.am 7750ed756d60aa61225667c129df35c6ec70f239 
  src/slave/containerizer/mesos/io/switchboard.hpp 
d3ab52d98ade067ccec63f3539073c152840099f 
  src/slave/containerizer/mesos/io/switchboard.cpp 
ae7a1fe3e24d5dc18269d9da951493700ee957d0 
  src/tests/containerizer/io_switchboard_tests.cpp PRE-CREATION 

Diff: https://reviews.apache.org/r/54147/diff/


Testing
---

GTEST_FILTER="" make -j check
sudo src/mesos-tests


Thanks,

Kevin Klues

Re: Review Request 54147: Added a server side component for the IOSwitchboard.

[Jie Yu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54147/#review157515
---




src/tests/containerizer/io_switchboard_tests.cpp (line 58)


close this and stderrFd as well?


- Jie Yu


On Nov. 30, 2016, 10:26 p.m., Kevin Klues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54147/
> ---
> 
> (Updated Nov. 30, 2016, 10:26 p.m.)
> 
> 
> Review request for mesos and Jie Yu.
> 
> 
> Bugs: MESOS-6639
> https://issues.apache.org/jira/browse/MESOS-6639
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The 'IOSwitchboardServer' component encapsulates the server side logic
> for redirecting the 'stdin/stdout/stderr' of a container to/from
> multiple sources/targets. For now, we only redirect IO from a
> container to the FDs supplied to us by the logger.  We also send the
> stdout/stderr data to a simple HTTP server that we launch on a unix
> domain socket set up by the agent.  Right now this server is just a
> stub and doesn't do anything useful.
> 
> In future commits, we will expand this HTTP server to handle
> 'ATTACH_CONTAINER_INPUT' and 'ATTACH_CONTAINER_OUTPUT' calls on behalf
> of a container. It will use the stdout/stderr messages passed to it to
> and send that data over the response stream to any clients connected
> with an 'ATTACH_CONTAINER_OUTPUT' call. Likewise, it will take any
> input streamed in over a 'ATTACH_CONTAINER_INPUT' request and write it
> to a container's stdin.
> 
> In 'local' mode, it will be run inside the agent itself. In
> 'non-local' mode, it will be run as an external process to survive
> agent restarts.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am d1cc1016b1642c504f52c27623bc12c7ddf07599 
>   src/slave/containerizer/mesos/io/switchboard.hpp 
> aaa3a35245b291f6003f519dbf8c0e1b82bc15fd 
>   src/slave/containerizer/mesos/io/switchboard.cpp 
> 25cbf2447d197134f0753b062b6f4130821005b2 
>   src/tests/containerizer/io_switchboard_tests.cpp PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/54147/diff/
> 
> 
> Testing
> ---
> 
> GTEST_FILTER="" make -j check
> sudo src/mesos-tests
> 
> 
> Thanks,
> 
> Kevin Klues
> 
>

Re: Review Request 54147: Added a server side component for the IOSwitchboard.

[Kevin Klues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54147/
---

(Updated Nov. 30, 2016, 10:26 p.m.)


Review request for mesos and Jie Yu.


Changes
---

Updated based on Jie's comments.


Bugs: MESOS-6639
https://issues.apache.org/jira/browse/MESOS-6639


Repository: mesos


Description
---

The 'IOSwitchboardServer' component encapsulates the server side logic
for redirecting the 'stdin/stdout/stderr' of a container to/from
multiple sources/targets. For now, we only redirect IO from a
container to the FDs supplied to us by the logger.  We also send the
stdout/stderr data to a simple HTTP server that we launch on a unix
domain socket set up by the agent.  Right now this server is just a
stub and doesn't do anything useful.

In future commits, we will expand this HTTP server to handle
'ATTACH_CONTAINER_INPUT' and 'ATTACH_CONTAINER_OUTPUT' calls on behalf
of a container. It will use the stdout/stderr messages passed to it to
and send that data over the response stream to any clients connected
with an 'ATTACH_CONTAINER_OUTPUT' call. Likewise, it will take any
input streamed in over a 'ATTACH_CONTAINER_INPUT' request and write it
to a container's stdin.

In 'local' mode, it will be run inside the agent itself. In
'non-local' mode, it will be run as an external process to survive
agent restarts.


Diffs (updated)
-

  src/Makefile.am d1cc1016b1642c504f52c27623bc12c7ddf07599 
  src/slave/containerizer/mesos/io/switchboard.hpp 
aaa3a35245b291f6003f519dbf8c0e1b82bc15fd 
  src/slave/containerizer/mesos/io/switchboard.cpp 
25cbf2447d197134f0753b062b6f4130821005b2 
  src/tests/containerizer/io_switchboard_tests.cpp PRE-CREATION 

Diff: https://reviews.apache.org/r/54147/diff/


Testing
---

GTEST_FILTER="" make -j check
sudo src/mesos-tests


Thanks,

Kevin Klues

Re: Review Request 54147: Added a server side component for the IOSwitchboard.

[Jie Yu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54147/#review157461
---


Fix it, then Ship it!





src/slave/containerizer/mesos/io/switchboard.hpp (line 111)


No need for explicit. You need explicit only for single parameter 
constructor.



src/slave/containerizer/mesos/io/switchboard.cpp (lines 297 - 302)


Try if this works `{defer(self(), ::outputHook, ...)}`



src/slave/containerizer/mesos/io/switchboard.cpp (lines 309 - 314)


Ditto.



src/slave/containerizer/mesos/io/switchboard.cpp (line 337)


onXXX accepts a void function. No need to 'return'.



src/slave/containerizer/mesos/io/switchboard.cpp (line 339)


Ignore this if you already did that in the following patch. We should 
terminate the process if promise is set or failed.



src/slave/containerizer/mesos/io/switchboard.cpp (line 345)


No need to return



src/slave/containerizer/mesos/io/switchboard.cpp (line 363)


Binding `this` without a defer is not safe. socket.accept() might finish 
after IOSwitchboardServerProcess terminates and is deleted.



src/slave/containerizer/mesos/io/switchboard.cpp (line 366)


No need to return for onAny



src/slave/containerizer/mesos/io/switchboard.cpp (line 376)


Ditto. This should be a defer.



src/tests/containerizer/io_switchboard_tests.cpp (line 40)


Why MesosTest? This can just be a TemporaryDirectoryTest



src/tests/containerizer/io_switchboard_tests.cpp (line 46)


Do you need this in this test?



src/tests/containerizer/io_switchboard_tests.cpp (lines 59 - 60)


If this is a TemporaryDirectoryTest, you can simply use `sandbox.get()`



src/tests/containerizer/io_switchboard_tests.cpp (line 118)


This should be EXPECT



src/tests/containerizer/io_switchboard_tests.cpp (line 123)


Ditto. this should be EXPECT


- Jie Yu


On Nov. 30, 2016, 9:35 p.m., Kevin Klues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54147/
> ---
> 
> (Updated Nov. 30, 2016, 9:35 p.m.)
> 
> 
> Review request for mesos and Jie Yu.
> 
> 
> Bugs: MESOS-6639
> https://issues.apache.org/jira/browse/MESOS-6639
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The 'IOSwitchboardServer' component encapsulates the server side logic
> for redirecting the 'stdin/stdout/stderr' of a container to/from
> multiple sources/targets. For now, we only redirect IO from a
> container to the FDs supplied to us by the logger.  We also send the
> stdout/stderr data to a simple HTTP server that we launch on a unix
> domain socket set up by the agent.  Right now this server is just a
> stub and doesn't do anything useful.
> 
> In future commits, we will expand this HTTP server to handle
> 'ATTACH_CONTAINER_INPUT' and 'ATTACH_CONTAINER_OUTPUT' calls on behalf
> of a container. It will use the stdout/stderr messages passed to it to
> and send that data over the response stream to any clients connected
> with an 'ATTACH_CONTAINER_OUTPUT' call. Likewise, it will take any
> input streamed in over a 'ATTACH_CONTAINER_INPUT' request and write it
> to a container's stdin.
> 
> In 'local' mode, it will be run inside the agent itself. In
> 'non-local' mode, it will be run as an external process to survive
> agent restarts.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am d1cc1016b1642c504f52c27623bc12c7ddf07599 
>   src/slave/containerizer/mesos/io/switchboard.hpp 
> aaa3a35245b291f6003f519dbf8c0e1b82bc15fd 
>   src/slave/containerizer/mesos/io/switchboard.cpp 
> 25cbf2447d197134f0753b062b6f4130821005b2 
>   src/tests/containerizer/io_switchboard_tests.cpp PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/54147/diff/
> 
> 
> Testing
> ---
> 
> GTEST_FILTER="" make -j check
> sudo src/mesos-tests
> 
> 
> Thanks,
> 
> Kevin Klues
> 
>

Re: Review Request 54147: Added a server side component for the IOSwitchboard.

[Kevin Klues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54147/
---

(Updated Nov. 30, 2016, 9:35 p.m.)


Review request for mesos and Jie Yu.


Changes
---

Updated to use a defer() in the redirect hooks, based on offline comments from 
Jie.


Bugs: MESOS-6639
https://issues.apache.org/jira/browse/MESOS-6639


Repository: mesos


Description
---

The 'IOSwitchboardServer' component encapsulates the server side logic
for redirecting the 'stdin/stdout/stderr' of a container to/from
multiple sources/targets. For now, we only redirect IO from a
container to the FDs supplied to us by the logger.  We also send the
stdout/stderr data to a simple HTTP server that we launch on a unix
domain socket set up by the agent.  Right now this server is just a
stub and doesn't do anything useful.

In future commits, we will expand this HTTP server to handle
'ATTACH_CONTAINER_INPUT' and 'ATTACH_CONTAINER_OUTPUT' calls on behalf
of a container. It will use the stdout/stderr messages passed to it to
and send that data over the response stream to any clients connected
with an 'ATTACH_CONTAINER_OUTPUT' call. Likewise, it will take any
input streamed in over a 'ATTACH_CONTAINER_INPUT' request and write it
to a container's stdin.

In 'local' mode, it will be run inside the agent itself. In
'non-local' mode, it will be run as an external process to survive
agent restarts.


Diffs (updated)
-

  src/Makefile.am d1cc1016b1642c504f52c27623bc12c7ddf07599 
  src/slave/containerizer/mesos/io/switchboard.hpp 
aaa3a35245b291f6003f519dbf8c0e1b82bc15fd 
  src/slave/containerizer/mesos/io/switchboard.cpp 
25cbf2447d197134f0753b062b6f4130821005b2 
  src/tests/containerizer/io_switchboard_tests.cpp PRE-CREATION 

Diff: https://reviews.apache.org/r/54147/diff/


Testing
---

GTEST_FILTER="" make -j check
sudo src/mesos-tests


Thanks,

Kevin Klues

Re: Review Request 54147: Added a server side component for the IOSwitchboard.

[Kevin Klues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54147/
---

(Updated Nov. 30, 2016, 6:53 p.m.)


Review request for mesos and Jie Yu.


Changes
---

Rebased on master.


Bugs: MESOS-6639
https://issues.apache.org/jira/browse/MESOS-6639


Repository: mesos


Description
---

The 'IOSwitchboardServer' component encapsulates the server side logic
for redirecting the 'stdin/stdout/stderr' of a container to/from
multiple sources/targets. For now, we only redirect IO from a
container to the FDs supplied to us by the logger.  We also send the
stdout/stderr data to a simple HTTP server that we launch on a unix
domain socket set up by the agent.  Right now this server is just a
stub and doesn't do anything useful.

In future commits, we will expand this HTTP server to handle
'ATTACH_CONTAINER_INPUT' and 'ATTACH_CONTAINER_OUTPUT' calls on behalf
of a container. It will use the stdout/stderr messages passed to it to
and send that data over the response stream to any clients connected
with an 'ATTACH_CONTAINER_OUTPUT' call. Likewise, it will take any
input streamed in over a 'ATTACH_CONTAINER_INPUT' request and write it
to a container's stdin.

In 'local' mode, it will be run inside the agent itself. In
'non-local' mode, it will be run as an external process to survive
agent restarts.


Diffs (updated)
-

  src/Makefile.am d1cc1016b1642c504f52c27623bc12c7ddf07599 
  src/slave/containerizer/mesos/io/switchboard.hpp 
aaa3a35245b291f6003f519dbf8c0e1b82bc15fd 
  src/slave/containerizer/mesos/io/switchboard.cpp 
25cbf2447d197134f0753b062b6f4130821005b2 
  src/tests/containerizer/io_switchboard_tests.cpp PRE-CREATION 

Diff: https://reviews.apache.org/r/54147/diff/


Testing
---

GTEST_FILTER="" make -j check
sudo src/mesos-tests


Thanks,

Kevin Klues

Re: Review Request 54147: Added a server side component for the IOSwitchboard.

[Kevin Klues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54147/
---

(Updated Nov. 30, 2016, 3:46 a.m.)


Review request for mesos and Jie Yu.


Changes
---

Renamed the test file, and moved the core logic of the server out of 
`initialize()` and into a `run()` function that must be explicitly called. The 
motivation for this was to ensure that the caller is able to *always* get a 
handle on the future used to mark termination of the server, even if it fails 
early on and calls terminate() on itself before the old `future()` call could 
be made.


Bugs: MESOS-6639
https://issues.apache.org/jira/browse/MESOS-6639


Repository: mesos


Description
---

The 'IOSwitchboardServer' component encapsulates the server side logic
for redirecting the 'stdin/stdout/stderr' of a container to/from
multiple sources/targets. For now, we only redirect IO from a
container to the FDs supplied to us by the logger.  We also send the
stdout/stderr data to a simple HTTP server that we launch on a unix
domain socket set up by the agent.  Right now this server is just a
stub and doesn't do anything useful.

In future commits, we will expand this HTTP server to handle
'ATTACH_CONTAINER_INPUT' and 'ATTACH_CONTAINER_OUTPUT' calls on behalf
of a container. It will use the stdout/stderr messages passed to it to
and send that data over the response stream to any clients connected
with an 'ATTACH_CONTAINER_OUTPUT' call. Likewise, it will take any
input streamed in over a 'ATTACH_CONTAINER_INPUT' request and write it
to a container's stdin.

In 'local' mode, it will be run inside the agent itself. In
'non-local' mode, it will be run as an external process to survive
agent restarts.


Diffs (updated)
-

  src/Makefile.am d1cc1016b1642c504f52c27623bc12c7ddf07599 
  src/slave/containerizer/mesos/io/switchboard.hpp 
aaa3a35245b291f6003f519dbf8c0e1b82bc15fd 
  src/slave/containerizer/mesos/io/switchboard.cpp 
25cbf2447d197134f0753b062b6f4130821005b2 
  src/tests/containerizer/io_switchboard_tests.cpp PRE-CREATION 

Diff: https://reviews.apache.org/r/54147/diff/


Testing
---

GTEST_FILTER="" make -j check
sudo src/mesos-tests


Thanks,

Kevin Klues

Re: Review Request 54147: Added a server side component for the IOSwitchboard.

[Kevin Klues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54147/
---

(Updated Nov. 29, 2016, 8:37 p.m.)


Review request for mesos and Jie Yu.


Changes
---

Updated to include a unit test. This changed the API slightly (for the better).


Bugs: MESOS-6639
https://issues.apache.org/jira/browse/MESOS-6639


Repository: mesos


Description
---

The 'IOSwitchboardServer' component encapsulates the server side logic
for redirecting the 'stdin/stdout/stderr' of a container to/from
multiple sources/targets. For now, we only redirect IO from a
container to the FDs supplied to us by the logger.  We also send the
stdout/stderr data to a simple HTTP server that we launch on a unix
domain socket set up by the agent.  Right now this server is just a
stub and doesn't do anything useful.

In future commits, we will expand this HTTP server to handle
'ATTACH_CONTAINER_INPUT' and 'ATTACH_CONTAINER_OUTPUT' calls on behalf
of a container. It will use the stdout/stderr messages passed to it to
and send that data over the response stream to any clients connected
with an 'ATTACH_CONTAINER_OUTPUT' call. Likewise, it will take any
input streamed in over a 'ATTACH_CONTAINER_INPUT' request and write it
to a container's stdin.

In 'local' mode, it will be run inside the agent itself. In
'non-local' mode, it will be run as an external process to survive
agent restarts.


Diffs (updated)
-

  src/Makefile.am 85eda538caf39f81f052896e744b7b0c724f81bb 
  src/slave/containerizer/mesos/io/switchboard.hpp 
aaa3a35245b291f6003f519dbf8c0e1b82bc15fd 
  src/slave/containerizer/mesos/io/switchboard.cpp 
25cbf2447d197134f0753b062b6f4130821005b2 
  src/tests/containerizer/io_switchboard.cpp PRE-CREATION 

Diff: https://reviews.apache.org/r/54147/diff/


Testing
---

GTEST_FILTER="" make -j check
sudo src/mesos-tests


Thanks,

Kevin Klues

Review Request 54147: Added a server side component for the IOSwitchboard.

[Kevin Klues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54147/
---

Review request for mesos and Jie Yu.


Bugs: MESOS-6639
https://issues.apache.org/jira/browse/MESOS-6639


Repository: mesos


Description
---

The 'IOSwitchboardServer' component encapsulates the server side logic
for redirecting the 'stdin/stdout/stderr' of a container to/from
multiple sources/targets. For now, we only redirect IO from a
container to the FDs supplied to us by the logger.  We also send the
stdout/stderr data to a simple HTTP server that we launch on a unix
domain socket set up by the agent.  Right now this server is just a
stub and doesn't do anything useful.

In future commits, we will expand this HTTP server to handle
'ATTACH_CONTAINER_INPUT' and 'ATTACH_CONTAINER_OUTPUT' calls on behalf
of a container. It will use the stdout/stderr messages passed to it to
and send that data over the response stream to any clients connected
with an 'ATTACH_CONTAINER_OUTPUT' call. Likewise, it will take any
input streamed in over a 'ATTACH_CONTAINER_INPUT' request and write it
to a container's stdin.

In 'local' mode, it will be run inside the agent itself. In
'non-local' mode, it will be run as an external process to survive
agent restarts.


Diffs
-

  src/slave/containerizer/mesos/io/switchboard.hpp 
aaa3a35245b291f6003f519dbf8c0e1b82bc15fd 
  src/slave/containerizer/mesos/io/switchboard.cpp 
25cbf2447d197134f0753b062b6f4130821005b2 

Diff: https://reviews.apache.org/r/54147/diff/


Testing
---

GTEST_FILTER="" make -j check
sudo src/mesos-tests


Thanks,

Kevin Klues