Re: Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.

[Jie Yu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/
---

(Updated Aug. 6, 2018, 11:42 p.m.)


Review request for mesos, Avinash sridharan, Chun-Hung Hsiao, and Greg Mann.


Changes
---

Addressed comments.


Bugs: MESOS-9127
https://issues.apache.org/jira/browse/MESOS-9127


Repository: mesos


Description (updated)
---

It is possible that the port mapping cleanup command will cause iptables
to deadlock if there are a lot of entires in the iptables, because the
`sed` won't process the next line while executing `iptables -w -t nat -D
...`. But the executing of `iptables -w -t nat -D ...` might get stuck
if the first command `iptables -w -t nat -S ` didn't finish
(because the xtables lock is not released). The first command might not
finish if it has a lot of output, filling the pipe that `sed` hasn't had
a chance to process yet. See more details in MESOS-9127.

This patch fixed the issue by writing the commands to a file and then
executing them.


Diffs (updated)
-

  
src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
 f1a3d263b7baa3ccbf270426745022d42fcc66ed 


Diff: https://reviews.apache.org/r/68158/diff/5/

Changes: https://reviews.apache.org/r/68158/diff/4-5/


Testing
---

sudo make check
```
[   OK ] CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper (8827 ms)
```


Thanks,

Jie Yu

Re: Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.

[Jie Yu]

> On Aug. 6, 2018, 8:40 p.m., Chun-Hung Hsiao wrote:
> > src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
> > Lines 382 (patched)
> > 
> >
> > If `iptables` prints something then exits abnormally,
> > do we want to exit this script immediately, or run `sh $FILE` to do 
> > partial cleanup?

I'd rather fail immediately. The output is not reliable if the iptables command 
exits abnormally.


- Jie


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/#review206904
---


On Aug. 6, 2018, 8:30 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68158/
> ---
> 
> (Updated Aug. 6, 2018, 8:30 p.m.)
> 
> 
> Review request for mesos, Avinash sridharan, Chun-Hung Hsiao, and Greg Mann.
> 
> 
> Bugs: MESOS-9127
> https://issues.apache.org/jira/browse/MESOS-9127
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> It is possible that the port mapping cleanup command will cause iptables
> to deadlock if there are a lot of entires in the iptables, because the
> `sed` won't process the next line while executing 'iptables -w -t nat -D
> ...'. But the executing of 'iptables -w -t nat -D ...' might get stuck
> if the first command 'iptables -w -t nat -S %s' didn't finish (because
> the xtables lock is not released). The first command might not finish if
> it has a lot of output, filling the pipe that `sed` hasn't had a chance
> to process yet. See more details in MESOS-9127.
> 
> This patch fixed the issue by writing the commands to a file and then
> executing them.
> 
> 
> Diffs
> -
> 
>   
> src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
>  f1a3d263b7baa3ccbf270426745022d42fcc66ed 
> 
> 
> Diff: https://reviews.apache.org/r/68158/diff/4/
> 
> 
> Testing
> ---
> 
> sudo make check
> ```
> [   OK ] CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper (8827 ms)
> ```
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.

[Chun-Hung Hsiao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/#review206904
---


Fix it, then Ship it!





src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
Lines 372 (patched)


Backticks instead of single-quotes? Ditto in the following two lines.



src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
Lines 374 (patched)


s/`%s`/``/



src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
Lines 382 (patched)


If `iptables` prints something then exits abnormally,
do we want to exit this script immediately, or run `sh $FILE` to do partial 
cleanup?


- Chun-Hung Hsiao


On Aug. 6, 2018, 8:30 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68158/
> ---
> 
> (Updated Aug. 6, 2018, 8:30 p.m.)
> 
> 
> Review request for mesos, Avinash sridharan, Chun-Hung Hsiao, and Greg Mann.
> 
> 
> Bugs: MESOS-9127
> https://issues.apache.org/jira/browse/MESOS-9127
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> It is possible that the port mapping cleanup command will cause iptables
> to deadlock if there are a lot of entires in the iptables, because the
> `sed` won't process the next line while executing 'iptables -w -t nat -D
> ...'. But the executing of 'iptables -w -t nat -D ...' might get stuck
> if the first command 'iptables -w -t nat -S %s' didn't finish (because
> the xtables lock is not released). The first command might not finish if
> it has a lot of output, filling the pipe that `sed` hasn't had a chance
> to process yet. See more details in MESOS-9127.
> 
> This patch fixed the issue by writing the commands to a file and then
> executing them.
> 
> 
> Diffs
> -
> 
>   
> src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
>  f1a3d263b7baa3ccbf270426745022d42fcc66ed 
> 
> 
> Diff: https://reviews.apache.org/r/68158/diff/4/
> 
> 
> Testing
> ---
> 
> sudo make check
> ```
> [   OK ] CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper (8827 ms)
> ```
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.

[Jie Yu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/
---

(Updated Aug. 6, 2018, 8:30 p.m.)


Review request for mesos, Avinash sridharan, Chun-Hung Hsiao, and Greg Mann.


Bugs: MESOS-9127
https://issues.apache.org/jira/browse/MESOS-9127


Repository: mesos


Description
---

It is possible that the port mapping cleanup command will cause iptables
to deadlock if there are a lot of entires in the iptables, because the
`sed` won't process the next line while executing 'iptables -w -t nat -D
...'. But the executing of 'iptables -w -t nat -D ...' might get stuck
if the first command 'iptables -w -t nat -S %s' didn't finish (because
the xtables lock is not released). The first command might not finish if
it has a lot of output, filling the pipe that `sed` hasn't had a chance
to process yet. See more details in MESOS-9127.

This patch fixed the issue by writing the commands to a file and then
executing them.


Diffs (updated)
-

  
src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
 f1a3d263b7baa3ccbf270426745022d42fcc66ed 


Diff: https://reviews.apache.org/r/68158/diff/4/

Changes: https://reviews.apache.org/r/68158/diff/3-4/


Testing
---

sudo make check
```
[   OK ] CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper (8827 ms)
```


Thanks,

Jie Yu

Re: Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.

[Chun-Hung Hsiao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/#review206903
---




src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
Lines 387 (patched)


You cannot use `%s` here, or you have to pass `getIptableRuleTag()` one 
more time (as the first argument) below.


- Chun-Hung Hsiao


On Aug. 6, 2018, 4:52 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68158/
> ---
> 
> (Updated Aug. 6, 2018, 4:52 p.m.)
> 
> 
> Review request for mesos, Avinash sridharan, Chun-Hung Hsiao, and Greg Mann.
> 
> 
> Bugs: MESOS-9127
> https://issues.apache.org/jira/browse/MESOS-9127
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> It is possible that the port mapping cleanup command will cause iptables
> to deadlock if there are a lot of entires in the iptables, because the
> `sed` won't process the next line while executing 'iptables -w -t nat -D
> ...'. But the executing of 'iptables -w -t nat -D ...' might get stuck
> if the first command 'iptables -w -t nat -S %s' didn't finish (because
> the xtables lock is not released). The first command might not finish if
> it has a lot of output, filling the pipe that `sed` hasn't had a chance
> to process yet. See more details in MESOS-9127.
> 
> This patch fixed the issue by writing the commands to a file and then
> executing them.
> 
> 
> Diffs
> -
> 
>   
> src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
>  f1a3d263b7baa3ccbf270426745022d42fcc66ed 
> 
> 
> Diff: https://reviews.apache.org/r/68158/diff/3/
> 
> 
> Testing
> ---
> 
> sudo make check
> ```
> [   OK ] CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper (8827 ms)
> ```
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.

[Chun-Hung Hsiao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/#review206892
---


Ship it!




Ship It!

- Chun-Hung Hsiao


On Aug. 6, 2018, 4:52 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68158/
> ---
> 
> (Updated Aug. 6, 2018, 4:52 p.m.)
> 
> 
> Review request for mesos, Avinash sridharan, Chun-Hung Hsiao, and Greg Mann.
> 
> 
> Bugs: MESOS-9127
> https://issues.apache.org/jira/browse/MESOS-9127
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> It is possible that the port mapping cleanup command will cause iptables
> to deadlock if there are a lot of entires in the iptables, because the
> `sed` won't process the next line while executing 'iptables -w -t nat -D
> ...'. But the executing of 'iptables -w -t nat -D ...' might get stuck
> if the first command 'iptables -w -t nat -S %s' didn't finish (because
> the xtables lock is not released). The first command might not finish if
> it has a lot of output, filling the pipe that `sed` hasn't had a chance
> to process yet. See more details in MESOS-9127.
> 
> This patch fixed the issue by writing the commands to a file and then
> executing them.
> 
> 
> Diffs
> -
> 
>   
> src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
>  f1a3d263b7baa3ccbf270426745022d42fcc66ed 
> 
> 
> Diff: https://reviews.apache.org/r/68158/diff/3/
> 
> 
> Testing
> ---
> 
> sudo make check
> ```
> [   OK ] CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper (8827 ms)
> ```
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.

[Deepak Goel]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/#review206887
---


Ship it!




Ship It!

- Deepak Goel


On Aug. 6, 2018, 4:52 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68158/
> ---
> 
> (Updated Aug. 6, 2018, 4:52 p.m.)
> 
> 
> Review request for mesos, Avinash sridharan, Chun-Hung Hsiao, and Greg Mann.
> 
> 
> Bugs: MESOS-9127
> https://issues.apache.org/jira/browse/MESOS-9127
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> It is possible that the port mapping cleanup command will cause iptables
> to deadlock if there are a lot of entires in the iptables, because the
> `sed` won't process the next line while executing 'iptables -w -t nat -D
> ...'. But the executing of 'iptables -w -t nat -D ...' might get stuck
> if the first command 'iptables -w -t nat -S %s' didn't finish (because
> the xtables lock is not released). The first command might not finish if
> it has a lot of output, filling the pipe that `sed` hasn't had a chance
> to process yet. See more details in MESOS-9127.
> 
> This patch fixed the issue by writing the commands to a file and then
> executing them.
> 
> 
> Diffs
> -
> 
>   
> src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
>  f1a3d263b7baa3ccbf270426745022d42fcc66ed 
> 
> 
> Diff: https://reviews.apache.org/r/68158/diff/3/
> 
> 
> Testing
> ---
> 
> sudo make check
> ```
> [   OK ] CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper (8827 ms)
> ```
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.

[Jie Yu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/
---

(Updated Aug. 6, 2018, 4:52 p.m.)


Review request for mesos, Avinash sridharan, Chun-Hung Hsiao, and Greg Mann.


Changes
---

Rebased.


Bugs: MESOS-9127
https://issues.apache.org/jira/browse/MESOS-9127


Repository: mesos


Description
---

It is possible that the port mapping cleanup command will cause iptables
to deadlock if there are a lot of entires in the iptables, because the
`sed` won't process the next line while executing 'iptables -w -t nat -D
...'. But the executing of 'iptables -w -t nat -D ...' might get stuck
if the first command 'iptables -w -t nat -S %s' didn't finish (because
the xtables lock is not released). The first command might not finish if
it has a lot of output, filling the pipe that `sed` hasn't had a chance
to process yet. See more details in MESOS-9127.

This patch fixed the issue by writing the commands to a file and then
executing them.


Diffs (updated)
-

  
src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
 f1a3d263b7baa3ccbf270426745022d42fcc66ed 


Diff: https://reviews.apache.org/r/68158/diff/3/

Changes: https://reviews.apache.org/r/68158/diff/2-3/


Testing
---

sudo make check
```
[   OK ] CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper (8827 ms)
```


Thanks,

Jie Yu

Re: Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.

[Jie Yu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/
---

(Updated Aug. 5, 2018, 5:30 a.m.)


Review request for mesos, Avinash sridharan, Chun-Hung Hsiao, and Greg Mann.


Bugs: MESOS-9127
https://issues.apache.org/jira/browse/MESOS-9127


Repository: mesos


Description
---

It is possible that the port mapping cleanup command will cause iptables
to deadlock if there are a lot of entires in the iptables, because the
`sed` won't process the next line while executing 'iptables -w -t nat -D
...'. But the executing of 'iptables -w -t nat -D ...' might get stuck
if the first command 'iptables -w -t nat -S %s' didn't finish (because
the xtables lock is not released). The first command might not finish if
it has a lot of output, filling the pipe that `sed` hasn't had a chance
to process yet. See more details in MESOS-9127.

This patch fixed the issue by writing the commands to a file and then
executing them.


Diffs
-

  
src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
 c40b57f78193520f9f0b901201b5c4c855cde8b3 


Diff: https://reviews.apache.org/r/68158/diff/2/


Testing
---

sudo make check
```
[   OK ] CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper (8827 ms)
```


Thanks,

Jie Yu

Re: Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.

[Jie Yu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/
---

(Updated Aug. 2, 2018, 7:17 p.m.)


Review request for mesos, Avinash sridharan, Chun-Hung Hsiao, and Greg Mann.


Changes
---

Fixed a stupid bug.


Bugs: MESOS-9127
https://issues.apache.org/jira/browse/MESOS-9127


Repository: mesos


Description (updated)
---

It is possible that the port mapping cleanup command will cause iptables
to deadlock if there are a lot of entires in the iptables, because the
`sed` won't process the next line while executing 'iptables -w -t nat -D
...'. But the executing of 'iptables -w -t nat -D ...' might get stuck
if the first command 'iptables -w -t nat -S %s' didn't finish (because
the xtables lock is not released). The first command might not finish if
it has a lot of output, filling the pipe that `sed` hasn't had a chance
to process yet. See more details in MESOS-9127.

This patch fixed the issue by writing the commands to a file and then
executing them.


Diffs (updated)
-

  
src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
 c40b57f78193520f9f0b901201b5c4c855cde8b3 


Diff: https://reviews.apache.org/r/68158/diff/2/

Changes: https://reviews.apache.org/r/68158/diff/1-2/


Testing
---

sudo make check
```
[   OK ] CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper (8827 ms)
```


Thanks,

Jie Yu

Re: Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/#review206785
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['68158']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2027/mesos-review-68158

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2027/mesos-review-68158/logs/mesos-tests-stdout.log):

```
[--] 9 tests from Endpoint/SlaveEndpointTest (982 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (31 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (35 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (73 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (1201 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (1222 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (1389 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (1411 ms total)

[--] Global test environment tear-down
[==] 1013 tests from 98 test cases ran. (700445 ms total)
[  PASSED  ] 1011 tests.
[  FAILED  ] 2 tests, listed below:
[  FAILED  ] DockerTest.ROOT_DOCKER_interface
[  FAILED  ] DockerTest.ROOT_DOCKER_kill

 2 FAILED TESTS
  YOU HAVE 222 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2027/mesos-review-68158/logs/mesos-tests-stderr.log):

```
I0802 07:06:49.881471 12444 slave.cpp:3939] Shutting down framework 
8297884e-f31b-4b26-9c8c-ccfb744f6787-
I0802 07:06:49.881471 11216 master.cpp:10963] Updating the state of task 
532adba1-32d4-44be-ad63-aba7a2b057a5 of framework 
8297884e-f31b-4b26-9c8c-ccfb744f6787- (latest state: TASK_KILLED, statI0802 
07:06:49.581436 12220 exec.cpp:162] Version: 1.7.0
I0802 07:06:49.606456  6056 exec.cpp:236] Executor registered on agent 
8297884e-f31b-4b26-9c8c-ccfb744f6787-S0
I0802 07:06:49.609436  7708 executor.cpp:182] Received SUBSCRIBED event
I0802 07:06:49.614432  7708 executor.cpp:186] Subscribed executor on 
windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0802 07:06:49.614432  7708 executor.cpp:182] Received LAUNCH event
I0802 07:06:49.619463  7708 executor.cpp:679] Starting task 
532adba1-32d4-44be-ad63-aba7a2b057a5
I0802 07:06:49.698460  7708 executor.cpp:499] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0802 07:06:49.853458  7708 executor.cpp:693] Forked command at 12168
I0802 07:06:49.883463  4152 exec.cpp:445] Executor asked to shutdown
I0802 07:06:49.884474 10908 executor.cpp:182] Received SHUTDOWN event
I0802 07:06:49.884474 10908 executor.cpp:796] Shutting down
I0802 07:06:49.884474 10908 executor.cpp:909] Sending SIGTERM to process tree 
at pid 12us update state: TASK_KILLED)
I0802 07:06:49.881471 12444 slave.cpp:6658] Shutting down executor 
'532adba1-32d4-44be-ad63-aba7a2b057a5' of framework 
8297884e-f31b-4b26-9c8c-ccfb744f6787- at executor(1)@192.10.1.6:60867
I0802 07:06:49.883463 12444 slave.cpp:931] Agent terminating
W0802 07:06:49.883463 12444 slave.cpp:3935] Ignoring shutdown framework 
8297884e-f31b-4b26-9c8c-ccfb744f6787- because it is terminating
I0802 07:06:49.883463 11216 master.cpp:11061] Removing task 
532adba1-32d4-44be-ad63-aba7a2b057a5 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 8297884e-f31b-4b26-9c8c-ccfb744f6787- on 
agent 8297884e-f31b-4b26-9c8c-ccfb744f6787-S0 at slave(462)@192.10.1.6:59164 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0802 07:06:49.887471 11216 master.cpp:1338] Agent 
8297884e-f31b-4b26-9c8c-ccfb744f6787-S0 at slave(462)@192.10.1.6:59164 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0802 07:06:49.887471 11216 master.cpp:3354] Disconnecting agent 
8297884e-f31b-4b26-9c8c-ccfb744f6787-S0 at slave(462)@192.10.1.6:59164 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0802 07:06:49.888473 11216 master.cpp:3373] Deactivating agent 
8297884e-f31b-4b26-9c8c-ccfb744f6787-S0 at slave(462)@192.10.1.6:59164 

Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.

[Jie Yu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/
---

Review request for mesos, Avinash sridharan, Chun-Hung Hsiao, and Greg Mann.


Bugs: MESOS-9127
https://issues.apache.org/jira/browse/MESOS-9127


Repository: mesos


Description
---

It is possible that the port mapping cleanup command will cause iptables
to deadlock if there are a lot of entires in the iptables, because the
`sed` won't process the next line while executing `iptables -w -t nat -D
...`. But the executing of `iptables -w -t nat -D ... ` might get stuck
if the first command `iptables -w -t nat -S %s` didn't finish (because
the xtables lock is not released). The first command might not finish if
it has a lot of output, filling the pipe that `sed` hasn't had a chance
to process yet. See more details in MESOS-9127.

This patch fixed the issue by writing the commands to a file and then
executing them.


Diffs
-

  
src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
 c40b57f78193520f9f0b901201b5c4c855cde8b3 


Diff: https://reviews.apache.org/r/68158/diff/1/


Testing
---

sudo make check
```
[   OK ] CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper (8827 ms)
```


Thanks,

Jie Yu