Re: Review Request 65683: Updated discard handling in Docker library functions.

[Greg Mann]

> On Feb. 23, 2018, 2:18 p.m., Andrei Budnik wrote:
> > src/docker/docker.cpp
> > Lines 1292 (patched)
> > 
> >
> > I think we don't need to discard the output, because the process will 
> > be killed, hence `io::read()` will be terminated after receiving EOF. Also, 
> > as we check `promise->future().hasDiscard()` before checking 
> > `output.isReady()` in `___inspect()`, the promise won't be set to `FAILED` 
> > state.

Ah good point, we can probably rely on the subprocess termination causing the 
read to end.


- Greg


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65683/#review198179
---


On Feb. 23, 2018, 7:46 a.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65683/
> ---
> 
> (Updated Feb. 23, 2018, 7:46 a.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Gilbert Song, Michael Park, and 
> Vinod Kone.
> 
> 
> Bugs: MESOS-8575
> https://issues.apache.org/jira/browse/MESOS-8575
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated discard handling in Docker library functions.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp d9e71f8841a868082170d28fc4f3d495e2eb1e61 
>   src/docker/docker.cpp 3df370e988fce12d323ff6b441da15dab27bdd28 
> 
> 
> Diff: https://reviews.apache.org/r/65683/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 65683: Updated discard handling in Docker library functions.

[Greg Mann]

> On Feb. 23, 2018, 9:14 a.m., Greg Mann wrote:
> > src/docker/docker.cpp
> > Lines 1287-1293 (patched)
> > 
> >
> > Whoops, this is not threadsafe :facepalm:
> > 
> > For example, we could call `output->discard()` from two different 
> > threads simultaneously (see `__inspect`).
> > 
> > I need to figure out a better way to do this...
> 
> Andrei Budnik wrote:
> One possible solution might be using `defer(self(), ...)` for callbacks.
> 
> Gilbert Song wrote:
> yea, you may need `defer`.

Unfortunately, the docker library is not a libprocess process :'( So we have 
nothing to defer to.


- Greg


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65683/#review198170
---


On Feb. 23, 2018, 7:46 a.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65683/
> ---
> 
> (Updated Feb. 23, 2018, 7:46 a.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Gilbert Song, Michael Park, and 
> Vinod Kone.
> 
> 
> Bugs: MESOS-8575
> https://issues.apache.org/jira/browse/MESOS-8575
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated discard handling in Docker library functions.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp d9e71f8841a868082170d28fc4f3d495e2eb1e61 
>   src/docker/docker.cpp 3df370e988fce12d323ff6b441da15dab27bdd28 
> 
> 
> Diff: https://reviews.apache.org/r/65683/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 65683: Updated discard handling in Docker library functions.

[Gilbert Song]

> On Feb. 23, 2018, 1:14 a.m., Greg Mann wrote:
> > src/docker/docker.cpp
> > Lines 1287-1293 (patched)
> > 
> >
> > Whoops, this is not threadsafe :facepalm:
> > 
> > For example, we could call `output->discard()` from two different 
> > threads simultaneously (see `__inspect`).
> > 
> > I need to figure out a better way to do this...
> 
> Andrei Budnik wrote:
> One possible solution might be using `defer(self(), ...)` for callbacks.

yea, you may need `defer`.


- Gilbert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65683/#review198170
---


On Feb. 22, 2018, 11:46 p.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65683/
> ---
> 
> (Updated Feb. 22, 2018, 11:46 p.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Gilbert Song, Michael Park, and 
> Vinod Kone.
> 
> 
> Bugs: MESOS-8575
> https://issues.apache.org/jira/browse/MESOS-8575
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated discard handling in Docker library functions.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp d9e71f8841a868082170d28fc4f3d495e2eb1e61 
>   src/docker/docker.cpp 3df370e988fce12d323ff6b441da15dab27bdd28 
> 
> 
> Diff: https://reviews.apache.org/r/65683/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 65683: Updated discard handling in Docker library functions.

[Gilbert Song]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65683/#review198188
---




src/docker/docker.cpp
Lines 151 (patched)


Should we do `killtree()` even if the status is not pending (e.g., ready or 
failure)?

If the semantic (caller invokes `.discard()` -> no longer care about the 
subprocess) is something we want to define for the library, should we always 
killtree regardless whatever status?



src/docker/docker.cpp
Lines 1199 (patched)


Should we use a lambda instead of `lambda::bind`? (I know we want 
consistency in this file since we used `lambda::bind` in two other places)

e.g.,
```
.onDiscard([=]() {
  return commandDiscarded();
}
```

We used `lambda::bind` before we have the lambda support in libprocess.



src/docker/docker.cpp
Lines 1641 (patched)


ditto.


- Gilbert Song


On Feb. 22, 2018, 11:46 p.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65683/
> ---
> 
> (Updated Feb. 22, 2018, 11:46 p.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Gilbert Song, Michael Park, and 
> Vinod Kone.
> 
> 
> Bugs: MESOS-8575
> https://issues.apache.org/jira/browse/MESOS-8575
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated discard handling in Docker library functions.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp d9e71f8841a868082170d28fc4f3d495e2eb1e61 
>   src/docker/docker.cpp 3df370e988fce12d323ff6b441da15dab27bdd28 
> 
> 
> Diff: https://reviews.apache.org/r/65683/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 65683: Updated discard handling in Docker library functions.

[Andrei Budnik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65683/#review198179
---




src/docker/docker.cpp
Lines 1292 (patched)


I think we don't need to discard the output, because the process will be 
killed, hence `io::read()` will be terminated after receiving EOF. Also, as we 
check `promise->future().hasDiscard()` before checking `output.isReady()` in 
`___inspect()`, the promise won't be set to `FAILED` state.


- Andrei Budnik


On Feb. 23, 2018, 7:46 a.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65683/
> ---
> 
> (Updated Feb. 23, 2018, 7:46 a.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Gilbert Song, Michael Park, and 
> Vinod Kone.
> 
> 
> Bugs: MESOS-8575
> https://issues.apache.org/jira/browse/MESOS-8575
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated discard handling in Docker library functions.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp d9e71f8841a868082170d28fc4f3d495e2eb1e61 
>   src/docker/docker.cpp 3df370e988fce12d323ff6b441da15dab27bdd28 
> 
> 
> Diff: https://reviews.apache.org/r/65683/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 65683: Updated discard handling in Docker library functions.

[Andrei Budnik]

> On Feb. 23, 2018, 9:14 a.m., Greg Mann wrote:
> > src/docker/docker.cpp
> > Lines 1287-1293 (patched)
> > 
> >
> > Whoops, this is not threadsafe :facepalm:
> > 
> > For example, we could call `output->discard()` from two different 
> > threads simultaneously (see `__inspect`).
> > 
> > I need to figure out a better way to do this...

One possible solution might be using `defer(self(), ...)` for callbacks.


- Andrei


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65683/#review198170
---


On Feb. 23, 2018, 7:46 a.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65683/
> ---
> 
> (Updated Feb. 23, 2018, 7:46 a.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Gilbert Song, Michael Park, and 
> Vinod Kone.
> 
> 
> Bugs: MESOS-8575
> https://issues.apache.org/jira/browse/MESOS-8575
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated discard handling in Docker library functions.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp d9e71f8841a868082170d28fc4f3d495e2eb1e61 
>   src/docker/docker.cpp 3df370e988fce12d323ff6b441da15dab27bdd28 
> 
> 
> Diff: https://reviews.apache.org/r/65683/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 65683: Updated discard handling in Docker library functions.

[Greg Mann]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65683/#review198170
---




src/docker/docker.cpp
Lines 1287-1293 (patched)


Whoops, this is not threadsafe :facepalm:

For example, we could call `output->discard()` from two different threads 
simultaneously (see `__inspect`).

I need to figure out a better way to do this...


- Greg Mann


On Feb. 23, 2018, 7:46 a.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65683/
> ---
> 
> (Updated Feb. 23, 2018, 7:46 a.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Gilbert Song, Michael Park, and 
> Vinod Kone.
> 
> 
> Bugs: MESOS-8575
> https://issues.apache.org/jira/browse/MESOS-8575
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated discard handling in Docker library functions.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp d9e71f8841a868082170d28fc4f3d495e2eb1e61 
>   src/docker/docker.cpp 3df370e988fce12d323ff6b441da15dab27bdd28 
> 
> 
> Diff: https://reviews.apache.org/r/65683/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 65683: Updated discard handling in Docker library functions.

[Greg Mann]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65683/
---

(Updated Feb. 23, 2018, 7:46 a.m.)


Review request for mesos, Andrei Budnik, Gilbert Song, Michael Park, and Vinod 
Kone.


Changes
---

Avoided potential memory leak with repeated `.onDiscard()`


Bugs: MESOS-8575
https://issues.apache.org/jira/browse/MESOS-8575


Repository: mesos


Description
---

Updated discard handling in Docker library functions.


Diffs (updated)
-

  src/docker/docker.hpp d9e71f8841a868082170d28fc4f3d495e2eb1e61 
  src/docker/docker.cpp 3df370e988fce12d323ff6b441da15dab27bdd28 


Diff: https://reviews.apache.org/r/65683/diff/5/

Changes: https://reviews.apache.org/r/65683/diff/4-5/


Testing
---


Thanks,

Greg Mann

Re: Review Request 65683: Updated discard handling in Docker library functions.

[Greg Mann]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65683/#review198154
---




src/docker/docker.cpp
Lines 1314-1315 (patched)


I'm thinking I need to do something different to clean these things up, 
since `_inspect()` is invoked repeatedly by the retry logic.

Probably better to register a single `onDiscard` callback in `inspect()`, 
which will perform this cleanup on a couple of persistent containers which we 
reassign in the body of `_inspect()`.


- Greg Mann


On Feb. 22, 2018, 12:36 a.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65683/
> ---
> 
> (Updated Feb. 22, 2018, 12:36 a.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Gilbert Song, Michael Park, and 
> Vinod Kone.
> 
> 
> Bugs: MESOS-8575
> https://issues.apache.org/jira/browse/MESOS-8575
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated discard handling in Docker library functions.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.cpp 3df370e988fce12d323ff6b441da15dab27bdd28 
> 
> 
> Diff: https://reviews.apache.org/r/65683/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 65683: Updated discard handling in Docker library functions.

[Greg Mann]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65683/
---

(Updated Feb. 22, 2018, 12:36 a.m.)


Review request for mesos, Andrei Budnik, Gilbert Song, Michael Park, and Vinod 
Kone.


Bugs: MESOS-8575
https://issues.apache.org/jira/browse/MESOS-8575


Repository: mesos


Description
---

Updated discard handling in Docker library functions.


Diffs (updated)
-

  src/docker/docker.cpp 3df370e988fce12d323ff6b441da15dab27bdd28 


Diff: https://reviews.apache.org/r/65683/diff/4/

Changes: https://reviews.apache.org/r/65683/diff/3-4/


Testing
---


Thanks,

Greg Mann

Re: Review Request 65683: Updated discard handling in Docker library functions.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65683/#review198083
---



FAIL: Failed to apply the current review.

Failed command: `python.exe .\support\apply-reviews.py -n -r 65683`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/65683

Relevant logs:

- 
[apply-review-65683-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/65683/logs/apply-review-65683-stdout.log):

```
error: patch failed: src/docker/docker.cpp:1308
error: src/docker/docker.cpp: patch does not apply
```

- Mesos Reviewbot Windows


On Feb. 21, 2018, 4:30 p.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65683/
> ---
> 
> (Updated Feb. 21, 2018, 4:30 p.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Gilbert Song, Michael Park, and 
> Vinod Kone.
> 
> 
> Bugs: MESOS-8575
> https://issues.apache.org/jira/browse/MESOS-8575
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated discard handling in Docker library functions.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.cpp 3df370e988fce12d323ff6b441da15dab27bdd28 
> 
> 
> Diff: https://reviews.apache.org/r/65683/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 65683: Updated discard handling in Docker library functions.

[Greg Mann]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65683/
---

(Updated Feb. 22, 2018, 12:30 a.m.)


Review request for mesos, Andrei Budnik and Gilbert Song.


Summary (updated)
-

Updated discard handling in Docker library functions.


Bugs: MESOS-8575
https://issues.apache.org/jira/browse/MESOS-8575


Repository: mesos


Description (updated)
---

Updated discard handling in Docker library functions.


Diffs (updated)
-

  src/docker/docker.cpp 3df370e988fce12d323ff6b441da15dab27bdd28 


Diff: https://reviews.apache.org/r/65683/diff/3/

Changes: https://reviews.apache.org/r/65683/diff/2-3/


Testing
---


Thanks,

Greg Mann