[
https://issues.apache.org/jira/browse/MESOS-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17190365#comment-17190365
]
Andrei Sekretenko commented on MESOS-10169:
-------------------------------------------
1.10.x
{noformat}
commit 51cf5a37570caa0405a1edd5b461f484384cb526
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 17:02:51 2020 +0200
Deduplicated concurrent image pulls by docker store.
This patch makes the docker store reuse a pending pull if asked for
an image that is already being pulled.
The pull caching follows the same approach as the earlier attempt in
https://reviews.apache.org/r/39331 . However, handing out futures to
the store callers and handling their discards are performed differently,
using a form of reference counting, so that the pull itself is discarded
only if all the futures returned by `Store::get()` have been discarded.
Review: https://reviews.apache.org/r/72790
commit d4e4654f82cd5dd99fc9a8318df02674c0ed6320
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 17:49:13 2020 +0200
Reverted removal of the PullingSameImageSimutanuously test.
Now that the docker store triggers pull at most once per multiple
simultaneous requests to the store, removal of the
`PullingSameImageSimutanuously` test in
33c61a1907129126f3b2e37b1f53827a04e89a34 can be reverted.
Review: https://reviews.apache.org/r/72791
commit 1fbbbea73b5c65ef916d4c16e87bde2f75951345
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 18:45:44 2020 +0200
Fixed typos in the name of `PullingSameImageSimutanuously` test.
Review: https://reviews.apache.org/r/72792
commit 0f8a37a3a27e5458083866c716dd44c9bc11b5c0
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 18:16:39 2020 +0200
Added discarding a future returned by get() into the docker store test.
This patch extends the `PullingSameImageSimultaneously` test
with discarding one of the futures returned by `store->get()`
and making sure that the pull still completes.
Review: https://reviews.apache.org/r/72793
commit 3ca3879d52ea0f9bff05443d331d63105b2cc4db
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 18:36:40 2020 +0200
Added a test for discarding image pull on discard of getting image.
This test checks that when all futures returned by docker store's
`get()` that are pending image pull are discarded by the callers,
the pull future (returned by the puller to the store) is discarded
by the store as well.
Review: https://reviews.apache.org/r/72794
{noformat}
1.9.x:
{noformat}
commit 30d63f89e5fbac10192d7daa33fcea4e38701241
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 17:02:51 2020 +0200
Deduplicated concurrent image pulls by docker store.
This patch makes the docker store reuse a pending pull if asked for
an image that is already being pulled.
The pull caching follows the same approach as the earlier attempt in
https://reviews.apache.org/r/39331 . However, handing out futures to
the store callers and handling their discards are performed differently,
using a form of reference counting, so that the pull itself is discarded
only if all the futures returned by `Store::get()` have been discarded.
Review: https://reviews.apache.org/r/72790
commit 0e54e8f51afc5d139496a0d0b5378a765f2cf9dd
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 17:49:13 2020 +0200
Reverted removal of the PullingSameImageSimutanuously test.
Now that the docker store triggers pull at most once per multiple
simultaneous requests to the store, removal of the
`PullingSameImageSimutanuously` test in
33c61a1907129126f3b2e37b1f53827a04e89a34 can be reverted.
Review: https://reviews.apache.org/r/72791
commit edd7930356250fb8cd42f57ac79d1749a815fb8a
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 18:45:44 2020 +0200
Fixed typos in the name of `PullingSameImageSimutanuously` test.
Review: https://reviews.apache.org/r/72792
commit 404f4650a6c0f68c535b4e4832103ecd2d288fec
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 18:16:39 2020 +0200
Added discarding a future returned by get() into the docker store test.
This patch extends the `PullingSameImageSimultaneously` test
with discarding one of the futures returned by `store->get()`
and making sure that the pull still completes.
Review: https://reviews.apache.org/r/72793
commit e98ced1e93751b78c3248221115fcf4db8a39b29
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 18:36:40 2020 +0200
Added a test for discarding image pull on discard of getting image.
This test checks that when all futures returned by docker store's
`get()` that are pending image pull are discarded by the callers,
the pull future (returned by the puller to the store) is discarded
by the store as well.
Review: https://reviews.apache.org/r/72794
{noformat}
1.8.x
{noformat}
commit cd49d821de12a3d2a237ae57844878b649b50295
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 17:02:51 2020 +0200
Deduplicated concurrent image pulls by docker store.
This patch makes the docker store reuse a pending pull if asked for
an image that is already being pulled.
The pull caching follows the same approach as the earlier attempt in
https://reviews.apache.org/r/39331 . However, handing out futures to
the store callers and handling their discards are performed differently,
using a form of reference counting, so that the pull itself is discarded
only if all the futures returned by `Store::get()` have been discarded.
Review: https://reviews.apache.org/r/72790
commit c42364a833df202752a15f27542e6e5ab6a78179
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 17:49:13 2020 +0200
Reverted removal of the PullingSameImageSimutanuously test.
Now that the docker store triggers pull at most once per multiple
simultaneous requests to the store, removal of the
`PullingSameImageSimutanuously` test in
33c61a1907129126f3b2e37b1f53827a04e89a34 can be reverted.
Review: https://reviews.apache.org/r/72791
commit 9aed797be27d6b831a7adf4b5e373c1c9e6640b6
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 18:45:44 2020 +0200
Fixed typos in the name of `PullingSameImageSimutanuously` test.
Review: https://reviews.apache.org/r/72792
commit 6afaf0957048aeec30ff2383213cca7dc0893d32
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 18:16:39 2020 +0200
Added discarding a future returned by get() into the docker store test.
This patch extends the `PullingSameImageSimultaneously` test
with discarding one of the futures returned by `store->get()`
and making sure that the pull still completes.
Review: https://reviews.apache.org/r/72793
commit d160015e544082ff310a7e4d2e94e3c4eae9499b
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 18:36:40 2020 +0200
Added a test for discarding image pull on discard of getting image.
This test checks that when all futures returned by docker store's
`get()` that are pending image pull are discarded by the callers,
the pull future (returned by the puller to the store) is discarded
by the store as well.
Review: https://reviews.apache.org/r/72794
{noformat}
1.7.x:
{noformat}
commit b4a49fd915cacd111cedc3c583d69bc2b9d6468e
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 17:02:51 2020 +0200
Deduplicated concurrent image pulls by docker store.
This patch makes the docker store reuse a pending pull if asked for
an image that is already being pulled.
The pull caching follows the same approach as the earlier attempt in
https://reviews.apache.org/r/39331 . However, handing out futures to
the store callers and handling their discards are performed differently,
using a form of reference counting, so that the pull itself is discarded
only if all the futures returned by `Store::get()` have been discarded.
Review: https://reviews.apache.org/r/72790
commit df5fcf75b1dd04daa44631b43441d1bd85dd0241
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 17:49:13 2020 +0200
Reverted removal of the PullingSameImageSimutanuously test.
Now that the docker store triggers pull at most once per multiple
simultaneous requests to the store, removal of the
`PullingSameImageSimutanuously` test in
33c61a1907129126f3b2e37b1f53827a04e89a34 can be reverted.
Review: https://reviews.apache.org/r/72791
commit 66f8ed162a2d6022b49af426144413830be6501c
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 18:45:44 2020 +0200
Fixed typos in the name of `PullingSameImageSimutanuously` test.
Review: https://reviews.apache.org/r/72792
commit f1008ce5bc0105d00e9061efc61a9fc5f61a289d
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 18:16:39 2020 +0200
Added discarding a future returned by get() into the docker store test.
This patch extends the `PullingSameImageSimultaneously` test
with discarding one of the futures returned by `store->get()`
and making sure that the pull still completes.
Review: https://reviews.apache.org/r/72793
commit 8fc28789a7441a24ce0165281f9cadedf5cce417
Author: Andrei Sekretenko <asekrete...@apache.org>
Date: Thu Aug 20 18:36:40 2020 +0200
Added a test for discarding image pull on discard of getting image.
This test checks that when all futures returned by docker store's
`get()` that are pending image pull are discarded by the callers,
the pull future (returned by the puller to the store) is discarded
by the store as well.
Review: https://reviews.apache.org/r/72794
{noformat}
> Reintroduce image fetch deduplication while keeping it possible to destroy
> UCR containers in PROVISIONING state
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MESOS-10169
> URL: https://issues.apache.org/jira/browse/MESOS-10169
> Project: Mesos
> Issue Type: Bug
> Affects Versions: 1.7.3, 1.8.2, 1.9.1, 1.10.0, 1.7.4, 1.11.0, 1.10.1
> Reporter: Andrei Sekretenko
> Assignee: Andrei Sekretenko
> Priority: Major
>
> The fix for https://issues.apache.org/jira/browse/MESOS-9964 had to drop the
> image fetch deduplication that was introduced long ago in
> https://issues.apache.org/jira/browse/MESOS-3736. We should reintroduce the
> deduplication back.
> UCR containers pulling the same image at the same time should pull the image
> only once.
> *This deduplication should not prevent killing some of the PROVISIONING
> containers, and such killing should not interfere with provisioning of other
> containers.*
> I would argue that it is OK for all the containers depending on a single
> image pull to hang if this single pull hangs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
