[jira] [Updated] (MESOS-3785) Use URI content modification time to trigger fetcher cache updates.
[ https://issues.apache.org/jira/browse/MESOS-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-3785: --- Sprint: Mesosphere Sprint 21, Mesosphere Sprint 22, Mesosphere Sprint 23 (was: Mesosphere Sprint 21, Mesosphere Sprint 22) > Use URI content modification time to trigger fetcher cache updates. > --- > > Key: MESOS-3785 > URL: https://issues.apache.org/jira/browse/MESOS-3785 > Project: Mesos > Issue Type: Improvement > Components: fetcher >Reporter: Bernd Mathiske >Assignee: Benjamin Bannier > Labels: mesosphere > > Instead of using checksums to trigger fetcher cache updates, we can for > starters use the content modification time (mtime), which is available for a > number of download protocols, e.g. HTTP and HDFS. > Proposal: Instead of just fetching the content size, we fetch both size and > mtime together. As before, if there is no size, then caching fails and we > fall back on direct downloading to the sandbox. > Assuming a size is given, we compare the mtime from the fetch URI with the > mtime known to the cache. If it differs, we update the cache. (As a defensive > measure, a difference in size should also trigger an update.) > Not having an mtime available at the fetch URI is simply treated as a unique > valid mtime value that differs from all others. This means that when > initially there is no mtime, cache content remains valid until there is one. > Thereafter, anew lack of an mtime invalidates the cache once. In other > words: any change from no mtime to having one or back is the same as > encountering a new mtime. > Note that this scheme does not require any new protobuf fields. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3785) Use URI content modification time to trigger fetcher cache updates.
[ https://issues.apache.org/jira/browse/MESOS-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Till Toenshoff updated MESOS-3785: -- Target Version/s: 0.27.0 (was: 0.26.0) > Use URI content modification time to trigger fetcher cache updates. > --- > > Key: MESOS-3785 > URL: https://issues.apache.org/jira/browse/MESOS-3785 > Project: Mesos > Issue Type: Improvement > Components: fetcher >Reporter: Bernd Mathiske >Assignee: Benjamin Bannier > Labels: mesosphere > > Instead of using checksums to trigger fetcher cache updates, we can for > starters use the content modification time (mtime), which is available for a > number of download protocols, e.g. HTTP and HDFS. > Proposal: Instead of just fetching the content size, we fetch both size and > mtime together. As before, if there is no size, then caching fails and we > fall back on direct downloading to the sandbox. > Assuming a size is given, we compare the mtime from the fetch URI with the > mtime known to the cache. If it differs, we update the cache. (As a defensive > measure, a difference in size should also trigger an update.) > Not having an mtime available at the fetch URI is simply treated as a unique > valid mtime value that differs from all others. This means that when > initially there is no mtime, cache content remains valid until there is one. > Thereafter, anew lack of an mtime invalidates the cache once. In other > words: any change from no mtime to having one or back is the same as > encountering a new mtime. > Note that this scheme does not require any new protobuf fields. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3785) Use URI content modification time to trigger fetcher cache updates.
[ https://issues.apache.org/jira/browse/MESOS-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-3785: --- Sprint: Mesosphere Sprint 21, Mesosphere Sprint 22 (was: Mesosphere Sprint 21) > Use URI content modification time to trigger fetcher cache updates. > --- > > Key: MESOS-3785 > URL: https://issues.apache.org/jira/browse/MESOS-3785 > Project: Mesos > Issue Type: Improvement > Components: fetcher >Reporter: Bernd Mathiske >Assignee: Benjamin Bannier > Labels: mesosphere > > Instead of using checksums to trigger fetcher cache updates, we can for > starters use the content modification time (mtime), which is available for a > number of download protocols, e.g. HTTP and HDFS. > Proposal: Instead of just fetching the content size, we fetch both size and > mtime together. As before, if there is no size, then caching fails and we > fall back on direct downloading to the sandbox. > Assuming a size is given, we compare the mtime from the fetch URI with the > mtime known to the cache. If it differs, we update the cache. (As a defensive > measure, a difference in size should also trigger an update.) > Not having an mtime available at the fetch URI is simply treated as a unique > valid mtime value that differs from all others. This means that when > initially there is no mtime, cache content remains valid until there is one. > Thereafter, anew lack of an mtime invalidates the cache once. In other > words: any change from no mtime to having one or back is the same as > encountering a new mtime. > Note that this scheme does not require any new protobuf fields. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3785) Use URI content modification time to trigger fetcher cache updates.
[ https://issues.apache.org/jira/browse/MESOS-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gilbert Song updated MESOS-3785: Labels: mesosphere (was: ) > Use URI content modification time to trigger fetcher cache updates. > --- > > Key: MESOS-3785 > URL: https://issues.apache.org/jira/browse/MESOS-3785 > Project: Mesos > Issue Type: Improvement > Components: fetcher >Reporter: Bernd Mathiske >Assignee: Benjamin Bannier > Labels: mesosphere > > Instead of using checksums to trigger fetcher cache updates, we can for > starters use the content modification time (mtime), which is available for a > number of download protocols, e.g. HTTP and HDFS. > Proposal: Instead of just fetching the content size, we fetch both size and > mtime together. As before, if there is no size, then caching fails and we > fall back on direct downloading to the sandbox. > Assuming a size is given, we compare the mtime from the fetch URI with the > mtime known to the cache. If it differs, we update the cache. (As a defensive > measure, a difference in size should also trigger an update.) > Not having an mtime available at the fetch URI is simply treated as a unique > valid mtime value that differs from all others. This means that when > initially there is no mtime, cache content remains valid until there is one. > Thereafter, anew lack of an mtime invalidates the cache once. In other > words: any change from no mtime to having one or back is the same as > encountering a new mtime. > Note that this scheme does not require any new protobuf fields. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
