Chesnay Schepler created FLINK-11464:
----------------------------------------
Summary: Implement DownloadCache
Key: FLINK-11464
URL: https://issues.apache.org/jira/browse/FLINK-11464
Project: Flink
Issue Type: Sub-task
Components: E2E Tests
Reporter: Chesnay Schepler
Assignee: Chesnay Schepler
Fix For: 1.8.0
h4. Problem
Several tests download archives for setting up systems, like Kafka of
Elasticsearch. We currently do not cache downloads in any way, resulting in
less stable tests (as mirrors aren't always available) and overall increased
test duration (since the downloads at times are quite slow). The duration issue
becomes especially apparent when running tests in a loop for debugging or
release-testing purposes.
Finally, it also puts unnecessary strain on the download mirrors.
h4. Proposal
Add a {{DownloadCache}} interface with a single {{Path getOrDownload(String
url, Path targetDir)}} method.
Access to and loading of implementations are handled like resources (see above).
The caching behavior is implementation-dependent.
A reasonable implementation should allow files may be cached in a user-provided
directory, with an optional time-to-live for long-term setups.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
