GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/6187
FLINK-9623][runtime] Move zipping logic out of blobservice
## What is the purpose of the change
This PR moves the zipping logic for local user directory artifacts out of
the blobservice.
Instead, directories are explicitly zipped by the `JobGraphGenerators` when
compiling the graph.
The zipping code was generalized and moved from `FileCache` and
`BlobClient` to `FileUtils`.
Additionally, `JobGraphTest` now extends `TestLogger`.
## Brief change log
* move zip compression/expansion as general utility methods to `FileUtils`
* modify `[Streaming]JobGraphGenerator` to zip local user directory
artifacts before registering them with the graph
* adjust `FileCacheDirectoryTest` to not manually create a mock zip file
* extend documentation of `FileCache`
* extend documentation of `DistributedCache`
* remove setting of user-artifacts in `TestStreamEnvironment` since it is
redundant
* remove odd verification/normalization code in `Plan#registerCachedFile`,
changes are covered by JobGraphGenerators
## Verifying this change
* zip compression/expansion is covered by new tests in `FileUtils`
* handling of local user directory artifacts by JobGraphGenerators covered
by new tests in `JobGraphGeneratorTest`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 9280_alpha
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6187.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6187
----
commit a8552a5fc85c4a3122eecd7739f5a02065dc93cd
Author: zentol <chesnay@...>
Date: 2018-06-13T14:10:16Z
[hotfix][tests] JobGraphTest extends TestLogger
commit 483ff525a7c047360a5c044fbb3c6bffb08a10c5
Author: zentol <chesnay@...>
Date: 2018-06-04T11:50:44Z
FLINK-9623][runtime] Move zipping logic out of blobservice
----
---
