[
https://issues.apache.org/jira/browse/FLINK-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155574#comment-16155574
]
ASF GitHub Bot commented on FLINK-7068:
---------------------------------------
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/4358#discussion_r137259815
--- Diff:
flink-fs-tests/src/test/java/org/apache/flink/hdfstests/HDFSTest.java ---
@@ -257,7 +265,88 @@ public void testBlobServerRecovery() throws Exception {
try {
blobStoreService =
BlobUtils.createBlobStoreFromConfig(config);
- BlobRecoveryITCase.testBlobServerRecovery(config,
blobStoreService);
+ BlobServerRecoveryTest.testBlobServerRecovery(config,
blobStoreService);
+ } finally {
+ if (blobStoreService != null) {
+ blobStoreService.closeAndCleanupAllData();
+ }
+ }
+ }
+
+ /**
+ * Tests that with {@link HighAvailabilityMode#ZOOKEEPER} distributed
corrupted JARs are
+ * recognised during the download via a {@link
org.apache.flink.runtime.blob.BlobServer}.
+ */
+ @Test
+ public void testBlobServerCorruptedFile() throws Exception {
+ org.apache.flink.configuration.Configuration
+ config = new
org.apache.flink.configuration.Configuration();
+ config.setString(HighAvailabilityOptions.HA_MODE, "ZOOKEEPER");
+ config.setString(CoreOptions.STATE_BACKEND, "ZOOKEEPER");
+ config.setString(BlobServerOptions.STORAGE_DIRECTORY,
+ temporaryFolder.newFolder().getAbsolutePath());
+ config.setString(HighAvailabilityOptions.HA_STORAGE_PATH,
hdfsURI);
+
+ BlobStoreService blobStoreService = null;
+
+ try {
+ blobStoreService =
BlobUtils.createBlobStoreFromConfig(config);
+
+
BlobServerCorruptionTest.testGetFailsFromCorruptFile(config, blobStoreService,
exception);
+ } finally {
+ if (blobStoreService != null) {
+ blobStoreService.closeAndCleanupAllData();
+ }
+ }
+ }
+
+ /**
+ * Tests that with {@link HighAvailabilityMode#ZOOKEEPER} distributed
JARs are recoverable from any
+ * participating BlobServer when uploaded via a {@link
org.apache.flink.runtime.blob.BlobCache}.
+ */
+ @Test
+ public void testBlobCacheRecovery() throws Exception {
+ org.apache.flink.configuration.Configuration
+ config = new
org.apache.flink.configuration.Configuration();
+ config.setString(HighAvailabilityOptions.HA_MODE, "ZOOKEEPER");
+ config.setString(CoreOptions.STATE_BACKEND, "ZOOKEEPER");
--- End diff --
Statebackend not defined.
> change BlobService sub-classes for permanent and transient BLOBs
> ----------------------------------------------------------------
>
> Key: FLINK-7068
> URL: https://issues.apache.org/jira/browse/FLINK-7068
> Project: Flink
> Issue Type: Sub-task
> Components: Distributed Coordination, Network
> Affects Versions: 1.4.0
> Reporter: Nico Kruber
> Assignee: Nico Kruber
>
> A {{PermanentBlobStore}} should resemble use cases for BLOBs that are
> permanently stored for a job's life time (HA and non-HA).
> A {{TransientBlobStore}} should reflect BLOB offloading for logs, RPC, etc.
> which even does not have to be reflected by files.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
