>From Ian Maxon <[email protected]>: Ian Maxon has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19869 )
Change subject: [ASTERIXDB-3618][STO] Close Library Downloader ...................................................................... [ASTERIXDB-3618][STO] Close Library Downloader - user model changes: no - storage format changes: no - interface changes: no Details: The IParallelDownloader instances used to page in libraries from S3 aren't being closed, which is not good. Fix this, and make IParallelDownloader an AutoCloseable. Ext-ref: MB-67071 Change-Id: I524cbfb6685bd3c123a6ad2db48482e5a2a1ce61 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19869 Reviewed-by: Ali Alsuliman <[email protected]> Reviewed-by: Ian Maxon <[email protected]> Tested-by: Jenkins <[email protected]> Integration-Tests: Jenkins <[email protected]> --- M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/IParallelDownloader.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/AbstractCloudIOManager.java 2 files changed, 38 insertions(+), 11 deletions(-) Approvals: Ian Maxon: Looks good to me, but someone else must approve Ali Alsuliman: Looks good to me, approved Jenkins: Verified; Verified diff --git a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/AbstractCloudIOManager.java b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/AbstractCloudIOManager.java index 0ef2011..ab93f39 100644 --- a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/AbstractCloudIOManager.java +++ b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/AbstractCloudIOManager.java @@ -196,19 +196,21 @@ @Override public void downloadLibrary(Collection<FileReference> libPath) throws HyracksDataException { - IParallelDownloader downloader = cloudClient.createParallelDownloader(bucket, localIoManager); - LOGGER.info("Downloading all files located in {}", libPath); - downloader.downloadDirectories(libPath); - LOGGER.info("Finished downloading {}", libPath); + try (IParallelDownloader downloader = cloudClient.createParallelDownloader(bucket, localIoManager)) { + LOGGER.info("Downloading all files located in {}", libPath); + downloader.downloadDirectories(libPath); + LOGGER.info("Finished downloading {}", libPath); + } } public void downloadAllLibraries() throws HyracksDataException { - IParallelDownloader downloader = cloudClient.createParallelDownloader(bucket, localIoManager); - FileReference appDir = resolveAbsolutePath( - localIoManager.getWorkspacePath(0).getPath() + File.separator + APPLICATION_ROOT_DIR_NAME); - LOGGER.info("Downloading all libraries in + {}", appDir); - downloader.downloadDirectories(Collections.singletonList(appDir)); - LOGGER.info("Finished downloading all libraries"); + try (IParallelDownloader downloader = cloudClient.createParallelDownloader(bucket, localIoManager)) { + FileReference appDir = resolveAbsolutePath( + localIoManager.getWorkspacePath(0).getPath() + File.separator + APPLICATION_ROOT_DIR_NAME); + LOGGER.info("Downloading all libraries in + {}", appDir); + downloader.downloadDirectories(Collections.singletonList(appDir)); + LOGGER.info("Finished downloading all libraries"); + } } /* diff --git a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/IParallelDownloader.java b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/IParallelDownloader.java index 184e015..22de1e9 100644 --- a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/IParallelDownloader.java +++ b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/IParallelDownloader.java @@ -23,7 +23,7 @@ import org.apache.hyracks.api.exceptions.HyracksDataException; import org.apache.hyracks.api.io.FileReference; -public interface IParallelDownloader { +public interface IParallelDownloader extends AutoCloseable { /** * Downloads files in all partitions -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19869 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I524cbfb6685bd3c123a6ad2db48482e5a2a1ce61 Gerrit-Change-Number: 19869 Gerrit-PatchSet: 3 Gerrit-Owner: Ian Maxon <[email protected]> Gerrit-Reviewer: Ali Alsuliman <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Peeyush Gupta <[email protected]> Gerrit-MessageType: merged
