jihoonson commented on a change in pull request #8823: Add InputSource and
InputFormat interfaces
URL: https://github.com/apache/incubator-druid/pull/8823#discussion_r344951548
##########
File path: core/src/main/java/org/apache/druid/data/input/ObjectSource.java
##########
@@ -50,41 +50,46 @@
File file();
}
- InputSplit<T> getSplit();
+ T getObject();
/**
- * Opens an {@link InputStream} on the split directly.
- * This is the basic way to read the given split.
+ * Opens an {@link InputStream} on the object directly.
+ * This is the basic way to read the given object.
*
* @see #fetch as an alternative way to read data.
*/
InputStream open() throws IOException;
/**
- * Fetches the split into the local storage.
+ * Fetches the object into the local storage.
* This method might be preferred instead of {@link #open()}, for example
*
* - {@link org.apache.druid.data.input.impl.InputFormat} requires expensive
random access on remote storage.
* - Holding a connection until you consume the entire InputStream is
expensive.
*
* @param temporaryDirectory to store temp data. This directory will be
removed automatically once
* the task finishes.
- * @param fetchBuffer is used to fetch remote split into local
storage.
+ * @param fetchBuffer is used to fetch remote object into local
storage.
*
* @see FileUtils#copyLarge
*/
default CleanableFile fetch(File temporaryDirectory, byte[] fetchBuffer)
throws IOException
{
- final File tempFile = File.createTempFile("druid-split", ".tmp",
temporaryDirectory);
- LOG.debug("Fetching split into file[%s]", tempFile.getAbsolutePath());
- FileUtils.copyLarge(
- open(),
- tempFile,
- fetchBuffer,
- getRetryCondition(),
- DEFAULT_MAX_FETCH_RETRY,
- StringUtils.format("Failed to fetch into [%s]",
tempFile.getAbsolutePath())
- );
+ final File tempFile = File.createTempFile("druid-object-source", ".tmp",
temporaryDirectory);
+ LOG.debug("Fetching object into file[%s]", tempFile.getAbsolutePath());
+ try (InputStream is = open()) {
+ FileUtils.copyLarge(
+ is,
+ tempFile,
+ fetchBuffer,
+ getRetryCondition(),
+ DEFAULT_MAX_FETCH_RETRY,
+ StringUtils.format("Failed to fetch into [%s]",
tempFile.getAbsolutePath())
+ );
+ }
+ catch (IOException e) {
+ throw new RuntimeException(e);
Review comment:
Removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
