mcvsubbu commented on a change in pull request #3849: Using PinotFS interface
instead of local file to download a segment f…
URL: https://github.com/apache/incubator-pinot/pull/3849#discussion_r271809995
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentUploadRestletResource.java
##########
@@ -182,19 +186,34 @@ public Response downloadSegment(
throw new ControllerApplicationException(LOGGER, e.getMessage(),
Response.Status.INTERNAL_SERVER_ERROR, e);
}
try {
- segmentName = URLDecoder.decode(segmentName, "UTF-8");
+ segmentName = URLDecoder.decode(segmentName, URL_ENCODING_SCHEME);
} catch (UnsupportedEncodingException e) {
String errStr = "Could not decode segment name '" + segmentName + "'";
throw new ControllerApplicationException(LOGGER, errStr,
Response.Status.BAD_REQUEST);
}
- final File dataFile = new File(provider.getBaseDataDir(),
StringUtil.join("/", tableName, segmentName));
- if (!dataFile.exists()) {
+
+ final java.net.URI segmentFileURI =
+ ControllerConf.getUriFromPath(StringUtil.join("/",
provider.getBaseDataDirURI().toString(),
+ tableName, URLEncoder.encode(segmentName, URL_ENCODING_SCHEME)));
+ PinotFS pinotFS =
PinotFSFactory.create(provider.getBaseDataDirURI().getScheme());
+
+ if (!pinotFS.exists(segmentFileURI)) {
throw new ControllerApplicationException(LOGGER,
- "Segment " + segmentName + " or table " + tableName + " not found",
Response.Status.NOT_FOUND);
+ "Segment " + segmentName + " or table " + tableName + " not found in
" + segmentFileURI.toString(), Response.Status.NOT_FOUND);
}
- Response.ResponseBuilder builder = Response.ok(dataFile);
- builder.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=" +
dataFile.getName());
- builder.header(HttpHeaders.CONTENT_LENGTH, dataFile.length());
+ File tmpSegmentFile = new File(StringUtil.join("/",
_controllerConf.getLocalTempDir(), tableName,
+ StringUtil.join("_", segmentName, String.valueOf(System.nanoTime()))));
+ pinotFS.copyToLocalFile(segmentFileURI, tmpSegmentFile);
Review comment:
This means an extra copy for local file system. Is there any way to avoid it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
