ruanwenjun commented on code in PR #10675:
URL: https://github.com/apache/dolphinscheduler/pull/10675#discussion_r910744602
##########
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/S3Utils.java:
##########
@@ -175,21 +175,36 @@ public String getFileName(ResourceType resourceType,
String tenantCode, String f
}
@Override
- public void download(String tenantCode, String srcFilePath, String
dstFile, boolean deleteSource, boolean overwrite) throws IOException {
+ public void download(String tenantCode, String srcFilePath, String
dstFilePath, boolean deleteSource, boolean overwrite) throws IOException {
+ File dstFile = new File(dstFilePath);
+ File dstFileParentDirectory = dstFile.getParentFile();
+ if (dstFile.exists()) {
+ if (dstFile.isDirectory()) {
+ logger.error("destination file must be a file, but {} is a
folder", dstFilePath);
+ throw new IOException("destination file must be a file, but "
+ dstFilePath +" is a folder");
+ } else if (!overwrite) {
+ logger.info("the destination file {} already exists, download
operation will be ignored", dstFilePath);
+ return;
+ }
+ } else {
+ if (!dstFileParentDirectory.mkdirs() &&
!dstFileParentDirectory.exists()) {
+ throw new IOException("failed to create parent directory of
destination file, directory is " + dstFileParentDirectory.getAbsolutePath());
+ }
+ }
S3Object o = s3Client.getObject(BUCKET_NAME, srcFilePath);
try (S3ObjectInputStream s3is = o.getObjectContent();
- FileOutputStream fos = new FileOutputStream(dstFile)) {
+ FileOutputStream fos = new FileOutputStream(dstFilePath)) {
byte[] readBuf = new byte[1024];
- int readLen = 0;
+ int readLen;
while ((readLen = s3is.read(readBuf)) > 0) {
fos.write(readBuf, 0, readLen);
}
} catch (AmazonServiceException e) {
- logger.error("the resource can`t be downloaded,the bucket is
{},and the src is {}", tenantCode, srcFilePath);
+ logger.error("the resource can`t be downloaded, the bucket is {},
and the src is {}", BUCKET_NAME, srcFilePath);
throw new IOException(e.getMessage());
} catch (FileNotFoundException e) {
- logger.error("the file isn`t exists");
- throw new IOException("the file isn`t exists");
+ logger.error("the destination file {} isn`t exists", dstFilePath);
+ throw new IOException("the destination file: " + dstFilePath + "
isn`t exists", e);
Review Comment:
Please log the stack of e in line 203 or directly throw it. And in fact, I
think directly throw FileNotFoundException in 206 is better than IOException.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
