sketchmind commented on code in PR #10675:
URL: https://github.com/apache/dolphinscheduler/pull/10675#discussion_r915527462
##########
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/S3Utils.java:
##########
@@ -175,21 +176,33 @@ public String getFileName(ResourceType resourceType,
String tenantCode, String f
}
@Override
- public void download(String tenantCode, String srcFilePath, String
dstFile, boolean deleteSource, boolean overwrite) throws IOException {
+ public void download(String tenantCode, String srcFilePath, String
dstFilePath, boolean deleteSource, boolean overwrite) throws IOException {
+ File dstFile = new File(dstFilePath);
+ if (dstFile.exists()) {
+ if (overwrite) {
+ if (dstFile.isDirectory()) {
+ Files.delete(dstFile.toPath());
+ }
+ } else {
+ logger.info("the destination path {} already exists, download
operation will be ignored", dstFilePath);
+ return;
Review Comment:
> Currently `overwrite` is always true. I'm not sure why there is an
overwrite parameter at present. And i think we should do this like
`org.apache.dolphinscheduler.common.utils.HadoopUtils#copyHdfsToLocal` using
`if (!dstPath.getParentFile().exists() && !dstPath.getParentFile().mkdirs()) {
return false; }`
The download method of interface StorageOperate has no return value. The
return value of
org.apache.dolphinscheduler.common.utils.HadoopUtils#copyHdfsToLocal is not
processed in download. Are you sure you want to return a boolean value in S3
download?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
