caishunfeng commented on code in PR #12076:
URL:
https://github.com/apache/dolphinscheduler/pull/12076#discussion_r1004148315
##########
dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/storage/impl/S3Utils.java:
##########
@@ -171,6 +183,25 @@ public String getResourceFileName(String tenantCode,
String fileName) {
return String.format(FORMAT_S_S, getS3ResDir(tenantCode), fileName);
}
+ @Override
+ public String getResourceFileName(String fullName) {
+ // here is a quick fix here to get fileName. We get the resource
upload path and
+ // get the index of the first appearance of resource upload path. The
index is put
+ // in the start index of the substring function and get the result
substring containing
+ // tenantcode and "resource" directory and the fileName.
+ // Then we split the result substring
+ // with "/" and join all elements except the first two elements
because they are
+ // tenantCode and "resource" directory.
+ String resourceUploadPath =
+ RESOURCE_UPLOAD_PATH.endsWith("/") ?
StringUtils.chop(RESOURCE_UPLOAD_PATH) : RESOURCE_UPLOAD_PATH;
+ // +1 because we want to skip the "/" after resource upload path as
well.
+ String pathContainingTenantNResource = fullName.substring(
+ fullName.indexOf(resourceUploadPath)
+ + resourceUploadPath.length() + 1);
+ String[] fileNameArr = pathContainingTenantNResource.split("/");
+ return
Joiner.on("/").join(Arrays.stream(fileNameArr).skip(2).collect(Collectors.toList()));
+ }
Review Comment:
```suggestion
public String getResourceFileName(String fullName) {
// here is a quick fix here to get fileName. We get the resource
upload path and
// get the index of the first appearance of resource upload path.
The index is put
// in the start index of the substring function and get the result
substring containing
// tenantcode and "resource" directory and the fileName.
// Then we split the result substring
// with "/" and join all elements except the first two elements
because they are
// tenantCode and "resource" directory.
String resourceUploadPath =
RESOURCE_UPLOAD_PATH.endsWith(Constnats.FOLDER_SEPARATOR) ?
StringUtils.chop(RESOURCE_UPLOAD_PATH) : RESOURCE_UPLOAD_PATH;
// +1 because we want to skip the "/" after resource upload path as
well.
String pathContainingTenantNResource = fullName.substring(
fullName.indexOf(resourceUploadPath)
+ resourceUploadPath.length() + 1);
String[] fileNameArr =
pathContainingTenantNResource.split(Constnats.FOLDER_SEPARATOR);
return
Joiner.on(Constnats.FOLDER_SEPARATOR).join(Arrays.stream(fileNameArr).skip(2).collect(Collectors.toList()));
}
```
##########
dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/storage/impl/HadoopUtils.java:
##########
@@ -290,14 +302,33 @@ public String getUdfDir(String tenantCode) {
*/
@Override
public boolean mkdir(String tenantCode, String hdfsPath) throws
IOException {
- return fs.mkdirs(new Path(hdfsPath));
+ return fs.mkdirs(new Path(addFolderSeparatorIfNotExisted(hdfsPath)));
}
@Override
public String getResourceFileName(String tenantCode, String fullName) {
return getHdfsResourceFileName(tenantCode, fullName);
}
+ @Override
+ public String getResourceFileName(String fullName) {
+ // here is a quick fix here to get fileName. We get the resource
upload path and
+ // get the index of the first appearance of resource upload path. The
index is put
+ // in the start index of the substring function and get the result
substring containing
+ // tenantcode and "resource" directory and the fileName.
+ // Then we split the result substring
+ // with "/" and join all elements except the first two elements
because they are
+ // tenantCode and "resource" directory.
+ String resourceUploadPath =
+ RESOURCE_UPLOAD_PATH.endsWith("/") ?
StringUtils.chop(RESOURCE_UPLOAD_PATH) : RESOURCE_UPLOAD_PATH;
+ // +1 because we want to skip the "/" after resource upload path as
well.
+ String pathContainingTenantNResource = fullName.substring(
+ fullName.indexOf(resourceUploadPath)
+ + resourceUploadPath.length() + 1);
+ String[] fileNameArr = pathContainingTenantNResource.split("/");
+ return
Joiner.on("/").join(Arrays.stream(fileNameArr).skip(2).collect(Collectors.toList()));
+ }
Review Comment:
replace "/" with `Constnats.FOLDER_SEPARATOR`
##########
dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/storage/impl/HadoopUtils.java:
##########
@@ -321,6 +352,11 @@ public void download(String bucketName, String
srcHdfsFilePath, String dstFile,
*/
@Override
public boolean copy(String srcPath, String dstPath, boolean deleteSource,
boolean overwrite) throws IOException {
+ // if(!exists(dstPath))
+ // {
+ // fs.mkdirs(new Path(dstPath));
+ // logger.info("created the destination path: {}", dstPath);
+ // }
Review Comment:
remove if not use.
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
