github-actions[bot] commented on issue #17688: URL: https://github.com/apache/dolphinscheduler/issues/17688#issuecomment-3539753983
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues. ### What happened When using Aliyun OSS as resource storage in DolphinScheduler 3.3.1, two related issues occur: **Issue 1: Resource validation error** - When selecting OSS resources in the task editing page, an error message appears - Error message: `project.node.useless_resources_tips: 未授权或者删除的资源` (Unauthorized or deleted resource) - This prevents users from selecting resource files for tasks **Issue 2: Duplicate folder display** - In the resource management page, the same folder appears multiple times in the list - When creating a new folder (e.g., "test") and opening it, the folder shows itself as a nested subfolder - This creates confusion as it appears there are multiple copies of the same folder Both issues occur consistently when using Aliyun OSS storage plugin. ### What you expected to happen 1. **Resource selection should work normally** - Users should be able to select OSS resource files in the task editing page without validation errors - Selected resources should be properly recognized and validated 2. **Each folder should appear only once** - In the resource list, each folder should display only one time - When opening a folder, it should show its actual contents, not the folder itself as a subfolder - No duplicate folder entries should appear in the resource tree ### How to reproduce **Prerequisites:** - DolphinScheduler 3.3.1 installed - Aliyun OSS configured as resource storage - Storage configuration in `common.properties`: ```properties resource.storage.type=OSS resource.alibaba.cloud.access.key.id=<your-access-key> resource.alibaba.cloud.access.key.secret=<your-secret-key> resource.alibaba.cloud.region=<your-region> resource.oss.bucket.name=<your-bucket> resource.oss.endpoint=<your-endpoint> ``` **Steps to reproduce Issue 1 (Resource validation error):** 1. Configure DolphinScheduler to use Aliyun OSS storage (as shown above) 2. Upload some resource files to OSS bucket (e.g., test.sh, data.txt) 3. Log in to DolphinScheduler web UI 4. Navigate to Project → Workflow → Create Workflow → Add Task 5. In the task configuration, try to select a resource file 6. Observe the error message: "未授权或者删除的资源" **Steps to reproduce Issue 2 (Duplicate folders):** 1. Configure DolphinScheduler to use Aliyun OSS storage 2. Log in to DolphinScheduler web UI 3. Navigate to Resource Center → File Management 4. Create a new folder named "test" 5. Observe that "test" folder appears multiple times in the folder list (duplicate icons) 6. Click to open the "test" folder 7. Observe that inside the "test" folder, there is another "test" subfolder (which should not exist) **Reproducibility:** - Issue 1: Every time when selecting resources - Issue 2: Every time when creating folders or viewing folder list --- ### Anything else **Root cause analysis:** After analyzing the source code, I found two root causes: **Root Cause 1 (Validation error):** - File: `dolphinscheduler-storage-plugin/dolphinscheduler-storage-oss/src/main/java/org/apache/dolphinscheduler/plugin/storage/oss/OssStorageOperator.java` - Method: `transformCommonPrefixToStorageEntity()` - Issue: Missing field assignments for `pfullName` and `relativePath` - The frontend validation logic depends on these fields to build the resource tree - When these fields are null, validation fails with "unauthorized or deleted resource" error **Root Cause 2 (Duplicate folders):** - File: Same as above - Method: `listStorageEntity()` - Issue: No deduplication logic for OSS directory markers - Explanation: - Aliyun OSS creates "directory marker objects" (empty objects with key ending with `/`) when folders are created - OSS API `listObjectsV2()` returns the same directory in both: - `CommonPrefixes`: list of subdirectories - `ObjectSummaries`: list of objects (including directory markers) - Without deduplication, folders appear twice in the resource tree - Additionally, the current directory itself appears in ObjectSummaries, causing self-reference **Comparison with other storage plugins:** The method `transformOSSObjectToStorageEntity()` in the same file correctly sets all fields including `pfullName` and `relativePath`, but `transformCommonPrefixToStorageEntity()` is missing these assignments, causing inconsistency. **Environment:** - DolphinScheduler Version: 3.3.1 - Storage Type: Aliyun OSS - JDK Version: 11 - Database: MySQL 8.0 / PostgreSQL - Browser: Chrome/Firefox (for UI issues) **Frequency:** - Both issues occur 100% of the time when using OSS storage **Logs:** No specific error logs in server logs, but the API response shows incomplete StorageEntity data: ```json { "code": 0, "msg": "success", "data": [ { "id": 1, "pid": -1, "name": "test", "fullName": "/tenant/resources/test/", "description": "", "type": "FILE", "directory": true, "pfullName": null, // ← Missing field "relativePath": null // ← Missing field } ] } ``` **Impact:** - Users cannot use OSS resources in workflow tasks (blocking issue) - Resource management UI is confusing with duplicate folders - Affects all users who configure Aliyun OSS as resource storage ### Version 3.3.1 ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
