nanxiuzi opened a new issue, #18632: URL: https://github.com/apache/dolphinscheduler/issues/18632
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues. ### What happened When the resource center uses the Aliyun OSS storage backend (`resource.storage.type=OSS`), the resource manager page only shows the first batch of entries under a directory. Files and sub-directories that exist in the OSS console (and are visible there) are missing from the UI. `OssStorageOperator#listStorageEntity` issues exactly one `listObjectsV2` call and never handles pagination: ```java ListObjectsV2Request listObjectsV2Request = new ListObjectsV2Request() .withBucketName(bucketName) .withDelimiter("/") .withPrefix(ossResourceAbsolutePath); ListObjectsV2Result listObjectsV2Result = ossClient.listObjectsV2(listObjectsV2Request); ``` No `max-keys` is set, so the OSS server default of **100** applies. It is also not possible to widen the window enough by hand: `max-keys` is capped at 1000 by the service. Because `Contents` (objects) and `CommonPrefixes` (directories) share that single quota, any directory with more than 100 combined entries loses everything past the first 100 (lexicographic order). The same code path is used by `listFileStorageEntityRecursively`, so nested listing is truncated as well. ### What you expected to happen The resource manager should list **all** entries under a directory, regardless of how many objects/directories exist, by following `IsTruncated` / `NextContinuationToken` until the result set is exhausted. ### How to reproduce 1. Configure DolphinScheduler with the OSS storage backend. 2. Upload more than 100 files/objects (or mix enough files and sub-directories to exceed 100 combined) under a single resource directory, e.g. via `ossutil` or the OSS console. 3. Open the resource center in the UI and browse that directory. 4. Only the first 100 entries are shown; the rest are invisible even though they exist in OSS. ### Anything else - Verified by decompiling `aliyun-sdk-oss` 3.15.1: `ListObjectsV2Request` does not inject a default `maxKeys` (it is `null`), and `populateListObjectsV2RequestParameters` skips sending `max-keys` when null, so the server default (100) is used. This means even without pagination the client cannot opt into a larger page. - `dolphinscheduler-storage-s3`'s `S3StorageOperator` has the same single-call structure; the symptom is less visible there only because the S3 default page size is 1000. - Callers of `listStorageEntity` are `ResourcesServiceImpl#pagingResourceItem` and `ResourcesServiceImpl#queryResourceFiles`. ### Version 3.4.2 ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
