cjlee38 opened a new issue, #7095:
URL: https://github.com/apache/opendal/issues/7095
### Describe the bug
When using a recursive lister, if a `PermissionDenied` error occurs while
iterating (e.g., accessing a protected directory), the lister immediately
returns `None` on the next `.next()` call, terminating the entire iteration.
This causes the listing to stop prematurely, even when there are many more
accessible files/directories remaining at the same level or in sibling
directories.
### Steps to Reproduce
```rust
use futures::StreamExt as _;
use opendal::Operator;
use opendal::ErrorKind;
#[tokio::main]
async fn main() {
opendal_reproduce().await;
}
async fn opendal_reproduce() -> Result<(), opendal::Error> {
let builder = opendal::services::Fs::default().root("/foo/bar/");
let op = Operator::new(builder)?.finish();
println!("Listing directory...");
let mut lister = op.lister_with("/").recursive(true).await.unwrap();
while let Some(entry_res) = lister.next().await {
match entry_res {
Ok(entry) => {
println!("Process entry: {}", entry.path());
}
Err(e) if e.kind() == ErrorKind::PermissionDenied => {
// I want to skip and continue
println!("Permission denied: {}", e);
continue;
}
Err(e) => {
println!("Error: {}", e);
return Err(e);
},
}
}
println!("Done listing directory.");
Ok(())
}
```
For example, if there are several files or folders(e.g. A, B, C) with
`MyForbiddenPath`, then the log output should be like below.
```
Listing directory...
Process entry: A
Process entry: B
Permission denied: PermissionDenied (permanent) at list, context: { service:
fs, path: MyForbiddenPath/ } => permission denied, source: Operation not
permitted (os error 1)
Done listing directory.
```
which skips the `C`
### Expected Behavior
After encountering a PermissionDenied error, the lister should skip the
inaccessible directory and continue returning the remaining entries from other
accessible directories.
The iteration should only return None when all accessible entries have been
exhausted.
### Additional Context
I encountered this issue with the `fs` service on macOS (hitting Photos
Library.photoslibrary), but I suspect this behavior applies to other services
as well since the recursive lister logic may be shared.
Also, I haven't verified this, but it's worth checking whether other error
types (not just PermissionDenied) also cause the lister to terminate
prematurely in the same way.
### Are you willing to submit a PR to fix this bug?
- [ ] Yes, I would like to submit a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
