Xuanwo commented on code in PR #3599:
URL:
https://github.com/apache/incubator-opendal/pull/3599#discussion_r1402865874
##########
core/src/types/list.rs:
##########
@@ -45,10 +44,20 @@ pub struct Lister {
/// required_metakey is the metakey required by users.
required_metakey: FlagSet<Metakey>,
- stating: Option<StatFuture>,
+ /// listing is used to indicate whether we are listing entries or stat
entries.
+ listing: bool,
+ /// task_queue is used to store tasks that are run in concurrent.
+ task_queue: VecDeque<StatTask>,
errored: bool,
}
+enum StatTask {
+ /// Handle is used to store the join handle of spawned task.
+ Handle(JoinHandle<(String, Result<RpStat>)>),
+ /// KnownEntry is used to store the entry that already contains the
required metakey.
+ KnownEntry(Box<Option<(String, Metadata)>>),
Review Comment:
Can we avoid this box?
##########
core/src/types/list.rs:
##########
@@ -80,44 +92,77 @@ impl Stream for Lister {
return Poll::Ready(None);
}
- if let Some(fut) = self.stating.as_mut() {
- let (path, rp) = ready!(fut.poll_unpin(cx));
-
- // Make sure we will not poll this future again.
- self.stating = None;
- let metadata = match rp {
- Ok(rp) => rp.into_metadata(),
- Err(err) => {
+ if self.listing && self.task_queue.len() < self.task_queue.capacity() {
Review Comment:
We can use `Option<Lister>` to make it more clear that the lister has been
consumed.
```rust
if let Some(lister) = self.lister.as_mut() {
if self.task_queue.len() < self.task_queue.capacity() {
match self.lister.poll_next(cx) {
Poll::Pending => {},
Poll::Ready(Ok(Some(oe))) => {
// build stat task
self.task_queue.push_back(task);
}
Poll::Ready(Ok(None)) => { self.lister = None; }
Poll::Ready(Ok(Err(err))) => { .. }
}
}
}
if let Some(task) = self.task_queue.front_mut() {
return match task {
StatTask::Handle(handle) => {
...
return Poll::Ready(Some(Ok(Entry::new(path, metadata))))
}
StatTask::KnownEntry(entry) => {
return Poll::Ready(Some(Ok(Entry::new(path, metadata))))
}
}
}
```
We don't keep look calling `poll_next`. I think this can make code more easy
to read.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
