Xuanwo commented on code in PR #6832:
URL: https://github.com/apache/opendal/pull/6832#discussion_r2572810979
##########
core/src/services/huggingface/lister.rs:
##########
@@ -89,3 +102,105 @@ impl oio::PageList for HuggingfaceLister {
Ok(())
}
}
+
+/// Parse the Link header to extract the next page URL.
+/// HuggingFace API returns pagination info in the Link header with rel="next".
+/// Example: <https://huggingface.co/api/models/.../tree?cursor=xxx>;
rel="next"
+fn parse_link_header(headers: &http::HeaderMap) -> Option<String> {
+ let link_header = headers.get(http::header::LINK)?;
+ let link_str = link_header.to_str().ok()?;
+
+ // Parse Link header format: <url>; rel="next"
+ for link in link_str.split(',') {
+ let link = link.trim();
+ if link.contains("rel=\"next\"") || link.contains("rel='next'") {
+ // Extract URL from <url>
+ if let Some(start) = link.find('<') {
+ if let Some(end) = link.find('>') {
+ return Some(link[start + 1..end].to_string());
+ }
+ }
Review Comment:
How about using:
```
let (_, rest) = link.split_once('<')?;
let (inside, _) = rest.split_once('>')?;
return Some(inside.to_string());
```
##########
core/src/services/huggingface/lister.rs:
##########
@@ -89,3 +102,105 @@ impl oio::PageList for HuggingfaceLister {
Ok(())
}
}
+
+/// Parse the Link header to extract the next page URL.
+/// HuggingFace API returns pagination info in the Link header with rel="next".
+/// Example: <https://huggingface.co/api/models/.../tree?cursor=xxx>;
rel="next"
+fn parse_link_header(headers: &http::HeaderMap) -> Option<String> {
+ let link_header = headers.get(http::header::LINK)?;
+ let link_str = link_header.to_str().ok()?;
+
+ // Parse Link header format: <url>; rel="next"
+ for link in link_str.split(',') {
+ let link = link.trim();
Review Comment:
I think We don't need to trim here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
