GitHub user cbisht31 edited a comment on the discussion: Native Pagination
Support for File Listing in OpenDAL
Thank you for the clarification regarding OpenDAL's internal pagination.
I'm using S3 and wanted to know how can I add offset and limit functionality to
this code. The API should support pagination for the frontend. For example, I
want to skip the first offset files and return only limit files in the
response. Does OpenDAL provide a way to achieve this directly?
Also, is there a way to get the total count of files in a directory without
listing all of them? For example, does OpenDAL provide a count function or a
similar method to retrieve the total number of files in a directory? Here is
the code snippet where I wish to add:
`async def list_files_with_metainfo(operator: opendal.Operator, subpath: str)
-> List[dict]:
"""
List files at the given subpath and retrieve full metainfo for each file.
"""
metainfo_list = []
try:
entries = await operator.list(subpath)
async for entry in entries:
name = entry.name if hasattr(entry, 'name') else str(entry)
full_path = entry.path
is_dir = False
error = None
try:
stat_info = await operator.stat(full_path)
if hasattr(stat_info, 'mode'):
is_dir = stat_info.mode.is_dir()
except Exception as e:
error = str(e)
file_info = {
"name": name,
"path": full_path,
"is_dir": is_dir,
"error": error
}
metainfo_list.append(file_info)
except Exception as e:
print(f"Error: {e}")
return metainfo_list`
GitHub link:
https://github.com/apache/opendal/discussions/6040#discussioncomment-12862971
----
This is an automatically sent email for dev@opendal.apache.org.
To unsubscribe, please send an email to: dev-unsubscr...@opendal.apache.org
