morristai commented on issue #5090: URL: https://github.com/apache/opendal/issues/5090#issuecomment-3050998325
Followed up on this issue — after some testing, I found that GDS over TCP/IP isn’t supported yet. I tested with both a remote S3 bucket and a local MinIO cluster. Using the latest `cudf-cu12 25.6.0` to read a remote S3 Parquet file (~1 GiB), it first reads the data into host RAM, and only after streaming the entire Parquet file does it copy the data into GPU VRAM. This means that even if we create a Kvikio Rust FFI binding, GDS for S3 services won’t work if the underlying library doesn’t support GDS over TCP/IP. However, `fs` should work though. ```python file_url = "s3://morris-testing-s3/00000-0-c5b97e11-c15c-4dcc-bab8-4f227a34a669-0-00001.parquet" df = cudf.read_parquet(file_url) ``` https://github.com/user-attachments/assets/4d910eaf-0da7-4c95-9d79-ec38c73c7b60 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@opendal.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
