[I] [BUG] Errors When Configuring celeborn.storage.s3.dir with s3:// and s3a:// Schemes [celeborn]

Tue, 11 Mar 2025 00:24:30 -0700 


shouwangyw opened a new issue, #3145:
URL: https://github.com/apache/celeborn/issues/3145

   ### What is the bug(with logs or screenshots)?
   A clear and concise description of what the bug is.
   
   ### How to reproduce the bug?
   1. **Using `s3://` Scheme:**
      - **Configuration**：`celeborn.storage.s3.dir s3://${bucket_name}`
      - **Celeborn worker cannot start and an error 
occurred**：org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem 
for scheme "s3"
      
   
![Image](https://github.com/user-attachments/assets/d9609a59-1ead-4ef8-9478-b3527468d08b)
   
   2. **Using `s3a://` Scheme:**
      - **Configuration**：`celeborn.storage.s3.dir s3a://${bucket_name}`
      - The Celeborn worker starts without issues, but an error occurs during 
the shuffle phase, specifically in the `createDiskFile` method. 
   
   
   
![Image](https://github.com/user-attachments/assets/d66d4918-9c29-45a6-87e7-e702ba6a5547)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to