pratham76 opened a new pull request, #13646:
URL: https://github.com/apache/gluten/pull/13646

   ## What changes are proposed in this pull request?
   
   Fixes #13645.
   
   `getS3HiveConfig` translates Spark per-bucket S3A options 
(`spark.hadoop.fs.s3a.bucket.<bucket>.<option>`) into Velox per-bucket keys 
(`s3.bucket.<bucket>.<option>`). It split the key at the first `.` after the 
prefix. For a bucket such as `my.data.bucket`, the bucket was parsed as `my` 
and the option as `data.bucket.endpoint`, which is not recognized, so the 
setting was silently dropped. The native scan then used the global `fs.s3a.*` 
values instead.
   
   This PR parses the key by matching a known option suffix (from the existing 
`sparkSuffixes` table) at the **end** of the key, as Hadoop S3A does with the 
real bucket name. The longest matching suffix wins (e.g. `endpoint.region` over 
`endpoint`), and everything before it is the bucket name. It also drops the 
`int` that stored the `size_t` result of `find()`.
   
   Bucket names without dots behave exactly as before, and unknown options are 
still ignored.
   
   ## How was this patch tested?
   
   Added `GlutenS3FileSystemTest.extractsPerBucketConfigs` (built when 
`ENABLE_S3=ON`). It covers:
   - an undotted bucket (`plain-bucket.endpoint`)
   - a dotted bucket with `endpoint`, `access.key`, `endpoint.region` and 
`iam.role.session.name`
   - an unknown option on a dotted bucket being ignored
   - no config being registered under the bucket name cut off at the first dot 
(`my`)
   
   ## Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: Claude claude-opus-5-5
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to