dwsmith1983 opened a new issue, #7128:
URL: https://github.com/apache/datafusion-comet/issues/7128
### Describe the bug
Found in #7127, which keeps URL passwords and SAS queries out of the Azure
store's errors and logs. Outside `azure.rs`, several native object-store
messages still print a whole URL, so a password in the userinfo or a token in
the query would reach task errors and logs. Line numbers are as of 39380cca8.
- `native/core/src/parquet/parquet_support.rs:913`: `object_store_backend`
runs `ObjectStoreScheme::parse(url)` on the raw URL before any store is built.
object_store 0.13.2 quotes the whole URL in `Unable to recognise URL "{url}"`.
- `native/core/src/parquet/objectstore/s3.rs:72` and `:76`: the same
`ObjectStoreScheme::parse(url)` echo, and `Scheme of URL is not S3: {url}`.
- `native/core/src/parquet/objectstore/s3_blob_fs_support.rs:81` and `:171`:
`Error parsing URL {url_str}` and `Error parsing URL {path}` print the raw
string when it fails to parse. `:138` prints the whole URL when the bucket name
is missing.
- `native/core/src/parquet/parquet_support.rs:736`: `No suitable object
store found for {url}` in `deregister_store`.
- `native/core/src/parquet/parquet_support.rs:1003` and `:1008`: the debug
lines print `url_key`, which `object_store_authority` (`:690`) builds from
`BeforeUsername` for ABFS URLs, so a password in the userinfo would appear. The
same string is the store cache key and its registration URL.
None of these is easy to reach from a normal Spark file listing, which
doesn't put credentials or a query in the paths it hands the native scan. So
this is defense in depth, like #7127.
### Steps to reproduce
Unit-level: call the functions above with a URL like
`s3://KEY:SECRET@bucket/path?sig=TOKEN`, or with an unrecognised scheme
carrying the same userinfo and query, and look at the error text.
### Expected behavior
Messages name a URL by its scheme and host, plus the ABFS container where
that helps. They never include the password or the query. The redaction helpers
added to `azure.rs` in #7127 show one way to do it.
### Additional context
The store cache key at `parquet_support.rs:987` and the registration URL
built by `object_store_registration_url` (`:1043`) would also carry an ABFS
password. That isn't printed outside the debug lines above, but dropping the
password from `object_store_authority` would cover all three.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
