dwsmith1983 opened a new issue, #7558:
URL: https://github.com/apache/datafusion-comet/issues/7558
### Describe the bug
Found while working on #7128. `ParquetWriterExec` in
`native/core/src/execution/operators/parquet_writer.rs` derives `Debug`, so
`{:?}` of the operator prints `output_path` and `object_store_options` as they
are. The options come from `NativeConfig.extractObjectStoreOptions` and can
hold credentials such as `fs.s3a.secret.key` or an Azure SAS token, and the
path can carry a password or a SAS query.
Comet does not format the plan with `{:?}` today, so nothing leaks yet. A
debug log, a `dbg!`, a test assertion or a DataFusion error that formats the
plan with `Debug` would print the secrets.
### Steps to reproduce
Format a `ParquetWriterExec` built with `fs.s3a.secret.key` in its options
with `{:?}`. The key's value appears in the output.
### Expected behavior
`Debug` for `ParquetWriterExec` prints the path without credentials (the
helpers #7127 adds) and only the keys of `object_store_options`, not their
values.
### Additional context
The plan display (`DisplayAs`) and the writer's messages are handled in the
#7128 pull requests; this is the remaining way the operator can print its
options.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
