andygrove opened a new issue, #2535: URL: https://github.com/apache/datafusion-ballista/issues/2535
**Describe the bug** S3 credentials set through `SET s3.secret_access_key` and `SET s3.session_token` are written to the logs in plain text when `debug` or `trace` logging is on. This happens on the client, the scheduler and the executors. | Level | Where | What is logged | | --- | --- | --- | | `debug` | `S3Options::set` ([object_store.rs#L255](https://github.com/apache/datafusion-ballista/blob/e25c2a55d70d9692c2982c1d07db90f5b9f61897/ballista/core/src/object_store.rs#L255)) | every `s3.*` key and value as it is set | | `debug` | `update_from_key_value_pair_mut` ([extension.rs#L685-L688](https://github.com/apache/datafusion-ballista/blob/e25c2a55d70d9692c2982c1d07db90f5b9f61897/ballista/core/src/extension.rs#L685-L688)) | key and value of any setting that fails to apply, which includes the `s3.*` keys on a process built without `S3Options` | | `trace` | `CustomObjectStoreRegistry::get_store` ([object_store.rs#L148](https://github.com/apache/datafusion-ballista/blob/e25c2a55d70d9692c2982c1d07db90f5b9f61897/ballista/core/src/object_store.rs#L148)) | the whole `S3RegistryConfiguration` via derived `Debug`, on every store lookup | | `trace` | `to_key_value_pairs` ([extension.rs#L660](https://github.com/apache/datafusion-ballista/blob/e25c2a55d70d9692c2982c1d07db90f5b9f61897/ballista/core/src/extension.rs#L660)) | every session config key and value being sent | | `trace` | `update_from_key_value_pair_mut` ([extension.rs#L678-L680](https://github.com/apache/datafusion-ballista/blob/e25c2a55d70d9692c2982c1d07db90f5b9f61897/ballista/core/src/extension.rs#L678-L680)) | every session config key and value being received | | `trace` | executor task launch ([execution_loop.rs#L333-L336](https://github.com/apache/datafusion-ballista/blob/e25c2a55d70d9692c2982c1d07db90f5b9f61897/ballista/executor/src/execution_loop.rs#L333-L336)) | all task props | The `debug` one matters most, since `debug` is a level people do turn on in production when chasing a problem. **To Reproduce** Set the options with a logger capturing `trace`, using a made-up secret: ``` TRACE ballista_core::extension setting up configuration key: `s3.secret_access_key`, value: `"SECRET_ONE"` DEBUG ballista_core::object_store set config, key:secret_access_key, value:SECRET_ONE TRACE ballista_core::object_store get_store: S3RegistryConfiguration { access_key_id: Some("KEY_ONE"), secret_access_key: Some("SECRET_ONE"), session_token: None, region: Some("us-east-1"), endpoint: None, allow_http: None } TRACE ballista_core::extension sending configuration key: `s3.secret_access_key`, value`Some("SECRET_ONE")` ``` **Expected behavior** Secret values are redacted in log output at every level. **Additional context** Some possible directions: - Give `S3RegistryConfiguration` a manual `Debug` that redacts `secret_access_key` and `session_token`, and drop the value from the `set` log line. - The generic config logging in `extension.rs` and the task props log don't know which keys are sensitive. They could log keys only, or redact by a key pattern (`secret`, `token`, `password`). Found while working on #2532. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
