GitHub user onursatici opened a pull request:
https://github.com/apache/spark/pull/19708
[SPARK-22479][SQL] Exclude credentials from
SaveintoDataSourceCommand.simpleString
## What changes were proposed in this pull request?
Do not include jdbc properties which may contain credentials in logging a
logical plan with `SaveIntoDataSourceCommand` in it.
## How was this patch tested?
building locally and trying to reproduce (per the steps in
https://issues.apache.org/jira/browse/SPARK-22479):
```
== Parsed Logical Plan ==
SaveIntoDataSourceCommand
org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider@10ffe32f,
ErrorIfExists
+- Range (0, 100, step=1, splits=Some(8))
== Analyzed Logical Plan ==
SaveIntoDataSourceCommand
org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider@10ffe32f,
ErrorIfExists
+- Range (0, 100, step=1, splits=Some(8))
== Optimized Logical Plan ==
SaveIntoDataSourceCommand
org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider@10ffe32f,
ErrorIfExists
+- Range (0, 100, step=1, splits=Some(8))
== Physical Plan ==
Execute SaveIntoDataSourceCommand
+- SaveIntoDataSourceCommand
org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider@10ffe32f,
ErrorIfExists
+- Range (0, 100, step=1, splits=Some(8))
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/onursatici/spark os/redact-jdbc-creds
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19708.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19708
----
commit 04aa9f0363f6202a5358e41587415da4fa5f425e
Author: osatici <[email protected]>
Date: 2017-11-09T14:06:05Z
do not log properties on SaveintoDataSourceCommand.simpleString
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
