GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/19985
[SPARK-22791] [SQL] [SS] Redact Output of Explain
## What changes were proposed in this pull request?
When calling explain on a query, the output can contain sensitive
information. We should provide an admin/user to redact such information.
```
== Physical Plan ==
*HashAggregate(keys=[value#6], functions=[count(1)], output=[value#6,
count(1)#12L])
+- StateStoreSave [value#6], state info [ checkpoint =
file:/private/var/folders/vx/j0ydl5rn0gd9mgrh1pljnw900000gn/T/temporary-91c6fac0-609f-4bc8-ad57-52c189f06797/state,
runId = 05a4b3af-f02c-40f8-9ff9-a3e18bae496f, opId = 0, ver = 0, numPartitions
= 5], Complete, 0
+- *HashAggregate(keys=[value#6], functions=[merge_count(1)],
output=[value#6, count#18L])
+- StateStoreRestore [value#6], state info [ checkpoint =
file:/private/var/folders/vx/j0ydl5rn0gd9mgrh1pljnw900000gn/T/temporary-91c6fac0-609f-4bc8-ad57-52c189f06797/state,
runId = 05a4b3af-f02c-40f8-9ff9-a3e18bae496f, opId = 0, ver = 0, numPartitions
= 5]
+- *HashAggregate(keys=[value#6], functions=[merge_count(1)],
output=[value#6, count#18L])
+- Exchange hashpartitioning(value#6, 5)
+- *HashAggregate(keys=[value#6],
functions=[partial_count(1)], output=[value#6, count#18L])
+- *SerializeFromObject [staticinvoke(class
org.apache.spark.unsafe.types.UTF8String, StringType, fromString, input[0,
java.lang.String, true], true, false) AS value#6]
+- *MapElements <function1>, obj#5: java.lang.String
+- *DeserializeToObject value#30.toString, obj#4:
java.lang.String
+- LocalTableScan [value#30]
```
## How was this patch tested?
Added a test case
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark redactPlan
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19985.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19985
----
commit 8f44edf5a4edcc8f1c42331cf3ab9b694fb01925
Author: gatorsmile <[email protected]>
Date: 2017-12-15T06:56:32Z
fix.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
