This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new 505248df16b [SPARK-39258][TESTS] Fix `Hide credentials in show create
table`
505248df16b is described below
commit 505248df16b9785e56a206db62129f6eff945483
Author: yangjie01 <[email protected]>
AuthorDate: Mon May 23 14:28:06 2022 -0700
[SPARK-39258][TESTS] Fix `Hide credentials in show create table`
### What changes were proposed in this pull request?
[SPARK-35378-FOLLOWUP](https://github.com/apache/spark/pull/36632) changes
the return value of `CommandResultExec.executeCollect()` from `InternalRow` to
`UnsafeRow`, this change causes the result of `r.tostring` in the following
code:
https://github.com/apache/spark/blob/de73753bb2e5fd947f237e731ff05aa9f2711677/sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala#L1143-L1148
change from
```
[CREATE TABLE tab1 (
NAME STRING,
THEID INT)
USING org.apache.spark.sql.jdbc
OPTIONS (
'dbtable' = 'TEST.PEOPLE',
'password' = '*********(redacted)',
'url' = '*********(redacted)',
'user' = 'testUser')
]
```
to
```
[0,10000000d5,5420455441455243,62617420454c4241,414e20200a282031,4e4952545320454d,45485420200a2c47,a29544e49204449,726f20474e495355,6568636170612e67,732e6b726170732e,a6362646a2e6c71,20534e4f4954504f,7462642720200a28,203d2027656c6261,45502e5453455427,200a2c27454c504f,6f77737361702720,2a27203d20276472,2a2a2a2a2a2a2a2a,6574636164657228,2720200a2c272964,27203d20276c7275,2a2a2a2a2a2a2a2a,746361646572282a,20200a2c27296465,3d20277265737527,7355747365742720,a29277265]
```
and the UT `JDBCSuite$Hide credentials in show create table` failed in
master branch.
This pr is change to use `executeCollectPublic()` instead of
`executeCollect()` to fix this UT.
### Why are the changes needed?
Fix UT failed in mater branch after
[SPARK-35378-FOLLOWUP](https://github.com/apache/spark/pull/36632)
### Does this PR introduce _any_ user-facing change?
NO.
### How was this patch tested?
- GitHub Action pass
- Manual test
Run `mvn clean install -DskipTests -pl sql/core -am -Dtest=none
-DwildcardSuites=org.apache.spark.sql.jdbc.JDBCSuite`
**Before**
```
- Hide credentials in show create table *** FAILED ***
"[0,10000000d5,5420455441455243,62617420454c4241,414e20200a282031,4e4952545320454d,45485420200a2c47,a29544e49204449,726f20474e495355,6568636170612e67,732e6b726170732e,a6362646a2e6c71,20534e4f4954504f,7462642720200a28,203d2027656c6261,45502e5453455427,200a2c27454c504f,6f77737361702720,2a27203d20276472,2a2a2a2a2a2a2a2a,6574636164657228,2720200a2c272964,27203d20276c7275,2a2a2a2a2a2a2a2a,746361646572282a,20200a2c27296465,3d20277265737527,7355747365742720,a29277265]"
did not contain "TES [...]
```
**After**
```
Run completed in 24 seconds, 868 milliseconds.
Total number of tests run: 93
Suites: completed 2, aborted 0
Tests: succeeded 93, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
```
Closes #36637 from LuciferYang/SPARK-39258.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 6eb15d12ae6bd77412dbfbf46eb8dbeec1eab466)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
index b1f5fd00868..a222391c06f 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
@@ -1141,7 +1141,7 @@ class JDBCSuite extends QueryTest
""".stripMargin)
val show = ShowCreateTableCommand(TableIdentifier(tableName),
ShowCreateTable.getoutputAttrs)
-
spark.sessionState.executePlan(show).executedPlan.executeCollect().foreach { r
=>
+
spark.sessionState.executePlan(show).executedPlan.executeCollectPublic().foreach
{ r =>
assert(!r.toString.contains(password))
assert(r.toString.contains(dbTable))
assert(r.toString.contains(userName))
@@ -1154,7 +1154,7 @@ class JDBCSuite extends QueryTest
}
withSQLConf(SQLConf.SQL_OPTIONS_REDACTION_PATTERN.key ->
"(?i)dbtable|user") {
-
spark.sessionState.executePlan(show).executedPlan.executeCollect().foreach { r
=>
+
spark.sessionState.executePlan(show).executedPlan.executeCollectPublic().foreach
{ r =>
assert(!r.toString.contains(password))
assert(!r.toString.contains(dbTable))
assert(!r.toString.contains(userName))
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
