wangyum opened a new pull request #25395: [SPARK-28675][SQL] Replace CatalogUtils.maskCredentials with SQLConf.get.redactOptions URL: https://github.com/apache/spark/pull/25395 ## What changes were proposed in this pull request? This PR replaces `CatalogUtils.maskCredentials` with `SQLConf.get.redactOptions` to consistent with redact logic. ## How was this patch tested? manual tests: Before this PR: ```sql spark-sql> desc extended mysql_federated_sample; TBL_ID bigint NULL CREATE_TIME int NULL DB_ID bigint NULL LAST_ACCESS_TIME int NULL OWNER string NULL RETENTION int NULL SD_ID bigint NULL TBL_NAME string NULL TBL_TYPE string NULL VIEW_EXPANDED_TEXT string NULL VIEW_ORIGINAL_TEXT string NULL IS_REWRITE_ENABLED boolean NULL # Detailed Table Information Database default Table mysql_federated_sample Owner root Created Time Thu Aug 08 16:53:43 GMT-07:00 2019 Last Access Wed Dec 31 17:00:00 GMT-07:00 1969 Created By Spark 3.0.0-SNAPSHOT Type MANAGED Provider org.apache.spark.sql.jdbc Location file:/user/hive/warehouse/mysql_federated_sample Serde Library org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe InputFormat org.apache.hadoop.mapred.SequenceFileInputFormat OutputFormat org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat Storage Properties [url=###, driver=com.mysql.jdbc.Driver, dbtable=TBLS] spark-sql> SHOW TABLE EXTENDED LIKE 'mysql_federated_sample'; 19/08/08 17:04:16 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException default mysql_federated_sample false Database: default Table: mysql_federated_sample Owner: root Created Time: Thu Aug 08 16:53:43 GMT-07:00 2019 Last Access: Wed Dec 31 17:00:00 GMT-07:00 1969 Created By: Spark 3.0.0-SNAPSHOT Type: MANAGED Provider: org.apache.spark.sql.jdbc Location: file:/user/hive/warehouse/mysql_federated_sample Serde Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe InputFormat: org.apache.hadoop.mapred.SequenceFileInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat Storage Properties: [url=###, driver=com.mysql.jdbc.Driver, dbtable=TBLS] Schema: root |-- TBL_ID: long (nullable = true) |-- CREATE_TIME: integer (nullable = true) |-- DB_ID: long (nullable = true) |-- LAST_ACCESS_TIME: integer (nullable = true) |-- OWNER: string (nullable = true) |-- RETENTION: integer (nullable = true) |-- SD_ID: long (nullable = true) |-- TBL_NAME: string (nullable = true) |-- TBL_TYPE: string (nullable = true) |-- VIEW_EXPANDED_TEXT: string (nullable = true) |-- VIEW_ORIGINAL_TEXT: string (nullable = true) |-- IS_REWRITE_ENABLED: boolean (nullable = true) ``` After this PR: ```sql spark-sql> desc formatted mysql_federated_sample; TBL_ID bigint NULL CREATE_TIME int NULL DB_ID bigint NULL LAST_ACCESS_TIME int NULL OWNER string NULL RETENTION int NULL SD_ID bigint NULL TBL_NAME string NULL TBL_TYPE string NULL VIEW_EXPANDED_TEXT string NULL VIEW_ORIGINAL_TEXT string NULL IS_REWRITE_ENABLED boolean NULL # Detailed Table Information Database default Table mysql_federated_sample Owner root Created Time Thu Aug 08 17:35:35 GMT-07:00 2019 Last Access Wed Dec 31 17:00:00 GMT-07:00 1969 Created By Spark 3.0.0-SNAPSHOT Type MANAGED Provider org.apache.spark.sql.jdbc Location file:/user/hive/warehouse/mysql_federated_sample Serde Library org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe InputFormat org.apache.hadoop.mapred.SequenceFileInputFormat OutputFormat org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat Storage Properties [url=*********(redacted), driver=com.mysql.jdbc.Driver, dbtable=TBLS] spark-sql> SHOW TABLE EXTENDED LIKE 'mysql_federated_sample'; 19/08/08 17:36:07 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException default mysql_federated_sample false Database: default Table: mysql_federated_sample Owner: root Created Time: Thu Aug 08 17:35:35 GMT-07:00 2019 Last Access: Wed Dec 31 17:00:00 GMT-07:00 1969 Created By: Spark 3.0.0-SNAPSHOT Type: MANAGED Provider: org.apache.spark.sql.jdbc Location: file:/user/hive/warehouse/mysql_federated_sample Serde Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe InputFormat: org.apache.hadoop.mapred.SequenceFileInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat Storage Properties: [url=*********(redacted), driver=com.mysql.jdbc.Driver, dbtable=TBLS] Schema: root |-- TBL_ID: long (nullable = true) |-- CREATE_TIME: integer (nullable = true) |-- DB_ID: long (nullable = true) |-- LAST_ACCESS_TIME: integer (nullable = true) |-- OWNER: string (nullable = true) |-- RETENTION: integer (nullable = true) |-- SD_ID: long (nullable = true) |-- TBL_NAME: string (nullable = true) |-- TBL_TYPE: string (nullable = true) |-- VIEW_EXPANDED_TEXT: string (nullable = true) |-- VIEW_ORIGINAL_TEXT: string (nullable = true) |-- IS_REWRITE_ENABLED: boolean (nullable = true) ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
