[GitHub] spark pull request #15458: [SPARK-17899][SQL] add a debug mode to keep raw t...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/15458#discussion_r83496528
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -915,4 +915,9 @@ object StaticSQLConf {
.internal()
.intConf
.createWithDefault(4000)
+
+ val DEBUG_MODE = buildConf("spark.sql.debug")
+.internal()
+.booleanConf
+.createWithDefault(false)
--- End diff --
Sure, I can do a quick PR to fix it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #15458: [SPARK-17899][SQL] add a debug mode to keep raw t...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/15458#discussion_r83467245
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -915,4 +915,9 @@ object StaticSQLConf {
.internal()
.intConf
.createWithDefault(4000)
+
+ val DEBUG_MODE = buildConf("spark.sql.debug")
+.internal()
+.booleanConf
+.createWithDefault(false)
--- End diff --
Let's add doc for this flag.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #15458: [SPARK-17899][SQL] add a debug mode to keep raw t...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15458 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #15458: [SPARK-17899][SQL] add a debug mode to keep raw t...
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/15458 [SPARK-17899][SQL] add a debug mode to keep raw table properties in HiveExternalCatalog ## What changes were proposed in this pull request? Currently `HiveExternalCatalog` will filter out the Spark SQL internal table properties, e.g. `spark.sql.sources.provider`, `spark.sql.sources.schema`, etc. This is reasonable for external users as they don't want to see these internal properties in `DESC TABLE`. However, as a Spark developer, sometimes we do wanna see the raw table properties. This PR adds a new internal SQL conf, `spark.sql.debug`, to enable debug mode and keep these raw table properties. This config can also be used in similar places where we wanna retain debug information in the future. ## How was this patch tested? new test in MetastoreDataSourcesSuite You can merge this pull request into a Git repository by running: $ git pull https://github.com/cloud-fan/spark debug Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/15458.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #15458 commit e821f1a9d19215fe180ffcbd8183aabd1185a316 Author: Wenchen Fan Date: 2016-10-13T04:21:01Z add a debug mode to keep raw table properties in HiveExternalCatalog --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
