[GitHub] HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL] Show actual class name of the writing command in CTAS explain
HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL]
Show actual class name of the writing command in CTAS explain
URL: https://github.com/apache/spark/pull/23582#discussion_r249049392
##
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
##
@@ -83,10 +84,12 @@ trait CreateHiveTableAsSelectBase extends
DataWritingCommand {
tableDesc: CatalogTable,
tableExists: Boolean): DataWritingCommand
+ def writingCommandClass: Class[_]
+
override def argString(maxFields: Int): String = {
s"[Database:${tableDesc.database}, " +
Review comment:
Looks we need a space between `Database:` and `${tableDesc.database}`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL] Show actual class name of the writing command in CTAS explain
HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL]
Show actual class name of the writing command in CTAS explain
URL: https://github.com/apache/spark/pull/23582#discussion_r249036620
##
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
##
@@ -83,10 +84,12 @@ trait CreateHiveTableAsSelectBase extends
DataWritingCommand {
tableDesc: CatalogTable,
tableExists: Boolean): DataWritingCommand
+ def writingCommandClass: Class[_]
+
override def argString(maxFields: Int): String = {
s"[Database:${tableDesc.database}, " +
s"TableName: ${tableDesc.identifier.table}, " +
-s"InsertIntoHiveTable]"
+s"${Utils.getSimpleName(writingCommandClass)}]"
Review comment:
Oh, I am sorry, it's not checking self.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL] Show actual class name of the writing command in CTAS explain
HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL]
Show actual class name of the writing command in CTAS explain
URL: https://github.com/apache/spark/pull/23582#discussion_r249036191
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveExplainSuite.scala
##
@@ -182,4 +183,27 @@ class HiveExplainSuite extends QueryTest with
SQLTestUtils with TestHiveSingleto
assert(output.toString.contains(s"Scan hive default.$tableName"))
}
}
+
+ test("SPARK-26661: Show actual class name of the writing command in CTAS
explain") {
+Seq(true, false).foreach { convertCTAS =>
+
+ withSQLConf(
+ HiveUtils.CONVERT_METASTORE_CTAS.key -> convertCTAS.toString,
+ HiveUtils.CONVERT_METASTORE_PARQUET.key -> convertCTAS.toString) {
+
+val tableName = "tab1"
Review comment:
or `OptimizedCreateHiveTableAsSelectCommand` and
`CreateHiveTableAsSelectCommand`. Checking one should be okay.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL] Show actual class name of the writing command in CTAS explain
HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL]
Show actual class name of the writing command in CTAS explain
URL: https://github.com/apache/spark/pull/23582#discussion_r249035982
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveExplainSuite.scala
##
@@ -182,4 +183,27 @@ class HiveExplainSuite extends QueryTest with
SQLTestUtils with TestHiveSingleto
assert(output.toString.contains(s"Scan hive default.$tableName"))
}
}
+
+ test("SPARK-26661: Show actual class name of the writing command in CTAS
explain") {
+Seq(true, false).foreach { convertCTAS =>
+
+ withSQLConf(
+ HiveUtils.CONVERT_METASTORE_CTAS.key -> convertCTAS.toString,
+ HiveUtils.CONVERT_METASTORE_PARQUET.key -> convertCTAS.toString) {
+
+val tableName = "tab1"
Review comment:
I would do:
```scala
val keyword = if (convertCTAS) {
Utils.getSimpleName(classOf[InsertIntoHadoopFsRelationCommand])
} else {
Utils.getSimpleName(classOf[InsertIntoHiveTable])
}
val df = sql("EXPLAIN CREATE TABLE tab1 STORED AS PARQUET AS SELECT
* FROM range(2)")
checkKeywordsExist(df, keyword)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL] Show actual class name of the writing command in CTAS explain
HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL]
Show actual class name of the writing command in CTAS explain
URL: https://github.com/apache/spark/pull/23582#discussion_r249036191
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveExplainSuite.scala
##
@@ -182,4 +183,27 @@ class HiveExplainSuite extends QueryTest with
SQLTestUtils with TestHiveSingleto
assert(output.toString.contains(s"Scan hive default.$tableName"))
}
}
+
+ test("SPARK-26661: Show actual class name of the writing command in CTAS
explain") {
+Seq(true, false).foreach { convertCTAS =>
+
+ withSQLConf(
+ HiveUtils.CONVERT_METASTORE_CTAS.key -> convertCTAS.toString,
+ HiveUtils.CONVERT_METASTORE_PARQUET.key -> convertCTAS.toString) {
+
+val tableName = "tab1"
Review comment:
or `OptimizedCreateHiveTableAsSelectCommand` and
`CreateHiveTableAsSelectCommand`. Checking one should be okay.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL] Show actual class name of the writing command in CTAS explain
HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL]
Show actual class name of the writing command in CTAS explain
URL: https://github.com/apache/spark/pull/23582#discussion_r248984540
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveExplainSuite.scala
##
@@ -182,4 +183,27 @@ class HiveExplainSuite extends QueryTest with
SQLTestUtils with TestHiveSingleto
assert(output.toString.contains(s"Scan hive default.$tableName"))
}
}
+
+ test("SPARK-26661: Show actual class name of the writing command in CTAS
explain") {
+Seq(true, false).foreach { convertCTAS =>
+
+ withSQLConf(
+ HiveUtils.CONVERT_METASTORE_CTAS.key -> convertCTAS.toString,
+ HiveUtils.CONVERT_METASTORE_PARQUET.key -> convertCTAS.toString) {
+
+val tableName = "tab1"
Review comment:
I would do:
```scala
val keyword = if (convertCTAS) {
Utils.getSimpleName(classOf[InsertIntoHadoopFsRelationCommand])
} else {
Utils.getSimpleName(classOf[InsertIntoHiveTable])
}
val df = sql("EXPLAIN CREATE TABLE tab1 STORED AS PARQUET AS SELECT
* FROM range(2)")
checkKeywordsExist(df, keyword)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL] Show actual class name of the writing command in CTAS explain
HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL]
Show actual class name of the writing command in CTAS explain
URL: https://github.com/apache/spark/pull/23582#discussion_r248984540
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveExplainSuite.scala
##
@@ -182,4 +183,27 @@ class HiveExplainSuite extends QueryTest with
SQLTestUtils with TestHiveSingleto
assert(output.toString.contains(s"Scan hive default.$tableName"))
}
}
+
+ test("SPARK-26661: Show actual class name of the writing command in CTAS
explain") {
+Seq(true, false).foreach { convertCTAS =>
+
+ withSQLConf(
+ HiveUtils.CONVERT_METASTORE_CTAS.key -> convertCTAS.toString,
+ HiveUtils.CONVERT_METASTORE_PARQUET.key -> convertCTAS.toString) {
+
+val tableName = "tab1"
Review comment:
I would do:
```scala
val keyword = if (convertCTAS) {
Utils.getSimpleName(classOf[InsertIntoHadoopFsRelationCommand])
} else {
Utils.getSimpleName(classOf[InsertIntoHiveTable])
}
val df = sql("EXPLAIN CREATE TABLE tab1 STORED AS PARQUET AS SELECT
* FROM range(2)")
checkKeywordsExist(df, keyword)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL] Show actual class name of the writing command in CTAS explain
HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL]
Show actual class name of the writing command in CTAS explain
URL: https://github.com/apache/spark/pull/23582#discussion_r248984540
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveExplainSuite.scala
##
@@ -182,4 +183,27 @@ class HiveExplainSuite extends QueryTest with
SQLTestUtils with TestHiveSingleto
assert(output.toString.contains(s"Scan hive default.$tableName"))
}
}
+
+ test("SPARK-26661: Show actual class name of the writing command in CTAS
explain") {
+Seq(true, false).foreach { convertCTAS =>
+
+ withSQLConf(
+ HiveUtils.CONVERT_METASTORE_CTAS.key -> convertCTAS.toString,
+ HiveUtils.CONVERT_METASTORE_PARQUET.key -> convertCTAS.toString) {
+
+val tableName = "tab1"
Review comment:
I would do:
```scala
val keyword = if (convertCTAS) {
Utils.getSimpleName(classOf[OptimizedCreateHiveTableAsSelectCommand])
} else {
Utils.getSimpleName(classOf[CreateHiveTableAsSelectCommand])
}
val df = sql("EXPLAIN CREATE TABLE tab1 STORED AS PARQUET AS SELECT
* FROM range(2)")
checkKeywordsExist(df, keyword)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL] Show actual class name of the writing command in CTAS explain
HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL]
Show actual class name of the writing command in CTAS explain
URL: https://github.com/apache/spark/pull/23582#discussion_r248984540
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveExplainSuite.scala
##
@@ -182,4 +183,27 @@ class HiveExplainSuite extends QueryTest with
SQLTestUtils with TestHiveSingleto
assert(output.toString.contains(s"Scan hive default.$tableName"))
}
}
+
+ test("SPARK-26661: Show actual class name of the writing command in CTAS
explain") {
+Seq(true, false).foreach { convertCTAS =>
+
+ withSQLConf(
+ HiveUtils.CONVERT_METASTORE_CTAS.key -> convertCTAS.toString,
+ HiveUtils.CONVERT_METASTORE_PARQUET.key -> convertCTAS.toString) {
+
+val tableName = "tab1"
Review comment:
I would do:
```scala
val keyword = if (convertCTAS) {
Utils.getSimpleName(classOf[InsertIntoHadoopFsRelationCommand])
} else {
Utils.getSimpleName(classOf[InsertIntoHiveTable])
}
val df = sql("EXPLAIN CREATE TABLE tab1 STORED AS PARQUET AS SELECT
* FROM range(2)")
checkKeywordsExist(df, keyword)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL] Show actual class name of the writing command in CTAS explain
HyukjinKwon commented on a change in pull request #23582: [SPARK-26661][SQL]
Show actual class name of the writing command in CTAS explain
URL: https://github.com/apache/spark/pull/23582#discussion_r248979249
##
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
##
@@ -83,10 +84,12 @@ trait CreateHiveTableAsSelectBase extends
DataWritingCommand {
tableDesc: CatalogTable,
tableExists: Boolean): DataWritingCommand
+ def writingCommandClass: Class[_]
+
override def argString(maxFields: Int): String = {
s"[Database:${tableDesc.database}, " +
s"TableName: ${tableDesc.identifier.table}, " +
-s"InsertIntoHiveTable]"
+s"${Utils.getSimpleName(writingCommandClass)}]"
Review comment:
I think we could simply `Utils.getSimpleName(this.getClass)}`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
