Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20803#discussion_r176313767
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
---
@@ -635,7 +637,8 @@ class SparkSession private(
* @since 2.0.0
*/
def sql(sqlText: String): DataFrame = {
- Dataset.ofRows(self, sessionState.sqlParser.parsePlan(sqlText))
+ Dataset.ofRows(self, sessionState.sqlParser.parsePlan(sqlText),
+ substitutor.substitute(sqlText))
--- End diff --
Hi, @LantaoJin .
What you need is just grapping the initial SQL text here, you can use Spark
extension. Please refer [Spark Atlas
Connector](https://github.com/hortonworks-spark/spark-atlas-connector/blob/master/spark-atlas-connector/src/main/scala/com/hortonworks/spark/atlas/sql/SparkExtension.scala)
for a sample code.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
