GitHub user yhuai opened a pull request:
https://github.com/apache/spark/pull/11825
[SPARK-13972] [SQL] [FOLLOW-UP] When creating the query execution for a
converted SQL query, we eagerly trigger analysis
## What changes were proposed in this pull request?
As part of testing generating SQL query from a analyzed SQL plan, we run
the generated SQL for tests in HiveComparisonTest. This PR makes the generated
SQL get eagerly analyzed. So, when a generated SQL has any analysis error, we
can see the error message created by
```
case NonFatal(e) => fail(
s"""Failed to analyze the converted SQL string:
|
|# Original HiveQL query string:
|$queryString
|
|# Resolved query plan:
|${originalQuery.analyzed.treeString}
|
|# Converted SQL query string:
|$convertedSQL
""".stripMargin, e)
```
Right now, if we can parse a generated SQL but fail to analyze it, we will
see error message generated by the following code (it only mentions that we
cannot execute the original query, i.e. `queryString`).
```
case e: Throwable =>
val errorMessage =
s"""
|Failed to execute query using catalyst:
|Error: ${e.getMessage}
|${stackTraceToString(e)}
|$queryString
|$query
|== HIVE - ${hive.size} row(s) ==
|${hive.mkString("\n")}
""".stripMargin
```
## How was this patch tested?
Existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yhuai/spark SPARK-13972-follow-up
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11825.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11825
----
commit a018f380a546ec8069f7e97cb1cdf01453b12c0e
Author: Yin Huai <[email protected]>
Date: 2016-03-18T17:24:10Z
[SPARK-13972] [SQL] [FOLLOW-UP] When creating the query execution for a
converted SQL query, we eagerly trigger analysis.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
