GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/18450
[SPARK-21238][SQL] allow nested SQL execution
## What changes were proposed in this pull request?
This is kind of another follow-up for
https://github.com/apache/spark/pull/18064 .
In #18064 , we wrap every SQL command with SQL execution, which makes
nested SQL execution very likely to happen. #18419 trid to improve it a little
bit, by introduing `SQLExecition.ignoreNestedExecutionId`. However, this is not
friendly to data source developers, they may need to update their code to use
this `ignoreNestedExecutionId` API.
This PR proposes a new solution, to just allow nested execution, with an
optimization that ignore SQL execution without assosicated Spark jobs and
metrics. So most of the SQL queries should still have only one execution
displayed in the UI.
## How was this patch tested?
existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark execution-id
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18450.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18450
----
commit d47433ed5a8598064f30244d2c7c72a65335292e
Author: Wenchen Fan <[email protected]>
Date: 2017-06-28T08:14:05Z
allow nested SQL execution
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
