GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/16238
[SPARK-18811] StreamSource resolution should happen in stream execution
thread
## What changes were proposed in this pull request?
When you start a stream, if we are trying to resolve the source of the
stream, for example if we need to resolve partition columns, this could take a
long time. This long execution time should not block the main thread where
`query.start()` was called on. It should happen in the stream execution thread
possibly before starting any triggers.
## How was this patch tested?
Unit test added. Made sure test fails with no code changes.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/brkyvz/spark SPARK-18811
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16238.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16238
----
commit 65eb42a5659be6b86519c5f09781d15dceb0bc18
Author: Burak Yavuz <[email protected]>
Date: 2016-12-09T21:40:41Z
ready for review
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
