GitHub user prokod opened a pull request:
https://github.com/apache/spark/pull/17883
[SPARK-20622][SQL]value only based directories parquet partition discovery
## What changes were proposed in this pull request?
A proposal for a solution which will allow Spark SQL to discover parquet
partitions for 'value only' named directories.
## How was this patch tested?
Based on existing test suite with added test cases
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/prokod/spark
feature/SPARK-20622/SQL/value_only_based_parquet_partition_discovery
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17883.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17883
----
commit 8d1d5c82c6f4f9c4a58e11191f20cf508be01f72
Author: Noam Asor <[email protected]>
Date: 2017-05-06T15:02:22Z
To allow Spark SQL to do parquet partition discovery for 'value only' named
dirs
Initial solution and tests
#SPARK-20622
commit 2219021c4597bd3e4b2fa0f35dea1808dab98fbe
Author: Noam Asor <[email protected]>
Date: 2017-05-06T15:42:55Z
To undup test code
#SPARK-20622
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
