GitHub user jayadevanmurali opened a pull request: https://github.com/apache/spark/pull/16635
[SPARK-19059] [SQL] Unable to retrieve data from parquet table whose name startswith underscore ## What changes were proposed in this pull request? The initial shouldFilterOut() method invocation filter the root path name(table name in the intial call) and remove if it contains _. I moved the check one level below, so it first list files/directories in the given root path and then apply filter. (Please fill in changes proposed in this fix) ## How was this patch tested? Added new test case for this scenario (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) Please review http://spark.apache.org/contributing.html before opening a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jayadevanmurali/spark branch-0.1-SPARK-19059 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/16635.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #16635 ---- commit 290b86ba76e8f6032824c95079748670c2257db0 Author: jayadevanmurali <jayadeva...@tcs.com> Date: 2016-01-31T02:28:52Z Merge pull request #1 from apache/master Update from original commit 9a7f87f7bd4b75b5149e826a7a5868f5549132cf Author: jayadevan <jayadeva...@tcs.com> Date: 2016-05-10T04:04:32Z Merge remote-tracking branch 'upstream/master' commit d8ed5862f887ced075bf73f58e1f9c13ff340527 Author: jayadevan <jayadeva...@tcs.com> Date: 2016-10-24T17:32:19Z Merge remote-tracking branch 'upstream/master' commit cffbec2d6c044a7fb568dcb19b28b07158b870ea Author: jayadevan <jayadeva...@tcs.com> Date: 2016-11-26T07:30:36Z Merge remote-tracking branch 'upstream/master' commit e6727500925be983bf7c345f1e3e6ad6756f19cb Author: jayadevan <jayadeva...@tcs.com> Date: 2017-01-11T15:43:52Z Merge remote-tracking branch 'upstream/master' commit f5999b9577aaaab0d2ca27c699dfc7a1662933dd Author: jayadevanmurali <jayadeva...@tcs.com> Date: 2017-01-18T18:25:15Z Added test case to handle SPARK-19059 commit 74e4a1a2740262fcf84e8c0704ddf2df57e614a0 Author: jayadevanmurali <jayadeva...@tcs.com> Date: 2017-01-18T18:38:36Z Updated listLeafFiles() handile SPARK-19059 commit dec96d848948d6c5f263b1e5535345d24c1058d3 Author: jayadevanmurali <jayadeva...@tcs.com> Date: 2017-01-18T18:52:05Z Update PartitioningAwareFileIndex.scala commit 4334b2ba2a882a94d59459a3d73b5fdb6fda6b80 Author: jayadevanmurali <jayadeva...@tcs.com> Date: 2017-01-18T18:58:22Z Update PartitioningAwareFileIndex.scala ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org