[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-12 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16500 Let me think whether we can improve the existing verification mechanism for both caches in the test cases. It can help us to know what the caches actually contain. --- If your project is set

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-11 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/16500 I guess the only purpose of the cache now is to associate file-status caches with specific table names. If we removed that, then tables would have to find their file-status cache by path, or we could

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16500 I'm wondering if we need the metadata cache anymore. Now we store partitions in the metastore, and have a cache for leaf files, what's the benefit of metadata cache? --- If your project is set

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16500 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71023/ Test PASSed. ---

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16500 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16500 **[Test build #71023 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71023/testReport)** for PR 16500 at commit

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16500 **[Test build #71023 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71023/testReport)** for PR 16500 at commit

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16500 cc @ericl @cloud-fan @mallman The actual code changes are just two lines. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as