Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r95084391
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r95074015
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r95073558
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r95073551
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r95073488
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r95073252
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/15539
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84568025
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala
---
@@ -38,14 +38,16 @@ class ListingFileCatalog(
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84508947
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala
---
@@ -42,24 +43,21 @@ class TableFileCatalog(
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84424330
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileCatalog.scala
---
@@ -294,7 +308,7 @@ object
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84424519
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala
---
@@ -64,11 +66,18 @@ class ListingFileCatalog(
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84424418
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala
---
@@ -42,24 +43,21 @@ class TableFileCatalog(
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84424682
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84409662
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84409377
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala
---
@@ -64,11 +66,18 @@ class ListingFileCatalog(
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84402516
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala
---
@@ -42,24 +43,21 @@ class TableFileCatalog(
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84355969
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84353307
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveTablePerfStatsSuite.scala
---
@@ -103,11 +92,103 @@ class HiveDataFrameSuite extends
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84353211
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala
---
@@ -87,21 +85,13 @@ class TableFileCatalog(
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84353179
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala ---
@@ -231,11 +231,16 @@ private[hive] class
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84352755
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala
---
@@ -64,11 +66,18 @@ class ListingFileCatalog(
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84352402
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84222074
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84203885
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveTablePerfStatsSuite.scala
---
@@ -103,11 +92,103 @@ class HiveDataFrameSuite extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84203441
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala ---
@@ -231,11 +231,16 @@ private[hive] class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84202841
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala
---
@@ -64,11 +66,18 @@ class ListingFileCatalog(
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84184588
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84161990
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84152855
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileCatalog.scala
---
@@ -276,15 +290,15 @@ object
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84152806
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala
---
@@ -38,14 +38,16 @@ class ListingFileCatalog(
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84152793
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84153565
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveTablePerfStatsSuite.scala
---
@@ -103,11 +92,84 @@ class HiveDataFrameSuite extends QueryTest
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84159294
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84118387
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala
---
@@ -32,13 +32,21 @@ import
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r83991043
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala
---
@@ -32,13 +32,21 @@ import
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r83990858
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveTablePerfStatsSuite.scala
---
@@ -103,11 +92,84 @@ class HiveDataFrameSuite extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r83990583
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r83988599
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala
---
@@ -38,14 +38,16 @@ class ListingFileCatalog(
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r83987399
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala
---
@@ -38,14 +38,16 @@ class ListingFileCatalog(
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r83987017
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileCatalog.scala
---
@@ -276,15 +290,15 @@ object
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r83986875
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the
Github user ericl commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r83981625
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -1486,10 +1485,10 @@ private[spark] object Utils extends Logging {
val
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r83980885
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -1486,10 +1485,10 @@ private[spark] object Utils extends Logging {
val
GitHub user ericl opened a pull request:
https://github.com/apache/spark/pull/15539
[SPARK-17994] [SQL] Add back a file status cache for catalog tables
## What changes were proposed in this pull request?
In SPARK-16980, we removed the full in-memory cache of table
44 matches
Mail list logo
