Eric Liang created SPARK-18679:

             Summary: Regression in file listing performance
                 Key: SPARK-18679
             Project: Spark
          Issue Type: Bug
            Reporter: Eric Liang
            Priority: Blocker

In Spark 2.1 ListingFileCatalog was significantly refactored (and renamed to 

It seems there is a performance regression here where we no longer performance 
listing in parallel for the non-root directory. This forces file listing to be 
completely serial when resolving datasource tables that are not backed by an 
external catalog.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to