Steve Loughran created SPARK-16737:
--------------------------------------
Summary: ListingFileCatalog comments about RPC calls in object
store isn't correct
Key: SPARK-16737
URL: https://issues.apache.org/jira/browse/SPARK-16737
Project: Spark
Issue Type: Bug
Components: SQL
Reporter: Steve Loughran
Priority: Trivial
The comment text which came in with SPARK-1612 says
{code}
- Although S3/S3A/S3N file system can be quite slow for remote file metadata
// operations, calling `getFileBlockLocations` does no harm here
since these file system
// implementations don't actually issue RPC for this method.
{code}
That doesn't hold for openstack swift , which can exhibit rack locality if the
swift nodes are spread across the same racks as the application. It is safest
to remove that comment:
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
