daryn-sharp commented on issue #24785: [SPARK-27937][CORE] Revert partial logic for auto namespace discovery URL: https://github.com/apache/spark/pull/24785#issuecomment-500484715 The client should specify all the required filesystems directly or indirectly via viewfs. It’s a very user specific use case to assume that not using viewfs means the config should be scanned for all federated/HA namespaces. A client should never distinguish between filesystem types when acquiring tokens. Viewfs just happens to be a handy way to implicitly collect tokens for all mounted filesystems rather than explicitly specifying them but it should not be the basis for any conditionalized behavior. Federation is a server-side only configuration for enabling multi-namespace DNs. A client cannot and should not distinguish between a federated and non-federated namespace. Technically a federation config doesn’t even belong in a client config and as such should never be used by a client. In short, the user’s use case is better solved specifying the filesystems. If the federation configs are leaked into the client configs, it should be easy to replace them with a conf key of required filesystems.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
