mgaido91 commented on issue #24785: [SPARK-27937][CORE] Revert partial logic for auto namespace discovery URL: https://github.com/apache/spark/pull/24785#issuecomment-500104501 @dhruve what you are calling a very specific use case, is what in the hadoop documentation is called federation: https://hadoop.apache.org/docs/r2.8.3/hadoop-project-dist/hadoop-hdfs/Federation.html#Federation_Configuration. Thanks for updating the JIRA. I am yet not very clear what the problem is in your case: I mean what was your configuration (`hdfs-site.xml`)? Anyway, why isn't setting `spark.kerberos.access.hadoopFileSystems` a solution for your use case? With that you can basically decide what to do and all the logic here is skipped.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
