dhruve opened a new pull request #24785: [SPARK-27937][SPARK-27937] Revert partial logic for auto namespace discovery URL: https://github.com/apache/spark/pull/24785 ## What changes were proposed in this pull request? This change reverts the logic which was introduced as a part of SPARK-24149 and a subsequent followup PR. With existing logic: - Spark fails to launch with HDFS federation enabled while trying to get a path to a logical nameservice. - It gets tokens for unrelated namespaces if they are used in HDFS Federation - Automatic namespace discovery is supported only if these are on the same cluster. Rationale for change: - For accessing data from related namespaces, viewfs should handle getting tokens for spark - For accessing data from unrelated namespaces(user explicitly specifies them using existing configs) as these could be on the same or different cluster. (Please fill in changes proposed in this fix) Revert the changes. ## How was this patch tested? Ran few manual tests and unit test.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
