mgaido91 commented on issue #24785: [SPARK-27937][CORE] Revert partial logic for auto namespace discovery URL: https://github.com/apache/spark/pull/24785#issuecomment-500032263 I am not sure this is the right approach. First, this change introduces a breaking change which can introduce regressions with federated namespaces: jobs running with the current code may break if they rely on this behavior after this PR. Moreover: > For accessing data from related namespaces, viewfs should handle getting tokens for spark This is already the case, please see the comment in the code you reverted. If there is a case when this is not true, we should rather fix it than revert the full change IMHO. > For accessing data from unrelated namespaces(user explicitly specifies them using existing configs) as these could be on the same or different cluster. This is still the case, if you want to access a namespace which is on a different cluster, you still have to use that config. Probably I am missing something, may you please provide a concrete use case where the current behavior is causing issues and which are the issues? Thanks.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
