Hi Community, I am trying to see if there’s a support for a property similar to “spark.yarn.access.hadoopFileSystems” (refer<https://spark.apache.org/docs/2.3.0/running-on-yarn.html>) in Tez? As I understand, with the current architecture, the file-system (anything other than hdfs) of the table being queried is initialized at the launch-time of a DAG, however, the file-system’s delegation tokens obtained in the context of the DAG aren’t renewed by yarn. As a result, long running queries may fail due to stale delegation tokens. If we can somehow get the delegation tokens at the time of yarn application launch and pass it in the application context to yarn, yarn may be able to renew those tokens.
Using tez.aux.uris may be a possible work-around, but I do not think this property was meant for this kind of a usage. Best, Ankur Khanna
