[GitHub] spark issue #19206: [SPARK-19206][yarn]Client and ApplicationMaster resolveP...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19206 I also meant that I don't think this is a problem and the "fix" is actually incorrect. I disagree with "this is a feature", because if the path resolves to an HA namespace, and YARN doesn't know about the namespace, things will still not work. If the path resolves to a specific namenode in an HA setup, and that namenode is the standby one, it will still not work. So YARN needs to know about the HA/federation configuration for this feature to work. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19206: [SPARK-19206][yarn]Client and ApplicationMaster resolveP...
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/19206 It says `Spark-19206` in your PR title but Spark-19206 is actually about `Update outdated parameter descriptions in external-kafka module`; so maybe you should reference a different JIRA. That's all what Sean and Marcelo meant. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19206: [SPARK-19206][yarn]Client and ApplicationMaster resolveP...
Github user Chaos-Ju commented on the issue: https://github.com/apache/spark/pull/19206 @vanzin @srowen thanks your notice and commentï¼ I means that if make Yarn Cluster know about the federated HDFS service, We should distribute the mount table XML in the cluster and by restart Yarn Cluster bring into effect. under this circumstancesï¼if We change the mount table or HA settingsï¼we should restart Yarn Cluster again. In additionï¼I dont think this is a bugï¼but its a feature . if it is possibleï¼We can make Yarn no need to know about the federated HDFS service like MR & Hive --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19206: [SPARK-19206][yarn]Client and ApplicationMaster resolveP...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19206 Also, doesn't this mean that your YARN service does not know about the federated HDFS service? Same thing can happen if you instead have an HA setup but YARN doesn't know about it. It will fail to resolve the namespace and YARN won't be able to download resources. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19206: [SPARK-19206][yarn]Client and ApplicationMaster resolveP...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19206 @Chaos-Ju Sean means the bug you reference in the title has nothing to do with the code. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19206: [SPARK-19206][yarn]Client and ApplicationMaster resolveP...
Github user Chaos-Ju commented on the issue: https://github.com/apache/spark/pull/19206 @srowen you think should close the jira and this is pointlessï¼ --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19206: [SPARK-19206][yarn]Client and ApplicationMaster resolveP...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19206 @Chaos-Ju this is connected to the wrong JIRA --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
