[jira] (SPARK-19404) Command line arguments warnings are written to stdout when using spark-submit and custom code

[Reekunj Parmar (JIRA)]

Title: Message Title

Reekunj Parmar updated an issue   Spark / SPARK-19404 Command line arguments warnings are written to stdout when using spark-submit and custom code Change By: Reekunj Parmar Expecting a clean stdout for piping results of an aggregation operation using spark-submit and custom code. Spark-defaults.conf has this value present in it ("spark.yarn.driver.memoryOverhead 384") which when present causes line #122 in this file https://github.com/apache/spark/blob/v1.6.1/yarn/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala#L122 to write directly to stdout , polluting our stream to the downstream process . All of these warnings   are   required  to be sent to the logger basically to file output instead of standard output. Even writing to Console.err or System.err will work. Add Comment   This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)