quot;).
>
>
> On Fri, Mar 30, 2018 at 11:41 AM, Koert Kuipers <ko...@tresata.com> wrote:
> > does anyone know why all spark settings end up being system properties,
> and
> > where this is done?
> >
> > for example when i pass "--conf spark.foo=bar&
, but user code can always break those.
Where: this is done by the SparkSubmit class (look for the Scala
version, "sys.props").
On Fri, Mar 30, 2018 at 11:41 AM, Koert Kuipers <ko...@tresata.com> wrote:
> does anyone know why all spark settings end up being system propert
does anyone know why all spark settings end up being system properties, and
where this is done?
for example when i pass "--conf spark.foo=bar" into spark-submit then
System.getProperty("spark.foo") will be equal to "bar"
i grepped the spark codebase for System.setPr
