[
https://issues.apache.org/jira/browse/SPARK-11862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen George updated SPARK-11862:
---------------------------------
Description: {{SparkHadoopUtil}} appears to be initialized with an empty
{{SparkConf}} object. Later, methods like {{globPath}} use a
{{HadoopConfiguration}} derived from this empty object. This means that Hadoop
parameters like AWS access keys are never set even though they're present in
the driver's {{SparkConf}} object. (was: `SparkHadoopUtil` appears to be
initialized with an empty `SparkConf` object. Later, methods like `globPath`
use a `HadoopConfiguration` derived from this empty object. This means that
Hadoop parameters like AWS access keys are never set even though they're
present in the driver's `SparkConf` object. )
> SparkHadoopUtil appears to ignore SparkConf
> -------------------------------------------
>
> Key: SPARK-11862
> URL: https://issues.apache.org/jira/browse/SPARK-11862
> Project: Spark
> Issue Type: Bug
> Affects Versions: 1.5.1
> Reporter: Allen George
>
> {{SparkHadoopUtil}} appears to be initialized with an empty {{SparkConf}}
> object. Later, methods like {{globPath}} use a {{HadoopConfiguration}}
> derived from this empty object. This means that Hadoop parameters like AWS
> access keys are never set even though they're present in the driver's
> {{SparkConf}} object.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
