Gavin Zhang created SPARK-3326:
----------------------------------
Summary: can't access a static variable after init in mapper
Key: SPARK-3326
URL: https://issues.apache.org/jira/browse/SPARK-3326
Project: Spark
Issue Type: Bug
Environment: CDH5.1.0
Spark1.0.0
Reporter: Gavin Zhang
I wrote a object like:
object Foo {
private Bar bar = null
def init(Bar bar){
this.bar = bar
}
def getSome(){
bar.someDef()
}
}
In Spark main def, I read some text from HDFS and init this object. And after
then calling getSome().
I was successful with this code:
sc.textFile(args(0)).take(10).map(println(Foo.getSome()))
However, when I changed it for write output to HDFS, I found the bar variable
in Foo object is null:
sc.textFile(args(0)).map(line=>Foo.getSome()).saveAsTextFile(args(1))
WHY?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
