Sure. Looking more closely at the code, I thought I might have had an error in
the flow of data structures in the R code, the line that extracts the words
from the corpus is now,
words <- distinct(SparkR:::flatMap(corpus function(line) {
strsplit(
gsub(“^\\s+|[[:punct:]]”, “”, tolower(line)),
“\\s”)[[1]]
}))
(just removes leading whitespace and all punctuation after having made the
whole line lowercase, then splits to a vector of words, ultimately flattening
the whole collection)
Counts works on the resultant words list, returning the value expected, so the
hang most likely occurs during the subtract. I should mention, the size of the
corpus is very small, just kb in size. The dictionary I subtract against is
also quite modest by Spark standards, just 4.8MB, and I’ve got 2G memory for
the Worker, which ought to be sufficient for such a small job.
The Scala analog runs quite fast, even with the subtract. If we look at the DAG
for the SparkR job and compare that against the event timeline for Stage 3, it
seems the job is stuck in Scheduler Delay (in 0/2 tasks completed) and never
begins the rest of the stage. Unfortunately, the executor log hangs up as well,
and doesn’t give much info.
[cid:F966AC39-9916-4CBD-B447-5BF1C136F67E]
Could you describe in a little more detail at what points data is actually held
in R’s internal process memory? I was under the impression that
SparkR:::textFile created an RDD object that would only be realized when a DAG
requiring it was executed, and would therefore be part of the memory managed by
Spark, and that memory would only be moved to R as an R object following a
collect(), take(), etc.
Thanks,
Alek Eskilson
From: Shivaram Venkataraman
<[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Wednesday, May 27, 2015 at 8:26 PM
To: Aleksander Eskilson
<[email protected]<mailto:[email protected]>>
Cc: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: SparkR Jobs Hanging in collectPartitions
Could you try to see which phase is causing the hang ? i.e. If you do a count()
after flatMap does that work correctly ? My guess is that the hang is somehow
related to data not fitting in the R process memory but its hard to say without
more diagnostic information.
Thanks
Shivaram
On Tue, May 26, 2015 at 7:28 AM, Eskilson,Aleksander
<[email protected]<mailto:[email protected]>> wrote:
I’ve been attempting to run a SparkR translation of a similar Scala job that
identifies words from a corpus not existing in a newline delimited dictionary.
The R code is:
dict <- SparkR:::textFile(sc, src1)
corpus <- SparkR:::textFile(sc, src2)
words <- distinct(SparkR:::flatMap(corpus, function(line) { gsub(“[[:punct:]]”,
“”, tolower(strsplit(line, “ |,|-“)[[1]]))}))
found <- subtract(words, dict)
(where src1, src2 are locations on HDFS)
Then attempting something like take(found, 10) or saveAsTextFile(found, dest)
should realize the collection, but that stage of the DAG hangs in Scheduler
Delay during the collectPartitions phase.
Synonymous Scala code however,
val corpus = sc.textFile(src1).flatMap(_.split(“ |,|-“))
val dict = sc.textFile(src2)
val words = corpus.map(word => word.filter(Character.isLetter(_))).disctinct()
val found = words.subtract(dict)
performs as expected. Any thoughts?
Thanks,
Alek Eskilson
CONFIDENTIALITY NOTICE This message and any included attachments are from
Cerner Corporation and are intended only for the addressee. The information
contained in this message is confidential and may constitute inside or
non-public information under international, federal, or state securities laws.
Unauthorized forwarding, printing, copying, distribution, or use of such
information is strictly prohibited and may be unlawful. If you are not the
addressee, please promptly delete this message and notify the sender of the
delivery error by e-mail or you may call Cerner's corporate offices in Kansas
City, Missouri, U.S.A at (+1) (816)221-1024<tel:%28%2B1%29%20%28816%29221-1024>.
