Well, we explicitly cleanup all intermediates that are no longer used. You
can use -explain to output the runtime plan, which includes rmvar (remove
variable), cpvar (copy variable) and mvvar (move variable) instructions
that internally cleanup intermediates. This cleanup removes data from
memory,
Hi,
Suppose I've this following code:
a = matrix(seq(1,10), 10,1)
for(i in 1:100){
b = a + 10
write (b, "path" + ".csv", format="csv")
}
So what I'm doing is for 100 items, I'm adding a constant to a matrix than
outputting it. And this operation occurs in spark using multiple core
