Got it.
Thanks for your help!!
Chieh-Yen
On Tue, Mar 25, 2014 at 6:51 PM, hequn cheng wrote:
> Hi~I wrote a program to test.The non-idempotent "compute" function in
> foreach does change the value of RDD. It may looks a little crazy to do so
> since modify the RDD will make it impossible to ke
Hi~I wrote a program to test.The non-idempotent "compute" function in
foreach does change the value of RDD. It may looks a little crazy to do so
since modify the RDD will make it impossible to keep RDD fault-tolerant in
spark :)
2014-03-25 11:11 GMT+08:00 林武康 :
> Hi hequn, I dig into the sourc
Hi hequn, I dig into the source of spark a bit deeper, and I got some ideas,
firstly, immutable is a feather of rdd but not a solid rule, there are ways to
change it, for excample, a rdd with non-idempotent "compute" function, though
it is really a bad design to make that function non-idempotent
