Thanks a lot for very prompt response. Then next questions are the following.
1. Can we conclude that Spark is NOT the solution for our requirement? Or 2. Is there a design approach to meet such requirements using Spark? From: Mayur Rustagi [mailto:mayur.rust...@gmail.com] Sent: 01 May 2014 18:22 To: user@spark.apache.org Cc: u...@spark.incubator.apache.org Subject: Re: update of RDDs RDD are immutable so cannot be updated. You can create new RDD containing updated entries(often not what you want to do). Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi <https://twitter.com/mayur_rustagi> On Thu, May 1, 2014 at 4:42 AM, narayanabhatla NarasimhaMurthy <nn.mur...@cmcltd.com> wrote: In our application, we need distributed RDDs containing key-value maps. We have operations that update RDDs by way of adding entries to the map, delete entries from the map as well as update value part of maps. We also have map reduce functions that operate on the RDDs.The questions are the following. 1. Can RDDs be updated? if Yes, what rae the methods? 2. If we update RDDs, will it happen in place or does it create new RDDs with almost double the original RDD size (original+newly created RDD)? Thank you very much. N.N.Murthy -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/update-of-RDDs-tp5132.html Sent from the Apache Spark User List mailing list archive at Nabble.com. ______________________________________________________________________________ DISCLAIMER The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or directly to netsupp...@cmcltd.com or telephone and immediately and permanently delete the message and any attachments. Thank you. ______________________________________________________________________________ This email has been scrubbed for your protection by SecureMX. For more information visit http://securemx.in _____________________________________________________________________________
