Thank you, I got it now I have to change the id values to see the same data as extra results (this is just for testing) amazing.
val df = spark.sql(SELECT monolitically_id() as id, name, department FROM json_person) df.write(append)... to ignite Thanks Sri On Fri, Aug 9, 2019 at 6:08 AM Andrei Aleksandrov <[email protected]> wrote: > Hi, > > Spark contains several *SaveModes *that will be applied if the table that > you are going to use exists: > > * *Overwrite *- with this option you *will try to re-create* existed > table or create new and load data there using IgniteDataStreamer > implementation > * *Append *- with this option you *will not try to re-create* existed > table or create new table and just load the data to existed table > > * *ErrorIfExists *- with this option you will get the exception if the > table that you are going to use exists > > * *Ignore *- with this option nothing will be done in case if the table > that you are going to use exists. If table already exists, the save > operation is expected to not save the contents of the DataFrame and to not > change the existing data. > According to your question: > > You should use the *Append *SaveMode for your spark integration in case > if you are going to store new data to cache and save the previous stored > data. > > Note, that in case if you will store the data for the same Primary Keys > then with data will be overwritten in Ignite table. For example: > > 1)Add person {id=1, name=Vlad, age=19} where id is the primary key > 2)Add person {id=1, name=Nikita, age=26} where id is the primary key > > In Ignite you will see only {id=1, name=Nikita, age=26}. > > Also here you can see the code sample for you and other information about > SaveModes: > > > https://apacheignite-fs.readme.io/docs/ignite-data-frame#section-saving-dataframes > > BR, > Andrei > > On 2019/08/08 17:33:39, sri hari kali charan Tummala <[email protected]> > <[email protected]> wrote: > > Hi All,> > > > > I am new to Apache Ignite community I am testing out ignite for > knowledge> > > sake in the below example the code reads a json file and writes to > ingite> > > in-memory table is it overwriting can I do append mode I did try spark> > > append mode .mode(org.apache.spark.sql.SaveMode.Append)> > > without stopping one ignite application inginte.stop which keeps the > cache> > > alive and tried to insert data to cache twice but I am still getting 4> > > records I was expecting 8 records , what would be the reason ?> > > > > > https://github.com/apache/ignite/blob/1f8cf042f67f523e23f795571f609a9c81726258/examples/src/main/spark/org/apache/ignite/examples/spark/IgniteDataFrameWriteExample.scala#L89> > > > > > -- > > > Thanks & Regards> > > Sri Tummala> > > > -- Thanks & Regards Sri Tummala
