Hi Andrey, I have similar requirement and I am using cache.putAll method to update existing entries or to insert new ones. I will be updating/inserting close to 3 million entries in one go.
I am using wrte through approach to update/insert/delete the data in oracle tables. I am using cachestores writeAll/ deleteAll method to achieve this. I am doing this in single ignite distributed transaction. Now the question is, 1a) Can I use streamer in ignite transaction? 1b) Can I use ignite jdbc bulk update, insert, delete with ignite distributed transaction? 2) if I use streamer will it invoke cache store writeAll method? I meant does write through approach work with streamer. 3) If I use Jdbc bulk mode for cache update and insert or delete, will it invoke cache store's wrieAll and deleteAll method? Does write through approach work with jdbc bulk update/insert/ delete? 4) Does ignite have any apis on cache only for update purpose? Put/putAll will insert or overwrite. What if I just want to update existing entries ? Thanks, Prasad On Fri, Mar 30, 2018, 11:12 PM Andrey Mashenkov <[email protected]> wrote: > Hi, > > Ignite has 2 JDBC drivers. > 1. Client driver [1] starts client node (has all failover features) and > you have to pass client node config in URL. > 2. Thin Driver [2] that connect directly to one of Ignite server node. > > So, you need a first one to be able to use streaming mode. > > [1] https://apacheignite-sql.readme.io/docs/jdbc-client-driver > [2] https://apacheignite-sql.readme.io/docs/jdbc-driver > > On Fri, Mar 30, 2018 at 1:16 PM, <[email protected]> wrote: > >> Hi Andrey, >> >> >> >> I am trying to run [2], as: >> >> // Register JDBC driver. >> >> Class.forName("org.apache.ignite.IgniteJdbcDriver"); >> >> // Opening connection in the streaming mode. >> >> Connection conn = >> DriverManager.getConnection("jdbc:ignite:cfg://streaming=true >> @file:///etc/config/ignite-jdbc.xml"); >> >> >> >> However, I'm a bit confused about that setting in [2] about the >> ignite-jdbc.xml. >> >> >> >> I do not know how to find or create the xml, and here I run the ignite >> node via JVM. >> >> >> >> If I can write java code to produce the ignite-jdbc or not? Or only >> complete Spring XML configuration? >> >> >> >> By the way, I have tried the [1], that worked well. >> >> >> >> Finally, I still need to use the SQL as a client node, and quick write >> data into cache. >> >> >> >> Thank you for helping me >> >> >> >> Rick >> >> >> >> >> >> *From:* Andrey Mashenkov [mailto:[email protected]] >> *Sent:* Thursday, March 29, 2018 6:20 PM >> *To:* [email protected] >> *Subject:* Re: How to insert multiple rows/data into Cache once >> >> >> >> Hi, >> >> >> >> Try to use DataStreamer for fast cache load [1]. >> >> If you need to use SQL, you can try to use bulk mode updated via JDBC [2] >> >> >> >> >> >> Also a COPY SQL command [3] will be available in next 2.5 release. >> >> The feature is already in master, you can try to build from it. See >> example [4] >> >> . >> >> >> >> [1] https://apacheignite.readme.io/docs/data-streamers >> >> [2] >> https://apacheignite.readme.io/v2.0/docs/jdbc-driver#section-streaming-mode >> >> [3] https://issues.apache.org/jira/browse/IGNITE-6917 >> >> [4] >> https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/sql/SqlJdbcCopyExample.java >> >> >> >> On Thu, Mar 29, 2018 at 11:30 AM, <[email protected]> wrote: >> >> Dear all, >> >> >> >> I am trying to use the SqlFieldsQuery sdk to insert data to one cache on >> Ignite. >> >> >> >> I can insert one data into one cache at a time. >> >> >> >> However, I have no idea to insert multiple rows/data into the cache once. >> >> >> >> For example, I would like to insert 1000 rows/data into the cache once. >> >> >> >> Here, I provide my code to everyone to reproduce my situation. >> >> *public* *class* IgniteCreateServer { >> >> *public* *class* Person { >> >> @QuerySqlField >> >> *private* String firstName; >> >> @QuerySqlField >> >> *private* String lastName; >> >> *public* Person(String firstName, String lastName) { >> >> *this*.firstName = firstName; >> >> *this*.lastName = lastName; >> >> } >> >> } >> >> *public* *static* *void* main(String[] args) { >> >> cacheConf.setName("igniteCache"); >> >> *cacheConf**.setIndexedTypes(String.**class**, String.**class**)*; >> >> cacheConf.setCacheMode(CacheMode.*REPLICATED*); >> >> cacheConf.setAtomicityMode(CacheAtomicityMode.*ATOMIC*); >> >> cfg.setCacheConfiguration(cacheConf); >> >> Ignite igniteNode = Ignition.*getOrStart*(cfg); >> >> *IgniteCache* cacheKeyvalue = *igniteNode**.getOrCreateCache(* >> *cacheConf**)*; >> >> >> >> *long* starttime, endtime; >> >> starttime = System.*currentTimeMillis*(); >> >> *int* datasize = 100000; >> >> *for* (*int* i = 0; i < datasize; i++) { >> >> cacheKeyvalue.put("key " + Integer.*toString*(i), Integer.*toString*( >> i)); >> >> } >> >> endtime = System.*currentTimeMillis*(); >> >> System.*out*.println("write " + datasize + " pairkeyvalue data: spend " >> + (endtime - starttime) + "milliseconds"); >> >> >> //================================================================================================================= >> >> >> >> cacheCfg.setName("personCache"); >> >> cacheCfg.setIndexedTypes(String.*class*, Person.*class*); >> >> cacheCfg.setCacheMode(CacheMode.*REPLICATED*); >> >> cacheCfg.setAtomicityMode(CacheAtomicityMode.*ATOMIC*); >> >> *IgniteCache* cacheKeyTable = igniteNode.getOrCreateCache(cacheCfg); >> >> >> >> *long* starttime1, endtime1; >> >> starttime1 = System.*currentTimeMillis*(); >> >> *for* (*int* i = 0; i < datasize; i++) { >> >> cacheKeyTable.query(*new* SqlFieldsQuery("INSERT INTO >> Person(_key,firstName,lastName) VALUES(?,?,?)") >> >> .setArgs(i, "key " + Integer.*toString*(i), Integer.*toString*(i))); >> >> } >> >> endtime1 = System.*currentTimeMillis*(); >> >> System.*out*.println("write" + datasize + " table data: spend " + ( >> endtime1 - starttime1) + "milliseconds"); >> >> } >> >> >> >> The my code show as: >> >> “write 100000 pairkeyvalue data: spend 4734 milliseconds >> >> write 100000 table data: spend 2846 milliseconds” >> >> >> >> From the above result, I feel that using the SQL to insert data to cache >> is faster than using cache.getall(). >> >> >> >> I am not sure if this is correct or not? >> >> >> >> In addition, that is important for me to insert data into cache via the >> use of SQL, >> >> so I would like to insert multiple rows/data to accelerate it. >> >> >> >> if any further information is needed, I am glad to be informed and will >> provide to you as soon as possible. >> >> >> >> Thanks >> >> >> >> Rick >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain >> confidential information. Please do not use or disclose it in any way and >> delete it if you are not the intended recipient. >> >> >> >> >> >> -- >> >> Best regards, >> Andrey V. Mashenkov >> >> >> -- >> 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain >> confidential information. Please do not use or disclose it in any way and >> delete it if you are not the intended recipient. >> > > > > -- > Best regards, > Andrey V. Mashenkov >
