Hi Team, I am using ignite 2.7.5.
I am trying to insert 25 million records in the ignite persistent table. Observations: I am observing that initial 2.5 million records took 5 mins to insert, next 2.5 million records took 8 mins to insert, following 2.5 records took 12 mins to insert and it kept on increasing . The last 2.5 million records took over 40 minutes to insert. The CPU of the client and server is around 30%, so nothing on that front. Heap usage looks normal. I have tried inserting to the table with and without indexes but nothing seems to make a difference. Configurations: I am running a 3 node cluster, 16 Cores 32 GB machines. Ignite Version: 2.7.5 Things I have tried: JDBI Implementation - 1. @SQLBatch does not work for ignite giving the below error org.skife.jdbi.v2.exceptions.TransactionException: Failed to start transaction at org.skife.jdbi.v2.tweak.transactions.LocalTransactionHandler.begin(LocalTransactionHandler.java:57) at org.skife.jdbi.v2.BasicHandle.begin(BasicHandle.java:159) 2. Tried ?streaming=true, but with no effect 3. Increasing threads to insert, but not much effect JDBC Implementation 1. Tried ?streaming=true, but with no effect 2. Tried batching, but the performance actually degraded 3. Tried ?streaming=true with and without batching but with no effect. Want to achieve a common pattern in insertions, like 5 mins or 8 mins constantly for 25 million insertions Please let me know your thoughts on the same. Your inputs could help me a lot. Thanks a lot. Regards, Shubham Agrawal
