First, Calcite support is still considered beta-quality, so that might be a factor.
However, more likely is your memory usage. A query like this would need a lot of Java heap space, so you would need to tune your garbage collector. > On 9 Aug 2023, at 15:48, Maxim Baglay <mbagla...@gmail.com> wrote: > > Hi, > Experiencing performance problems on writing data to Ignite stack with next > queries > > MERGE INTO ... VALUES ((...)...) ON () WHEN MATCHED THEN UPDATE SET ... WHEN > NOT MATCHED THEN INSERT ... VALUES (...) > > with 1-2 values it takes about 100-150ms in average (and growing to 600-700ms > and more) when expected to be under 10ms. Average execution time constantly > grows. > > Additional details: > Connecting as thin jdbc client. > Use Calcite as query engine. > Ingite stack has 40 data nodes. > Stack sync is done with zookeeper. > Connecting using r53 address. > The table is partitioned with 1 backup. > Query execution time are from SYS.SQL_QUERY_HISTORY > > The questions here are: > why this query takes so much time? > and what are recomendations for stack to save dozens of millions of rows > (amount of nodes, amount of CPUs, heap size...)? > > Tried to turn off persistance - that didn't change exectuion time > Tried to use partitionAwareness=true - that didn't change exectuion time > Tried to set skipReducerOnUpdate tp true - that didn't help > Tried providing full set of nodes in connection string instead of r53 - that > didn't help > > Thanks in advance for the help!
