Re: Traditional RDBMS PARTITION concept on Ignite

2022-07-21 Thread Mengliao(Mike) Wang
Hi Denis and Stephen, Thanks a lot for the great information and suggestions. I think we are clear on the directions we are heading, but just one more question on the calcite engine Stephen mentioned. From what I read this is replacing the current H2-based engine starting from Ignite 2.13, and fro

Re: Traditional RDBMS PARTITION concept on Ignite

2022-07-20 Thread Denis Magda
Hey folks, let me chime in to clarify the matters. What Wen Bo and Mengliao are really asking for is a combination of two features "partitioning + sharding". In the relational world (Postgres, MySQL, Oracle) partitioning is used to split a large table into smaller tables (called partitions). For e

Re: Traditional RDBMS PARTITION concept on Ignite

2022-07-20 Thread Stephen Darlington
Ignite’s SQL is ANSI 99 compliant. Windowing functions such as PARTITION BY came in SQL 2003 (and later). It’s possible that the new Calcite engine (sql-calcite ) supports the keywords, but I have not checked. > While querying we can only sc

Re: Traditional RDBMS PARTITION concept on Ignite

2022-07-18 Thread Mengliao(Mike) Wang
Hi Stephen, What we are looking for is the table partition with SQL in particular, instead of the data partition people mostly refer to in Ignite which is more from the infrastructure perspective. A.k.a the "PARTITION BY" keyword in traditional RDBMS. In the Ignite official document ( https://igni

Re: Traditional RDBMS PARTITION concept on Ignite

2022-07-14 Thread Stephen Darlington
As you say, partitions in Ignite are about the distribution of data. You can group together related data using affinity keys, but if you only have three distinct values that would be a really bad idea. You can’t change the number of partitions after a table has been created. Either of your othe

Traditional RDBMS PARTITION concept on Ignite

2022-07-13 Thread Wen Bo (Bill) Li
Hi, The traditional RDBMS has the concept of partitioning a table into different chunks, but that isn't really partitioning data to different nodes as described in the Ignite document. Our team is trying to partition a table based on the values of one column and query data based on these values. F