Hi Sergey, Your changes look useful from the application developer perspective. However, I'm curious why would the one change some low-level IGNITE_MAX_INDEX_PAYLOAD_SIZE parameter when it's advised to pass INLINE_SIZE to CREATE TABLE to change the index size cluster-wide.
- Denis On Mon, Apr 27, 2020 at 5:38 AM Sergey Antonov <antonovserge...@gmail.com> wrote: > Hi, Igniters! > > I'd like to share a new small feature in AI [1]. > > For different reasons, the cluster could have a different SQL index inline > size [2] on cluster nodes. For example due to > different IGNITE_MAX_INDEX_PAYLOAD_SIZE [3] value on cluster nodes. > > The difference in index inline size may lead to performance degradation. > I think we must compare inline sizes on node join and warn if difference > found. Also, We should have the ability to check inline sizes on demand. > > I've implemented this check on node join and new command in control.sh > > Look at warning message and utility command output: > > Warn message on a node in the cluster during new node join: > > [2020-04-27 15:36:21,185][WARN ][tcp-disco-msg-worker-[6ba0b823 > 127.0.0.1:47502 > crd]-#17%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root] Inline > sizes on local node and node 5bf6ca48-34a0-4aff-8db2-c0b6df303d3f are > different. Please drop and create again these indexes to avoid performance > problems with SQL queries. Problem indexes: > > PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2) > > Warn messages on a joining node, if difference found: > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea > 127.0.0.1:47501 > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root] > Inline sizes on local node and node a86e9cea-63e8-42af-a897-cec4be500001 > are different. Please drop and create again these indexes to avoid > performance problems with SQL queries. Problem indexes: > > PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2) > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea > 127.0.0.1:47501 > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root] > Inline sizes on local node and node a08de16d-df05-48af-a0b9-5596d9c00002 > are different. Please drop and create again these indexes to avoid > performance problems with SQL queries. Problem indexes: > > PUBLIC#TEST_TABLE#L_IDX(1,3),PUBLIC#TEST_TABLE#S1_IDX(1,3),PUBLIC#TEST_TABLE#I_IDX(1,3) > > > Utility output, if a difference in inline sizes was found: > > Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV] > 2020 Copyright(C) Apache Software Foundation > User: santonov > Time: 2020-04-27T15:32:25.759 > Command [CACHE] started > Arguments: --cache check_index_inline_sizes --yes > > -------------------------------------------------------------------------------- > Found 4 secondary indexes. > 3 index(es) have different effective inline size on nodes. It can lead to > performance degradation in SQL queries. > Index(es): > Full index name: PUBLIC#TEST_TABLE#L_IDX nodes: > [ca1d2bae-89d4-4e8d-ae11-6c68f3900000] inline size: 1, nodes: > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2 > Full index name: PUBLIC#TEST_TABLE#S1_IDX nodes: > [ca1d2bae-89d4-4e8d-ae11-6c68f3900000] inline size: 1, nodes: > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2 > Full index name: PUBLIC#TEST_TABLE#I_IDX nodes: > [ca1d2bae-89d4-4e8d-ae11-6c68f3900000] inline size: 1, nodes: > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2 > > Recommendations: > Check that value of property IGNITE_MAX_INDEX_PAYLOAD_SIZE are the same > on all nodes. > Recreate indexes (execute DROP INDEX, CREATE INDEX commands) with > different inline size. > Command [CACHE] finished with code: 0 > Control utility has completed execution at: 2020-04-27T15:32:28.025 > Execution time: 2266 ms > > Utility output, if all indexes have the same inline size: > > Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV] > 2020 Copyright(C) Apache Software Foundation > User: santonov > Time: 2020-04-27T15:30:20.950 > Command [CACHE] started > Arguments: --cache check_index_inline_sizes --yes > > -------------------------------------------------------------------------------- > Found 2 secondary indexes. > All secondary indexes have the same effective inline size on all cluster > nodes. > Command [CACHE] finished with code: 0 > Control utility has completed execution at: 2020-04-27T15:30:23.428 > Execution time: 2478 ms > > Any objections? > > [1] https://issues.apache.org/jira/browse/IGNITE-12942 > [2] https://apacheignite-sql.readme.io/docs/create-index > [3] > > https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteSystemProperties.html#IGNITE_MAX_INDEX_PAYLOAD_SIZE > > -- > BR, Sergey Antonov >
