[jira] [Comment Edited] (IGNITE-16760) Performance degradation of IgniteTables#tables after configuration changes
[
https://issues.apache.org/jira/browse/IGNITE-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563376#comment-17563376
]
Evgeny Stanilovsky edited comment on IGNITE-16760 at 7/7/22 5:03 AM:
-
[~Denis Chudov] tests with annotation
@Disabled("https://issues.apache.org/jira/browse/IGNITE-16760;) still present
in main, can you take a look ? Additionally i still observe degradation on
annotated test, is it another known issue ?
was (Author: zstan):
[~Denis Chudov] tests with annotation
@Disabled("https://issues.apache.org/jira/browse/IGNITE-16760;) still present
in main, can you take a look ?
> Performance degradation of IgniteTables#tables after configuration changes
> --
>
> Key: IGNITE-16760
> URL: https://issues.apache.org/jira/browse/IGNITE-16760
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha4
>Reporter: Taras Ledkov
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Performance degradation of configuration changes:
> Steps to reproduce:
> 1. Start cluster with 3 nodes
> 2. Run in the loop
> {code}
> CREATE TABLE TEST(ID INTEGER PRIMARY KEY, V INTEGER)
> for (Table t : ign.tables().tables()) {
> ;
> }
> {code}
> On begin {{IgniteTables#tables}} takes ~ 0.7 sec.
> The time of the operation is grown.
> The time after ~100 iteration is about 20 sec.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-16760) Performance degradation of IgniteTables#tables after configuration changes
[
https://issues.apache.org/jira/browse/IGNITE-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17561378#comment-17561378
]
Denis Chudov edited comment on IGNITE-16760 at 7/4/22 11:05 AM:
After some discussions, we decided to make only tombstone skipping within the
work on this ticket. Other improvements will be done under other tickets (
IGNITE-17291 ).
was (Author: denis chudov):
After some discussions, we decided to make only tombstone skipping within the
work on this ticket. Other improvements will be done under other tickets (
IGNITE-16760 ).
> Performance degradation of IgniteTables#tables after configuration changes
> --
>
> Key: IGNITE-16760
> URL: https://issues.apache.org/jira/browse/IGNITE-16760
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha4
>Reporter: Taras Ledkov
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3
>
> Performance degradation of configuration changes:
> Steps to reproduce:
> 1. Start cluster with 3 nodes
> 2. Run in the loop
> {code}
> CREATE TABLE TEST(ID INTEGER PRIMARY KEY, V INTEGER)
> for (Table t : ign.tables().tables()) {
> ;
> }
> {code}
> On begin {{IgniteTables#tables}} takes ~ 0.7 sec.
> The time of the operation is grown.
> The time after ~100 iteration is about 20 sec.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-16760) Performance degradation of IgniteTables#tables after configuration changes
[
https://issues.apache.org/jira/browse/IGNITE-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17561378#comment-17561378
]
Denis Chudov edited comment on IGNITE-16760 at 7/1/22 11:10 AM:
After some discussions, we decided to make only tombstone skipping within the
work on this ticket. Other improvements will be done under other tickets (
IGNITE-16760 ).
was (Author: denis chudov):
After some discussions, we decided to make only tombstone skipping within the
work on this ticket. Other improvements will be done under other tickets.
> Performance degradation of IgniteTables#tables after configuration changes
> --
>
> Key: IGNITE-16760
> URL: https://issues.apache.org/jira/browse/IGNITE-16760
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha4
>Reporter: Taras Ledkov
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3
>
> Performance degradation of configuration changes:
> Steps to reproduce:
> 1. Start cluster with 3 nodes
> 2. Run in the loop
> {code}
> CREATE TABLE TEST(ID INTEGER PRIMARY KEY, V INTEGER)
> for (Table t : ign.tables().tables()) {
> ;
> }
> {code}
> On begin {{IgniteTables#tables}} takes ~ 0.7 sec.
> The time of the operation is grown.
> The time after ~100 iteration is about 20 sec.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-16760) Performance degradation of IgniteTables#tables after configuration changes
[
https://issues.apache.org/jira/browse/IGNITE-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17517327#comment-17517327
]
Denis Chudov edited comment on IGNITE-16760 at 4/5/22 9:23 AM:
---
The growth of duration of IgniteTables#tables method happens due to growth of
records count having prefix "table.tables" stored in distributed configuration.
They need to be read for retrieving all tables' information. See
{{TableManager#directTableIds}} which eventually calls
{{DistributedConfigurationStorage#readAllLatest}}.
There are two problems:
- cursor which is created for {{readAllLatest}} retrieves records one by one,
requesting the metastorage raft group on each {{next()}} call and receiving
SingleEntryResponse, see processing of CursorNextCommand in
MetaStorageListener. We can implement batch responses to reduce the count of
network hops;
- for this particular case, there are too many records with tombstones that are
read but seem to be useless. The cursor could skip them, possibly we can
introduce an option for creating cursor on range which should skip tombstones.
Also, the impact of such problems can be reduced when metastorage compaction
will be implemented.
was (Author: denis chudov):
The growth of duration of IgniteTables#tables method happens due to growth of
records count having prefix "table.tables" stored in distributed configuration.
They need to be read for retrieving all tables' information. See
{{TableManager#directTableIds}} which eventually calls
{{DistributedConfigurationStorage#readAllLatest}}.
There are two problems:
- cursor which is created for {{readAllLatest}} retrieves records one by one,
requesting the metastorage raft group on each {{next()}} call and receiving
SingleEntryResponse, see processing of CursorNextCommand in
MetaStorageListener. We can implement batch responses to reduce the count of
network hops;
- for this particular case, there are too many records with tombstones that are
read but seem to be useless. The cursor could skip them, possibly we can
introduce an option for creating cursor on range which should skip tombstones.
Also, the impact of such problems can be reduced when metastorage compaction
will be implemented.
> Performance degradation of IgniteTables#tables after configuration changes
> --
>
> Key: IGNITE-16760
> URL: https://issues.apache.org/jira/browse/IGNITE-16760
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha4
>Reporter: Taras Ledkov
>Priority: Major
> Labels: ignite-3
>
> Performance degradation of configuration changes:
> Steps to reproduce:
> 1. Start cluster with 3 nodes
> 2. Run in the loop
> {code}
> CREATE TABLE TEST(ID INTEGER PRIMARY KEY, V INTEGER)
> DROP TABLE TEST
> {code}
> On begin {{IgniteTables#tables}} takes ~ 0.7 sec.
> The time of the operation is grown.
> The time after ~100 iteration is about 20 sec.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (IGNITE-16760) Performance degradation of IgniteTables#tables after configuration changes
[
https://issues.apache.org/jira/browse/IGNITE-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17517327#comment-17517327
]
Denis Chudov edited comment on IGNITE-16760 at 4/5/22 9:22 AM:
---
The growth of duration of IgniteTables#tables method happens due to growth of
records count having prefix "table.tables" stored in distributed configuration.
They need to be read for retrieving all tables' information. See
{{TableManager#directTableIds}} which eventually calls
{{DistributedConfigurationStorage#readAllLatest}}.
There are two problems:
- cursor which is created for {{readAllLatest}} retrieves records one by one,
requesting the metastorage raft group on each {{next()}} call and receiving
SingleEntryResponse, see processing of CursorNextCommand in
MetaStorageListener. We can implement batch responses to reduce the count of
network hops;
- for this particular case, there are too many records with tombstones that are
read but seem to be useless. The cursor could skip them, possibly we can
introduce an option for creating cursor on range which should skip tombstones.
Also, the impact of such problems can be reduced when metastorage compaction
will be implemented.
was (Author: denis chudov):
The growth of duration of IgniteTables#tables method happens due to growth of
records count having prefix "table.tables" stored in distributed configuration.
They need to be read for retrieving all tables' information. See
`TableManager#directTableIds` which eventually calls
`DistributedConfigurationStorage#readAllLatest`.
There are two problems:
- cursor which is created for `readAllLatest` retrieves records one by one,
requesting the metastorage raft group on each `next()` call and receiving
`SingleEntryResponse`, see processing of CursorNextCommand in
MetaStorageListener. We can implement batch responses to reduce the count of
network hops;
- for this particular case, there are too many records with tombstones that are
read but seem to be useless. The cursor could skip them, possibly we can
introduce an option for creating cursor on range which should skip tombstones.
Also, the impact of such problems can be reduced when metastorage compaction
will be implemented.
> Performance degradation of IgniteTables#tables after configuration changes
> --
>
> Key: IGNITE-16760
> URL: https://issues.apache.org/jira/browse/IGNITE-16760
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha4
>Reporter: Taras Ledkov
>Priority: Major
> Labels: ignite-3
>
> Performance degradation of configuration changes:
> Steps to reproduce:
> 1. Start cluster with 3 nodes
> 2. Run in the loop
> {code}
> CREATE TABLE TEST(ID INTEGER PRIMARY KEY, V INTEGER)
> DROP TABLE TEST
> {code}
> On begin {{IgniteTables#tables}} takes ~ 0.7 sec.
> The time of the operation is grown.
> The time after ~100 iteration is about 20 sec.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
