[jira] [Commented] (IGNITE-21280) Add DATA_PAGE_FRAGMENTED_UPDATE_RECORD WAL record type placeholder
[ https://issues.apache.org/jira/browse/IGNITE-21280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811155#comment-17811155 ] Nikolay Izhikov commented on IGNITE-21280: -- Hello [~slava.koptilin], [~ktkale...@gridgain.com] Can you, please, clarify the intention of reservation? Do we have some related ticket to use new WAL record type? > Add DATA_PAGE_FRAGMENTED_UPDATE_RECORD WAL record type placeholder > -- > > Key: IGNITE-21280 > URL: https://issues.apache.org/jira/browse/IGNITE-21280 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Assignee: Vyacheslav Koptilin >Priority: Major > Fix For: 2.17 > > Time Spent: 20m > Remaining Estimate: 0h > > Reserve new WAL type for encrypted DATA_PAGE_FRAGMENTED_UPDATE_RECORD record. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21125) Prevent duplicate keys in dump
[
https://issues.apache.org/jira/browse/IGNITE-21125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-21125:
-
Fix Version/s: 2.17
> Prevent duplicate keys in dump
> --
>
> Key: IGNITE-21125
> URL: https://issues.apache.org/jira/browse/IGNITE-21125
> Project: Ignite
> Issue Type: Task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-109, ise, pull-request-available
> Fix For: 2.17
>
>
> Currently, cache dump can contain the same key several times.
> In the read time we must maintain Set of read keys to filter out duplicates.
> So dump read leads to high GC pressure.
> We can overcome this issue and filter out duplicates in the write time:
> * Iterator don't write keys that was written by {{beforeChange}} listener,
> already.
> * Partition Iterator returns key in ascending order. Order set by
> {{CacheDataTree#compare}} method.
> * {{beforeChange}} listener must not write keys that was already written by
> the iterator.
> Cheap algorithm is the following:
> * Store last key written by iterator.
> * If changed key (in {{beforeChange}}) less then last key written by iterator
> then it must be skiped. Because, it already saved by the iterator.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-21125) Prevent duplicate keys in dump
[
https://issues.apache.org/jira/browse/IGNITE-21125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-21125:
-
Description:
Currently, cache dump can contain the same key several times.
In the read time we must maintain Set of read keys to filter out duplicates.
So dump read leads to high GC pressure.
We can overcome this issue and filter out duplicates in the write time:
* Iterator don't write keys that was written by {{beforeChange}} listener,
already.
* Partition Iterator returns key in ascending order. Order set by
{{CacheDataTree#compare}} method.
* {{beforeChange}} listener must not write keys that was already written by the
iterator.
Cheap algorithm is the following:
* Store last key written by iterator.
* If changed key (in {{beforeChange}}) less then last key written by iterator
then it must be skiped. Because, it already saved by the iterator.
was:
Currently, cache dump can contain the same key several times.
In the read time we must maintain Set of read keys to filter out duplicates.
So dump read leads to high GC pressure.
We can overcome this issue and filter out duplicates in the write time:
* Iterator don't write keys that was written by {{beforeChange}} listener,
already.
* Partition Iterator returns key in ascending order. Order set by
{{CacheDataTree#compare}} method.
* {{beforeChange}} listener must not write keys that was already written by the
iterator.
Cheap algorithm is the following:
> Prevent duplicate keys in dump
> --
>
> Key: IGNITE-21125
> URL: https://issues.apache.org/jira/browse/IGNITE-21125
> Project: Ignite
> Issue Type: Task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-109, ise, pull-request-available
>
> Currently, cache dump can contain the same key several times.
> In the read time we must maintain Set of read keys to filter out duplicates.
> So dump read leads to high GC pressure.
> We can overcome this issue and filter out duplicates in the write time:
> * Iterator don't write keys that was written by {{beforeChange}} listener,
> already.
> * Partition Iterator returns key in ascending order. Order set by
> {{CacheDataTree#compare}} method.
> * {{beforeChange}} listener must not write keys that was already written by
> the iterator.
> Cheap algorithm is the following:
> * Store last key written by iterator.
> * If changed key (in {{beforeChange}}) less then last key written by iterator
> then it must be skiped. Because, it already saved by the iterator.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-21125) Prevent duplicate keys in dump
Nikolay Izhikov created IGNITE-21125:
Summary: Prevent duplicate keys in dump
Key: IGNITE-21125
URL: https://issues.apache.org/jira/browse/IGNITE-21125
Project: Ignite
Issue Type: Task
Reporter: Nikolay Izhikov
Assignee: Nikolay Izhikov
Currently, cache dump can contain the same key several times.
In the read time we must maintain Set of read keys to filter out duplicates.
So dump read leads to high GC pressure.
We can overcome this issue and filter out duplicates in the write time:
* Iterator don't write keys that was written by {{beforeChange}} listener,
already.
* Partition Iterator returns key in ascending order. Order set by
{{CacheDataTree#compare}} method.
* {{beforeChange}} listener must not write keys that was already written by the
iterator.
Cheap algorithm is the following:
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (IGNITE-21056) Use thread local buffer for encrypted dump
[ https://issues.apache.org/jira/browse/IGNITE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov resolved IGNITE-21056. -- Resolution: Fixed > Use thread local buffer for encrypted dump > -- > > Key: IGNITE-21056 > URL: https://issues.apache.org/jira/browse/IGNITE-21056 > Project: Ignite > Issue Type: Task >Reporter: Yuri Naryshkin >Assignee: Yuri Naryshkin >Priority: Minor > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 20m > Remaining Estimate: 0h > > When encrypted dump taken, expanded byte buffer doesn't replace thread local > one. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21056) Use thread local buffer for encrypted dump
[ https://issues.apache.org/jira/browse/IGNITE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-21056: - Fix Version/s: 2.16 > Use thread local buffer for encrypted dump > -- > > Key: IGNITE-21056 > URL: https://issues.apache.org/jira/browse/IGNITE-21056 > Project: Ignite > Issue Type: Task >Reporter: Yuri Naryshkin >Assignee: Yuri Naryshkin >Priority: Minor > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 10m > Remaining Estimate: 0h > > When encrypted dump taken, expanded byte buffer doesn't replace thread local > one. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20614) Use snapshot rate limiter for dumps
[
https://issues.apache.org/jira/browse/IGNITE-20614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20614:
-
Fix Version/s: 2.16
> Use snapshot rate limiter for dumps
> ---
>
> Key: IGNITE-20614
> URL: https://issues.apache.org/jira/browse/IGNITE-20614
> Project: Ignite
> Issue Type: Bug
>Reporter: Nikolay Izhikov
>Assignee: Maksim Timonin
>Priority: Major
> Labels: IEP-109, ise
> Fix For: 2.16
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Snapshots supports {{snapshotTransferRate}} distributed property to limit
> amount of bytes written by snapshot creation process to the disk.
> Dump must use same property to limit disc usage while creating dumps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-20614) Use snapshot rate limiter for dumps
[
https://issues.apache.org/jira/browse/IGNITE-20614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794653#comment-17794653
]
Nikolay Izhikov commented on IGNITE-20614:
--
Cherry-picked to 2.16
> Use snapshot rate limiter for dumps
> ---
>
> Key: IGNITE-20614
> URL: https://issues.apache.org/jira/browse/IGNITE-20614
> Project: Ignite
> Issue Type: Bug
>Reporter: Nikolay Izhikov
>Assignee: Maksim Timonin
>Priority: Major
> Labels: IEP-109, ise
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Snapshots supports {{snapshotTransferRate}} distributed property to limit
> amount of bytes written by snapshot creation process to the disk.
> Dump must use same property to limit disc usage while creating dumps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-21013) Support EncryptionSpi for Cache dump
[
https://issues.apache.org/jira/browse/IGNITE-21013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794652#comment-17794652
]
Nikolay Izhikov commented on IGNITE-21013:
--
Merged to master
Cherry-picked to 2.16
> Support EncryptionSpi for Cache dump
>
>
> Key: IGNITE-21013
> URL: https://issues.apache.org/jira/browse/IGNITE-21013
> Project: Ignite
> Issue Type: Task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-109, ise
> Fix For: 2.16
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Ignite has {{EncryptionSpi}} that provides feature to encrypt data on disk.
> Cache dumps must use this SPI to protect data in dump.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-21013) Support EncryptionSpi for Cache dump
[
https://issues.apache.org/jira/browse/IGNITE-21013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-21013:
-
Description:
Ignite has {{EncryptionSpi}} that provides feature to encrypt data on disk.
Cache dumps must use this SPI to protect data in dump.
was:
{{DumpEntry}} must contains entry version.
* Entry version is the same for every copy of the enty.
* Entry version can be used to recover CDC state after CDC based replication
implemented in cdc-ext fails for some reason.
Use-case:
* cdc replication fails.
* backup cluster cleared.
* starts cdc -> changes replicated to the backup cluster.
* create dump on the primary cluster.
* restore dump on the backup cluster with the {{putConflict}} calls.
Goal of this ticket is to add version to the {{DumpEntry}}
> Support EncryptionSpi for Cache dump
>
>
> Key: IGNITE-21013
> URL: https://issues.apache.org/jira/browse/IGNITE-21013
> Project: Ignite
> Issue Type: Task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-109, ise
> Fix For: 2.16
>
>
> Ignite has {{EncryptionSpi}} that provides feature to encrypt data on disk.
> Cache dumps must use this SPI to protect data in dump.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-21013) Support EncryptionSpi for Cache dump
Nikolay Izhikov created IGNITE-21013:
Summary: Support EncryptionSpi for Cache dump
Key: IGNITE-21013
URL: https://issues.apache.org/jira/browse/IGNITE-21013
Project: Ignite
Issue Type: Task
Reporter: Nikolay Izhikov
Assignee: Nikolay Izhikov
Fix For: 2.16
{{DumpEntry}} must contains entry version.
* Entry version is the same for every copy of the enty.
* Entry version can be used to recover CDC state after CDC based replication
implemented in cdc-ext fails for some reason.
Use-case:
* cdc replication fails.
* backup cluster cleared.
* starts cdc -> changes replicated to the backup cluster.
* create dump on the primary cluster.
* restore dump on the backup cluster with the {{putConflict}} calls.
Goal of this ticket is to add version to the {{DumpEntry}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-20917) Add entry version in the DumpEntry
[
https://issues.apache.org/jira/browse/IGNITE-20917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790052#comment-17790052
]
Nikolay Izhikov commented on IGNITE-20917:
--
cherry-picked to 2.16
> Add entry version in the DumpEntry
> --
>
> Key: IGNITE-20917
> URL: https://issues.apache.org/jira/browse/IGNITE-20917
> Project: Ignite
> Issue Type: Task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-109, ise
> Fix For: 2.16
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{DumpEntry}} must contains entry version.
> * Entry version is the same for every copy of the enty.
> * Entry version can be used to recover CDC state after CDC based replication
> implemented in cdc-ext fails for some reason.
> Use-case:
> * cdc replication fails.
> * backup cluster cleared.
> * starts cdc -> changes replicated to the backup cluster.
> * create dump on the primary cluster.
> * restore dump on the backup cluster with the {{putConflict}} calls.
> Goal of this ticket is to add version to the {{DumpEntry}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-20836) Support zipping of dump files
[ https://issues.apache.org/jira/browse/IGNITE-20836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790051#comment-17790051 ] Nikolay Izhikov commented on IGNITE-20836: -- cherry-picked to 2.16 > Support zipping of dump files > - > > Key: IGNITE-20836 > URL: https://issues.apache.org/jira/browse/IGNITE-20836 > Project: Ignite > Issue Type: Task >Reporter: Yuri Naryshkin >Assignee: Yuri Naryshkin >Priority: Major > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 8h 50m > Remaining Estimate: 0h > > For additional space saving, need to introduce a mode in which dump files are > compressed during the creation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20836) Support zipping of dump files
[ https://issues.apache.org/jira/browse/IGNITE-20836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20836: - Fix Version/s: 2.16 (was: 2.17) > Support zipping of dump files > - > > Key: IGNITE-20836 > URL: https://issues.apache.org/jira/browse/IGNITE-20836 > Project: Ignite > Issue Type: Task >Reporter: Yuri Naryshkin >Assignee: Yuri Naryshkin >Priority: Major > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 8h 50m > Remaining Estimate: 0h > > For additional space saving, need to introduce a mode in which dump files are > compressed during the creation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20917) Add entry version in the DumpEntry
[
https://issues.apache.org/jira/browse/IGNITE-20917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20917:
-
Description:
{{DumpEntry}} must contains entry version.
* Entry version is the same for every copy of the enty.
* Entry version can be used to recover CDC state after CDC based replication
implemented in cdc-ext fails for some reason.
Use-case:
* cdc replication fails.
* backup cluster cleared.
* starts cdc -> changes replicated to the backup cluster.
* create dump on the primary cluster.
* restore dump on the backup cluster with the {{putConflict}} calls.
Goal of this ticket is to add version to the {{DumpEntry}}
was:
{{DumpEntry}} must contains entry version.
* Entry version is the same for every copy of the enty.
* Entry version can be used to recover CDC state after CDC based replication
implemented in cdc-ext fails for some reason.
Use-case:
* cdc replication fails.
* backup cluster cleared.
* starts cdc -> changes replicated to the backup cluster.
* create dump on the primary cluster.
* restore dump on the backup cluster with the {{putConflict}} calls.
> Add entry version in the DumpEntry
> --
>
> Key: IGNITE-20917
> URL: https://issues.apache.org/jira/browse/IGNITE-20917
> Project: Ignite
> Issue Type: Task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-109, ise
> Fix For: 2.16
>
>
> {{DumpEntry}} must contains entry version.
> * Entry version is the same for every copy of the enty.
> * Entry version can be used to recover CDC state after CDC based replication
> implemented in cdc-ext fails for some reason.
> Use-case:
> * cdc replication fails.
> * backup cluster cleared.
> * starts cdc -> changes replicated to the backup cluster.
> * create dump on the primary cluster.
> * restore dump on the backup cluster with the {{putConflict}} calls.
> Goal of this ticket is to add version to the {{DumpEntry}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20917) Add entry version in the DumpEntry
[
https://issues.apache.org/jira/browse/IGNITE-20917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20917:
-
Description:
{{DumpEntry}} must contains entry version.
* Entry version is the same for every copy of the enty.
* Entry version can be used to recover CDC state after CDC based replication
implemented in cdc-ext fails for some reason.
Use-case:
* cdc replication fails.
* backup cluster cleared.
* starts cdc -> changes replicated to the backup cluster.
* create dump on the primary cluster.
* restore dump on the backup cluster with the {{putConflict}} calls.
was:Don't create snapshot directories for in-memory cluster until first dump
created.
> Add entry version in the DumpEntry
> --
>
> Key: IGNITE-20917
> URL: https://issues.apache.org/jira/browse/IGNITE-20917
> Project: Ignite
> Issue Type: Task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-109, ise
> Fix For: 2.16
>
>
> {{DumpEntry}} must contains entry version.
> * Entry version is the same for every copy of the enty.
> * Entry version can be used to recover CDC state after CDC based replication
> implemented in cdc-ext fails for some reason.
> Use-case:
> * cdc replication fails.
> * backup cluster cleared.
> * starts cdc -> changes replicated to the backup cluster.
> * create dump on the primary cluster.
> * restore dump on the backup cluster with the {{putConflict}} calls.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20917) Add entry version in the DumpEntry
Nikolay Izhikov created IGNITE-20917: Summary: Add entry version in the DumpEntry Key: IGNITE-20917 URL: https://issues.apache.org/jira/browse/IGNITE-20917 Project: Ignite Issue Type: Task Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.16 Don't create snapshot directories for in-memory cluster until first dump created. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20722) Support skipCopies for DumpReader
[ https://issues.apache.org/jira/browse/IGNITE-20722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20722: - Fix Version/s: 2.16 > Support skipCopies for DumpReader > - > > Key: IGNITE-20722 > URL: https://issues.apache.org/jira/browse/IGNITE-20722 > Project: Ignite > Issue Type: Task >Reporter: Yuri Naryshkin >Assignee: Yuri Naryshkin >Priority: Minor > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 1h > Remaining Estimate: 0h > > Dump can store several copy of each partition. > DumpReader must provide an option to skip copies and process only one copy > for each partition. > This will reduce restore time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-20718) DumpIterator for primary copies
[
https://issues.apache.org/jira/browse/IGNITE-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov resolved IGNITE-20718.
--
Resolution: Invalid
> DumpIterator for primary copies
> ---
>
> Key: IGNITE-20718
> URL: https://issues.apache.org/jira/browse/IGNITE-20718
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-109, ise
>
> Primary copies of partitions don't contain entry duplicates.
> Because, entries can be filtered based on version on the creation time.
> So, during iteration we don't need to track possible duplicates inside
> {{DumpedPartitionIterator}} implementation in {{partKeys}} variable. This
> will decrease memory consumption and improve performance of dump iteration
> and therefore dump check procedures.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20708) Support cacheGroupNames for DumpReader
[ https://issues.apache.org/jira/browse/IGNITE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20708: - Fix Version/s: 2.16 > Support cacheGroupNames for DumpReader > -- > > Key: IGNITE-20708 > URL: https://issues.apache.org/jira/browse/IGNITE-20708 > Project: Ignite > Issue Type: Task >Reporter: Yuri Naryshkin >Assignee: Yuri Naryshkin >Priority: Minor > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > Need to support cacheGroupNames option to filter out cache groups that will > be analyzed when the DumpReader runs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20708) Support cacheGroupNames for DumpReader
[ https://issues.apache.org/jira/browse/IGNITE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779385#comment-17779385 ] Nikolay Izhikov commented on IGNITE-20708: -- [~yuri.naryshkin] merged to master. Thanks for the contribution! > Support cacheGroupNames for DumpReader > -- > > Key: IGNITE-20708 > URL: https://issues.apache.org/jira/browse/IGNITE-20708 > Project: Ignite > Issue Type: Task >Reporter: Yuri Naryshkin >Assignee: Yuri Naryshkin >Priority: Minor > Labels: IEP-109, ise > Time Spent: 2h 40m > Remaining Estimate: 0h > > Need to support cacheGroupNames option to filter out cache groups that will > be analyzed when the DumpReader runs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20718) DumpIterator for primary copies
[
https://issues.apache.org/jira/browse/IGNITE-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20718:
-
Issue Type: Improvement (was: Bug)
> DumpIterator for primary copies
> ---
>
> Key: IGNITE-20718
> URL: https://issues.apache.org/jira/browse/IGNITE-20718
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-109, ise
>
> Primary copies of partitions don't contain entry duplicates.
> Because, entries can be filtered based on version on the creation time.
> So, during iteration we don't need to track possible duplicates inside
> {{DumpedPartitionIterator}} implementation in {{partKeys}} variable. This
> will decrease memory consumption and improve performance of dump iteration
> and therefore dump check procedures.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20718) DumpIterator for primary copies
[
https://issues.apache.org/jira/browse/IGNITE-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20718:
-
Description:
Primary copies of partitions don't contain entry duplicates.
Because, entries can be filtered based on version on the creation time.
So, during iteration we don't need to track possible duplicates inside
{{DumpedPartitionIterator}} implementation in {{partKeys}} variable. This will
decrease memory consumption and improve performance of dump iteration and
therefore dump check procedures.
was:
Snapshots supports {{snapshotTransferRate}} distributed property to limit
amount of bytes written by snapshot creation process to the disk.
Dump must use same property to limit disc usage while creating dumps.
> DumpIterator for primary copies
> ---
>
> Key: IGNITE-20718
> URL: https://issues.apache.org/jira/browse/IGNITE-20718
> Project: Ignite
> Issue Type: Bug
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-109, ise
>
> Primary copies of partitions don't contain entry duplicates.
> Because, entries can be filtered based on version on the creation time.
> So, during iteration we don't need to track possible duplicates inside
> {{DumpedPartitionIterator}} implementation in {{partKeys}} variable. This
> will decrease memory consumption and improve performance of dump iteration
> and therefore dump check procedures.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20718) DumpIterator for primary copies
Nikolay Izhikov created IGNITE-20718:
Summary: DumpIterator for primary copies
Key: IGNITE-20718
URL: https://issues.apache.org/jira/browse/IGNITE-20718
Project: Ignite
Issue Type: Bug
Reporter: Nikolay Izhikov
Assignee: Nikolay Izhikov
Snapshots supports {{snapshotTransferRate}} distributed property to limit
amount of bytes written by snapshot creation process to the disk.
Dump must use same property to limit disc usage while creating dumps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20528) CDC doesn't work if the "Cache objects transformation" is applied
[
https://issues.apache.org/jira/browse/IGNITE-20528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20528:
-
Fix Version/s: 2.16
> CDC doesn't work if the "Cache objects transformation" is applied
> -
>
> Key: IGNITE-20528
> URL: https://issues.apache.org/jira/browse/IGNITE-20528
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Korotkov
>Assignee: Anton Vinogradov
>Priority: Major
> Labels: iep-97, ise
> Fix For: 2.16
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> CDC doesn't work If some cache objects transformation is applied (see the
> [https://cwiki.apache.org/confluence/display/IGNITE/IEP-97+Cache+objects+transformation|https://cwiki.apache.org/confluence/display/IGNITE/IEP-97+Cache+objects+transformation]).
> ignite_cdc.sh utility produces the NPE (see below). The immediate reason of
> the NPE is that ignite_cdc.sh uses the reduced version of the context
> (StandaloneGridKernalContext), which doesn't contain the GridCacheProcessor.
>
> {noformat}
> [2023-10-02T10:43:32,017][ERROR][Thread-1][] Unable to convert value
> [CacheObjectImpl [val=null, hasValBytes=true]]
> java.lang.NullPointerException: null
> at
> org.apache.ignite.internal.processors.cache.CacheObjectTransformerUtils.transformer(CacheObjectTransformerUtils.java:32)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.CacheObjectTransformerUtils.restoreIfNecessary(CacheObjectTransformerUtils.java:120)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.CacheObjectAdapter.valueFromValueBytes(CacheObjectAdapter.java:73)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.CacheObjectImpl.value(CacheObjectImpl.java:92)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.CacheObjectImpl.value(CacheObjectImpl.java:58)
> ~[classes/:?]
> at
> org.apache.ignite.internal.pagemem.wal.record.UnwrapDataEntry.unwrappedValue(UnwrapDataEntry.java:104)
> ~[classes/:?]
> at
> org.apache.ignite.internal.cdc.WalRecordsConsumer.lambda$static$c56580e2$1(WalRecordsConsumer.java:99)
> ~[classes/:?]
> at
> org.apache.ignite.internal.util.lang.gridfunc.TransformFilteringIterator.nextX(TransformFilteringIterator.java:119)
> [classes/:?]
> at
> org.apache.ignite.internal.util.lang.GridIteratorAdapter.next(GridIteratorAdapter.java:35)
> [classes/:?]
> at
> org.apache.ignite.internal.util.lang.gridfunc.TransformFilteringIterator.hasNextX(TransformFilteringIterator.java:85)
> [classes/:?]
> at
> org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
> [classes/:?]
> at
> org.apache.ignite.cdc.AbstractCdcEventsApplier.apply(AbstractCdcEventsApplier.java:71)
> [ignite-cdc-ext-1.0.0-SNAPSHOT.jar:?]
> at
> org.apache.ignite.cdc.AbstractIgniteCdcStreamer.onEvents(AbstractIgniteCdcStreamer.java:118)
> [ignite-cdc-ext-1.0.0-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:142)
> [classes/:?]
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeSegmentActively(CdcMain.java:557)
> [classes/:?]
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:496)
> [classes/:?]
> at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:344)
> [classes/:?]
> at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:283) [classes/:?]
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20708) Support cacheGroupNames for DumpReader
[ https://issues.apache.org/jira/browse/IGNITE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20708: - Description: Need to support cacheGroupNames option to filter out cache groups that will be analyzed when the DumpReader runs. (was: Need to support --cache-group-names option to filter out cache groups that will be analyzed when the utility runs, so that administrator can restore only specific caches) > Support cacheGroupNames for DumpReader > -- > > Key: IGNITE-20708 > URL: https://issues.apache.org/jira/browse/IGNITE-20708 > Project: Ignite > Issue Type: Task >Reporter: Yuri Naryshkin >Assignee: Yuri Naryshkin >Priority: Minor > Labels: IEP-109, ise > Time Spent: 10m > Remaining Estimate: 0h > > Need to support cacheGroupNames option to filter out cache groups that will > be analyzed when the DumpReader runs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20708) Support --cache-group-names for restore utility
[ https://issues.apache.org/jira/browse/IGNITE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20708: - Labels: IEP-109 ise (was: ise) > Support --cache-group-names for restore utility > --- > > Key: IGNITE-20708 > URL: https://issues.apache.org/jira/browse/IGNITE-20708 > Project: Ignite > Issue Type: Task >Reporter: Yuri Naryshkin >Assignee: Yuri Naryshkin >Priority: Minor > Labels: IEP-109, ise > Time Spent: 10m > Remaining Estimate: 0h > > Need to support --cache-group-names option to filter out cache groups that > will be analyzed when the utility runs, so that administrator can restore > only specific caches -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20708) Support --cache-group-names for restore utility
[ https://issues.apache.org/jira/browse/IGNITE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20708: - Labels: ise (was: ) > Support --cache-group-names for restore utility > --- > > Key: IGNITE-20708 > URL: https://issues.apache.org/jira/browse/IGNITE-20708 > Project: Ignite > Issue Type: Task >Reporter: Yuri Naryshkin >Assignee: Yuri Naryshkin >Priority: Minor > Labels: ise > Time Spent: 10m > Remaining Estimate: 0h > > Need to support --cache-group-names option to filter out cache groups that > will be analyzed when the utility runs, so that administrator can restore > only specific caches -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20708) Support cacheGroupNames for DumpReader
[ https://issues.apache.org/jira/browse/IGNITE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20708: - Summary: Support cacheGroupNames for DumpReader (was: Support --cache-group-names for restore utility) > Support cacheGroupNames for DumpReader > -- > > Key: IGNITE-20708 > URL: https://issues.apache.org/jira/browse/IGNITE-20708 > Project: Ignite > Issue Type: Task >Reporter: Yuri Naryshkin >Assignee: Yuri Naryshkin >Priority: Minor > Labels: IEP-109, ise > Time Spent: 10m > Remaining Estimate: 0h > > Need to support --cache-group-names option to filter out cache groups that > will be analyzed when the utility runs, so that administrator can restore > only specific caches -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20392) Move cassandra module to extensions
[ https://issues.apache.org/jira/browse/IGNITE-20392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov reassigned IGNITE-20392: Assignee: Nikolay Izhikov > Move cassandra module to extensions > --- > > Key: IGNITE-20392 > URL: https://issues.apache.org/jira/browse/IGNITE-20392 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Daschinsky >Assignee: Nikolay Izhikov >Priority: Major > Fix For: 2.16 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20470) Ducktape to check dump performance
[ https://issues.apache.org/jira/browse/IGNITE-20470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20470: - Fix Version/s: 2.16 > Ducktape to check dump performance > -- > > Key: IGNITE-20470 > URL: https://issues.apache.org/jira/browse/IGNITE-20470 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Sergey Korotkov >Priority: Major > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 1h > Remaining Estimate: 0h > > Dump creation can affect transactions performance with change listener and > disc operations. We must create ducktape test to check this. > Example test scenario: > * Start nodes > * Start transaction operations: insert, update, remove. > * Create dump > * Check dump consistency. > Measure > * Transaction performance penalty. > * GC utilization. > * Disc utilization. > Params: > * All types of caches(ATOMIC, TRANSACTIONAL) must be tested. > * In-memory caches are primary use-case, but persistence has to be tested, > also. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20470) Ducktape to check dump performance
[ https://issues.apache.org/jira/browse/IGNITE-20470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776622#comment-17776622 ] Nikolay Izhikov commented on IGNITE-20470: -- Merged to master. [~serge.korotkov] Thanks for contribution! > Ducktape to check dump performance > -- > > Key: IGNITE-20470 > URL: https://issues.apache.org/jira/browse/IGNITE-20470 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Sergey Korotkov >Priority: Major > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 1h > Remaining Estimate: 0h > > Dump creation can affect transactions performance with change listener and > disc operations. We must create ducktape test to check this. > Example test scenario: > * Start nodes > * Start transaction operations: insert, update, remove. > * Create dump > * Check dump consistency. > Measure > * Transaction performance penalty. > * GC utilization. > * Disc utilization. > Params: > * All types of caches(ATOMIC, TRANSACTIONAL) must be tested. > * In-memory caches are primary use-case, but persistence has to be tested, > also. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20566) CDC doesn't replicate complex objects when keepBinary is set to the false
[
https://issues.apache.org/jira/browse/IGNITE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov reassigned IGNITE-20566:
Assignee: (was: Nikolay Izhikov)
> CDC doesn't replicate complex objects when keepBinary is set to the false
> -
>
> Key: IGNITE-20566
> URL: https://issues.apache.org/jira/browse/IGNITE-20566
> Project: Ignite
> Issue Type: Bug
>Reporter: Anton Vinogradov
>Priority: Major
>
> To reproduce just change
> {{org.apache.ignite.cdc.CdcConfiguration#DFLT_KEEP_BINARY}} to the {{false}}.
> {{org.apache.ignite.cdc.AbstractReplicationTest#testActivePassiveReplication}}
> still will be successfull since uses promitive key/val.
> {{org.apache.ignite.cdc.AbstractReplicationTest#testActivePassiveReplicationComplexKeyWithKeyValue}}
> will stuck, transaction on destination cluster will never be finished.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20601) Support custom location for dumps
[ https://issues.apache.org/jira/browse/IGNITE-20601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20601: - Fix Version/s: 2.16 > Support custom location for dumps > - > > Key: IGNITE-20601 > URL: https://issues.apache.org/jira/browse/IGNITE-20601 > Project: Ignite > Issue Type: Bug >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 20m > Remaining Estimate: 0h > > User may want to create cache dump in any specific folder, like regular PDS > snapshots. > Ignite need to support this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20597) Support cache group filter for dump
[ https://issues.apache.org/jira/browse/IGNITE-20597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20597: - Fix Version/s: 2.16 > Support cache group filter for dump > --- > > Key: IGNITE-20597 > URL: https://issues.apache.org/jira/browse/IGNITE-20597 > Project: Ignite > Issue Type: Bug >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 40m > Remaining Estimate: 0h > > User may want to create cache dump only for specific set of cache groups. > Ignite need to support this -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20470) Ducktape to check dump performance
[ https://issues.apache.org/jira/browse/IGNITE-20470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20470: - Description: Dump creation can affect transactions performance with change listener and disc operations. We must create ducktape test to check this. Example test scenario: * Start nodes * Start transaction operations: insert, update, remove. * Create dump * Check dump consistency. Measure * Transaction performance penalty. * GC utilization. * Disc utilization. Params: * All types of caches(ATOMIC, TRANSACTIONAL) must be tested. * In-memory caches are primary use-case, but persistence has to be tested, also. was: Dump creation can affect transactions performance with change listener and disc operations. We must create ducktape test to check this. Example test scenario: * Start nodes * Start transaction operations: insert, update, remove. * Create dump * Check dump consistency. Measure * Transaction performance penalty. * GC utilization. * Disc utilization. > Ducktape to check dump performance > -- > > Key: IGNITE-20470 > URL: https://issues.apache.org/jira/browse/IGNITE-20470 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Sergey Korotkov >Priority: Major > Labels: IEP-109, ise > Time Spent: 0.5h > Remaining Estimate: 0h > > Dump creation can affect transactions performance with change listener and > disc operations. We must create ducktape test to check this. > Example test scenario: > * Start nodes > * Start transaction operations: insert, update, remove. > * Create dump > * Check dump consistency. > Measure > * Transaction performance penalty. > * GC utilization. > * Disc utilization. > Params: > * All types of caches(ATOMIC, TRANSACTIONAL) must be tested. > * In-memory caches are primary use-case, but persistence has to be tested, > also. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20614) Use snapshot rate limiter for dumps
[
https://issues.apache.org/jira/browse/IGNITE-20614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20614:
-
Description:
Snapshots supports {{snapshotTransferRate}} distributed property to limit
amount of bytes written by snapshot creation process to the disk.
Dump must use same property to limit disc usage while creating dumps.
was:
User may want to create cache dump in any specific folder, like regular PDS
snapshots.
Ignite need to support this.
> Use snapshot rate limiter for dumps
> ---
>
> Key: IGNITE-20614
> URL: https://issues.apache.org/jira/browse/IGNITE-20614
> Project: Ignite
> Issue Type: Bug
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-109, ise
>
> Snapshots supports {{snapshotTransferRate}} distributed property to limit
> amount of bytes written by snapshot creation process to the disk.
> Dump must use same property to limit disc usage while creating dumps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20614) Use snapshot rate limiter for dumps
Nikolay Izhikov created IGNITE-20614: Summary: Use snapshot rate limiter for dumps Key: IGNITE-20614 URL: https://issues.apache.org/jira/browse/IGNITE-20614 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov User may want to create cache dump in any specific folder, like regular PDS snapshots. Ignite need to support this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20589) Lazy creation of snapshot dir for in-memory cluster
[ https://issues.apache.org/jira/browse/IGNITE-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20589: - Fix Version/s: 2.16 > Lazy creation of snapshot dir for in-memory cluster > --- > > Key: IGNITE-20589 > URL: https://issues.apache.org/jira/browse/IGNITE-20589 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Don't create snapshot directories for in-memory cluster until first dump > created. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20595) Split snapshot suite into three
[ https://issues.apache.org/jira/browse/IGNITE-20595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20595: - Fix Version/s: 2.16 > Split snapshot suite into three > --- > > Key: IGNITE-20595 > URL: https://issues.apache.org/jira/browse/IGNITE-20595 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 20m > Remaining Estimate: 0h > > Right now snapshot suite runs 1.5 hours or even longer. > We need split it to reduce Run All timing -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20595) Split snapshot suite into three
[ https://issues.apache.org/jira/browse/IGNITE-20595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20595: - Summary: Split snapshot suite into three (was: Split snapshot suite into two) > Split snapshot suite into three > --- > > Key: IGNITE-20595 > URL: https://issues.apache.org/jira/browse/IGNITE-20595 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > Time Spent: 10m > Remaining Estimate: 0h > > Right now snapshot suite runs 1.5 hours or even longer. > We need split it to reduce Run All timing -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20472) API to consume cache dumps
[ https://issues.apache.org/jira/browse/IGNITE-20472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20472: - Summary: API to consume cache dumps (was: API to consumer cache dumps) > API to consume cache dumps > -- > > Key: IGNITE-20472 > URL: https://issues.apache.org/jira/browse/IGNITE-20472 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 20m > Remaining Estimate: 0h > > Ignite must provide API to consume cache dumps by user defined consumer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20580) Support only primary mode
[ https://issues.apache.org/jira/browse/IGNITE-20580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20580: - Fix Version/s: 2.16 > Support only primary mode > - > > Key: IGNITE-20580 > URL: https://issues.apache.org/jira/browse/IGNITE-20580 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > Fix For: 2.16 > > > Support only-primary mode during cache dump createion -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20472) API to consumer cache dumps
[ https://issues.apache.org/jira/browse/IGNITE-20472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20472: - Fix Version/s: 2.16 > API to consumer cache dumps > --- > > Key: IGNITE-20472 > URL: https://issues.apache.org/jira/browse/IGNITE-20472 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 20m > Remaining Estimate: 0h > > Ignite must provide API to consume cache dumps by user defined consumer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20429) Support dump check command
[ https://issues.apache.org/jira/browse/IGNITE-20429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20429: - Fix Version/s: 2.16 > Support dump check command > -- > > Key: IGNITE-20429 > URL: https://issues.apache.org/jira/browse/IGNITE-20429 > Project: Ignite > Issue Type: Bug >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > Fix For: 2.16 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20601) Support custom location for dumps
[ https://issues.apache.org/jira/browse/IGNITE-20601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20601: - Description: User may want to create cache dump in any specific folder, like regular PDS snapshots. Ignite need to support this. was: User may want to create cache dump only for specific set of cache groups. Ignite need to support this > Support custom location for dumps > - > > Key: IGNITE-20601 > URL: https://issues.apache.org/jira/browse/IGNITE-20601 > Project: Ignite > Issue Type: Bug >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > > User may want to create cache dump in any specific folder, like regular PDS > snapshots. > Ignite need to support this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20601) Support custom location for dumps
Nikolay Izhikov created IGNITE-20601: Summary: Support custom location for dumps Key: IGNITE-20601 URL: https://issues.apache.org/jira/browse/IGNITE-20601 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov User may want to create cache dump only for specific set of cache groups. Ignite need to support this -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19950) Cache dumps
[ https://issues.apache.org/jira/browse/IGNITE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-19950: - Fix Version/s: 2.16 > Cache dumps > --- > > Key: IGNITE-19950 > URL: https://issues.apache.org/jira/browse/IGNITE-19950 > Project: Ignite > Issue Type: New Feature >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, iep-43, ise > Fix For: 2.16 > > Time Spent: 6.5h > Remaining Estimate: 0h > > Ignite can create snapshot for persistent cache groups only. > It will be very usefull for a user to have ability to create and restore > snapshot of in-memory cache groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20597) Support cache group filter for dump
[ https://issues.apache.org/jira/browse/IGNITE-20597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20597: - Description: User may want to create cache dump only for specific set of cache groups. Ignite need to support this > Support cache group filter for dump > --- > > Key: IGNITE-20597 > URL: https://issues.apache.org/jira/browse/IGNITE-20597 > Project: Ignite > Issue Type: Bug >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > > User may want to create cache dump only for specific set of cache groups. > Ignite need to support this -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20597) Support cache group filter for dump
Nikolay Izhikov created IGNITE-20597: Summary: Support cache group filter for dump Key: IGNITE-20597 URL: https://issues.apache.org/jira/browse/IGNITE-20597 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20596) Support dump cancel command
[ https://issues.apache.org/jira/browse/IGNITE-20596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20596: - Summary: Support dump cancel command (was: Support dump status command) > Support dump cancel command > --- > > Key: IGNITE-20596 > URL: https://issues.apache.org/jira/browse/IGNITE-20596 > Project: Ignite > Issue Type: Bug >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20596) Support dump status command
Nikolay Izhikov created IGNITE-20596: Summary: Support dump status command Key: IGNITE-20596 URL: https://issues.apache.org/jira/browse/IGNITE-20596 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20595) Split snapshot suite into two
[ https://issues.apache.org/jira/browse/IGNITE-20595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20595: - Description: Right now snapshot suite runs 1.5 hours or even longer. We need split it to reduce Run All timing was:Don't create snapshot directories for in-memory cluster until first dump created. > Split snapshot suite into two > - > > Key: IGNITE-20595 > URL: https://issues.apache.org/jira/browse/IGNITE-20595 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > > Right now snapshot suite runs 1.5 hours or even longer. > We need split it to reduce Run All timing -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20595) Split snapshot suite into two
Nikolay Izhikov created IGNITE-20595: Summary: Split snapshot suite into two Key: IGNITE-20595 URL: https://issues.apache.org/jira/browse/IGNITE-20595 Project: Ignite Issue Type: Task Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Don't create snapshot directories for in-memory cluster until first dump created. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20594) Replace ThreadLocal ByteBuffer with PooledAllocator
Nikolay Izhikov created IGNITE-20594: Summary: Replace ThreadLocal ByteBuffer with PooledAllocator Key: IGNITE-20594 URL: https://issues.apache.org/jira/browse/IGNITE-20594 Project: Ignite Issue Type: Task Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Current approach uses thread local ByteBuffer to write data while dumping caches. This can lead to memory overhead when only some threads from pool modifies data (call writeChanged) and for each thread separate ByteBuffer allocated. We have PooledAllocator that can provide ability to reduce memory overhead. Need to study if it can be used in this case. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-19950) Cache dumps
[ https://issues.apache.org/jira/browse/IGNITE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767918#comment-17767918 ] Nikolay Izhikov edited comment on IGNITE-19950 at 10/6/23 2:28 PM: --- Merged to cache_dumps branch. was (Author: nizhikov): Merged on cache_dumps branch. > Cache dumps > --- > > Key: IGNITE-19950 > URL: https://issues.apache.org/jira/browse/IGNITE-19950 > Project: Ignite > Issue Type: New Feature >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, iep-43, ise > Time Spent: 3.5h > Remaining Estimate: 0h > > Ignite can create snapshot for persistent cache groups only. > It will be very usefull for a user to have ability to create and restore > snapshot of in-memory cache groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (IGNITE-19950) Snapshot of in-memory cache groups
[ https://issues.apache.org/jira/browse/IGNITE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov reopened IGNITE-19950: -- > Snapshot of in-memory cache groups > -- > > Key: IGNITE-19950 > URL: https://issues.apache.org/jira/browse/IGNITE-19950 > Project: Ignite > Issue Type: New Feature >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, iep-43, ise > Time Spent: 3.5h > Remaining Estimate: 0h > > Ignite can create snapshot for persistent cache groups only. > It will be very usefull for a user to have ability to create and restore > snapshot of in-memory cache groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19950) Cache dumps
[ https://issues.apache.org/jira/browse/IGNITE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-19950: - Summary: Cache dumps (was: Snapshot of in-memory cache groups) > Cache dumps > --- > > Key: IGNITE-19950 > URL: https://issues.apache.org/jira/browse/IGNITE-19950 > Project: Ignite > Issue Type: New Feature >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, iep-43, ise > Time Spent: 3.5h > Remaining Estimate: 0h > > Ignite can create snapshot for persistent cache groups only. > It will be very usefull for a user to have ability to create and restore > snapshot of in-memory cache groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20589) Lazy creation of snapshot dir for in-memory cluster
[ https://issues.apache.org/jira/browse/IGNITE-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20589: - Description: Don't create snapshot directories for in-memory cluster until first dump created. (was: Support only-primary mode during cache dump createion) > Lazy creation of snapshot dir for in-memory cluster > --- > > Key: IGNITE-20589 > URL: https://issues.apache.org/jira/browse/IGNITE-20589 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > > Don't create snapshot directories for in-memory cluster until first dump > created. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20589) Lazy creation of snapshot dir for in-memory cluster
Nikolay Izhikov created IGNITE-20589: Summary: Lazy creation of snapshot dir for in-memory cluster Key: IGNITE-20589 URL: https://issues.apache.org/jira/browse/IGNITE-20589 Project: Ignite Issue Type: Task Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Support only-primary mode during cache dump createion -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-20580) Support only primary mode
[ https://issues.apache.org/jira/browse/IGNITE-20580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov resolved IGNITE-20580. -- Resolution: Fixed > Support only primary mode > - > > Key: IGNITE-20580 > URL: https://issues.apache.org/jira/browse/IGNITE-20580 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > > Support only-primary mode during cache dump createion -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20580) Support only primary mode
[ https://issues.apache.org/jira/browse/IGNITE-20580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20580: - Labels: IEP-109 ise (was: ) > Support only primary mode > - > > Key: IGNITE-20580 > URL: https://issues.apache.org/jira/browse/IGNITE-20580 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > > Support only-primary mode during cache dump createion -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20580) Support only primary mode
Nikolay Izhikov created IGNITE-20580: Summary: Support only primary mode Key: IGNITE-20580 URL: https://issues.apache.org/jira/browse/IGNITE-20580 Project: Ignite Issue Type: Task Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Support only-primary mode during cache dump createion -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20528) CDC doesn't work if the "Cache objects transformation" is applied
[
https://issues.apache.org/jira/browse/IGNITE-20528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20528:
-
Labels: ise (was: )
> CDC doesn't work if the "Cache objects transformation" is applied
> -
>
> Key: IGNITE-20528
> URL: https://issues.apache.org/jira/browse/IGNITE-20528
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Korotkov
>Priority: Major
> Labels: ise
>
> CDC doesn't work If some cache objects transformation is applied (see the
> [https://cwiki.apache.org/confluence/display/IGNITE/IEP-97+Cache+objects+transformation).|https://cwiki.apache.org/confluence/display/IGNITE/IEP-97+Cache+objects+transformation)]
> ignite_cdc.sh utility produces the NPE (see below). The immediate reason of
> the NPE is that ignite_cdc.sh uses the reduced version of the context
> (StandaloneGridKernalContext), which doesn't contain the GridCacheProcessor.
>
> {noformat}
> [2023-10-02T10:43:32,017][ERROR][Thread-1][] Unable to convert value
> [CacheObjectImpl [val=null, hasValBytes=true]]
> java.lang.NullPointerException: null
> at
> org.apache.ignite.internal.processors.cache.CacheObjectTransformerUtils.transformer(CacheObjectTransformerUtils.java:32)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.CacheObjectTransformerUtils.restoreIfNecessary(CacheObjectTransformerUtils.java:120)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.CacheObjectAdapter.valueFromValueBytes(CacheObjectAdapter.java:73)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.CacheObjectImpl.value(CacheObjectImpl.java:92)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.CacheObjectImpl.value(CacheObjectImpl.java:58)
> ~[classes/:?]
> at
> org.apache.ignite.internal.pagemem.wal.record.UnwrapDataEntry.unwrappedValue(UnwrapDataEntry.java:104)
> ~[classes/:?]
> at
> org.apache.ignite.internal.cdc.WalRecordsConsumer.lambda$static$c56580e2$1(WalRecordsConsumer.java:99)
> ~[classes/:?]
> at
> org.apache.ignite.internal.util.lang.gridfunc.TransformFilteringIterator.nextX(TransformFilteringIterator.java:119)
> [classes/:?]
> at
> org.apache.ignite.internal.util.lang.GridIteratorAdapter.next(GridIteratorAdapter.java:35)
> [classes/:?]
> at
> org.apache.ignite.internal.util.lang.gridfunc.TransformFilteringIterator.hasNextX(TransformFilteringIterator.java:85)
> [classes/:?]
> at
> org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
> [classes/:?]
> at
> org.apache.ignite.cdc.AbstractCdcEventsApplier.apply(AbstractCdcEventsApplier.java:71)
> [ignite-cdc-ext-1.0.0-SNAPSHOT.jar:?]
> at
> org.apache.ignite.cdc.AbstractIgniteCdcStreamer.onEvents(AbstractIgniteCdcStreamer.java:118)
> [ignite-cdc-ext-1.0.0-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:142)
> [classes/:?]
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeSegmentActively(CdcMain.java:557)
> [classes/:?]
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:496)
> [classes/:?]
> at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:344)
> [classes/:?]
> at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:283) [classes/:?]
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20528) CDC doesn't work if the "Cache objects transformation" is applied
[
https://issues.apache.org/jira/browse/IGNITE-20528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20528:
-
Labels: iep-97 ise (was: ise)
> CDC doesn't work if the "Cache objects transformation" is applied
> -
>
> Key: IGNITE-20528
> URL: https://issues.apache.org/jira/browse/IGNITE-20528
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Korotkov
>Priority: Major
> Labels: iep-97, ise
>
> CDC doesn't work If some cache objects transformation is applied (see the
> [https://cwiki.apache.org/confluence/display/IGNITE/IEP-97+Cache+objects+transformation).|https://cwiki.apache.org/confluence/display/IGNITE/IEP-97+Cache+objects+transformation)]
> ignite_cdc.sh utility produces the NPE (see below). The immediate reason of
> the NPE is that ignite_cdc.sh uses the reduced version of the context
> (StandaloneGridKernalContext), which doesn't contain the GridCacheProcessor.
>
> {noformat}
> [2023-10-02T10:43:32,017][ERROR][Thread-1][] Unable to convert value
> [CacheObjectImpl [val=null, hasValBytes=true]]
> java.lang.NullPointerException: null
> at
> org.apache.ignite.internal.processors.cache.CacheObjectTransformerUtils.transformer(CacheObjectTransformerUtils.java:32)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.CacheObjectTransformerUtils.restoreIfNecessary(CacheObjectTransformerUtils.java:120)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.CacheObjectAdapter.valueFromValueBytes(CacheObjectAdapter.java:73)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.CacheObjectImpl.value(CacheObjectImpl.java:92)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.CacheObjectImpl.value(CacheObjectImpl.java:58)
> ~[classes/:?]
> at
> org.apache.ignite.internal.pagemem.wal.record.UnwrapDataEntry.unwrappedValue(UnwrapDataEntry.java:104)
> ~[classes/:?]
> at
> org.apache.ignite.internal.cdc.WalRecordsConsumer.lambda$static$c56580e2$1(WalRecordsConsumer.java:99)
> ~[classes/:?]
> at
> org.apache.ignite.internal.util.lang.gridfunc.TransformFilteringIterator.nextX(TransformFilteringIterator.java:119)
> [classes/:?]
> at
> org.apache.ignite.internal.util.lang.GridIteratorAdapter.next(GridIteratorAdapter.java:35)
> [classes/:?]
> at
> org.apache.ignite.internal.util.lang.gridfunc.TransformFilteringIterator.hasNextX(TransformFilteringIterator.java:85)
> [classes/:?]
> at
> org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
> [classes/:?]
> at
> org.apache.ignite.cdc.AbstractCdcEventsApplier.apply(AbstractCdcEventsApplier.java:71)
> [ignite-cdc-ext-1.0.0-SNAPSHOT.jar:?]
> at
> org.apache.ignite.cdc.AbstractIgniteCdcStreamer.onEvents(AbstractIgniteCdcStreamer.java:118)
> [ignite-cdc-ext-1.0.0-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:142)
> [classes/:?]
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeSegmentActively(CdcMain.java:557)
> [classes/:?]
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:496)
> [classes/:?]
> at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:344)
> [classes/:?]
> at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:283) [classes/:?]
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-20470) Ducktape to check dump performance
[ https://issues.apache.org/jira/browse/IGNITE-20470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770340#comment-17770340 ] Nikolay Izhikov commented on IGNITE-20470: -- https://github.com/apache/ignite/pull/10953 - cache_dumps is a feature branch to add test > Ducktape to check dump performance > -- > > Key: IGNITE-20470 > URL: https://issues.apache.org/jira/browse/IGNITE-20470 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > > Dump creation can affect transactions performance with change listener and > disc operations. We must create ducktape test to check this. > Example test scenario: > * Start nodes > * Start transaction operations: insert, update, remove. > * Create dump > * Check dump consistency. > Measure > * Transaction performance penalty. > * GC utilization. > * Disc utilization. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20472) API to consumer cache dumps
[ https://issues.apache.org/jira/browse/IGNITE-20472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov reassigned IGNITE-20472: Assignee: Nikolay Izhikov > API to consumer cache dumps > --- > > Key: IGNITE-20472 > URL: https://issues.apache.org/jira/browse/IGNITE-20472 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > > Ignite must provide API to consume cache dumps by user defined consumer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20472) API to consumer cache dumps
Nikolay Izhikov created IGNITE-20472: Summary: API to consumer cache dumps Key: IGNITE-20472 URL: https://issues.apache.org/jira/browse/IGNITE-20472 Project: Ignite Issue Type: Task Reporter: Nikolay Izhikov Ignite must provide API to consume cache dumps by user defined consumer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20470) Ducktape to check dump performance
[ https://issues.apache.org/jira/browse/IGNITE-20470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-20470: - Labels: IEP-109 ise (was: IEP-109) > Ducktape to check dump performance > -- > > Key: IGNITE-20470 > URL: https://issues.apache.org/jira/browse/IGNITE-20470 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > > Dump creation can affect transactions performance with change listener and > disc operations. We must create ducktape test to check this. > Example test scenario: > * Start nodes > * Start transaction operations: insert, update, remove. > * Create dump > * Check dump consistency. > Measure > * Transaction performance penalty. > * GC utilization. > * Disc utilization. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20470) Ducktape to check dump performance
Nikolay Izhikov created IGNITE-20470: Summary: Ducktape to check dump performance Key: IGNITE-20470 URL: https://issues.apache.org/jira/browse/IGNITE-20470 Project: Ignite Issue Type: Task Reporter: Nikolay Izhikov Dump creation can affect transactions performance with change listener and disc operations. We must create ducktape test to check this. Example test scenario: * Start nodes * Start transaction operations: insert, update, remove. * Create dump * Check dump consistency. Measure * Transaction performance penalty. * GC utilization. * Disc utilization. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19950) Snapshot of in-memory cache groups
[ https://issues.apache.org/jira/browse/IGNITE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766824#comment-17766824 ] Nikolay Izhikov commented on IGNITE-19950: -- Design decisions we can changed during performance testing: * Use per thread buffer to write entries: currently - buffer per partition file. * Use fully non blocking algorithm based on max entry version - like in incremental snapshots. * User one file for whole dump to use sequential writes - currently, file per partition. > Snapshot of in-memory cache groups > -- > > Key: IGNITE-19950 > URL: https://issues.apache.org/jira/browse/IGNITE-19950 > Project: Ignite > Issue Type: New Feature >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, iep-43, ise > Time Spent: 1h > Remaining Estimate: 0h > > Ignite can create snapshot for persistent cache groups only. > It will be very usefull for a user to have ability to create and restore > snapshot of in-memory cache groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20429) Support dump check command
Nikolay Izhikov created IGNITE-20429: Summary: Support dump check command Key: IGNITE-20429 URL: https://issues.apache.org/jira/browse/IGNITE-20429 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20429) Support dump check command
[ https://issues.apache.org/jira/browse/IGNITE-20429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov reassigned IGNITE-20429: Assignee: Nikolay Izhikov > Support dump check command > -- > > Key: IGNITE-20429 > URL: https://issues.apache.org/jira/browse/IGNITE-20429 > Project: Ignite > Issue Type: Bug >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, ise > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20309) Snapshot creation returns OK even if check failed
[
https://issues.apache.org/jira/browse/IGNITE-20309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20309:
-
Fix Version/s: 2.16
> Snapshot creation returns OK even if check failed
> -
>
> Key: IGNITE-20309
> URL: https://issues.apache.org/jira/browse/IGNITE-20309
> Project: Ignite
> Issue Type: Bug
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: ise
> Fix For: 2.16
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Currently, {{SnapshotPartitionsQuickVerifyHandler#complete}} don't fail
> snapshot even if {{invoke}} throws.
> Reproducer:
>
> {noformat}
> /**
> * Test ensures that snapshot fail if during check some files are absent.
> * @see SnapshotPartitionsQuickVerifyHandler
> */
> @Test
> public void testHandlerExceptionFailSnapshot() throws Exception {
> handlers.add(new SnapshotHandler() {
> @Override public SnapshotHandlerType type() {
> return SnapshotHandlerType.CREATE;
> }
> @Override public Void invoke(SnapshotHandlerContext ctx) {
> // Someone remove snapshot files during creation.
> // In this case snapshot must fail.
> U.delete(ctx.snapshotDirectory());
> return null;
> }
> });
> IgniteEx ignite = startGridsWithCache(1, CACHE_KEYS_RANGE,
> valueBuilder(), dfltCacheCfg);
> assertThrows(
> null,
> () -> snp(ignite).createSnapshot("must_fail", null, false,
> onlyPrimary).get(getTestTimeout()),
> IgniteException.class,
> "Snapshot data doesn't contain required cache group partition"
> );
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20309) Snapshot creation returns OK even if check failed
[
https://issues.apache.org/jira/browse/IGNITE-20309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20309:
-
Labels: ise (was: )
> Snapshot creation returns OK even if check failed
> -
>
> Key: IGNITE-20309
> URL: https://issues.apache.org/jira/browse/IGNITE-20309
> Project: Ignite
> Issue Type: Bug
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: ise
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Currently, {{SnapshotPartitionsQuickVerifyHandler#complete}} don't fail
> snapshot even if {{invoke}} throws.
> Reproducer:
>
> {noformat}
> /**
> * Test ensures that snapshot fail if during check some files are absent.
> * @see SnapshotPartitionsQuickVerifyHandler
> */
> @Test
> public void testHandlerExceptionFailSnapshot() throws Exception {
> handlers.add(new SnapshotHandler() {
> @Override public SnapshotHandlerType type() {
> return SnapshotHandlerType.CREATE;
> }
> @Override public Void invoke(SnapshotHandlerContext ctx) {
> // Someone remove snapshot files during creation.
> // In this case snapshot must fail.
> U.delete(ctx.snapshotDirectory());
> return null;
> }
> });
> IgniteEx ignite = startGridsWithCache(1, CACHE_KEYS_RANGE,
> valueBuilder(), dfltCacheCfg);
> assertThrows(
> null,
> () -> snp(ignite).createSnapshot("must_fail", null, false,
> onlyPrimary).get(getTestTimeout()),
> IgniteException.class,
> "Snapshot data doesn't contain required cache group partition"
> );
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (IGNITE-20309) Snapshot creation returns OK even if check failed
[
https://issues.apache.org/jira/browse/IGNITE-20309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov resolved IGNITE-20309.
--
Resolution: Fixed
> Snapshot creation returns OK even if check failed
> -
>
> Key: IGNITE-20309
> URL: https://issues.apache.org/jira/browse/IGNITE-20309
> Project: Ignite
> Issue Type: Bug
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Currently, {{SnapshotPartitionsQuickVerifyHandler#complete}} don't fail
> snapshot even if {{invoke}} throws.
> Reproducer:
>
> {noformat}
> /**
> * Test ensures that snapshot fail if during check some files are absent.
> * @see SnapshotPartitionsQuickVerifyHandler
> */
> @Test
> public void testHandlerExceptionFailSnapshot() throws Exception {
> handlers.add(new SnapshotHandler() {
> @Override public SnapshotHandlerType type() {
> return SnapshotHandlerType.CREATE;
> }
> @Override public Void invoke(SnapshotHandlerContext ctx) {
> // Someone remove snapshot files during creation.
> // In this case snapshot must fail.
> U.delete(ctx.snapshotDirectory());
> return null;
> }
> });
> IgniteEx ignite = startGridsWithCache(1, CACHE_KEYS_RANGE,
> valueBuilder(), dfltCacheCfg);
> assertThrows(
> null,
> () -> snp(ignite).createSnapshot("must_fail", null, false,
> onlyPrimary).get(getTestTimeout()),
> IgniteException.class,
> "Snapshot data doesn't contain required cache group partition"
> );
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-20309) Snapshot creation returns OK even if check failed
[
https://issues.apache.org/jira/browse/IGNITE-20309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov reassigned IGNITE-20309:
Assignee: Nikolay Izhikov
> Snapshot creation returns OK even if check failed
> -
>
> Key: IGNITE-20309
> URL: https://issues.apache.org/jira/browse/IGNITE-20309
> Project: Ignite
> Issue Type: Bug
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>
> Currently, {{SnapshotPartitionsQuickVerifyHandler#complete}} don't fail
> snapshot even if {{invoke}} throws.
> Reproducer:
>
> {noformat}
> /**
> * Test ensures that snapshot fail if during check some files are absent.
> * @see SnapshotPartitionsQuickVerifyHandler
> */
> @Test
> public void testHandlerExceptionFailSnapshot() throws Exception {
> handlers.add(new SnapshotHandler() {
> @Override public SnapshotHandlerType type() {
> return SnapshotHandlerType.CREATE;
> }
> @Override public Void invoke(SnapshotHandlerContext ctx) {
> // Someone remove snapshot files during creation.
> // In this case snapshot must fail.
> U.delete(ctx.snapshotDirectory());
> return null;
> }
> });
> IgniteEx ignite = startGridsWithCache(1, CACHE_KEYS_RANGE,
> valueBuilder(), dfltCacheCfg);
> assertThrows(
> null,
> () -> snp(ignite).createSnapshot("must_fail", null, false,
> onlyPrimary).get(getTestTimeout()),
> IgniteException.class,
> "Snapshot data doesn't contain required cache group partition"
> );
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20309) Snapshot creation returns OK even if check failed
Nikolay Izhikov created IGNITE-20309:
Summary: Snapshot creation returns OK even if check failed
Key: IGNITE-20309
URL: https://issues.apache.org/jira/browse/IGNITE-20309
Project: Ignite
Issue Type: Bug
Reporter: Nikolay Izhikov
Currently, {{SnapshotPartitionsQuickVerifyHandler#complete}} don't fail
snapshot even if {{invoke}} throws.
Reproducer:
{noformat}
/**
* Test ensures that snapshot fail if during check some files are absent.
* @see SnapshotPartitionsQuickVerifyHandler
*/
@Test
public void testHandlerExceptionFailSnapshot() throws Exception {
handlers.add(new SnapshotHandler() {
@Override public SnapshotHandlerType type() {
return SnapshotHandlerType.CREATE;
}
@Override public Void invoke(SnapshotHandlerContext ctx) {
// Someone remove snapshot files during creation.
// In this case snapshot must fail.
U.delete(ctx.snapshotDirectory());
return null;
}
});
IgniteEx ignite = startGridsWithCache(1, CACHE_KEYS_RANGE,
valueBuilder(), dfltCacheCfg);
assertThrows(
null,
() -> snp(ignite).createSnapshot("must_fail", null, false,
onlyPrimary).get(getTestTimeout()),
IgniteException.class,
"Snapshot data doesn't contain required cache group partition"
);
}
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-20297) NPE in kafka-to-ignite if metadata topic unknown
[
https://issues.apache.org/jira/browse/IGNITE-20297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov reassigned IGNITE-20297:
Assignee: Nikolay Izhikov
> NPE in kafka-to-ignite if metadata topic unknown
>
>
> Key: IGNITE-20297
> URL: https://issues.apache.org/jira/browse/IGNITE-20297
> Project: Ignite
> Issue Type: New Feature
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-59, ise
>
> kafka-to-ignite fails with NPE instead of readable error message if metadata
> topic unknown:
> {noformat}
> class org.apache.ignite.IgniteException: null
> at
> org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.run(AbstractKafkaToIgniteCdcStreamer.java:127)
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteCdcStreamer.run(KafkaToIgniteCdcStreamer.java:71)
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteCommandLineStartup.main(KafkaToIgniteCommandLineStartup.java:68)
> Caused by: java.lang.NullPointerException
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteMetadataUpdater.(KafkaToIgniteMetadataUpdater.java:107)
> at
> org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.runAppliers(AbstractKafkaToIgniteCdcStreamer.java:145)
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteCdcStreamer.runx(KafkaToIgniteCdcStreamer.java:110)
> at
> org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.run(AbstractKafkaToIgniteCdcStreamer.java:124)
> ... 2 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20297) NPE in kafka-to-ignite if metadata topic unknown
[
https://issues.apache.org/jira/browse/IGNITE-20297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20297:
-
Labels: IEP-80 ise (was: ise)
> NPE in kafka-to-ignite if metadata topic unknown
>
>
> Key: IGNITE-20297
> URL: https://issues.apache.org/jira/browse/IGNITE-20297
> Project: Ignite
> Issue Type: New Feature
>Reporter: Nikolay Izhikov
>Priority: Major
> Labels: IEP-80, ise
>
> kafka-to-ignite fails with NPE instead of readable error message if metadata
> topic unknown:
> {noformat}
> class org.apache.ignite.IgniteException: null
> at
> org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.run(AbstractKafkaToIgniteCdcStreamer.java:127)
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteCdcStreamer.run(KafkaToIgniteCdcStreamer.java:71)
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteCommandLineStartup.main(KafkaToIgniteCommandLineStartup.java:68)
> Caused by: java.lang.NullPointerException
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteMetadataUpdater.(KafkaToIgniteMetadataUpdater.java:107)
> at
> org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.runAppliers(AbstractKafkaToIgniteCdcStreamer.java:145)
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteCdcStreamer.runx(KafkaToIgniteCdcStreamer.java:110)
> at
> org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.run(AbstractKafkaToIgniteCdcStreamer.java:124)
> ... 2 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20297) NPE in kafka-to-ignite if metadata topic unknown
[
https://issues.apache.org/jira/browse/IGNITE-20297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20297:
-
Labels: IEP-59 ise (was: IEP-80 ise)
> NPE in kafka-to-ignite if metadata topic unknown
>
>
> Key: IGNITE-20297
> URL: https://issues.apache.org/jira/browse/IGNITE-20297
> Project: Ignite
> Issue Type: New Feature
>Reporter: Nikolay Izhikov
>Priority: Major
> Labels: IEP-59, ise
>
> kafka-to-ignite fails with NPE instead of readable error message if metadata
> topic unknown:
> {noformat}
> class org.apache.ignite.IgniteException: null
> at
> org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.run(AbstractKafkaToIgniteCdcStreamer.java:127)
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteCdcStreamer.run(KafkaToIgniteCdcStreamer.java:71)
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteCommandLineStartup.main(KafkaToIgniteCommandLineStartup.java:68)
> Caused by: java.lang.NullPointerException
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteMetadataUpdater.(KafkaToIgniteMetadataUpdater.java:107)
> at
> org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.runAppliers(AbstractKafkaToIgniteCdcStreamer.java:145)
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteCdcStreamer.runx(KafkaToIgniteCdcStreamer.java:110)
> at
> org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.run(AbstractKafkaToIgniteCdcStreamer.java:124)
> ... 2 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20297) NPE in kafka-to-ignite if metadata topic unknown
Nikolay Izhikov created IGNITE-20297:
Summary: NPE in kafka-to-ignite if metadata topic unknown
Key: IGNITE-20297
URL: https://issues.apache.org/jira/browse/IGNITE-20297
Project: Ignite
Issue Type: New Feature
Reporter: Nikolay Izhikov
kafka-to-ignite fails with NPE instead of readable error message if metadata
topic unknown:
{noformat}
class org.apache.ignite.IgniteException: null
at
org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.run(AbstractKafkaToIgniteCdcStreamer.java:127)
at
org.apache.ignite.cdc.kafka.KafkaToIgniteCdcStreamer.run(KafkaToIgniteCdcStreamer.java:71)
at
org.apache.ignite.cdc.kafka.KafkaToIgniteCommandLineStartup.main(KafkaToIgniteCommandLineStartup.java:68)
Caused by: java.lang.NullPointerException
at
org.apache.ignite.cdc.kafka.KafkaToIgniteMetadataUpdater.(KafkaToIgniteMetadataUpdater.java:107)
at
org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.runAppliers(AbstractKafkaToIgniteCdcStreamer.java:145)
at
org.apache.ignite.cdc.kafka.KafkaToIgniteCdcStreamer.runx(KafkaToIgniteCdcStreamer.java:110)
at
org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.run(AbstractKafkaToIgniteCdcStreamer.java:124)
... 2 more
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20297) NPE in kafka-to-ignite if metadata topic unknown
[
https://issues.apache.org/jira/browse/IGNITE-20297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-20297:
-
Labels: ise (was: )
> NPE in kafka-to-ignite if metadata topic unknown
>
>
> Key: IGNITE-20297
> URL: https://issues.apache.org/jira/browse/IGNITE-20297
> Project: Ignite
> Issue Type: New Feature
>Reporter: Nikolay Izhikov
>Priority: Major
> Labels: ise
>
> kafka-to-ignite fails with NPE instead of readable error message if metadata
> topic unknown:
> {noformat}
> class org.apache.ignite.IgniteException: null
> at
> org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.run(AbstractKafkaToIgniteCdcStreamer.java:127)
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteCdcStreamer.run(KafkaToIgniteCdcStreamer.java:71)
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteCommandLineStartup.main(KafkaToIgniteCommandLineStartup.java:68)
> Caused by: java.lang.NullPointerException
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteMetadataUpdater.(KafkaToIgniteMetadataUpdater.java:107)
> at
> org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.runAppliers(AbstractKafkaToIgniteCdcStreamer.java:145)
> at
> org.apache.ignite.cdc.kafka.KafkaToIgniteCdcStreamer.runx(KafkaToIgniteCdcStreamer.java:110)
> at
> org.apache.ignite.cdc.kafka.AbstractKafkaToIgniteCdcStreamer.run(AbstractKafkaToIgniteCdcStreamer.java:124)
> ... 2 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19950) Snapshot of in-memory cache groups
[ https://issues.apache.org/jira/browse/IGNITE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-19950: - Labels: IEP-109 iep-43 ise (was: iep-43 ise) > Snapshot of in-memory cache groups > -- > > Key: IGNITE-19950 > URL: https://issues.apache.org/jira/browse/IGNITE-19950 > Project: Ignite > Issue Type: New Feature >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-109, iep-43, ise > Time Spent: 20m > Remaining Estimate: 0h > > Ignite can create snapshot for persistent cache groups only. > It will be very usefull for a user to have ability to create and restore > snapshot of in-memory cache groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19950) Snapshot of in-memory cache groups
[ https://issues.apache.org/jira/browse/IGNITE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-19950: - Labels: iep-43 ise (was: iep-43) > Snapshot of in-memory cache groups > -- > > Key: IGNITE-19950 > URL: https://issues.apache.org/jira/browse/IGNITE-19950 > Project: Ignite > Issue Type: New Feature >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: iep-43, ise > > Ignite can create snapshot for persistent cache groups only. > It will be very usefull for a user to have ability to create and restore > snapshot of in-memory cache groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19950) Snapshot of in-memory cache groups
[ https://issues.apache.org/jira/browse/IGNITE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-19950: - Labels: iep-43 (was: ) > Snapshot of in-memory cache groups > -- > > Key: IGNITE-19950 > URL: https://issues.apache.org/jira/browse/IGNITE-19950 > Project: Ignite > Issue Type: New Feature >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: iep-43 > > Ignite can create snapshot for persistent cache groups only. > It will be very usefull for a user to have ability to create and restore > snapshot of in-memory cache groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19950) Snapshot of in-memory cache groups
Nikolay Izhikov created IGNITE-19950: Summary: Snapshot of in-memory cache groups Key: IGNITE-19950 URL: https://issues.apache.org/jira/browse/IGNITE-19950 Project: Ignite Issue Type: New Feature Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Ignite can create snapshot for persistent cache groups only. It will be very usefull for a user to have ability to create and restore snapshot of in-memory cache groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19918) idle_verify compactFooter info
[ https://issues.apache.org/jira/browse/IGNITE-19918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-19918: - Fix Version/s: 2.16 > idle_verify compactFooter info > -- > > Key: IGNITE-19918 > URL: https://issues.apache.org/jira/browse/IGNITE-19918 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: ise > Fix For: 2.16 > > Time Spent: 40m > Remaining Estimate: 0h > > Ignite has issues with default value of compactFooter parameter on nodes and > thin client. > Default has been changed lately IGNITE-16038 which break compatibility of pre > 2.14 and >= 2.14 thin clients. > For debug and operations purposes it will be convenient to have information > about entry compactFooter values in PDS. > Such information must be included in idle_verify report. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19915) Remove obsolete IgniteCacheSnapshotManager
[ https://issues.apache.org/jira/browse/IGNITE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-19915: - Labels: IEP-80 iep-43 ise (was: IEP-80 iep-43) > Remove obsolete IgniteCacheSnapshotManager > -- > > Key: IGNITE-19915 > URL: https://issues.apache.org/jira/browse/IGNITE-19915 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-80, iep-43, ise > Fix For: 2.16 > > Time Spent: 1.5h > Remaining Estimate: 0h > > IgniteSnapshotManager implements snapshotting features. > IgniteCacheSnapshotManager is obsolete and can be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19915) Remove obsolete IgniteCacheSnapshotManager
[ https://issues.apache.org/jira/browse/IGNITE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-19915: - Fix Version/s: 2.16 > Remove obsolete IgniteCacheSnapshotManager > -- > > Key: IGNITE-19915 > URL: https://issues.apache.org/jira/browse/IGNITE-19915 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-80, iep-43 > Fix For: 2.16 > > Time Spent: 1.5h > Remaining Estimate: 0h > > IgniteSnapshotManager implements snapshotting features. > IgniteCacheSnapshotManager is obsolete and can be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-19918) idle_verify compactFooter info
[ https://issues.apache.org/jira/browse/IGNITE-19918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov reassigned IGNITE-19918: Assignee: Nikolay Izhikov > idle_verify compactFooter info > -- > > Key: IGNITE-19918 > URL: https://issues.apache.org/jira/browse/IGNITE-19918 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: ise > > Ignite has issues with default value of compactFooter parameter on nodes and > thin client. > Default has been changed lately IGNITE-16038 which break compatibility of pre > 2.14 and >= 2.14 thin clients. > For debug and operations purposes it will be convenient to have information > about entry compactFooter values in PDS. > Such information must be included in idle_verify report. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19918) idle_verify compactFooter info
Nikolay Izhikov created IGNITE-19918: Summary: idle_verify compactFooter info Key: IGNITE-19918 URL: https://issues.apache.org/jira/browse/IGNITE-19918 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Ignite has issues with default value of compactFooter parameter on nodes and thin client. Default has been changed lately IGNITE-16038 which break compatibility of pre 2.14 and >= 2.14 thin clients. For debug and operations purposes it will be convenient to have information about entry compactFooter values in PDS. Such information must be included in idle_verify report. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19915) Remove obsolete IgniteCacheSnapshotManager
Nikolay Izhikov created IGNITE-19915: Summary: Remove obsolete IgniteCacheSnapshotManager Key: IGNITE-19915 URL: https://issues.apache.org/jira/browse/IGNITE-19915 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov IgniteSnapshotManager implements snapshotting features. IgniteCacheSnapshotManager is obsolete and can be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-19746) control.sh --performance-statistics status doesn't not print actual status
[
https://issues.apache.org/jira/browse/IGNITE-19746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov resolved IGNITE-19746.
--
Resolution: Fixed
> control.sh --performance-statistics status doesn't not print actual status
> --
>
> Key: IGNITE-19746
> URL: https://issues.apache.org/jira/browse/IGNITE-19746
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Korotkov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-81
>
> The status sub-command of the control.sh --performance-statistics doesn't not
> print the actual status to console.
> Previously it was like (note the *Disabled.* word):
> {noformat}
> Control utility [ver. 15.0.0-SNAPSHOT#20230422-sha1:7f80003d]
> 2023 Copyright(C) Apache Software Foundation
> User: ducker
> Time: 2023-04-23T22:17:12.489
> Command [PERFORMANCE-STATISTICS] started
> Arguments: --host x.x.x.x --performance-statistics status --user admin
> --password *
>
> Disabled.
> Command [PERFORMANCE-STATISTICS] finished with code: 0
> Control utility has completed execution at: 2023-04-23T22:17:13.271
> Execution time: 782 ms
> {noformat}
>
> Now it's like (note the absence of the *Disabled.* word):
> {noformat}
> Control utility [ver. 15.0.0-SNAPSHOT#20230613-sha1:cacee58d]
> 2023 Copyright(C) Apache Software Foundation
> User: ducker
> Time: 2023-06-15T15:46:41.586
> Command [PERFORMANCE-STATISTICS] started
> Arguments: --host x.x.x.x --performance-statistics status --user admin
> --password *
>
> Command [PERFORMANCE-STATISTICS] finished with code: 0
> Control utility has completed execution at: 2023-06-15T15:46:42.523
> Execution time: 937 ms
> {noformat}
>
> Outputs of other sub-commands are also need to be checked.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19676) JMX Commands invoker
[ https://issues.apache.org/jira/browse/IGNITE-19676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-19676: - Fix Version/s: 2.16 > JMX Commands invoker > > > Key: IGNITE-19676 > URL: https://issues.apache.org/jira/browse/IGNITE-19676 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-81, important, ise > Fix For: 2.16 > > Time Spent: 3.5h > Remaining Estimate: 0h > > After creation of Management API Ignite must provide JMX command invoker. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19676) JMX Commands invoker
[ https://issues.apache.org/jira/browse/IGNITE-19676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736469#comment-17736469 ] Nikolay Izhikov commented on IGNITE-19676: -- https://ci2.ignite.apache.org/test/-8869604011192050851?currentProjectId=IgniteTests24Java8=true=%3Cdefault%3E - failures unrelated. Tests fails in master. > JMX Commands invoker > > > Key: IGNITE-19676 > URL: https://issues.apache.org/jira/browse/IGNITE-19676 > Project: Ignite > Issue Type: Task >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-81, important, ise > Time Spent: 2h 50m > Remaining Estimate: 0h > > After creation of Management API Ignite must provide JMX command invoker. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19661) In-memory CDC should not produce WAL records when disabled.
[ https://issues.apache.org/jira/browse/IGNITE-19661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-19661: - Labels: IEP-59 ise (was: ise) > In-memory CDC should not produce WAL records when disabled. > --- > > Key: IGNITE-19661 > URL: https://issues.apache.org/jira/browse/IGNITE-19661 > Project: Ignite > Issue Type: Task >Reporter: Nikita Amelchev >Assignee: Nikita Amelchev >Priority: Major > Labels: IEP-59, ise > Fix For: 2.16 > > Time Spent: 40m > Remaining Estimate: 0h > > In-memory CDC should not produce WAL records when disabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-19734) SystemView - move Task class next to Command class
[
https://issues.apache.org/jira/browse/IGNITE-19734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov resolved IGNITE-19734.
--
Resolution: Fixed
> SystemView - move Task class next to Command class
> --
>
> Key: IGNITE-19734
> URL: https://issues.apache.org/jira/browse/IGNITE-19734
> Project: Ignite
> Issue Type: Sub-task
>Affects Versions: 2.16
>Reporter: Valery Shorin
>Priority: Major
> Labels: IEP-81
> Fix For: 2.16
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Need to move the corresponding Task class and helper classes next to
> {{org.apache.ignite.internal.management.SystemViewCommand}} class and rename
> them according to the following naming strategy:
> {{SystemViewCommand}}
> {{SystemViewCommandArg}}
> {{SystemViewTask}}
> {{SystemViewTaskResult}}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-19734) SystemView - move Task class next to Command class
[
https://issues.apache.org/jira/browse/IGNITE-19734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov reassigned IGNITE-19734:
Assignee: Valery Shorin
> SystemView - move Task class next to Command class
> --
>
> Key: IGNITE-19734
> URL: https://issues.apache.org/jira/browse/IGNITE-19734
> Project: Ignite
> Issue Type: Sub-task
>Affects Versions: 2.16
>Reporter: Valery Shorin
>Assignee: Valery Shorin
>Priority: Major
> Labels: IEP-81
> Fix For: 2.16
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Need to move the corresponding Task class and helper classes next to
> {{org.apache.ignite.internal.management.SystemViewCommand}} class and rename
> them according to the following naming strategy:
> {{SystemViewCommand}}
> {{SystemViewCommandArg}}
> {{SystemViewTask}}
> {{SystemViewTaskResult}}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19753) Move task and command class to the same packages
[
https://issues.apache.org/jira/browse/IGNITE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-19753:
-
Labels: IEP-81 (was: )
> Move task and command class to the same packages
>
>
> Key: IGNITE-19753
> URL: https://issues.apache.org/jira/browse/IGNITE-19753
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Priority: Major
> Labels: IEP-81
>
> Right now command of Management API in
> {{org.apache.ignite.internal.management}} package and tasks that implements
> command logic in {{org.apache.ignite.internal.visor}} package.
> Visor not existing anymore in Ignite so it's seems logical to move tasks and
> their dependencies to commands.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
