[jira] [Comment Edited] (IGNITE-16809) .NET: CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615285#comment-17615285 ] Jerome Isaac Haltom edited comment on IGNITE-16809 at 10/10/22 7:43 PM: For instance, it's well accepted that you should be able to cancel a PUT request when using HTTP client, but that you should except the operation to be idempotent: that your client might try the operation again, without knowing whether the previous operation completed. This matches exactly the putCache etc methods on Ignite. was (Author: JIRAUSER287703): For instance, it's well accepted that you should be able to cancel a PUT request when using HTTP client, but that you should except the operation to be idempotent: that your client might try the operation again, without knowing whether the previous operation completed. This matches exactly the putCache etc methods on Ignote. > .NET: CancellationToken on Async methods > > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: platforms, thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Assignee: Pavel Tupitsyn >Priority: Minor > Labels: .NET > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-16809) .NET: CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615285#comment-17615285 ] Jerome Isaac Haltom commented on IGNITE-16809: -- For instance, it's well accepted that you should be able to cancel a PUT request when using HTTP client, but that you should except the operation to be idempotent: that your client might try the operation again, without knowing whether the previous operation completed. This matches exactly the putCache etc methods on Ignote. > .NET: CancellationToken on Async methods > > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: platforms, thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Assignee: Pavel Tupitsyn >Priority: Minor > Labels: .NET > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-16809) .NET: CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615284#comment-17615284 ] Jerome Isaac Haltom commented on IGNITE-16809: -- Last part of that paragraph: >or find a convenient place that you can stop halfway through but in a valid >condition, before then throwing OperationCanceledException. In other words, >the caller must be able to recover to a known consistent state after >cancelling your work, or realize that cancellation was not responded to and >that the caller then must decide whether to accept the work, or revert its >successful completion on its own. A "consistent state" isn't meant to convey that the entire operation should be atomic. That's a job for something larger, like transactions. If you were to interpret the paragraph how you want, all idempotent HTTP operations should be uncancellable. > .NET: CancellationToken on Async methods > > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: platforms, thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Assignee: Pavel Tupitsyn >Priority: Minor > Labels: .NET > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-16809) .NET: CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615281#comment-17615281 ] Pavel Tupitsyn commented on IGNITE-16809: - On the other hand, this guide https://devblogs.microsoft.com/premier-developer/recommended-patterns-for-cancellationtoken/ says *"Don’t cancel if you’ve already incurred side-effects that your method isn’t prepared to revert"*, which is true for all Ignite APIs that modify data. So we can only cancel the network response processing part for APIs that do not modify data, like *cache.get*. > .NET: CancellationToken on Async methods > > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: platforms, thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Assignee: Pavel Tupitsyn >Priority: Minor > Labels: .NET > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16809) .NET: CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-16809: Component/s: platforms > .NET: CancellationToken on Async methods > > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: platforms, thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Priority: Minor > Labels: .NET > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-16809) .NET CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615278#comment-17615278 ] Pavel Tupitsyn commented on IGNITE-16809: - [~jhaltom] agree, we will look into this. > .NET CancellationToken on Async methods > --- > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Priority: Minor > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16809) .NET: CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-16809: Summary: .NET: CancellationToken on Async methods (was: .NET CancellationToken on Async methods) > .NET: CancellationToken on Async methods > > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Priority: Minor > Labels: .NET > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16809) .NET CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-16809: Labels: .NET (was: ) > .NET CancellationToken on Async methods > --- > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Priority: Minor > Labels: .NET > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-16809) .NET: CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn reassigned IGNITE-16809: --- Assignee: Pavel Tupitsyn > .NET: CancellationToken on Async methods > > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: platforms, thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Assignee: Pavel Tupitsyn >Priority: Minor > Labels: .NET > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-16809) .NET CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615273#comment-17615273 ] Jerome Isaac Haltom edited comment on IGNITE-16809 at 10/10/22 6:59 PM: It is generally accepted that signaling a cancellation may not take effect immediately, only that the state of the application be left consistent. That the user be capable of cancelling the network portions of the call seems fine to me. That can be a significant amount of time in an application shutdown situation, especially if the shutdown is unorderly, and the remote Ignite service goes down first, leaving the client in a state of waiting for a timeout. This is actually my situation at present. On shutdown, all of my applications receive a request to terminate at the same time. They signal cancellation tokens and shutdown and what is essentially random timings from each other. was (Author: JIRAUSER287703): It is generally accepted that signaling a cancellation may not take effect immediately, only that the state of the application be left consistent. That the user be capable of cancelling the network portions of the call seems fine to me. That can be a significant amount of time in an application shutdown situation, especially if the shutdown is unorderly, and the remote Ignite service goes down first, leaving the client in a state of waiting for a timeout. > .NET CancellationToken on Async methods > --- > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Priority: Minor > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-16809) .NET CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615273#comment-17615273 ] Jerome Isaac Haltom edited comment on IGNITE-16809 at 10/10/22 6:57 PM: It is generally accepted that signaling a cancellation may not take effect immediately, only that the state of the application be left consistent. That the user be capable of cancelling the network portions of the call seems fine to me. That can be a significant amount of time in an application shutdown situation, especially if the shutdown is unorderly, and the remote Ignite service goes down first, leaving the client in a state of waiting for a timeout. was (Author: JIRAUSER287703): It is generally accepted that signaling a cancellation may not take effect immediately, only that the state of the application be left consistent. That the user be capable of cancelling the network portions of the call seems fine to me. That can be a significant amount of time in an application shutdown situation, especially if the shutdown is unorderly, and the remote Ignite service goes down first, leaving the client in a state of waiting for a timeout. > .NET CancellationToken on Async methods > --- > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Priority: Minor > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-16809) .NET CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615273#comment-17615273 ] Jerome Isaac Haltom commented on IGNITE-16809: -- It is generally accepted that signaling a cancellation may not take effect immediately, only that the state of the application be left consistent. That the user be capable of cancelling the network portions of the call seems fine to me. That can be a significant amount of time in an application shutdown situation, especially if the shutdown is unorderly, and the remote Ignite service goes down first, leaving the client in a state of waiting for a timeout. > .NET CancellationToken on Async methods > --- > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Priority: Minor > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-16809) .NET CancellationToken on Async methods
[ https://issues.apache.org/jira/browse/IGNITE-16809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615215#comment-17615215 ] Pavel Tupitsyn commented on IGNITE-16809: - [~jhaltom] In the underlying Ignite Java API only Compute supports cancellation, as far as I see. And ExecuteJavaTaskAsync takes CancellationToken. Other things like cache operations can't really be cancelled. Providing CancellationToken for them can be misleading (even though we could cancel the network part of the operation). > .NET CancellationToken on Async methods > --- > > Key: IGNITE-16809 > URL: https://issues.apache.org/jira/browse/IGNITE-16809 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.12 >Reporter: Jerome Isaac Haltom >Priority: Minor > > The .NET ThinClient API has numerous async methods, but none seem to support > cancellation. I suspect they could and probably should. Each should accept a > CancellationToken parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-17830) .NET: Thin 3.0: Incorrect Guid serialization
[ https://issues.apache.org/jira/browse/IGNITE-17830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615210#comment-17615210 ] Pavel Tupitsyn edited comment on IGNITE-17830 at 10/10/22 4:49 PM: --- Merged to main: 3e2687b68f73bcd192f219d2c589c183da6540bd was (Author: ptupitsyn): https://github.com/apache/ignite-3/commit/3e2687b68f73bcd192f219d2c589c183da6540bd > .NET: Thin 3.0: Incorrect Guid serialization > > > Key: IGNITE-17830 > URL: https://issues.apache.org/jira/browse/IGNITE-17830 > Project: Ignite > Issue Type: Bug > Components: platforms, thin client >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET, ignite-3 > Fix For: 3.0.0-alpha6 > > Time Spent: 10m > Remaining Estimate: 0h > > Guid <-> UUID serialization should ensure that string representations in Java > and C# are equal. > See Ignite 2.x code as a reference > https://github.com/apache/ignite/blob/789feff34f78f8544bf7498f22352bc4e22e352a/modules/platforms/dotnet/Apache.Ignite.Core/Impl/Binary/BinaryUtils.cs#L948 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17830) .NET: Thin 3.0: Incorrect Guid serialization
[ https://issues.apache.org/jira/browse/IGNITE-17830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615210#comment-17615210 ] Pavel Tupitsyn commented on IGNITE-17830: - https://github.com/apache/ignite-3/commit/3e2687b68f73bcd192f219d2c589c183da6540bd > .NET: Thin 3.0: Incorrect Guid serialization > > > Key: IGNITE-17830 > URL: https://issues.apache.org/jira/browse/IGNITE-17830 > Project: Ignite > Issue Type: Bug > Components: platforms, thin client >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET, ignite-3 > Fix For: 3.0.0-alpha6 > > Time Spent: 10m > Remaining Estimate: 0h > > Guid <-> UUID serialization should ensure that string representations in Java > and C# are equal. > See Ignite 2.x code as a reference > https://github.com/apache/ignite/blob/789feff34f78f8544bf7498f22352bc4e22e352a/modules/platforms/dotnet/Apache.Ignite.Core/Impl/Binary/BinaryUtils.cs#L948 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-17830) .NET: Thin 3.0: Incorrect Guid serialization
[ https://issues.apache.org/jira/browse/IGNITE-17830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615210#comment-17615210 ] Pavel Tupitsyn edited comment on IGNITE-17830 at 10/10/22 4:49 PM: --- Merged to main: [3e2687b68f73bcd192f219d2c589c183da6540bd|https://github.com/apache/ignite-3/commit/3e2687b68f73bcd192f219d2c589c183da6540bd] was (Author: ptupitsyn): Merged to main: 3e2687b68f73bcd192f219d2c589c183da6540bd > .NET: Thin 3.0: Incorrect Guid serialization > > > Key: IGNITE-17830 > URL: https://issues.apache.org/jira/browse/IGNITE-17830 > Project: Ignite > Issue Type: Bug > Components: platforms, thin client >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET, ignite-3 > Fix For: 3.0.0-alpha6 > > Time Spent: 10m > Remaining Estimate: 0h > > Guid <-> UUID serialization should ensure that string representations in Java > and C# are equal. > See Ignite 2.x code as a reference > https://github.com/apache/ignite/blob/789feff34f78f8544bf7498f22352bc4e22e352a/modules/platforms/dotnet/Apache.Ignite.Core/Impl/Binary/BinaryUtils.cs#L948 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17830) .NET: Thin 3.0: Incorrect Guid serialization
[ https://issues.apache.org/jira/browse/IGNITE-17830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615197#comment-17615197 ] Igor Sapego commented on IGNITE-17830: -- [~ptupitsyn] Good, ship it! > .NET: Thin 3.0: Incorrect Guid serialization > > > Key: IGNITE-17830 > URL: https://issues.apache.org/jira/browse/IGNITE-17830 > Project: Ignite > Issue Type: Bug > Components: platforms, thin client >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET, ignite-3 > Fix For: 3.0.0-alpha6 > > > Guid <-> UUID serialization should ensure that string representations in Java > and C# are equal. > See Ignite 2.x code as a reference > https://github.com/apache/ignite/blob/789feff34f78f8544bf7498f22352bc4e22e352a/modules/platforms/dotnet/Apache.Ignite.Core/Impl/Binary/BinaryUtils.cs#L948 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17537) Make units of 'timeout' and 'duration' more explicit in control.sh --tx output
[
https://issues.apache.org/jira/browse/IGNITE-17537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Pavlov updated IGNITE-17537:
---
Priority: Minor (was: Trivial)
> Make units of 'timeout' and 'duration' more explicit in control.sh --tx output
> --
>
> Key: IGNITE-17537
> URL: https://issues.apache.org/jira/browse/IGNITE-17537
> Project: Ignite
> Issue Type: Improvement
> Components: control.sh
>Affects Versions: 2.13
>Reporter: Ilya Shishkov
>Assignee: Julia Bakulina
>Priority: Minor
> Labels: ise, newbie
> Fix For: 2.14
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Execution of {{control.sh --tx}} command produces output of matching
> transactions, eg.:
> {quote}Matching transactions:
> Tx: [xid=fdc4d720281--0fd8-0177--0012, label=null,
> state=ACTIVE, startTime=2022-07-06 05:05:07.432,
> {*}{color:#ff}duration=778{color}{*}, isolation=REPEATABLE_READ,
> concurrency=PESSIMISTIC, topVer=AffinityTopologyVersion [topVer=199,
> minorTopVer=0], {*}{color:#ff}timeout=4{color}{*}, ...]
> {quote}
> But, from the above line it is unclear, that in fact, duration is printed in
> seconds [1], while timeout is printed in milliseconds.
> We can improve output in one of the following ways:
> # Explicitly append unit for seconds and milliseconds.
> # Print duration and timeout in same units: both in seconds or both in
> milliseconds.
> Links:
> #
> https://github.com/apache/ignite/blob/bf9a460eccd07701cacac2a414c65707243350f1/modules/core/src/main/java/org/apache/ignite/internal/visor/tx/VisorTxInfo.java#L286
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17865) Disable partition ranges in log messages about rebalance or PME
[
https://issues.apache.org/jira/browse/IGNITE-17865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-17865:
---
Labels: ise (was: )
> Disable partition ranges in log messages about rebalance or PME
> ---
>
> Key: IGNITE-17865
> URL: https://issues.apache.org/jira/browse/IGNITE-17865
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Ilya Shishkov
>Priority: Minor
> Labels: ise
>
> When you need to analyze cases of partitions inconsistency, full list of
> partitions is needed, but there is an optimization which replaces list of
> consecutive partitions with ranges. So, as you can see below, we can not find
> partition 2 by simple text search:
> {quote}
> 2021-08-11 23:12:11.338 [WARN
> ][sys-#213][org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl]
> Partitions have been scheduled for rebalancing due to outdated update
> counter [grp=group, readyTopVer=AffinityTopologyVersion [topVer=33,
> minorTopVer=0], topVer=AffinityTopologyVersion [topVer=33, minorTopVer=0],
> nodeId=8e05997a-4ff7-4fdb-9480-cfbcefa8bbf5,
> partsFull=[*{color:red}1-3{color}*, ...], partsHistorical=[]]
> {quote}
> {quote}
> 2021-08-11 23:12:11.338 [WARN
> ][sys-#213][org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture]
> Partitions weren't present in any history reservation: [[[grp=grp2
> part=[[*{color:red}1-3{color}*]]]
> {quote}
> We need to remove this optimization, because it can lead to mistakes in logs
> analysis.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17865) Disable partition ranges in log messages about rebalance or PME
[
https://issues.apache.org/jira/browse/IGNITE-17865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-17865:
---
Description:
When you need to analyze cases of partitions inconsistency, full list of
partitions is needed, but there is an optimization which replaces list of
consecutive partitions with ranges. So, as you can see below, we can not find
partition 2 by simple text search:
{quote}
2021-08-11 23:12:11.338 [WARN
][sys-#213][org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl]
Partitions have been scheduled for rebalancing due to outdated update counter
[grp=group, readyTopVer=AffinityTopologyVersion [topVer=33, minorTopVer=0],
topVer=AffinityTopologyVersion [topVer=33, minorTopVer=0],
nodeId=8e05997a-4ff7-4fdb-9480-cfbcefa8bbf5,
partsFull=[{color:red}*1-3*{color}, ...], partsHistorical=[]]
{quote}
{quote}
2021-08-11 23:12:11.338 [WARN
][sys-#213][org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture]
Partitions weren't present in any history reservation: [[[grp=grp2
part=[[{color:red}*1-3*{color}]]]
{quote}
We need to remove this optimization, because it can lead to mistakes in logs
analysis.
was:
When you need to analyze cases of partitions inconsistency, full list of
partitions is needed, but there is an optimization which replaces list of
consecutive partitions with ranges. So, as you can see below, we can not find
partition 2 by simple text search:
{quote}
2021-08-11 23:12:11.338 [WARN
][sys-#213][org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl]
Partitions have been scheduled for rebalancing due to outdated update counter
[grp=group, readyTopVer=AffinityTopologyVersion [topVer=33, minorTopVer=0],
topVer=AffinityTopologyVersion [topVer=33, minorTopVer=0],
nodeId=8e05997a-4ff7-4fdb-9480-cfbcefa8bbf5,
partsFull=[*{color:red}1-3{color}*, ...], partsHistorical=[]]
{quote}
{quote}
2021-08-11 23:12:11.338 [WARN
][sys-#213][org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture]
Partitions weren't present in any history reservation: [[[grp=grp2
part=[[*{color:red}1-3{color}*]]]
{quote}
We need to remove this optimization, because it can lead to mistakes in logs
analysis.
> Disable partition ranges in log messages about rebalance or PME
> ---
>
> Key: IGNITE-17865
> URL: https://issues.apache.org/jira/browse/IGNITE-17865
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Ilya Shishkov
>Priority: Minor
> Labels: ise
>
> When you need to analyze cases of partitions inconsistency, full list of
> partitions is needed, but there is an optimization which replaces list of
> consecutive partitions with ranges. So, as you can see below, we can not find
> partition 2 by simple text search:
> {quote}
> 2021-08-11 23:12:11.338 [WARN
> ][sys-#213][org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl]
> Partitions have been scheduled for rebalancing due to outdated update
> counter [grp=group, readyTopVer=AffinityTopologyVersion [topVer=33,
> minorTopVer=0], topVer=AffinityTopologyVersion [topVer=33, minorTopVer=0],
> nodeId=8e05997a-4ff7-4fdb-9480-cfbcefa8bbf5,
> partsFull=[{color:red}*1-3*{color}, ...], partsHistorical=[]]
> {quote}
> {quote}
> 2021-08-11 23:12:11.338 [WARN
> ][sys-#213][org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture]
> Partitions weren't present in any history reservation: [[[grp=grp2
> part=[[{color:red}*1-3*{color}]]]
> {quote}
> We need to remove this optimization, because it can lead to mistakes in logs
> analysis.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17865) Disable partition ranges in log messages about rebalance or PME
Ilya Shishkov created IGNITE-17865:
--
Summary: Disable partition ranges in log messages about rebalance
or PME
Key: IGNITE-17865
URL: https://issues.apache.org/jira/browse/IGNITE-17865
Project: Ignite
Issue Type: Sub-task
Reporter: Ilya Shishkov
When you need to analyze cases of partitions inconsistency, full list of
partitions is needed, but there is an optimization which replaces list of
consecutive partitions with ranges. So, as you can see below, we can not find
partition 2 by simple text search:
{quote}
2021-08-11 23:12:11.338 [WARN
][sys-#213][org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl]
Partitions have been scheduled for rebalancing due to outdated update counter
[grp=group, readyTopVer=AffinityTopologyVersion [topVer=33, minorTopVer=0],
topVer=AffinityTopologyVersion [topVer=33, minorTopVer=0],
nodeId=8e05997a-4ff7-4fdb-9480-cfbcefa8bbf5,
partsFull=[*{color:red}1-3{color}*, ...], partsHistorical=[]]
{quote}
{quote}
2021-08-11 23:12:11.338 [WARN
][sys-#213][org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture]
Partitions weren't present in any history reservation: [[[grp=grp2
part=[[*{color:red}1-3{color}*]]]
{quote}
We need to remove this optimization, because it can lead to mistakes in logs
analysis.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-17254) Storage API for RAFT snapshot streaming
[
https://issues.apache.org/jira/browse/IGNITE-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Puchkovskiy reassigned IGNITE-17254:
--
Assignee: Roman Puchkovskiy
> Storage API for RAFT snapshot streaming
> ---
>
> Key: IGNITE-17254
> URL: https://issues.apache.org/jira/browse/IGNITE-17254
> Project: Ignite
> Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
> Labels: ignite-3
>
> Current PartitionStorage has a method for taking snapshots (and a method for
> restoring from them). The methods work with files. In IGNITE-17253, we are
> going to switch to streaming protocol (which does not use files at all). So
> we need streaming snapshot-related methods on MvPartitionStorage.
> h3. UPDATE
> From my understanding, we basically need two more methods in the MV partition
> storage to manipulate data:
> {code:java}
> // This one represents a combination of "addWrite" and "commitWrite", but a
> faster one.
> // It will be used in incoming snapshot copier to write data.
> void addWriteCommitted(RowId rowId, BinaryRow row, HybridTimestamp
> commitTimestamp);
> // This one will be used to get all versions of a particular row and form a
> SnapshotMvDataResponse message.
> Cursor scanVersionsEx(RowId rowId);{code}
> Both of them are very simple and, for the most part, match their counterparts
> in every storage.
> What's not as simple is the fail-over scenario. We shouldn't treat
> partially-downloaded partition as a valid set of data. So, the easiest was
> (in My opinion) to achieve this is to:
> * set "lastAppliedIndex" value to "-1" before writing any the data
> * drop old data (these two steps may be swapped, it's not that important)
> * download and write everything
> * set valid value for "lastAppliedIndex"
> This is a crude solution, but should work well, especially if properly
> documented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17864) Optimize scan(HybridTimestamp.MAX_VALUE) and read(HybridTimestamp.MAX_VALUE)
[ https://issues.apache.org/jira/browse/IGNITE-17864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17864: - Description: Please check https://issues.apache.org/jira/browse/IGNITE-17856 for more details about scan(UUID txId) and read(RowId rowId, UUID txId) removal. Within given ticket it's expected that timestamp based scans and reads will be optimized in order to perform HybridTimestamp.MAX_VALUE bouned operations in a most efficient way. (was: Please check https://issues.apache.org/jira/browse/IGNITE-17856 for more details about scan(UUID txId) and read(RowId rowId, UUID txId) removal. Within given ticket it's expected that timestamp based scans will be optimized in order to perform) > Optimize scan(HybridTimestamp.MAX_VALUE) and read(HybridTimestamp.MAX_VALUE) > > > Key: IGNITE-17864 > URL: https://issues.apache.org/jira/browse/IGNITE-17864 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > > Please check https://issues.apache.org/jira/browse/IGNITE-17856 for more > details about scan(UUID txId) and read(RowId rowId, UUID txId) removal. > Within given ticket it's expected that timestamp based scans and reads will > be optimized in order to perform HybridTimestamp.MAX_VALUE bouned operations > in a most efficient way. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17864) Optimize scan(HybridTimestamp.MAX_VALUE) and read(HybridTimestamp.MAX_VALUE)
[ https://issues.apache.org/jira/browse/IGNITE-17864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17864: - Description: Please check https://issues.apache.org/jira/browse/IGNITE-17856 for more details about scan(UUID txId) and read(RowId rowId, UUID txId) removal. Within given ticket it's expected that timestamp based scans will be optimized in order to perform (was: Please check https://issues.apache.org/jira/browse/IGNITE-17856 for more details about) > Optimize scan(HybridTimestamp.MAX_VALUE) and read(HybridTimestamp.MAX_VALUE) > > > Key: IGNITE-17864 > URL: https://issues.apache.org/jira/browse/IGNITE-17864 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > > Please check https://issues.apache.org/jira/browse/IGNITE-17856 for more > details about scan(UUID txId) and read(RowId rowId, UUID txId) removal. > Within given ticket it's expected that timestamp based scans will be > optimized in order to perform -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17864) Optimize scan(HybridTimestamp.MAX_VALUE) and read(HybridTimestamp.MAX_VALUE)
[ https://issues.apache.org/jira/browse/IGNITE-17864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17864: - Description: Please check https://issues.apache.org/jira/browse/IGNITE-17856 for more details about > Optimize scan(HybridTimestamp.MAX_VALUE) and read(HybridTimestamp.MAX_VALUE) > > > Key: IGNITE-17864 > URL: https://issues.apache.org/jira/browse/IGNITE-17864 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > > Please check https://issues.apache.org/jira/browse/IGNITE-17856 for more > details about -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17864) Optimize scan(HybridTimestamp.MAX_VALUE) and read(HybridTimestamp.MAX_VALUE)
[ https://issues.apache.org/jira/browse/IGNITE-17864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17864: - Summary: Optimize scan(HybridTimestamp.MAX_VALUE) and read(HybridTimestamp.MAX_VALUE) (was: Properly implement scan(HybridTimestamp.MAX_VALUE)) > Optimize scan(HybridTimestamp.MAX_VALUE) and read(HybridTimestamp.MAX_VALUE) > > > Key: IGNITE-17864 > URL: https://issues.apache.org/jira/browse/IGNITE-17864 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17864) Properly implement scan(HybridTimestamp.MAX_VALUE)
Alexander Lapin created IGNITE-17864: Summary: Properly implement scan(HybridTimestamp.MAX_VALUE) Key: IGNITE-17864 URL: https://issues.apache.org/jira/browse/IGNITE-17864 Project: Ignite Issue Type: Improvement Reporter: Alexander Lapin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17863) ThreadNameValidationTest fails if run alone (or first in suite)
Roman Puchkovskiy created IGNITE-17863: -- Summary: ThreadNameValidationTest fails if run alone (or first in suite) Key: IGNITE-17863 URL: https://issues.apache.org/jira/browse/IGNITE-17863 Project: Ignite Issue Type: Bug Reporter: Roman Puchkovskiy Assignee: Roman Puchkovskiy Fix For: 2.15 This seems to happen because code that configures Log4j logging system triggers creation of a thread factory after the test saves the count of created thread factories, so the test thinks that there is some production code doing so, even though it's an artifact of our test code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17862) Restore grade build by adding newly added replicas module to the flow
[ https://issues.apache.org/jira/browse/IGNITE-17862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Pochatkin updated IGNITE-17862: --- Component/s: build > Restore grade build by adding newly added replicas module to the flow > - > > Key: IGNITE-17862 > URL: https://issues.apache.org/jira/browse/IGNITE-17862 > Project: Ignite > Issue Type: Bug > Components: build >Reporter: Alexander Lapin >Assignee: Mikhail Pochatkin >Priority: Major > > New replicator module, that was added during tx RW implementation, wasn't > added to the gradle build scripts and resources. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17862) Restore grade build by adding newly added replicas module to the flow
[ https://issues.apache.org/jira/browse/IGNITE-17862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Pochatkin updated IGNITE-17862: --- Labels: ignite-3 (was: ) > Restore grade build by adding newly added replicas module to the flow > - > > Key: IGNITE-17862 > URL: https://issues.apache.org/jira/browse/IGNITE-17862 > Project: Ignite > Issue Type: Bug > Components: build >Reporter: Alexander Lapin >Assignee: Mikhail Pochatkin >Priority: Major > Labels: ignite-3 > > New replicator module, that was added during tx RW implementation, wasn't > added to the gradle build scripts and resources. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-17852) an error caused the server node to exit; "CRC hasn't been calculated"
[
https://issues.apache.org/jira/browse/IGNITE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov resolved IGNITE-17852.
--
Resolution: Fixed
> an error caused the server node to exit; "CRC hasn't been calculated"
> -
>
> Key: IGNITE-17852
> URL: https://issues.apache.org/jira/browse/IGNITE-17852
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.12
>Reporter: Denis Kuznetsov
>Assignee: Nikolay Izhikov
>Priority: Minor
> Labels: ise
> Time Spent: 20m
> Remaining Estimate: 0h
>
> an error caused the server node to exit:
> {code:java}
> 2022-10-04 20:00:48.828 [ERROR][db-checkpoint-thread-#220][] Critical system
> error detected. Will be handled accordingly to configured handler
> [hnd=StopNodeOrHaltFailure
> Handler [tryStop=false, timeout=0, super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failur
> eCtx=FailureContext [type=CRITICAL_ERROR, err=class
> o.a.i.IgniteCheckedException: Compound exception for CountDownFuture.]]
> org.apache.ignite.IgniteCheckedException: Compound exception for
> CountDownFuture.
> at
> org.apache.ignite.internal.util.future.CountDownFuture.addError(CountDownFuture.java:72)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.util.future.CountDownFuture.onDone(CountDownFuture.java:46)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.util.future.CountDownFuture.onDone(CountDownFuture.java:28)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:478)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:166)
> ~[ignite-core-2.12.0-p6.jar:2.12.
> 0-p6]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> ~[?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> ~[?:?]
> at java.lang.Thread.run(Thread.java:829) [?:?]
> Suppressed: java.lang.AssertionError: CRC hasn't been calculated,
> crc=0
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.write(FilePageStore.java:737)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageReadWriteManagerImpl.write(PageReadWriteManagerImpl.java:116)
> ~[ignite-core-2.12.0
> -p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.write(FilePageStoreManager.java:637)
> ~[ignite-core-2.12.0-p6.jar:2.1
> 2.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointManager.lambda$new$0(CheckpointManager.java:175)
> ~[ignite-core-2.12.0-p6.
> jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter$1.writePage(CheckpointPagesWriter.java:266)
> ~[ignite-core-2.1
> 2.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.copyPageForCheckpoint(PageMemoryImpl.java:1343)
> ~[ignite-core-2.12.0-p6
> .jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1250)
> ~[ignite-core-2.12.0-p6.j
> ar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.writePages(CheckpointPagesWriter.java:207)
> ~[ignite-core-2.12
> .0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:151)
> ~[ignite-core-2.12.0-p6.j
> ar:2.12.0-p6]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> ~[?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> ~[?:?]
> at java.lang.Thread.run(Thread.java:829) [?:?] {code}
> node configured with compression:
> {code:java}
> {code}
> the node exited during the snapshot and caused the snapshot to stop:
> {code:java}
> 20221005_092448 ignite.log:2022-10-05 02:58:50.787
> [ERROR][disco-event-worker-#190][org.apache.ignite.internal.processors.cache.p
> ersistence.snapshot.AbstractSnapshotFutureTask] Snapshot task has accepted
> exception to stop at
>
[jira] [Updated] (IGNITE-17852) an error caused the server node to exit; "CRC hasn't been calculated"
[
https://issues.apache.org/jira/browse/IGNITE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-17852:
-
Fix Version/s: 2.15
> an error caused the server node to exit; "CRC hasn't been calculated"
> -
>
> Key: IGNITE-17852
> URL: https://issues.apache.org/jira/browse/IGNITE-17852
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.12
>Reporter: Denis Kuznetsov
>Assignee: Nikolay Izhikov
>Priority: Minor
> Labels: ise
> Fix For: 2.15
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> an error caused the server node to exit:
> {code:java}
> 2022-10-04 20:00:48.828 [ERROR][db-checkpoint-thread-#220][] Critical system
> error detected. Will be handled accordingly to configured handler
> [hnd=StopNodeOrHaltFailure
> Handler [tryStop=false, timeout=0, super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failur
> eCtx=FailureContext [type=CRITICAL_ERROR, err=class
> o.a.i.IgniteCheckedException: Compound exception for CountDownFuture.]]
> org.apache.ignite.IgniteCheckedException: Compound exception for
> CountDownFuture.
> at
> org.apache.ignite.internal.util.future.CountDownFuture.addError(CountDownFuture.java:72)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.util.future.CountDownFuture.onDone(CountDownFuture.java:46)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.util.future.CountDownFuture.onDone(CountDownFuture.java:28)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:478)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:166)
> ~[ignite-core-2.12.0-p6.jar:2.12.
> 0-p6]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> ~[?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> ~[?:?]
> at java.lang.Thread.run(Thread.java:829) [?:?]
> Suppressed: java.lang.AssertionError: CRC hasn't been calculated,
> crc=0
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.write(FilePageStore.java:737)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageReadWriteManagerImpl.write(PageReadWriteManagerImpl.java:116)
> ~[ignite-core-2.12.0
> -p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.write(FilePageStoreManager.java:637)
> ~[ignite-core-2.12.0-p6.jar:2.1
> 2.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointManager.lambda$new$0(CheckpointManager.java:175)
> ~[ignite-core-2.12.0-p6.
> jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter$1.writePage(CheckpointPagesWriter.java:266)
> ~[ignite-core-2.1
> 2.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.copyPageForCheckpoint(PageMemoryImpl.java:1343)
> ~[ignite-core-2.12.0-p6
> .jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1250)
> ~[ignite-core-2.12.0-p6.j
> ar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.writePages(CheckpointPagesWriter.java:207)
> ~[ignite-core-2.12
> .0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:151)
> ~[ignite-core-2.12.0-p6.j
> ar:2.12.0-p6]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> ~[?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> ~[?:?]
> at java.lang.Thread.run(Thread.java:829) [?:?] {code}
> node configured with compression:
> {code:java}
> {code}
> the node exited during the snapshot and caused the snapshot to stop:
> {code:java}
> 20221005_092448 ignite.log:2022-10-05 02:58:50.787
> [ERROR][disco-event-worker-#190][org.apache.ignite.internal.processors.cache.p
> ersistence.snapshot.AbstractSnapshotFutureTask] Snapshot task has accepted
> exception to stop at
[jira] [Updated] (IGNITE-17839) Improve semantics of implicit txn management for async transactions
[
https://issues.apache.org/jira/browse/IGNITE-17839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Scherbakov updated IGNITE-17839:
---
Release Note: New API for async implicit transactions:
runInTransactionAsync (was: TBD)
> Improve semantics of implicit txn management for async transactions
> ---
>
> Key: IGNITE-17839
> URL: https://issues.apache.org/jira/browse/IGNITE-17839
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0
>
>
> There is a usability issue for transaction implicit management API.
> For async transaction tx.commit must be called after all async ops are
> enlisted to the transaction. This is a responsibility of a user in case of
> explicit management.
> But, for implicit management, tx.commit must be called automatically. This is
> not generally not possible for async flows, because by the end of a closure
> some operations are not yet started.
> The example:
> runInTransactionAsync(tx -> {
> return opAsync().thenComposeAsync(res -> otherOpAsync());
> })
> We can fix this by introducing async context for implicit management and
> require a user to return last completion stage in the chain, like:
> CompletableFuture runInTransactionAsync(Function CompletableFuture> clo);
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17859) Update indexes on data modifications
[
https://issues.apache.org/jira/browse/IGNITE-17859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17859:
-
Description:
h3. Motivation
For the sake of better performance and common sense, it is necessary to
integrate indexes into data manipulation flow, which implies
* Integrating indexes into the process of efficient evaluation keys to rowsIds
both within pure read/scan operations and as part of modification operations in
order to find proper rowId of a row to be modified.
* Integrating indexes into the process of data modification itself, meaning
that not only data should be updated but also all corresponding indexes should
be populated with updated entries along with cleanup on transaction rollback.
Given Jira issue is about second part.
*Definition* *of Done*
* All indexes with relevant schema version are populated with modified rows.
* All pending index entries are removed on tx rollback.
h3. Implementation Notes
# Seems, that it has sense to introduce new Index abstractions that will
manage update logic internally. Something like following:
** Index
** HashIndex extends Index
** HashUniqueIndex extends HashIndex
** SortedIndex extneds Index
** SorteUniquedIndex extneds SortedIndex
# In order to define which indexes to update both during update itself or
during rollback it'll be useful to add extra parameter _schemaVersion_ to each
operation enlisted into transaction. All in all, that will allow to select only
proper set of indexes with relevant schema versions.
# During transaction rollback it'll be necessary to cleanup outdated pending
entries that were touched during tx lifetime.
h4. More details about *first* item.
Index itself may declare update() and other methods that will have
index-type-specific lock management and uniqueness processing logic, e.g. for
HashIndex.update following is suggested:
{code:java}
@Override
public CompletableFuture update(UUID txId, TxState txState, Tuple oldRow, Tuple
newRow, VersionChain rowId) {
Tuple oldVal = oldRow == Tuple.TOMBSTONE ? Tuple.TOMBSTONE :
oldRow.select(col);
Tuple newVal = newRow == Tuple.TOMBSTONE ? Tuple.TOMBSTONE :
newRow.select(col);
List futs = new ArrayList<>();
if (!oldVal.equals(newVal)) {
if (oldVal.length() > 0) {
Lock lock0 = lockTable.getOrAddEntry(oldVal);
txState.addLock(lock0);
futs.add(lock0.acquire(txId, LockMode.IX));
// Do not remove bookmarks due to multi-versioning.
}
if (newVal.length() > 0) {
Lock lock0 = lockTable.getOrAddEntry(newVal);
txState.addLock(lock0);
futs.add(lock0.acquire(txId, LockMode.IX).thenAccept(ignored0 -> {
if (index.insert(newVal, rowId)) {
txState.addUndo(() -> index.remove(newVal, rowId));
}
}));
}
}
return CompletableFuture.allOf(futs.toArray(new CompletableFuture[0]));
} {code}
Further details could be found in [https://github.com/ascherbakoff/ai3-txn-mvp]
Detailed lock management design is described in [IEP-91-Locking
model|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol#IEP91:Transactionprotocol-Lockingmodel]
See index related sections, e.g. for HashIndex following lock flow is
suggested:
{code:java}
Non-unique locks
// scan
S_commit(key)
// insert
IX_commit(key)
// delete
IX_commit(key) {code}
h4. More details about *third* item.
Please see cleanup flow described in
https://issues.apache.org/jira/browse/IGNITE-17673
It might have sense to consider all three items as separate tickets.
It also worth to mention that index modification is triggered from
PartitionListener
* handleUpdateCommand
* handleUpdateAllCommand
* handleTxCleanupCommand
was:
h3. Motivation
For the sake of better performance and common sense, it is necessary to
integrate indexes into data manipulation flow, which implies
* Integrating indexes into the process of efficient evaluation keys to rowsIds
both within pure read/scan operations and as part of modification operations in
order to find proper rowId of a row to be modified.
* Integrating indexes into the process of data modification itself, meaning
that not only data should be updated but also all corresponding indexes should
be populated with updated entries along with cleanup on transaction rollback.
Given Jira issue is about second part.
*Definition* *of Done*
* All indexes with relevant schema version are populated with modified rows.
* All pending index entries are removed on tx rollback.
h3. Implementation Notes
# Seems, that it has sense to introduce new Index abstractions that will
manage update logic internally. Something like following:
** Index
** HashIndex extends Index
** HashUniqueIndex extends HashIndex
** SortedIndex extneds Index
[jira] [Updated] (IGNITE-17859) Update indexes on data modifications
[
https://issues.apache.org/jira/browse/IGNITE-17859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17859:
-
Description:
h3. Motivation
For the sake of better performance and common sense, it is necessary to
integrate indexes into data manipulation flow, which implies
* Integrating indexes into the process of efficient evaluation keys to rowsIds
both within pure read/scan operations and as part of modification operations in
order to find proper rowId of a row to be modified.
* Integrating indexes into the process of data modification itself, meaning
that not only data should be updated but also all corresponding indexes should
be populated with updated entries along with cleanup on transaction rollback.
Given Jira issue is about second part.
*Definition* *of Done*
* All indexes with relevant schema version are populated with modified rows.
* All pending index entries are removed on tx rollback.
h3. Implementation Notes
# Seems, that it has sense to introduce new Index abstractions that will
manage update logic internally. Something like following:
** Index
** HashIndex extends Index
** HashUniqueIndex extends HashIndex
** SortedIndex extneds Index
** SorteUniquedIndex extneds SortedIndex
# In order to define which indexes to update both during update itself or
during rollback it'll be useful to add extra parameter _schemaVersion_ to each
operation enlisted into transaction. All in all, that will allow to select only
proper set of indexes with relevant schema versions.
# During transaction rollback it'll be necessary to cleanup outdated pending
entries that were touched during tx lifetime.
h4. More details about *first* item.
Index itself may declare update() and other methods that will have
index-type-specific lock management and uniqueness processing logic, e.g. for
HashIndex.update following is suggested:
{code:java}
@Override
public CompletableFuture update(UUID txId, TxState txState, Tuple oldRow, Tuple
newRow, VersionChain rowId) {
Tuple oldVal = oldRow == Tuple.TOMBSTONE ? Tuple.TOMBSTONE :
oldRow.select(col);
Tuple newVal = newRow == Tuple.TOMBSTONE ? Tuple.TOMBSTONE :
newRow.select(col);
List futs = new ArrayList<>();
if (!oldVal.equals(newVal)) {
if (oldVal.length() > 0) {
Lock lock0 = lockTable.getOrAddEntry(oldVal);
txState.addLock(lock0);
futs.add(lock0.acquire(txId, LockMode.IX));
// Do not remove bookmarks due to multi-versioning.
}
if (newVal.length() > 0) {
Lock lock0 = lockTable.getOrAddEntry(newVal);
txState.addLock(lock0);
futs.add(lock0.acquire(txId, LockMode.IX).thenAccept(ignored0 -> {
if (index.insert(newVal, rowId)) {
txState.addUndo(() -> index.remove(newVal, rowId));
}
}));
}
}
return CompletableFuture.allOf(futs.toArray(new CompletableFuture[0]));
} {code}
Further details could be found in [https://github.com/ascherbakoff/ai3-txn-mvp]
Detailed lock management design is described in [IEP-91-Locking
model|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol#IEP91:Transactionprotocol-Lockingmodel]
See index related sections, e.g. for HashIndex following lock flow is
suggested:
{code:java}
Non-unique locks
// scan
S_commit(key)
// insert
IX_commit(key)
// delete
IX_commit(key) {code}
h4. More details about *third* item.
Please see cleanup flow described in
https://issues.apache.org/jira/browse/IGNITE-17673
It might have sense to consider all three items as separate tickets.
It also worth to mention that index modification is triggered from
PartitionListener
* handleUpdateCommand
* handleUpdateAllCommand
* handleTxCleanupCommand
was:
h3. Motivation
For the sake of better performance and common sense, it is necessary to
integrate indexes into data manipulation flow, which implies
* Integrating indexes into the process of efficient evaluation keys to rowsIds
both within pure read/scan operations and as part of modification operations in
order to find proper rowId of a row to be modified.
* Integrating indexes into the process of data modification itself, meaning
that not only data should be updated but also all corresponding indexes should
be populated with updated entries along with cleanup on transaction rollback.
Given Jira issue is about second part.
*Definition* *of Done*
* All indexes with relevant schema version are populated with modified rows.
* All pending index entries are removed on tx rollback.
h3. Implementation Notes
# Seems, that it has sense to introduce new Index abstractions that will
manage update logic internally. Something like following:
** Index
** HashIndex extends Index
** HashUniqueIndex extends HashIndex
** SortedIndex extneds
[jira] [Assigned] (IGNITE-17417) Node name support in CLI
[
https://issues.apache.org/jira/browse/IGNITE-17417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Gagarkin reassigned IGNITE-17417:
--
Assignee: Ivan Gagarkin
> Node name support in CLI
>
>
> Key: IGNITE-17417
> URL: https://issues.apache.org/jira/browse/IGNITE-17417
> Project: Ignite
> Issue Type: Task
> Components: cli
>Reporter: Aleksandr
>Assignee: Ivan Gagarkin
>Priority: Major
> Labels: ignite-3, ignite-3-cli-tool
>
> Now there is only one way to point at the ignite node in the CLI – `node-url`
> or `cluster-url` options. They are represented as an URL that sometimes is
> too annoying to type and remember. It would be much more user-friendly to
> have the second option here. I think node name could be used as a CLI option.
> I propose to add `node-name` option to every command that requires
> `cluster-url` or `node-url`. So, the following user story would be possible:
> {code:bash}
> > ignite
> [disconnected]> connect node2
> [node2]> connect node2
> [node2]> node config show --node-name node1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17847) Dump threads if afterTest() times out
[ https://issues.apache.org/jira/browse/IGNITE-17847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17614999#comment-17614999 ] Roman Puchkovskiy commented on IGNITE-17847: Thank you for the review > Dump threads if afterTest() times out > - > > Key: IGNITE-17847 > URL: https://issues.apache.org/jira/browse/IGNITE-17847 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: MakeTeamcityGreenAgain > Fix For: 2.15 > > Time Spent: 40m > Remaining Estimate: 0h > > afterTest() is not covered by timeout. It might be unsafe to cancel > afterTest(), but it might be useful to at least dump threads if we see that > it got stuck. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17859) Update indexes on data modifications
[
https://issues.apache.org/jira/browse/IGNITE-17859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17859:
-
Description:
h3. Motivation
For the sake of better performance and common sense, it is necessary to
integrate indexes into data manipulation flow, which implies
* Integrating indexes into the process of efficient evaluation keys to rowsIds
both within pure read/scan operations and as part of modification operations in
order to find proper rowId of a row to be modified.
* Integrating indexes into the process of data modification itself, meaning
that not only data should be updated but also all corresponding indexes should
be populated with updated entries along with cleanup on transaction rollback.
Given Jira issue is about second part.
*Definition* *of Done*
* All indexes with relevant schema version are populated with modified rows.
* All pending index entries are removed on tx rollback.
h3. Implementation Notes
# Seems, that it has sense to introduce new Index abstractions that will
manage update logic internally. Something like following:
** Index
** HashIndex extends Index
** HashUniqueIndex extends HashIndex
** SortedIndex extneds Index
** SorteUniquedIndex extneds SortedIndex
# In order to define which indexes to update both during update itself or
during rollback it'll be useful to add extra parameter _schemaVersion_ to each
operation enlisted into transaction. All in all, that will allow to select only
proper set of indexes with relevant schema versions.
# During transaction rollback it'll be necessary to cleanup outdated pending
entries that were touched during tx lifetime.
h4. More details about *first* item.
Index itself may declare update() and other methods that will have
index-type-specific lock management and uniqueness processing logic, e.g. for
HashIndex.update following is suggested:
{code:java}
@Override
public CompletableFuture update(UUID txId, TxState txState, Tuple oldRow, Tuple
newRow, VersionChain rowId) {
Tuple oldVal = oldRow == Tuple.TOMBSTONE ? Tuple.TOMBSTONE :
oldRow.select(col);
Tuple newVal = newRow == Tuple.TOMBSTONE ? Tuple.TOMBSTONE :
newRow.select(col);
List futs = new ArrayList<>();
if (!oldVal.equals(newVal)) {
if (oldVal.length() > 0) {
Lock lock0 = lockTable.getOrAddEntry(oldVal);
txState.addLock(lock0);
futs.add(lock0.acquire(txId, LockMode.IX));
// Do not remove bookmarks due to multi-versioning.
}
if (newVal.length() > 0) {
Lock lock0 = lockTable.getOrAddEntry(newVal);
txState.addLock(lock0);
futs.add(lock0.acquire(txId, LockMode.IX).thenAccept(ignored0 -> {
if (index.insert(newVal, rowId)) {
txState.addUndo(() -> index.remove(newVal, rowId));
}
}));
}
}
return CompletableFuture.allOf(futs.toArray(new CompletableFuture[0]));
} {code}
Further details could be found in [https://github.com/ascherbakoff/ai3-txn-mvp]
Detailed lock management design is described in [IEP-91-Locking
model|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol#IEP91:Transactionprotocol-Lockingmodel]
See index related sections, e.g. for HashIndex following lock flow is
suggested:
{code:java}
Non-unique locks
// scan
S_commit(key)
// insert
IX_commit(key)
// delete
IX_commit(key) {code}
h4. More details about *third* item.
Please see cleanup flow described in
https://issues.apache.org/jira/browse/IGNITE-17673
It might have sense to consider all three items as separate tickets.
was:
h3. Motivation
For the sake of better performance and common sense, it is necessary to
integrate indexes into data manipulation flow, which implies
* Integrating indexes into the process of efficient evaluation keys to rowsIds
both within pure read/scan operations and as part of modification operations in
order to find proper rowId of a row to be modified.
* Integrating indexes into the process of data modification itself, meaning
that not only data should be updated but also all corresponding indexes should
be populated with updated entries along with cleanup on transaction rollback.
Given Jira issue is about second part.
*Definition* *of Done*
* All indexes with relevant schema version are populated with modified rows.
* All pending index entries are removed on tx rollback.
h3. Implementation Notes
# Seems, that it has sense to introduce new Index abstractions that will
manage update logic internally. Something like following:
** Index
** HashIndex extends Index
** HashUniqueIndex extends HashIndex
** SortedIndex extneds Index
** SorteUniquedIndex extneds SortedIndex
# In order to define which indexes to update both during update itself or
during rollback it'll be useful to add extra
[jira] [Updated] (IGNITE-17859) Update indexes on data modifications
[
https://issues.apache.org/jira/browse/IGNITE-17859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17859:
-
Description:
h3. Motivation
For the sake of better performance and common sense, it is necessary to
integrate indexes into data manipulation flow, which implies
* Integrating indexes into the process of efficient evaluation keys to rowsIds
both within pure read/scan operations and as part of modification operations in
order to find proper rowId of a row to be modified.
* Integrating indexes into the process of data modification itself, meaning
that not only data should be updated but also all corresponding indexes should
be populated with updated entries along with cleanup on transaction rollback.
Given Jira issue is about second part.
*Definition* *of Done*
* All indexes with relevant schema version are populated with modified rows.
* All pending index entries are removed on tx rollback.
h3. Implementation Notes
# Seems, that it has sense to introduce new Index abstractions that will
manage update logic internally. Something like following:
** Index
** HashIndex extends Index
** HashUniqueIndex extends HashIndex
** SortedIndex extneds Index
** SorteUniquedIndex extneds SortedIndex
# In order to define which indexes to update both during update itself or
during rollback it'll be useful to add extra parameter _schemaVersion_ to each
operation enlisted into transaction. All in all, that will allow to select only
proper set of indexes with relevant schema versions.
# During transaction rollback it'll be necessary to cleanup outdated pending
entries that were touched during tx lifetime.
h4. More details about *first* item.
Index itself may declare update() and other methods that will have
index-type-specific lock management and uniqueness processing logic, e.g. for
HashIndex.update following is suggested:
{code:java}
@Override
public CompletableFuture update(UUID txId, TxState txState, Tuple oldRow, Tuple
newRow, VersionChain rowId) {
Tuple oldVal = oldRow == Tuple.TOMBSTONE ? Tuple.TOMBSTONE :
oldRow.select(col);
Tuple newVal = newRow == Tuple.TOMBSTONE ? Tuple.TOMBSTONE :
newRow.select(col);
List futs = new ArrayList<>();
if (!oldVal.equals(newVal)) {
if (oldVal.length() > 0) {
Lock lock0 = lockTable.getOrAddEntry(oldVal);
txState.addLock(lock0);
futs.add(lock0.acquire(txId, LockMode.IX));
// Do not remove bookmarks due to multi-versioning.
}
if (newVal.length() > 0) {
Lock lock0 = lockTable.getOrAddEntry(newVal);
txState.addLock(lock0);
futs.add(lock0.acquire(txId, LockMode.IX).thenAccept(ignored0 -> {
if (index.insert(newVal, rowId)) {
txState.addUndo(() -> index.remove(newVal, rowId));
}
}));
}
}
return CompletableFuture.allOf(futs.toArray(new CompletableFuture[0]));
} {code}
Further details could be found in [https://github.com/ascherbakoff/ai3-txn-mvp]
Detailed lock management design is described in [IEP-91-Locking
model|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol#IEP91:Transactionprotocol-Lockingmodel]
See index related sections, e.g. for HashIndex following lock flow is
suggested:
{code:java}
Non-unique locks
// scan
S_commit(key)
// insert
IX_commit(key)
// delete
IX_commit(key) {code}
h4. More details about *third* item.
Please see cleanup flow described in
https://issues.apache.org/jira/browse/IGNITE-17673
> Update indexes on data modifications
>
>
> Key: IGNITE-17859
> URL: https://issues.apache.org/jira/browse/IGNITE-17859
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> h3. Motivation
> For the sake of better performance and common sense, it is necessary to
> integrate indexes into data manipulation flow, which implies
> * Integrating indexes into the process of efficient evaluation keys to
> rowsIds both within pure read/scan operations and as part of modification
> operations in order to find proper rowId of a row to be modified.
> * Integrating indexes into the process of data modification itself, meaning
> that not only data should be updated but also all corresponding indexes
> should be populated with updated entries along with cleanup on transaction
> rollback.
> Given Jira issue is about second part.
> *Definition* *of Done*
> * All indexes with relevant schema version are populated with modified rows.
> * All pending index entries are removed on tx rollback.
> h3. Implementation Notes
> # Seems, that it has sense to introduce new Index abstractions that will
> manage
[jira] [Comment Edited] (IGNITE-17369) Snapshot is inconsistent under streamed loading with 'allowOverwrite==false'.
[ https://issues.apache.org/jira/browse/IGNITE-17369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612278#comment-17612278 ] Vladimir Steshin edited comment on IGNITE-17369 at 10/10/22 9:09 AM: - Snapshot can begin work with different state of kin partitions. The shapshot process waits for the datastreamer futures. (_GridCacheMvccManager.addDataStreamerFuture()_). The problem is that these futures are created separately and concurrently on primary and backups nodes by _IsolatedUpdater_. As result, at the checkpoint some backups might be written without the primaries. And opposite. There are no updates accepted during checkpoint. Late streamer updates is not written to snapshoting partitions. Solutions: 1) V1 (PR 10285). PR brings watching _DataStreamer_ futures in snapshot process. The futures are created before writing streamer batch on any node. We cannot relay on the future as on final and consistent write for streamer batch or certain entry. But we know that datastreamer is in progress at the checkpoint and that it is on pause. We can invalidate snapshot at this moment. In theory the solution is not resilent. On streamer batch could've been entirely written before snapshot. Second batch after. First batch writes partition on primaries or backups. Second writes the rest. Snapshot is inconsistent. 2) V2 (PR 10286). _IsolatedUpdater_ could just notify snapshot process, if exists, that concurrent inconsistent update is on. A notification of at least one entry on any node wound be enough. Should work in practice. In theory the solution is not resilent. On streamer batch could've been entirely written before snapshot. Second batch after. First batch writes partition on primaries or backups. Second writes the rest. Snapshot is inconsistent. 3) V3 (PR 10284). We could mark that _DataStreamer_ is on on any first streamer batch received. And unmark somehow later. If _DataStreamer_ is marked as active, the snapshot process could check this mark. Since the mark is set before writting data, it is set before the datastreamer future which is being waited for in the snapshot process. This guaraties the mark are visible before the snapshot. The problem is how to close such mark. When the streaming node left? Node can live forever. Send special closing request? The streamer node can do not close streamer at all. Meaning no _close()_ is invoked. Moreoever, _DataStreamer_ works through _CommunicationSPI_. Which doesn't guarantee delivery. We can't be sure that closing request is delivered and streamer is unmarked on the accepting node. Do we need to set this mark with a timeout and re-set with next datastreamer batche? Which timeout? Bind to what? On closing requests, a rebalance can happen. Should be processed too. Looks like we need a discovery closing message. Much simpler and reliable. Also, datastreamer can be canceled. Meaning one batches were written before snapshot. Other won't ever. 4) V4 (PR 10299). We could watch streamer is already registered before snapshot and simultaneously. The snapshot init message is accepted on client. We may adjust snapshot proces so that it would watch datastreamer even on client node. was (Author: vladsz83): Snapshot can begin work with different state of kin partitions. The shapshot process waits for the datastreamer futures. (_GridCacheMvccManager.addDataStreamerFuture()_). The problem is that these futures are created separately and concurrently on primary and backups nodes by _IsolatedUpdater_. As result, at the checkpoint some backups might be written without the primaries. And opposite. There are no updates accepted during checkpoint. Late streamer updates is not written to snapshoting partitions. Solutions: 1) V1 (PR 10285). PR brings watching _DataStreamer_ futures in snapshot process. The futures are created before writing streamer batch on any node. We cannot relay on the future as on final and consistent write for streamer batch or certain entry. But we know that datastreamer is in progress at the checkpoint and that it is on pause. We can invalidate snapshot at this moment. In theory the solution is not resilent. On streamer batch could've been entirely written before snapshot. Second batch after. First batch writes partition on primaries or backups. Second writes the rest. Snapshot is inconsistent. 2) V2 (PR 10286). _IsolatedUpdater_ could just notify snapshot process, if exists, that concurrent inconsistent update is on. A notification of at least one entry on any node wound be enough. Should work in practice. In theory the solution is not resilent. On streamer batch could've been entirely written before snapshot. Second batch after. First batch writes partition on primaries or backups. Second writes the rest. Snapshot is inconsistent. 3) V3 (PR 10284). We could mark that _DataStreamer_ is on on any
[jira] [Assigned] (IGNITE-17862) Restore grade build by adding newly added replicas module to the flow
[ https://issues.apache.org/jira/browse/IGNITE-17862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Pochatkin reassigned IGNITE-17862: -- Assignee: Mikhail Pochatkin > Restore grade build by adding newly added replicas module to the flow > - > > Key: IGNITE-17862 > URL: https://issues.apache.org/jira/browse/IGNITE-17862 > Project: Ignite > Issue Type: Bug >Reporter: Alexander Lapin >Assignee: Mikhail Pochatkin >Priority: Major > > New replicator module, that was added during tx RW implementation, wasn't > added to the gradle build scripts and resources. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17655) Sql. Provide integration for index scan
[
https://issues.apache.org/jira/browse/IGNITE-17655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17614974#comment-17614974
]
Konstantin Orlov commented on IGNITE-17655:
---
[~amashenkov], LGTM!
> Sql. Provide integration for index scan
> ---
>
> Key: IGNITE-17655
> URL: https://issues.apache.org/jira/browse/IGNITE-17655
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Konstantin Orlov
>Assignee: Andrey Mashenkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>
> After IGNITE-17431 indexes became a part of query plan. So, now we need to
> provide appropriate runtime to make an execution of index scans possible.
> Integration points for sql engine will be interface
> org.apache.ignite.internal.index.Index and derived
> org.apache.ignite.internal.index.SortedIndex. A few changes is required to
> those interfaces though: the {{scan}} methods should be reworked in the
> following way:
> {code:java}
> // for Index.java scan should look like this one
> Publisher scan(int partId, InternalTransaction tx, BinaryTuple
> key, BitSet columnsToInclude)
> // for SortedIndex.java range scan should look like this
> Publisher scan(int partId, InternalTransaction tx, @Nullable
> BinaryTuple leftBound, @Nullable BinaryTuple rightBound, int flags, BitSet
> columnsToInclude)
> {code}
> Besides, we need to reconsider the way we work with an index conditions.
> Currently we stick with Ignite-2 way, where search row is represented by a
> tuple of the same size as table row, but only indexed columns is set to a
> particular values while others are set to null. For example for table (id,
> project_id, command_id, name) and index (command_id, project_id) a search
> tuple looks like (null, , ,
> null). We need to rework this in a way to make a search tuple match the
> actual order of indexed columns. With that said, the search tuple for the
> example above should look like (,
> ).
> Finally, since both table and index scan have the very similar API, it seems
> reasonable to rework
> {{org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode}} in a way to
> cover both cases (table scan and index scan).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17862) Restore grade build by adding newly added replicas module to the flow
Alexander Lapin created IGNITE-17862: Summary: Restore grade build by adding newly added replicas module to the flow Key: IGNITE-17862 URL: https://issues.apache.org/jira/browse/IGNITE-17862 Project: Ignite Issue Type: Bug Reporter: Alexander Lapin New replicator module, that was added during tx RW implementation, wasn't added to the gradle build scripts and resources. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-17841) Update list of PMC members and Committers
[ https://issues.apache.org/jira/browse/IGNITE-17841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erlan Aytpaev resolved IGNITE-17841. Resolution: Fixed > Update list of PMC members and Committers > - > > Key: IGNITE-17841 > URL: https://issues.apache.org/jira/browse/IGNITE-17841 > Project: Ignite > Issue Type: Task > Components: website >Reporter: Kseniya Romanova >Assignee: Erlan Aytpaev >Priority: Trivial > > Please add to the page > [https://ignite.apache.org/our-community.html#community] > > 1. new PMC member: > Ivan Daschinsky [https://whimsy.apache.org/roster/committer/ivandasch] > [https://github.com/ivandasch] > > 2. new Committers: > Kirill Tkalenko [https://whimsy.apache.org/roster/committer/tkalkirill] > [https://github.com/tkalkirill] > Mikhail Petrov [https://whimsy.apache.org/roster/committer/mpetrov] > > Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17861) It's possible to enlist an operation into already finishing or finished transaction
[ https://issues.apache.org/jira/browse/IGNITE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17861: - Summary: It's possible to enlist an operation into already finishing or finished transaction (was: It's possible to enlist an operation into already finishing transaction) > It's possible to enlist an operation into already finishing or finished > transaction > --- > > Key: IGNITE-17861 > URL: https://issues.apache.org/jira/browse/IGNITE-17861 > Project: Ignite > Issue Type: Bug >Reporter: Alexander Lapin >Assignee: Alexander Lapin >Priority: Major > > TBD -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17861) It's possible to enlist an operation into already finishing transaction
Alexander Lapin created IGNITE-17861: Summary: It's possible to enlist an operation into already finishing transaction Key: IGNITE-17861 URL: https://issues.apache.org/jira/browse/IGNITE-17861 Project: Ignite Issue Type: Bug Reporter: Alexander Lapin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-17861) It's possible to enlist an operation into already finishing transaction
[ https://issues.apache.org/jira/browse/IGNITE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin reassigned IGNITE-17861: Assignee: Alexander Lapin > It's possible to enlist an operation into already finishing transaction > --- > > Key: IGNITE-17861 > URL: https://issues.apache.org/jira/browse/IGNITE-17861 > Project: Ignite > Issue Type: Bug >Reporter: Alexander Lapin >Assignee: Alexander Lapin >Priority: Major > > TBD -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17861) It's possible to enlist an operation into already finishing transaction
[ https://issues.apache.org/jira/browse/IGNITE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17861: - Description: TBD > It's possible to enlist an operation into already finishing transaction > --- > > Key: IGNITE-17861 > URL: https://issues.apache.org/jira/browse/IGNITE-17861 > Project: Ignite > Issue Type: Bug >Reporter: Alexander Lapin >Priority: Major > > TBD -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17861) It's possible to enlist an operation into already finishing transaction
[ https://issues.apache.org/jira/browse/IGNITE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17861: - Epic Link: IGNITE-15081 > It's possible to enlist an operation into already finishing transaction > --- > > Key: IGNITE-17861 > URL: https://issues.apache.org/jira/browse/IGNITE-17861 > Project: Ignite > Issue Type: Bug >Reporter: Alexander Lapin >Assignee: Alexander Lapin >Priority: Major > > TBD -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17860) Wrong request example for the 'Compare-And-Swap' REST API command
Igor Gusev created IGNITE-17860: --- Summary: Wrong request example for the 'Compare-And-Swap' REST API command Key: IGNITE-17860 URL: https://issues.apache.org/jira/browse/IGNITE-17860 Project: Ignite Issue Type: Task Reporter: Igor Gusev [https://ignite.apache.org/docs/2.13.0/restapi#compare-and-swap] The 'Compare-And-Swap' Ignite REST API request example mentions authentificate command instead of 'Compare-And-Swap' command. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17860) Wrong request example for the 'Compare-And-Swap' REST API command
[ https://issues.apache.org/jira/browse/IGNITE-17860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Gusev updated IGNITE-17860: Component/s: documentation > Wrong request example for the 'Compare-And-Swap' REST API command > - > > Key: IGNITE-17860 > URL: https://issues.apache.org/jira/browse/IGNITE-17860 > Project: Ignite > Issue Type: Task > Components: documentation >Reporter: Igor Gusev >Priority: Major > > [https://ignite.apache.org/docs/2.13.0/restapi#compare-and-swap] > The 'Compare-And-Swap' Ignite REST API request example mentions > authentificate command instead of 'Compare-And-Swap' command. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-17852) an error caused the server node to exit; "CRC hasn't been calculated"
[
https://issues.apache.org/jira/browse/IGNITE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov reassigned IGNITE-17852:
Assignee: Nikolay Izhikov
> an error caused the server node to exit; "CRC hasn't been calculated"
> -
>
> Key: IGNITE-17852
> URL: https://issues.apache.org/jira/browse/IGNITE-17852
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.12
>Reporter: Denis Kuznetsov
>Assignee: Nikolay Izhikov
>Priority: Minor
> Labels: ise
>
> an error caused the server node to exit:
> {code:java}
> 2022-10-04 20:00:48.828 [ERROR][db-checkpoint-thread-#220][] Critical system
> error detected. Will be handled accordingly to configured handler
> [hnd=StopNodeOrHaltFailure
> Handler [tryStop=false, timeout=0, super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failur
> eCtx=FailureContext [type=CRITICAL_ERROR, err=class
> o.a.i.IgniteCheckedException: Compound exception for CountDownFuture.]]
> org.apache.ignite.IgniteCheckedException: Compound exception for
> CountDownFuture.
> at
> org.apache.ignite.internal.util.future.CountDownFuture.addError(CountDownFuture.java:72)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.util.future.CountDownFuture.onDone(CountDownFuture.java:46)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.util.future.CountDownFuture.onDone(CountDownFuture.java:28)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:478)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:166)
> ~[ignite-core-2.12.0-p6.jar:2.12.
> 0-p6]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> ~[?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> ~[?:?]
> at java.lang.Thread.run(Thread.java:829) [?:?]
> Suppressed: java.lang.AssertionError: CRC hasn't been calculated,
> crc=0
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.write(FilePageStore.java:737)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageReadWriteManagerImpl.write(PageReadWriteManagerImpl.java:116)
> ~[ignite-core-2.12.0
> -p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.write(FilePageStoreManager.java:637)
> ~[ignite-core-2.12.0-p6.jar:2.1
> 2.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointManager.lambda$new$0(CheckpointManager.java:175)
> ~[ignite-core-2.12.0-p6.
> jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter$1.writePage(CheckpointPagesWriter.java:266)
> ~[ignite-core-2.1
> 2.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.copyPageForCheckpoint(PageMemoryImpl.java:1343)
> ~[ignite-core-2.12.0-p6
> .jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1250)
> ~[ignite-core-2.12.0-p6.j
> ar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.writePages(CheckpointPagesWriter.java:207)
> ~[ignite-core-2.12
> .0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:151)
> ~[ignite-core-2.12.0-p6.j
> ar:2.12.0-p6]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> ~[?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> ~[?:?]
> at java.lang.Thread.run(Thread.java:829) [?:?] {code}
> node configured with compression:
> {code:java}
> {code}
> the node exited during the snapshot and caused the snapshot to stop:
> {code:java}
> 20221005_092448 ignite.log:2022-10-05 02:58:50.787
> [ERROR][disco-event-worker-#190][org.apache.ignite.internal.processors.cache.p
> ersistence.snapshot.AbstractSnapshotFutureTask] Snapshot task has accepted
> exception to stop at
>
[jira] [Assigned] (IGNITE-17856) MvPartitionStorage#scan(UUID) and read(UUID) should return PartitionTimestampCursor and ReadResult
[
https://issues.apache.org/jira/browse/IGNITE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin reassigned IGNITE-17856:
Assignee: Alexander Lapin
> MvPartitionStorage#scan(UUID) and read(UUID) should return
> PartitionTimestampCursor and ReadResult
> --
>
> Key: IGNITE-17856
> URL: https://issues.apache.org/jira/browse/IGNITE-17856
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> h3. Motivation
> Tx protocol design assumes that data entry exists in the form of a _write
> intent_ from the moment the entry is changed within transaction to the moment
> it is either converted to the regular value or removed during tx cleanup
> phase.
> Read Only transactions because of their lock-less nature may see such
> writeIntents (corresponding changes were implemented within IGNITE-17720 and
> IGNITE-17627) and ...
> ... according to tx design updates so does Read Write transactions.
> Let's clarify this in greater detail:
> * Originally tx finish process firstly converted the write intents into
> regular values, and only then released the locks. Thus, other RW transactions
> could never see write intents as they were always waiting for locks to be
> released.
> * In the current version of the [TX
> IEP|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol],
> the order has been changed to the opposite: locks are released before write
> Intents conversion, so that not only a RO but also a RW transactions may see
> the intents.
> All in all, that means that both read and scan RW operations may use
> timestamp based reads and scans with a properly distant timestamp in the
> future.
> *Definition of Done*
> {code:java}
> Cursor scan(UUID txId) {code}
> and
> {code:java}
> BinaryRow read(RowId rowId, UUID txId) {code}
> should be removed.
> {code:java}
> PartitionTimestampCursor scan(HybridTimestamp timestamp) {code}
> and
> {code:java}
> ReadResult read(RowId rowId, HybridTimestamp timestamp) {code}
> should be used instead.
> h3. Implementation notes
> Solution is pretty straight forward
> * Removing deprecated methods along with underneath implementations.
> * Propagating RW reads and scans to timestamp based MvPartitionStorage reads
> and scans with some sort of HybridTimestamp.MAX_VALUE as a timestamp paramter.
> * Verifying that within RW transactions returned ReadResults contain values
> with matching txIds (see TxIdMismatchException). That should be guaranteed by
> locks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17856) RW reads and scans should use timestamp based MVPartitionStorage read and scan operations
[
https://issues.apache.org/jira/browse/IGNITE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17856:
-
Summary: RW reads and scans should use timestamp based MVPartitionStorage
read and scan operations (was: MvPartitionStorage#scan(UUID) and read(UUID)
should return PartitionTimestampCursor and ReadResult)
> RW reads and scans should use timestamp based MVPartitionStorage read and
> scan operations
> -
>
> Key: IGNITE-17856
> URL: https://issues.apache.org/jira/browse/IGNITE-17856
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> h3. Motivation
> Tx protocol design assumes that data entry exists in the form of a _write
> intent_ from the moment the entry is changed within transaction to the moment
> it is either converted to the regular value or removed during tx cleanup
> phase.
> Read Only transactions because of their lock-less nature may see such
> writeIntents (corresponding changes were implemented within IGNITE-17720 and
> IGNITE-17627) and ...
> ... according to tx design updates so does Read Write transactions.
> Let's clarify this in greater detail:
> * Originally tx finish process firstly converted the write intents into
> regular values, and only then released the locks. Thus, other RW transactions
> could never see write intents as they were always waiting for locks to be
> released.
> * In the current version of the [TX
> IEP|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol],
> the order has been changed to the opposite: locks are released before write
> Intents conversion, so that not only a RO but also a RW transactions may see
> the intents.
> All in all, that means that both read and scan RW operations may use
> timestamp based reads and scans with a properly distant timestamp in the
> future.
> *Definition of Done*
> {code:java}
> Cursor scan(UUID txId) {code}
> and
> {code:java}
> BinaryRow read(RowId rowId, UUID txId) {code}
> should be removed.
> {code:java}
> PartitionTimestampCursor scan(HybridTimestamp timestamp) {code}
> and
> {code:java}
> ReadResult read(RowId rowId, HybridTimestamp timestamp) {code}
> should be used instead.
> h3. Implementation notes
> Solution is pretty straight forward
> * Removing deprecated methods along with underneath implementations.
> * Propagating RW reads and scans to timestamp based MvPartitionStorage reads
> and scans with some sort of HybridTimestamp.MAX_VALUE as a timestamp paramter.
> * Verifying that within RW transactions returned ReadResults contain values
> with matching txIds (see TxIdMismatchException). That should be guaranteed by
> locks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17856) MvPartitionStorage#scan(UUID) and read(UUID) should return PartitionTimestampCursor and ReadResult
[
https://issues.apache.org/jira/browse/IGNITE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17856:
-
Description:
h3. Motivation
Tx protocol design assumes that data entry exists in the form of a _write
intent_ from the moment the entry is changed within transaction to the moment
it is either converted to the regular value or removed during tx cleanup phase.
Read Only transactions because of their lock-less nature may see such
writeIntents (corresponding changes were implemented within IGNITE-17720 and
IGNITE-17627) and ...
... according to tx design updates so does Read Write transactions.
Let's clarify this in greater detail:
* Originally tx finish process firstly converted the write intents into
regular values, and only then released the locks. Thus, other RW transactions
could never see write intents as they were always waiting for locks to be
released.
* In the current version of the [TX
IEP|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol],
the order has been changed to the opposite: locks are released before write
Intents conversion, so that not only a RO but also a RW transactions may see
the intents.
All in all, that means that both read and scan RW operations may use timestamp
based reads and scans with a properly distant timestamp in the future.
*Definition of Done*
{code:java}
Cursor scan(UUID txId) {code}
and
{code:java}
BinaryRow read(RowId rowId, UUID txId) {code}
should be removed.
{code:java}
PartitionTimestampCursor scan(HybridTimestamp timestamp) {code}
and
{code:java}
ReadResult read(RowId rowId, HybridTimestamp timestamp) {code}
should be used instead.
h3. Implementation notes
Solution is pretty straight forward
* Removing deprecated methods along with underneath implementations.
* Propagating RW reads and scans to timestamp based MvPartitionStorage reads
and scans with some sort of HybridTimestamp.MAX_VALUE as a timestamp paramter.
* Verifying that within RW transactions returned ReadResults contain values
with matching txIds (see TxIdMismatchException). That should be guaranteed by
locks.
was:
h3. Motivation
Tx protocol design assumes that data entry exists in the form of a _write
intent_ from the moment the entry is changed within transaction to the moment
it is either converted to the regular value or removed during tx cleanup phase.
Read Only transactions because of their lock-less nature may see such
writeIntents (corresponding changes were implemented within IGNITE-17720 and
IGNITE-17627) and ...
... according to tx design updates so does Read Write transactions.
Let's clarify this in greater detail:
* Originally tx finish process firstly converted the write intents into
regular values, and only then released the locks. Thus, other RW transactions
could never see write intents as they were always waiting for locks to be
released.
* In the current version of the [TX
IEP|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol],
the order has been changed to the opposite: locks are released before write
Intents conversion, so that not only a RO but also a RW transactions may see
the intents.
All in all, that means that both read and scan RW operations should be updated
in a way similar to RO reads and scans were:
*Definition of Done*
{code:java}
Cursor scan(UUID txId) {code}
should be changed as follows
{code:java}
PartitionTimestampCursor scan(UUID txId) {code}
in order to produce an ability of definition whether the row is writeIntent or
not and reading latest non-intent row if the intent belongs to pending
transaction.
Same is about read operation.
{code:java}
BinaryRow read(RowId rowId, UUID txId) {code}
should be changed as follows
{code:java}
ReadResult read(RowId rowId, UUID txId) {code}
> MvPartitionStorage#scan(UUID) and read(UUID) should return
> PartitionTimestampCursor and ReadResult
> --
>
> Key: IGNITE-17856
> URL: https://issues.apache.org/jira/browse/IGNITE-17856
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> h3. Motivation
> Tx protocol design assumes that data entry exists in the form of a _write
> intent_ from the moment the entry is changed within transaction to the moment
> it is either converted to the regular value or removed during tx cleanup
> phase.
> Read Only transactions because of their lock-less nature may see such
> writeIntents (corresponding changes were implemented within IGNITE-17720 and
> IGNITE-17627) and ...
> ... according to tx design updates so does Read Write transactions.
> Let's clarify this in greater detail:
> * Originally tx finish process
[jira] [Updated] (IGNITE-17856) MvPartitionStorage#scan(UUID) and read(UUID) should return PartitionTimestampCursor and ReadResult
[
https://issues.apache.org/jira/browse/IGNITE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17856:
-
Description:
h3. Motivation
Tx protocol design assumes that data entry exists in the form of a _write
intent_ from the moment the entry is changed within transaction to the moment
it is either converted to the regular value or removed during tx cleanup phase.
Read Only transactions because of their lock-less nature may see such
writeIntents (corresponding changes were implemented within IGNITE-17720 and
IGNITE-17627) and ...
... according to tx design updates so does Read Write transactions.
Let's clarify this in greater detail:
* Originally tx finish process firstly converted the write intents into
regular values, and only then released the locks. Thus, other RW transactions
could never see write intents as they were always waiting for locks to be
released.
* In the current version of the [TX
IEP|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol],
the order has been changed to the opposite: locks are released before write
Intents conversion, so that not only a RO but also a RW transactions may see
the intents.
All in all, that means that both read and scan RW operations may use timestamp
based reads and scans with a properly distant timestamp in the future.
*Definition of Done*
{code:java}
Cursor scan(UUID txId) {code}
and
{code:java}
BinaryRow read(RowId rowId, UUID txId) {code}
should be removed.
{code:java}
PartitionTimestampCursor scan(HybridTimestamp timestamp) {code}
and
{code:java}
ReadResult read(RowId rowId, HybridTimestamp timestamp) {code}
should be used instead.
h3. Implementation notes
Solution is pretty straight forward
* Removing deprecated methods along with underneath implementations.
* Propagating RW reads and scans to timestamp based MvPartitionStorage reads
and scans with some sort of HybridTimestamp.MAX_VALUE as a timestamp paramter.
* Verifying that within RW transactions returned ReadResults contain values
with matching txIds (see TxIdMismatchException). That should be guaranteed by
locks.
was:
h3. Motivation
Tx protocol design assumes that data entry exists in the form of a _write
intent_ from the moment the entry is changed within transaction to the moment
it is either converted to the regular value or removed during tx cleanup phase.
Read Only transactions because of their lock-less nature may see such
writeIntents (corresponding changes were implemented within IGNITE-17720 and
IGNITE-17627) and ...
... according to tx design updates so does Read Write transactions.
Let's clarify this in greater detail:
* Originally tx finish process firstly converted the write intents into
regular values, and only then released the locks. Thus, other RW transactions
could never see write intents as they were always waiting for locks to be
released.
* In the current version of the [TX
IEP|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol],
the order has been changed to the opposite: locks are released before write
Intents conversion, so that not only a RO but also a RW transactions may see
the intents.
All in all, that means that both read and scan RW operations may use timestamp
based reads and scans with a properly distant timestamp in the future.
*Definition of Done*
{code:java}
Cursor scan(UUID txId) {code}
and
{code:java}
BinaryRow read(RowId rowId, UUID txId) {code}
should be removed.
{code:java}
PartitionTimestampCursor scan(HybridTimestamp timestamp) {code}
and
{code:java}
ReadResult read(RowId rowId, HybridTimestamp timestamp) {code}
should be used instead.
h3. Implementation notes
Solution is pretty straight forward
* Removing deprecated methods along with underneath implementations.
* Propagating RW reads and scans to timestamp based MvPartitionStorage reads
and scans with some sort of HybridTimestamp.MAX_VALUE as a timestamp paramter.
* Verifying that within RW transactions returned ReadResults contain values
with matching txIds (see TxIdMismatchException). That should be guaranteed by
locks.
> MvPartitionStorage#scan(UUID) and read(UUID) should return
> PartitionTimestampCursor and ReadResult
> --
>
> Key: IGNITE-17856
> URL: https://issues.apache.org/jira/browse/IGNITE-17856
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> h3. Motivation
> Tx protocol design assumes that data entry exists in the form of a _write
> intent_ from the moment the entry is changed within transaction to the moment
> it is either converted to the regular value or removed during tx cleanup
> phase.
> Read Only
[jira] [Commented] (IGNITE-17826) .NET: Thin 3.0: Incomplete exception stack traces
[
https://issues.apache.org/jira/browse/IGNITE-17826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17614918#comment-17614918
]
Pavel Tupitsyn commented on IGNITE-17826:
-
Merged to main: 7467c0873e0c882bce8291c0753d4bb40bc4f7d6
> .NET: Thin 3.0: Incomplete exception stack traces
> -
>
> Key: IGNITE-17826
> URL: https://issues.apache.org/jira/browse/IGNITE-17826
> Project: Ignite
> Issue Type: Bug
> Components: platforms, thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
> Labels: .NET, ignite-3
> Fix For: 3.0.0-alpha6
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There was some intermittent test failure on TC, and the stack trace is:
> {code}
> OneTimeSetUp: Apache.Ignite.IgniteException : IGN-CMN-65535
> TraceId:ad65ae38-a17b-4147-a289-dc571678f327 Failed to read storage data
> > Apache.Ignite.IgniteException :
> org.apache.ignite.lang.IgniteInternalException
> {code}
> Issues:
> * .NET stack trace looks incomplete
> * There is no details about this error in Java server logs (misconfigured
> ClientHandler?)
> * Java stack trace is ignored in .NET (see ClientSocket.ReadError)
> Additionally, we should enable server-side stack traces for tests.
> https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunAllTests/6810719?logFilter=debug
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17859) Update indexes on data modifications
Alexander Lapin created IGNITE-17859: Summary: Update indexes on data modifications Key: IGNITE-17859 URL: https://issues.apache.org/jira/browse/IGNITE-17859 Project: Ignite Issue Type: Improvement Reporter: Alexander Lapin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17826) .NET: Thin 3.0: Incomplete exception stack traces
[
https://issues.apache.org/jira/browse/IGNITE-17826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17614908#comment-17614908
]
Igor Sapego commented on IGNITE-17826:
--
[~ptupitsyn] looks good to me.
> .NET: Thin 3.0: Incomplete exception stack traces
> -
>
> Key: IGNITE-17826
> URL: https://issues.apache.org/jira/browse/IGNITE-17826
> Project: Ignite
> Issue Type: Bug
> Components: platforms, thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
> Labels: .NET, ignite-3
> Fix For: 3.0.0-alpha6
>
>
> There was some intermittent test failure on TC, and the stack trace is:
> {code}
> OneTimeSetUp: Apache.Ignite.IgniteException : IGN-CMN-65535
> TraceId:ad65ae38-a17b-4147-a289-dc571678f327 Failed to read storage data
> > Apache.Ignite.IgniteException :
> org.apache.ignite.lang.IgniteInternalException
> {code}
> Issues:
> * .NET stack trace looks incomplete
> * There is no details about this error in Java server logs (misconfigured
> ClientHandler?)
> * Java stack trace is ignored in .NET (see ClientSocket.ReadError)
> Additionally, we should enable server-side stack traces for tests.
> https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunAllTests/6810719?logFilter=debug
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17858) ignite-geospatial-ext fails to compile due to wrong package
Kishor Bachhav created IGNITE-17858:
---
Summary: ignite-geospatial-ext fails to compile due to wrong
package
Key: IGNITE-17858
URL: https://issues.apache.org/jira/browse/IGNITE-17858
Project: Ignite
Issue Type: Bug
Components: build
Affects Versions: 2.14
Reporter: Kishor Bachhav
mvn clean install -DskipTests -Pcheckstyle fails to compile
ignite-geospatial-ext because of wrong package name.
"package org.apache.ignite.internal.processors.query.schema.management does not
exist"
{code:java}
// [INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 01:23 min
[INFO] Finished at: 2022-10-10T11:12:21+05:30
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile)
on project ignite-geospatial-ext: Compilation failure: Compilation failure:
[ERROR]
/Users/admin/Kishor/SxT/repos/ignite-extensions/modules/geospatial-ext/geospatial/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GeoSpatialUtils.java:[25,69]
package org.apache.ignite.internal.processors.query.schema.management does not
exist
[ERROR]
/Users/admin/Kishor/SxT/repos/ignite-extensions/modules/geospatial-ext/geospatial/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GeoSpatialUtils.java:[26,69]
package org.apache.ignite.internal.processors.query.schema.management does not
exist
[ERROR]
/Users/admin/Kishor/SxT/repos/ignite-extensions/modules/geospatial-ext/geospatial/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GeoSpatialUtils.java:[40,64]
cannot find symbol
[ERROR] symbol: class IndexDescriptor
[ERROR] location: class
org.apache.ignite.internal.processors.query.h2.opt.GeoSpatialUtils
[ERROR]
/Users/admin/Kishor/SxT/repos/ignite-extensions/modules/geospatial-ext/geospatial/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GeoSpatialIndexDescriptorFactory.java:[36,69]
package org.apache.ignite.internal.processors.query.schema.management does not
exist
[ERROR]
/Users/admin/Kishor/SxT/repos/ignite-extensions/modules/geospatial-ext/geospatial/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GeoSpatialIndexDescriptorFactory.java:[37,69]
package org.apache.ignite.internal.processors.query.schema.management does not
exist
[ERROR]
/Users/admin/Kishor/SxT/repos/ignite-extensions/modules/geospatial-ext/geospatial/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GeoSpatialIndexDescriptorFactory.java:[38,69]
package org.apache.ignite.internal.processors.query.schema.management does not
exist
[ERROR]
/Users/admin/Kishor/SxT/repos/ignite-extensions/modules/geospatial-ext/geospatial/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GeoSpatialIndexDescriptorFactory.java:[44,55]
cannot find symbol
[ERROR] symbol: class AbstractIndexDescriptorFactory
[ERROR]
/Users/admin/Kishor/SxT/repos/ignite-extensions/modules/geospatial-ext/geospatial/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GeoSpatialIndexDescriptorFactory.java:[55,9]
cannot find symbol
[ERROR] symbol: class TableDescriptor
[ERROR] location: class
org.apache.ignite.internal.processors.query.h2.opt.GeoSpatialIndexDescriptorFactory
[ERROR]
/Users/admin/Kishor/SxT/repos/ignite-extensions/modules/geospatial-ext/geospatial/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GeoSpatialIndexDescriptorFactory.java:[52,22]
cannot find symbol
[ERROR] symbol: class IndexDescriptor
[ERROR] location: class
org.apache.ignite.internal.processors.query.h2.opt.GeoSpatialIndexDescriptorFactory
[ERROR]
/Users/admin/Kishor/SxT/repos/ignite-extensions/modules/geospatial-ext/geospatial/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GeoSpatialUtils.java:[35,9]
cannot find symbol
[ERROR] symbol: variable SchemaManager
[ERROR] location: class
org.apache.ignite.internal.processors.query.h2.opt.GeoSpatialUtils
[ERROR]
/Users/admin/Kishor/SxT/repos/ignite-extensions/modules/geospatial-ext/geospatial/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GeoSpatialIndexDescriptorFactory.java:[52,5]
method does not override or implement a method from a supertype
[ERROR]
/Users/admin/Kishor/SxT/repos/ignite-extensions/modules/geospatial-ext/geospatial/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GeoSpatialIndexDescriptorFactory.java:[60,61]
cannot find symbol
[ERROR] symbol: method
indexDescriptorToKeysDefinition(org.apache.ignite.internal.processors.query.GridQueryIndexDescriptor,org.apache.ignite.internal.processors.query.GridQueryTypeDescriptor)
[ERROR] location: class
