[jira] [Updated] (IGNITE-17457) Cluster locks after the transaction recovery procedure if the tx primary node fail
[
https://issues.apache.org/jira/browse/IGNITE-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Korotkov updated IGNITE-17457:
-
Description:
Ignite cluster may be locked (all client operations would block) after the tx
recovery procedure executed on the tx primary node failure.
The prepared transaction may remain un-commited on the backup node after the tx
recovery. So the partition exchange wouldn't complete. So cluster would be
locked.
The Immediate reason is the race condition in the method:
{code:java}
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter::markFinalizing(RECOVERY_FINISH){code}
It may be called concurrently for the same transaction both from the recovery
procedure:
{code:java}
IgniteTxManager::commitIfPrepared{code}
and from the tx recovery request handler:
{code:java}
IgniteTxHandler::processCheckPreparedTxRequest{code}
Details {color:#ff}TBD{color}
{color:#172b4d}Reproducer is in the pull request.{color}
was:
Ignite cluster may be locked (all client operations would block) after the tx
recovery procedure executed on the tx primary node failure.
The prepared transaction may remain un-commited on the backup node after the tx
recovery. So the partition exchange wouldn't complete. So cluster would be
locked.
The Immediate reason is the race condition in the method:
{code:java}
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter::markFinalizing(RECOVERY_FINISH){code}
It may be called concurrently for the same transaction both from the recovery
procedure:
{code:java}
IgniteTxManager::commitIfPrepared{code}
and from the tx recovery request handler:
{code:java}
IgniteTxHandler::processCheckPreparedTxRequest{code}
Details {color:#ff}TBD{color}
{color:#172b4d}Rreproducer is in the pull request.{color}
> Cluster locks after the transaction recovery procedure if the tx primary node
> fail
> --
>
> Key: IGNITE-17457
> URL: https://issues.apache.org/jira/browse/IGNITE-17457
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Korotkov
>Assignee: Sergey Korotkov
>Priority: Major
>
> Ignite cluster may be locked (all client operations would block) after the tx
> recovery procedure executed on the tx primary node failure.
> The prepared transaction may remain un-commited on the backup node after the
> tx recovery. So the partition exchange wouldn't complete. So cluster would
> be locked.
> The Immediate reason is the race condition in the method:
> {code:java}
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter::markFinalizing(RECOVERY_FINISH){code}
> It may be called concurrently for the same transaction both from the recovery
> procedure:
> {code:java}
> IgniteTxManager::commitIfPrepared{code}
> and from the tx recovery request handler:
> {code:java}
> IgniteTxHandler::processCheckPreparedTxRequest{code}
>
> Details {color:#ff}TBD{color}
> {color:#172b4d}Reproducer is in the pull request.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17457) Cluster locks after the transaction recovery procedure if the tx primary node fail
[
https://issues.apache.org/jira/browse/IGNITE-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Korotkov updated IGNITE-17457:
-
Description:
Ignite cluster may be locked (all client operations would block) after the tx
recovery procedure executed on the tx primary node failure.
The prepared transaction may remain un-commited on the backup node after the tx
recovery. So the partition exchange wouldn't complete. So cluster would be
locked.
The Immediate reason is the race condition in the method:
{code:java}
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter::markFinalizing(RECOVERY_FINISH){code}
It may be called concurrently for the same transaction both from the recovery
procedure:
{code:java}
IgniteTxManager::commitIfPrepared{code}
and from the tx recovery request handler:
{code:java}
IgniteTxHandler::processCheckPreparedTxRequest{code}
Details {color:#ff}TBD{color}
{color:#172b4d}Rreproducer is in the pull request.{color}
was:
Ignite cluster may be locked (all client operations would block) after the tx
recovery procedure executed on the tx primary node failure.
The prepared transaction may remain un-commited on the backup node after the tx
recovery. So the partition exchange wouldn't complete. So cluster would be
locked.
The Immediate reason is the race condition in the method:
{code:java}
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter::markFinalizing(RECOVERY_FINISH){code}
It may be called concurrently for the same transaction both from the recovery
procedure:
{code:java}
IgniteTxManager::commitIfPrepared{code}
and from the tx recovery request handler:
{code:java}
IgniteTxHandler::processCheckPreparedTxRequest{code}
Details and reproducer {color:#ff}TBD{color}
> Cluster locks after the transaction recovery procedure if the tx primary node
> fail
> --
>
> Key: IGNITE-17457
> URL: https://issues.apache.org/jira/browse/IGNITE-17457
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Korotkov
>Assignee: Sergey Korotkov
>Priority: Major
>
> Ignite cluster may be locked (all client operations would block) after the tx
> recovery procedure executed on the tx primary node failure.
> The prepared transaction may remain un-commited on the backup node after the
> tx recovery. So the partition exchange wouldn't complete. So cluster would
> be locked.
> The Immediate reason is the race condition in the method:
> {code:java}
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter::markFinalizing(RECOVERY_FINISH){code}
> It may be called concurrently for the same transaction both from the recovery
> procedure:
> {code:java}
> IgniteTxManager::commitIfPrepared{code}
> and from the tx recovery request handler:
> {code:java}
> IgniteTxHandler::processCheckPreparedTxRequest{code}
>
> Details {color:#ff}TBD{color}
> {color:#172b4d}Rreproducer is in the pull request.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-17457) Cluster locks after the transaction recovery procedure if the tx primary node fail
[
https://issues.apache.org/jira/browse/IGNITE-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Korotkov reassigned IGNITE-17457:
Assignee: Sergey Korotkov
> Cluster locks after the transaction recovery procedure if the tx primary node
> fail
> --
>
> Key: IGNITE-17457
> URL: https://issues.apache.org/jira/browse/IGNITE-17457
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Korotkov
>Assignee: Sergey Korotkov
>Priority: Major
>
> Ignite cluster may be locked (all client operations would block) after the tx
> recovery procedure executed on the tx primary node failure.
> The prepared transaction may remain un-commited on the backup node after the
> tx recovery. So the partition exchange wouldn't complete. So cluster would
> be locked.
> The Immediate reason is the race condition in the method:
> {code:java}
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter::markFinalizing(RECOVERY_FINISH){code}
> It may be called concurrently for the same transaction both from the recovery
> procedure:
> {code:java}
> IgniteTxManager::commitIfPrepared{code}
> and from the tx recovery request handler:
> {code:java}
> IgniteTxHandler::processCheckPreparedTxRequest{code}
>
> Details and reproducer {color:#ff}TBD{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17457) Cluster locks after the transaction recovery procedure if the tx primary node fail
Sergey Korotkov created IGNITE-17457:
Summary: Cluster locks after the transaction recovery procedure if
the tx primary node fail
Key: IGNITE-17457
URL: https://issues.apache.org/jira/browse/IGNITE-17457
Project: Ignite
Issue Type: Bug
Reporter: Sergey Korotkov
Ignite cluster may be locked (all client operations would block) after the tx
recovery procedure executed on the tx primary node failure.
The prepared transaction may remain un-commited on the backup node after the tx
recovery. So the partition exchange wouldn't complete. So cluster would be
locked.
The Immediate reason is the race condition in the method:
{code:java}
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter::markFinalizing(RECOVERY_FINISH){code}
It may be called concurrently for the same transaction both from the recovery
procedure:
{code:java}
IgniteTxManager::commitIfPrepared{code}
and from the tx recovery request handler:
{code:java}
IgniteTxHandler::processCheckPreparedTxRequest{code}
Details and reproducer {color:#ff}TBD{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17206) .NET: Thin client: Add IgniteSet
[
https://issues.apache.org/jira/browse/IGNITE-17206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574289#comment-17574289
]
Ignite TC Bot commented on IGNITE-17206:
{panel:title=Branch: [pull/10177/head] Base: [master] : Possible Blockers
(36)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Control Utility{color} [[tests 6 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=6707811]]
* IgniteControlUtilityTestSuite:
GridCommandHandlerTest.testCacheIdleVerifyChecksGapsAtomic - Test has low fail
rate in base branch 0,0% and is not flaky
* IgniteControlUtilityTestSuite:
GridCommandHandlerTest.testChangeSnapshotTransferRateInRuntime - Test has low
fail rate in base branch 0,0% and is not flaky
* IgniteControlUtilityTestSuite: GridCommandHandlerTest.testSuccessStopWarmUp -
Test has low fail rate in base branch 0,0% and is not flaky
* IgniteControlUtilityTestSuite:
GridCommandHandlerTest.testBaselineRemoveOnNotActiveCluster - Test has low fail
rate in base branch 0,0% and is not flaky
* IgniteControlUtilityTestSuite:
GridCommandHandlerTest.testCacheIdleVerifyCrcWithCorruptedPartition - Test has
low fail rate in base branch 0,0% and is not flaky
* IgniteControlUtilityTestSuite:
GridCommandHandlerTest.testBaselineAutoAdjustmentAutoRemoveNode - Test has low
fail rate in base branch 0,0% and is not flaky
{color:#d04437}PDS (Indexing){color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=6707842]]
{color:#d04437}Platform .NET (Windows){color} [[tests
2|https://ci.ignite.apache.org/viewLog.html?buildId=6707848]]
* exe: IgniteSetClientTestsColocated.TestCopyToInvalidArguments - History for
base branch is absent.
* exe: IgniteSetClientTests.TestCopyToInvalidArguments - History for base
branch is absent.
{color:#d04437}ZooKeeper (Discovery) 1{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6707875]]
* ZookeeperDiscoverySpiTestSuite1:
ZookeeperDiscoveryRandomStopOrFailConcurrentTest.testStopOrFailConcurrently[stop
mode = RANDOM, with crd = true] - Test has low fail rate in base branch 0,0%
and is not flaky
{color:#d04437}Continuous Query 4{color} [[tests
23|https://ci.ignite.apache.org/viewLog.html?buildId=6707810]]
* IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testScanQuery[pageSize=100, clientType=SERVER] -
Test has low fail rate in base branch 0,0% and is not flaky
* IgniteCacheQuerySelfTestSuite6:
CacheContinuousQueryAsyncFilterListenerTest.testNonDeadLockInFilterReplicatedJCacheApiMvcc
- Test has low fail rate in base branch 0,0% and is not flaky
* IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testDdlAndDmlQueries[pageSize=100,
clientType=SERVER] - Test has low fail rate in base branch 0,0% and is not flaky
* IgniteCacheQuerySelfTestSuite6:
CacheContinuousQueryAsyncFilterListenerTest.testNonDeadLockInListenerMvcc -
Test has low fail rate in base branch 0,0% and is not flaky
* IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testSqlFieldsQuery[pageSize=100,
clientType=SERVER] - Test has low fail rate in base branch 0,0% and is not flaky
* IgniteCacheQuerySelfTestSuite6:
CacheContinuousQueryAsyncFilterListenerTest.testNonDeadLockInFilterMvccTxJCacheApi
- Test has low fail rate in base branch 0,0% and is not flaky
* IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testMultipleStatementsSql[pageSize=100,
clientType=SERVER] - Test has low fail rate in base branch 0,0% and is not flaky
* IgniteCacheQuerySelfTestSuite6:
CacheContinuousQueryAsyncFilterListenerTest.testNonDeadLockInFilterSyncFilterMvcc
- Test has low fail rate in base branch 0,0% and is not flaky
* IgniteCacheQuerySelfTestSuite6:
StaticCacheDdlKeepStaticConfigurationTest.testDropColumn - Test has low fail
rate in base branch 0,0% and is not flaky
* IgniteCacheQuerySelfTestSuite6:
StaticCacheDdlKeepStaticConfigurationTest.testAddColumn - Test has low fail
rate in base branch 0,0% and is not flaky
* IgniteCacheQuerySelfTestSuite6: StaticCacheDdlTest.testAddIndex - Test has
low fail rate in base branch 0,0% and is not flaky
... and 12 tests blockers
{color:#d04437}Queries 2{color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=6707851]]
{color:#d04437}Cache (Tx Recovery){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6707799]]
* IgniteCacheTxRecoverySelfTestSuite:
GridCachePartitionedTxOriginatingNodeFailureSelfTest.testTxAllNodes - Test has
low fail rate in base branch 0,0% and is not flaky
{panel}
{panel:title=Branch: [pull/10177/head] Base: [master] : New Tests
(104)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Platform .NET (Windows){color} [[tests
52|https://ci.ignite.apache.org/viewLog.html?buildId=6707848]]
* {color:#013220}exe:
[jira] [Assigned] (IGNITE-17308) Revisit SortedIndexMvStorage interface
[
https://issues.apache.org/jira/browse/IGNITE-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtcev reassigned IGNITE-17308:
Assignee: Aleksandr Polovtcev
> Revisit SortedIndexMvStorage interface
> --
>
> Key: IGNITE-17308
> URL: https://issues.apache.org/jira/browse/IGNITE-17308
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Bessonov
>Assignee: Aleksandr Polovtcev
>Priority: Major
> Labels: ignite-3
>
> Currently, SortedIndexMvStorage is a very weird mixture of many things. Its
> contract is far from obvious and it's only used in tests as a part of
> "reference implementation".
> Originally, it was implemented when the vision of MV store wasn't fully
> solidified.
> h3. API changes
> * {{IndexRowEx}} should disappear. It was a quick and dirty solution. It
> should be replaced with {{{}InternalTuple{}}}, with the requirement that
> every internal tuple can be converted into a IEP-92 format.
> * {{scan}} should not return rows, but only indexed rows and RowId
> instances. Index scan should NOT by itself filter-out invalid rows, this will
> be performed outside of scan.
> * TxId / Timestamp parameters are no longer applicable, given that index
> does not perform rows validation.
> * Partition filter should be removed as well. To simplify things, every
> partition will be indexed {+}independently{+}.
> * {{supportsBackwardsScan}} and {{supportsIndexOnlyScan}} can be removed for
> now. Former can be brought back in the future, while latter makes no sense
> considering that indexes are not multiversioned.
> * new methods, like {{update}} and {{remove}} should be added to API.
> h3. New API for removed functions
> * There should be a new entity on top of partition and index store. It
> updates indexes and filters scan queries. There's no point in fully designing
> it right now, all we need is working tests for now. Porting current tests to
> new API is up to a developer.
> h3. Other
> I would say that effective InternalTuple comparison is out of scope. We could
> just adapt current test code somehow.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-16892) Update lock manager in order to soupport S, X and I locks
[
https://issues.apache.org/jira/browse/IGNITE-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-16892:
-
Description:
1. Lock management logic will be introduced in IGNITE-17255
2. Lock storages will be introduced in IGNITE-15932
Given ticket is a sort of a bridge between 1 and 2 that will adjust LockManager
methods along with corresponding implementation with LockMode parameter that
will clarify whether lock is
* Exclusive
* Shared
* IntentExclusive
* IntentShared
* SharedAndIntentExclusive
{code:java}
CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
LockException;
Iterator locks(UUID txId) {code}
lockName is a sort of lock locator, that either:
* RowId for data storage locks.
* UUID or similar to table/index commonly intent locks.
* Index keys.
was:
1. Lock management logic will be introduced in IGNITE-17255
2. Lock storages will be introduced in IGNITE-15932
Given ticket is a sort of a bridge between 1 and 2 that will adjust LockManager
methods along with corresponding implementation with LockMode parameter that
will clarify whether lock is
* Exclusive
* Shared
* IntentExclusive
* IntentShared
* SharedAndIntentExclusive
{code:java}
CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
LockException;
Iterator locks(UUID txId) {code}
lockName is a sort of lock locator, that either:
* RowId for data storage locks.
* UUID or similar to table/index commonly intent locks.
* Index keys.
Details will be elaborated during IGNITE-17255 implementation.
> Update lock manager in order to soupport S, X and I locks
> -
>
> Key: IGNITE-16892
> URL: https://issues.apache.org/jira/browse/IGNITE-16892
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction3_rw
> Time Spent: 20m
> Remaining Estimate: 0h
>
> 1. Lock management logic will be introduced in IGNITE-17255
> 2. Lock storages will be introduced in IGNITE-15932
> Given ticket is a sort of a bridge between 1 and 2 that will adjust
> LockManager methods along with corresponding implementation with LockMode
> parameter that will clarify whether lock is
> * Exclusive
> * Shared
> * IntentExclusive
> * IntentShared
> * SharedAndIntentExclusive
> {code:java}
> CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
> void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
> LockException;
> Iterator locks(UUID txId) {code}
> lockName is a sort of lock locator, that either:
> * RowId for data storage locks.
> * UUID or similar to table/index commonly intent locks.
> * Index keys.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] (IGNITE-17206) .NET: Thin client: Add IgniteSet
[ https://issues.apache.org/jira/browse/IGNITE-17206 ]
Pavel Tupitsyn deleted comment on IGNITE-17206:
-
was (Author: ignitetcbot):
{panel:title=Branch: [pull/10177/head] Base: [master] : Possible Blockers
(100)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Snapshots{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707695]]
{color:#d04437}Control Utility{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707645]]
{color:#d04437}PDS (Indexing){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707676]]
{color:#d04437}Cache 5{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707617]]
{color:#d04437}Cache 6{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707618]]
{color:#d04437}Cache 2{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707614]]
{color:#d04437}PDS 5{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707671]]
{color:#d04437}SPI (Discovery){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707698]]
{color:#d04437}Cache 13{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707613]]
{color:#d04437}PDS 6{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707672]]
{color:#d04437}Cassandra Store{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707635]]
{color:#d04437}Basic 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707604]]
{color:#d04437}Control Utility (Zookeeper){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707646]]
{color:#d04437}ZooKeeper (Discovery) 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707709]]
{color:#d04437}PDS (Compatibility){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707675]]
{color:#d04437}Snapshots With Indexes{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707696]]
{color:#d04437}Queries 3{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707687]]
{color:#d04437}Java Client{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707656]]
{color:#d04437}Continuous Query 2{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707642]]
{color:#d04437}Cache 9{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707621]]
{color:#d04437}Cache 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707609]]
{color:#d04437}PDS 2{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707668]]
{color:#d04437}Cache 12{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707612]]
{color:#d04437}JDBC Driver{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707659]]
{color:#d04437}Queries 3 (lazy=true){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707688]]
{color:#d04437}Cache 7{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707619]]
{color:#d04437}Binary Objects{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707608]]
{color:#d04437}Continuous Query 4{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707644]]
{color:#d04437}Calcite SQL{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707634]]
{color:#d04437}Thin Client: Java{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707703]]
{color:#d04437}Cache (Failover) 2{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707625]]
{color:#d04437}Cache 3{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707615]]
{color:#d04437}PDS 4{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707670]]
{color:#d04437}Cache (Restarts) 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707631]]
{color:#d04437}Cache 8{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707620]]
{color:#d04437}SPI{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707697]]
{color:#d04437}PDS 3{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707669]]
{color:#d04437}Start Nodes{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707701]]
{color:#d04437}Cache (Failover SSL){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707627]]
{color:#d04437}ZooKeeper (Discovery) 3{color} [[tests 0
[jira] [Updated] (IGNITE-17391) speed up index-reader.sh
[ https://issues.apache.org/jira/browse/IGNITE-17391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-17391: - Fix Version/s: 2.14 > speed up index-reader.sh > > > Key: IGNITE-17391 > URL: https://issues.apache.org/jira/browse/IGNITE-17391 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr >Assignee: Nikolay Izhikov >Priority: Minor > Fix For: 2.14 > > Time Spent: 20m > Remaining Estimate: 0h > > Utility index-reader.sh when analyzing files index.bin larger works longer > than we would like, for example: the analysis of indexes for 100 GB took more > than 15 hours, provided that the virtual machine is idle according to the > main indicators (CPU/MEM/HDD). We need to be able to speed up the application. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-16559) Node's log contains "Failed to refresh a leader" messages.
[
https://issues.apache.org/jira/browse/IGNITE-16559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574242#comment-17574242
]
Mirza Aliev commented on IGNITE-16559:
--
[~Sergey Uttsel] LGTM
> Node's log contains "Failed to refresh a leader" messages.
> ---
>
> Key: IGNITE-16559
> URL: https://issues.apache.org/jira/browse/IGNITE-16559
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Assignee: Sergey Uttsel
>Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We noticed that when we run
> {{ItMixedQueriesTest.testIgniteSchemaAwaresAlterTableCommand}} on TC it is
> possible that log contain such messages:
> {noformat}
> 2022-02-15 12:36:43:568 +0300
> [ERROR][%ItMixedQueriesTest_null_1%Raft-Group-Client-0][RaftGroupServiceImpl]
> Failed to refresh a leader
> [groupId=8e71fc5e-6b24-4b69-ba5a-6eae4c2165cf_part_16]
> java.util.concurrent.CompletionException:
> java.util.concurrent.TimeoutException
> at
> java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
> at
> java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)
> at
> java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:632)
> at
> java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
> at
> java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
> at
> org.apache.ignite.raft.jraft.rpc.impl.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:502)
> at
> org.apache.ignite.raft.jraft.rpc.impl.RaftGroupServiceImpl$1.lambda$accept$1(RaftGroupServiceImpl.java:544)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.util.concurrent.TimeoutException
> {noformat}
> Possible root cause:
> Seems, that we get TimeoutException when we try to get a leader from a client
> for a group, for which leader has not been elected yet. If you check the
> logs, you can see, that we get timeout exception and after that leader for
> the corresponding group has been elected.
> Note that we have only one node and 10 partitions for a table in the test,
> but raft leaders are elected sequentially on a node, so electing 10 leaders
> for raft groups on one node might take a little bit longer.
> Possible solution:
> Increase timeout for a client to get a leader for the first time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17168) Ducktests: bump the LATEST release version
[
https://issues.apache.org/jira/browse/IGNITE-17168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Pavlov updated IGNITE-17168:
---
Labels: ducktests ise (was: ducktests)
> Ducktests: bump the LATEST release version
> --
>
> Key: IGNITE-17168
> URL: https://issues.apache.org/jira/browse/IGNITE-17168
> Project: Ignite
> Issue Type: Test
>Reporter: Sergey Korotkov
>Priority: Minor
> Labels: ducktests, ise
>
> Most ducktests are run twice. One time against the current development
> version and another one against the most recent released version. It's
> configured by the decorator applied to test class:
> {code:python}
> @ignite_versions(str(DEV_BRANCH), str(LATEST))
> {code}
> The task is to update the LATEST to current most recent release (2.13)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17206) .NET: Thin client: Add IgniteSet
[
https://issues.apache.org/jira/browse/IGNITE-17206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574228#comment-17574228
]
Ignite TC Bot commented on IGNITE-17206:
{panel:title=Branch: [pull/10177/head] Base: [master] : Possible Blockers
(100)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Snapshots{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707695]]
{color:#d04437}Control Utility{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707645]]
{color:#d04437}PDS (Indexing){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707676]]
{color:#d04437}Cache 5{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707617]]
{color:#d04437}Cache 6{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707618]]
{color:#d04437}Cache 2{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707614]]
{color:#d04437}PDS 5{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707671]]
{color:#d04437}SPI (Discovery){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707698]]
{color:#d04437}Cache 13{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707613]]
{color:#d04437}PDS 6{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707672]]
{color:#d04437}Cassandra Store{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707635]]
{color:#d04437}Basic 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707604]]
{color:#d04437}Control Utility (Zookeeper){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707646]]
{color:#d04437}ZooKeeper (Discovery) 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707709]]
{color:#d04437}PDS (Compatibility){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707675]]
{color:#d04437}Snapshots With Indexes{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707696]]
{color:#d04437}Queries 3{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707687]]
{color:#d04437}Java Client{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707656]]
{color:#d04437}Continuous Query 2{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707642]]
{color:#d04437}Cache 9{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707621]]
{color:#d04437}Cache 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707609]]
{color:#d04437}PDS 2{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707668]]
{color:#d04437}Cache 12{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707612]]
{color:#d04437}JDBC Driver{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707659]]
{color:#d04437}Queries 3 (lazy=true){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707688]]
{color:#d04437}Cache 7{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707619]]
{color:#d04437}Binary Objects{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707608]]
{color:#d04437}Continuous Query 4{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707644]]
{color:#d04437}Calcite SQL{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707634]]
{color:#d04437}Thin Client: Java{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707703]]
{color:#d04437}Cache (Failover) 2{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707625]]
{color:#d04437}Cache 3{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707615]]
{color:#d04437}PDS 4{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707670]]
{color:#d04437}Cache (Restarts) 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707631]]
{color:#d04437}Cache 8{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707620]]
{color:#d04437}SPI{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707697]]
{color:#d04437}PDS 3{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707669]]
{color:#d04437}Start Nodes{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707701]]
{color:#d04437}Cache (Failover SSL){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=6707627]]
{color:#d04437}ZooKeeper (Discovery) 3{color} [[tests 0
[jira] [Comment Edited] (IGNITE-17398) Opencensus metrics do not work well with Prometheus since it do not have tags
[
https://issues.apache.org/jira/browse/IGNITE-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574198#comment-17574198
]
Sergey Kadaner edited comment on IGNITE-17398 at 8/2/22 12:25 PM:
--
I see your point about it not being free, however, they say "Each labelset is
an additional time series". As I understand it, that means that it is the same
cost as creating the same number of time series. I.e. _cache_MY_CACHE_NAME1_
and _cache_MY_CACHE_NAME2_ metrics have the same cost as
_cache\{name="MY_CACHE_NAME1"}_ and {_}cache\{name="MY_CACHE_NAME2"}{_}. So
basically there is no additional cost in this case.
In fact, the previous section to the one you mentioned explains [when to use
labels|https://prometheus.io/docs/practices/instrumentation/#use-labels] and it
matches this case exactly.
It is hard to say how it affects the GC pressure without actually measuring it,
but AFAIK modern garbage collectors (G1 and newer) are very good with
short-lived objects and should not have issues reclaiming them.
The issue is also not simply a convenience problem unless you can suggest how
to draw diagrams without the tags and without knowing cache names in advance.
was (Author: sergeykad):
I see your point about it not being free, however, they say "Each labelset is
an additional time series". As I understand it, that means that it is the same
cost as creating the same number of time series. I.e. _cache_MY_CACHE_NAME1_
and _cache_MY_CACHE_NAME2_ metrics have the same cost as
_cache\{name="MY_CACHE_NAME1"}_ and {_}cache\{name="MY_CACHE_NAME2"}{_}. So
basically there is no additional cost in this case.
It is hard to say how it affects the GC pressure without actually measuring it,
but AFAIK modern garbage collectors (G1 and newer) are very good with
short-lived objects and should not have issues reclaiming them.
The issue is also not simply a convenience problem unless you can suggest how
to draw diagrams without the tags and without knowing cache names in advance.
> Opencensus metrics do not work well with Prometheus since it do not have tags
> -
>
> Key: IGNITE-17398
> URL: https://issues.apache.org/jira/browse/IGNITE-17398
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.9.1
>Reporter: Sergey Kadaner
>Priority: Major
> Labels: metrics
> Attachments: image-2022-07-20-17-51-07-217.png
>
>
> The metrics created by the [new metrics
> system|https://ignite.apache.org/docs/latest/monitoring-metrics/new-metrics]
> are very inconvenient to use with Prometheus since it does not use tags.
> For example, Spring metric generated for the same cache looks like the
> following in Prometheus and is very convenient to use:
>
> {noformat}
> cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
> 1387.0{noformat}
>
> The native Ignite metric looks like the following:
>
> {noformat}
> cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
> The Spring reported statistics can be easily filtered by cache name and other
> attributes, while the build-in Ignite metrics do not provide an easy way to
> access cache names. The only option seems to be parsing the
> "cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by
> Grafana.
>
> For example with tags it is very easy to get a graph in Grafana with a cache
> hit percentage that includes every existing cache. It automatically extracts
> the cache name from the "cache" tag. Unfortunately, it is not possible if
> there are no tags.
> !image-2022-07-20-17-51-07-217.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17146) Add support for alpha, beta, etc. releases in IgniteProductVersion
[
https://issues.apache.org/jira/browse/IGNITE-17146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirill Tkalenko updated IGNITE-17146:
-
Labels: ignite-3 tech-debt (was: ignite-3)
> Add support for alpha, beta, etc. releases in IgniteProductVersion
> --
>
> Key: IGNITE-17146
> URL: https://issues.apache.org/jira/browse/IGNITE-17146
> Project: Ignite
> Issue Type: Task
>Reporter: Kirill Tkalenko
>Priority: Major
> Labels: ignite-3, tech-debt
> Fix For: 3.0.0-alpha6
>
>
> {code:java}
> org.apache.ignite.internal.properties.IgniteProductVersion#alphaVersion{code}
> has been added for the upcoming 3.0.0-alpha5 release, we need to fix it so
> that we can work with alpha, beta and other releases.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-17398) Opencensus metrics do not work well with Prometheus since it do not have tags
[
https://issues.apache.org/jira/browse/IGNITE-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574198#comment-17574198
]
Sergey Kadaner edited comment on IGNITE-17398 at 8/2/22 12:20 PM:
--
I see your point about it not being free, however, they say "Each labelset is
an additional time series". As I understand it, that means that it is the same
cost as creating the same number of time series. I.e. _cache_MY_CACHE_NAME1_
and _cache_MY_CACHE_NAME2_ metrics have the same cost as
_cache\{name="MY_CACHE_NAME1"}_ and {_}cache\{name="MY_CACHE_NAME2"}{_}. So
basically there is no additional cost in this case.
It is hard to say how it affects the GC pressure without actually measuring it,
but AFAIK modern garbage collectors (G1 and newer) are very good with
short-lived objects and should not have issues reclaiming them.
The issue is also not simply a convenience problem unless you can suggest how
to draw diagrams without the tags and without knowing cache names in advance.
was (Author: sergeykad):
I see your point about it not being free, however, they say "Each labelset is
an additional time series". As I understand it, that means that it is the same
cost as creating the same number of time series. I.e. _cache_MY_CACHE_NAME1_
and _cache_MY_CACHE_NAME2_ metrics have the same cost as
_cache\{name="MY_CACHE_NAME1"}_ and {_}cache\{name="MY_CACHE_NAME2"}{_}. So
basically there is no additional cost in this case.
It is hard to say how it affects the GC pressure without actually measuring it,
but AFAIK modern garbage collectors (G1 and newer) are very good with
short-lived objects and should not have issues reclaiming them.
The issue is also not simply a convenience problem unless you can suggest how
to draw diagrams without knowing cache names in advance without the tags.
> Opencensus metrics do not work well with Prometheus since it do not have tags
> -
>
> Key: IGNITE-17398
> URL: https://issues.apache.org/jira/browse/IGNITE-17398
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.9.1
>Reporter: Sergey Kadaner
>Priority: Major
> Labels: metrics
> Attachments: image-2022-07-20-17-51-07-217.png
>
>
> The metrics created by the [new metrics
> system|https://ignite.apache.org/docs/latest/monitoring-metrics/new-metrics]
> are very inconvenient to use with Prometheus since it does not use tags.
> For example, Spring metric generated for the same cache looks like the
> following in Prometheus and is very convenient to use:
>
> {noformat}
> cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
> 1387.0{noformat}
>
> The native Ignite metric looks like the following:
>
> {noformat}
> cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
> The Spring reported statistics can be easily filtered by cache name and other
> attributes, while the build-in Ignite metrics do not provide an easy way to
> access cache names. The only option seems to be parsing the
> "cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by
> Grafana.
>
> For example with tags it is very easy to get a graph in Grafana with a cache
> hit percentage that includes every existing cache. It automatically extracts
> the cache name from the "cache" tag. Unfortunately, it is not possible if
> there are no tags.
> !image-2022-07-20-17-51-07-217.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-17398) Opencensus metrics do not work well with Prometheus since it do not have tags
[
https://issues.apache.org/jira/browse/IGNITE-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574198#comment-17574198
]
Sergey Kadaner edited comment on IGNITE-17398 at 8/2/22 12:19 PM:
--
I see your point about it not being free, however, they say "Each labelset is
an additional time series". As I understand it, that means that it is the same
cost as creating the same number of time series. I.e. _cache_MY_CACHE_NAME1_
and _cache_MY_CACHE_NAME2_ metrics have the same cost as
_cache\{name="MY_CACHE_NAME1"}_ and {_}cache\{name="MY_CACHE_NAME2"}{_}. So
basically there is no additional cost in this case.
It is hard to say how it affects the GC pressure without actually measuring it,
but AFAIK modern garbage collectors (G1 and newer) are very good with
short-lived objects and should not have issues reclaiming them.
The issue is also not simply a convenience problem unless you can suggest how
to draw diagrams without knowing cache names in advance without the tags.
was (Author: sergeykad):
I see your point about it not being free, however, they say "Each labelset is
an additional time series". As I understand it, that means that it is the same
cost as creating the same number of time series. I.e. _cache_MY_CACHE_NAME1_
and _cache_MY_CACHE_NAME2_ metrics have the same cost as
_cache\{name="MY_CACHE_NAME1"}_ and {_}cache\{name="MY_CACHE_NAME2"}{_}. So
basically there is no additional cost in this case.
It is hard to say how it affects the GC pressure without actually measuring it,
but AFAIK modern garbage collectors (G1 and newer) are very good with
short-lived objects and should not have issues reclaiming them.
The issue is also not simply a convenience problem unless you can suggest how
to draw diagrams without knowing cache names in advance.
> Opencensus metrics do not work well with Prometheus since it do not have tags
> -
>
> Key: IGNITE-17398
> URL: https://issues.apache.org/jira/browse/IGNITE-17398
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.9.1
>Reporter: Sergey Kadaner
>Priority: Major
> Labels: metrics
> Attachments: image-2022-07-20-17-51-07-217.png
>
>
> The metrics created by the [new metrics
> system|https://ignite.apache.org/docs/latest/monitoring-metrics/new-metrics]
> are very inconvenient to use with Prometheus since it does not use tags.
> For example, Spring metric generated for the same cache looks like the
> following in Prometheus and is very convenient to use:
>
> {noformat}
> cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
> 1387.0{noformat}
>
> The native Ignite metric looks like the following:
>
> {noformat}
> cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
> The Spring reported statistics can be easily filtered by cache name and other
> attributes, while the build-in Ignite metrics do not provide an easy way to
> access cache names. The only option seems to be parsing the
> "cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by
> Grafana.
>
> For example with tags it is very easy to get a graph in Grafana with a cache
> hit percentage that includes every existing cache. It automatically extracts
> the cache name from the "cache" tag. Unfortunately, it is not possible if
> there are no tags.
> !image-2022-07-20-17-51-07-217.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17456) After node restart, there are duplicate messages about WAL segment compression.
[
https://issues.apache.org/jira/browse/IGNITE-17456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-17456:
--
Description:
If you enable compression of WAL segments and at the same time set the WAL
archive size too small, then compression will not actually occur, but after
each node restart, "fake" notifications about segment compression are
sequentially written to the log from the beginning (see log example below).
{noformat}
[2022-08-02T14:46:33,757][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Resolved write ahead log work directory:
/home/xtern/src/java/ignite/work/db/wal/node00-63fb6fa2-fcea-42aa-a3c8-b36cd330ba7c
[2022-08-02T14:46:33,757][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Resolved write ahead log archive directory:
/home/xtern/src/java/ignite/work/db/wal/archive/node00-63fb6fa2-fcea-42aa-a3c8-b36cd330ba7c
[2022-08-02T14:46:33,759][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=0]
[2022-08-02T14:46:33,759][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=1]
[2022-08-02T14:46:33,759][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=2]
[2022-08-02T14:46:33,759][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=3]
...
[2022-08-02T14:46:33,761][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=49]
[2022-08-02T14:46:33,761][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileHandleManagerImpl]
Initialized write-ahead log manager [mode=LOG_ONLY]
...
[2022-08-02T14:46:34,084][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-1-#133%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=0]
[2022-08-02T14:46:34,084][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=1]
[2022-08-02T14:46:34,084][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=2]
[2022-08-02T14:46:34,084][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=3]
...
[2022-08-02T14:46:34,092][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=49]
[2022-08-02T14:46:34,093][INFO
][exchange-worker-#127%wal.WalCompactionAfterRestartTest0%][GridCacheProcessor]
Finished recovery for cache [cache=ignite-sys-cache, grp=ignite-sys-cache,
startVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
{noformat}
Reproducer:
{code:java}
private final ListeningTestLogger logger = new ListeningTestLogger(log);
/** {@inheritDoc} */
@Override protected IgniteConfiguration getConfiguration(String name)
throws Exception {
return super.getConfiguration(name)
.setGridLogger(logger)
.setDataStorageConfiguration(new DataStorageConfiguration()
.setDefaultDataRegionConfiguration(new DataRegionConfiguration()
.setPersistenceEnabled(true))
.setWalSegmentSize(512 * 1024)
.setMaxWalArchiveSize(512 * 1024)
.setWalCompactionEnabled(true)
);
}
@Test
public void testCompactionDuplicationNotificationsOnRestart() throws
Exception {
String msg = "Segment compressed notification [idx=0]";
LogListener exactOnceMsg = LogListener.matches(msg).times(1).build();
LogListener duplicationMsgCheck =
LogListener.matches(msg).times(0).build();
logger.registerListener(exactOnceMsg);
IgniteEx ignite = startGrid(0);
ignite.cluster().state(ClusterState.ACTIVE);
for (int i = 0; i < 1_000; i++)
ignite.getOrCreateCache(DEFAULT_CACHE_NAME).put(i, i);
assertTrue(ignite.context().cache().context().wal().lastCompactedSegment() > 0);
assertTrue(exactOnceMsg.check());
// Restart grid.
stopGrid(0);
logger.registerListener(duplicationMsgCheck);
ignite = startGrid(0);
ignite.cluster().state(ClusterState.ACTIVE);
assertTrue("Duplicate notification present after restart.",
duplicationMsgCheck.check());
}
{code}
This happens because we don't update {{lastEnqueuedToCompressIdx}} in
[jira] [Updated] (IGNITE-17456) After node restart, there are duplicate messages about WAL segment compression.
[
https://issues.apache.org/jira/browse/IGNITE-17456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-17456:
--
Description:
If you enable compression of WAL segments and at the same time set the WAL
archive size too small, then compression will not actually occur, but after
each node restart, "fake" notifications about segment compression are
sequentially written to the log from the beginning (see log example below).
{noformat}
[2022-08-02T14:46:33,757][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Resolved write ahead log work directory:
/home/xtern/src/java/ignite/work/db/wal/node00-63fb6fa2-fcea-42aa-a3c8-b36cd330ba7c
[2022-08-02T14:46:33,757][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Resolved write ahead log archive directory:
/home/xtern/src/java/ignite/work/db/wal/archive/node00-63fb6fa2-fcea-42aa-a3c8-b36cd330ba7c
[2022-08-02T14:46:33,759][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=0]
[2022-08-02T14:46:33,759][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=1]
[2022-08-02T14:46:33,759][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=2]
[2022-08-02T14:46:33,759][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=3]
...
[2022-08-02T14:46:33,761][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=49]
[2022-08-02T14:46:33,761][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileHandleManagerImpl]
Initialized write-ahead log manager [mode=LOG_ONLY]
...
[2022-08-02T14:46:34,084][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-1-#133%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=0]
[2022-08-02T14:46:34,084][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=1]
[2022-08-02T14:46:34,084][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=2]
[2022-08-02T14:46:34,084][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=3]
...
[2022-08-02T14:46:34,092][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=49]
[2022-08-02T14:46:34,093][INFO
][exchange-worker-#127%wal.WalCompactionAfterRestartTest0%][GridCacheProcessor]
Finished recovery for cache [cache=ignite-sys-cache, grp=ignite-sys-cache,
startVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
{noformat}
Reproducer:
{code:java}
private final ListeningTestLogger logger = new ListeningTestLogger(log);
/** {@inheritDoc} */
@Override protected IgniteConfiguration getConfiguration(String name)
throws Exception {
return super.getConfiguration(name)
.setGridLogger(logger)
.setDataStorageConfiguration(new DataStorageConfiguration()
.setDefaultDataRegionConfiguration(new DataRegionConfiguration()
.setPersistenceEnabled(true))
.setWalSegmentSize(512 * 1024)
.setMaxWalArchiveSize(512 * 1024)
.setWalCompactionEnabled(true)
);
}
@Test
public void testCompactionDuplicationNotificationsOnRestart() throws
Exception {
String msg = "Segment compressed notification [idx=0]";
LogListener exactOnceMsg = LogListener.matches(msg).times(1).build();
LogListener duplicationMsgCheck =
LogListener.matches(msg).times(0).build();
logger.registerListener(exactOnceMsg);
IgniteEx ignite = startGrid(0);
ignite.cluster().state(ClusterState.ACTIVE);
for (int i = 0; i < 1_000; i++)
ignite.getOrCreateCache(DEFAULT_CACHE_NAME).put(i, i);
assertTrue(ignite.context().cache().context().wal().lastCompactedSegment() > 0);
assertTrue(exactOnceMsg.check());
// Restart grid.
stopGrid(0);
logger.registerListener(duplicationMsgCheck);
ignite = startGrid(0);
ignite.cluster().state(ClusterState.ACTIVE);
assertTrue("Duplicate notification present after restart.",
duplicationMsgCheck.check());
}
{code}
was:
If you enable compression of WAL segments and at the same time set
[jira] [Commented] (IGNITE-17398) Opencensus metrics do not work well with Prometheus since it do not have tags
[
https://issues.apache.org/jira/browse/IGNITE-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574198#comment-17574198
]
Sergey Kadaner commented on IGNITE-17398:
-
I see your point about it not being free, however, they say "Each labelset is
an additional time series". As I understand it, that means that it is the same
cost as creating the same number of time series. I.e. _cache_MY_CACHE_NAME1_
and _cache_MY_CACHE_NAME2_ metrics have the same cost as
_cache\{name="MY_CACHE_NAME1"}_ and {_}cache\{name="MY_CACHE_NAME2"}{_}. So
basically there is no additional cost in this case.
It is hard to say how it affects the GC pressure without actually measuring it,
but AFAIK modern garbage collectors (G1 and newer) are very good with
short-lived objects and should not have issues reclaiming them.
The issue is also not simply a convenience problem unless you can suggest how
to draw diagrams without knowing cache names in advance.
> Opencensus metrics do not work well with Prometheus since it do not have tags
> -
>
> Key: IGNITE-17398
> URL: https://issues.apache.org/jira/browse/IGNITE-17398
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.9.1
>Reporter: Sergey Kadaner
>Priority: Major
> Labels: metrics
> Attachments: image-2022-07-20-17-51-07-217.png
>
>
> The metrics created by the [new metrics
> system|https://ignite.apache.org/docs/latest/monitoring-metrics/new-metrics]
> are very inconvenient to use with Prometheus since it does not use tags.
> For example, Spring metric generated for the same cache looks like the
> following in Prometheus and is very convenient to use:
>
> {noformat}
> cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
> 1387.0{noformat}
>
> The native Ignite metric looks like the following:
>
> {noformat}
> cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
> The Spring reported statistics can be easily filtered by cache name and other
> attributes, while the build-in Ignite metrics do not provide an easy way to
> access cache names. The only option seems to be parsing the
> "cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by
> Grafana.
>
> For example with tags it is very easy to get a graph in Grafana with a cache
> hit percentage that includes every existing cache. It automatically extracts
> the cache name from the "cache" tag. Unfortunately, it is not possible if
> there are no tags.
> !image-2022-07-20-17-51-07-217.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17456) After node restart, there are duplicate messages about WAL segment compression.
[
https://issues.apache.org/jira/browse/IGNITE-17456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-17456:
--
Labels: ise (was: )
> After node restart, there are duplicate messages about WAL segment
> compression.
> ---
>
> Key: IGNITE-17456
> URL: https://issues.apache.org/jira/browse/IGNITE-17456
> Project: Ignite
> Issue Type: Bug
>Reporter: Pavel Pereslegin
>Assignee: Pavel Pereslegin
>Priority: Minor
> Labels: ise
>
> If you enable compression of WAL segments and at the same time set the WAL
> archive size too small, then compression will not actually occur, but after
> each node restart, "fake" notifications about segment compression are
> sequentially written to the log from the beginning (see log example below).
> {noformat}
> [2022-08-02T14:46:33,757][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Resolved write ahead log work directory:
> /home/xtern/src/java/ignite/work/db/wal/node00-63fb6fa2-fcea-42aa-a3c8-b36cd330ba7c
> [2022-08-02T14:46:33,757][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Resolved write ahead log archive directory:
> /home/xtern/src/java/ignite/work/db/wal/archive/node00-63fb6fa2-fcea-42aa-a3c8-b36cd330ba7c
> [2022-08-02T14:46:33,759][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Enqueuing segment for compression [idx=0]
> [2022-08-02T14:46:33,759][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Enqueuing segment for compression [idx=1]
> [2022-08-02T14:46:33,759][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Enqueuing segment for compression [idx=2]
> [2022-08-02T14:46:33,759][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Enqueuing segment for compression [idx=3]
> ...
> [2022-08-02T14:46:33,761][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Enqueuing segment for compression [idx=49]
> [2022-08-02T14:46:33,761][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileHandleManagerImpl]
> Initialized write-ahead log manager [mode=LOG_ONLY]
> ...
> [2022-08-02T14:46:34,084][INFO
> ][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-1-#133%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
> Segment compressed notification [idx=0]
> [2022-08-02T14:46:34,084][INFO
> ][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
> Segment compressed notification [idx=1]
> [2022-08-02T14:46:34,084][INFO
> ][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
> Segment compressed notification [idx=2]
> [2022-08-02T14:46:34,084][INFO
> ][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
> Segment compressed notification [idx=3]
> ...
> [2022-08-02T14:46:34,092][INFO
> ][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
> Segment compressed notification [idx=49]
> [2022-08-02T14:46:34,093][INFO
> ][exchange-worker-#127%wal.WalCompactionAfterRestartTest0%][GridCacheProcessor]
> Finished recovery for cache [cache=ignite-sys-cache, grp=ignite-sys-cache,
> startVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
> {noformat}
> Reproducer:
> {code:java}
> private final ListeningTestLogger logger = new ListeningTestLogger(log);
> /** {@inheritDoc} */
> @Override protected IgniteConfiguration getConfiguration(String name)
> throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(name);
> cfg.setGridLogger(logger)
> .setDataStorageConfiguration(new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(new
> DataRegionConfiguration()
> .setPersistenceEnabled(true))
> .setWalSegmentSize(512 * 1024)
> .setMaxWalArchiveSize(512 * 1024)
> .setWalCompactionEnabled(true)
> );
> return cfg;
> }
> @Test
> public void testCompactionDuplicationNotificationsOnRestart() throws
> Exception {
> String msg = "Segment compressed notification [idx=0]";
> LogListener exactOnceMsg = LogListener.matches(msg).times(1).build();
> LogListener duplicationMsgCheck =
> LogListener.matches(msg).times(0).build();
> logger.registerListener(exactOnceMsg);
> IgniteEx ignite =
[jira] [Updated] (IGNITE-11368) use the same information about indexes for JDBC drivers as for system view INDEXES
[
https://issues.apache.org/jira/browse/IGNITE-11368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Pavlov updated IGNITE-11368:
---
Labels: ise newbie (was: newbie)
> use the same information about indexes for JDBC drivers as for system view
> INDEXES
> --
>
> Key: IGNITE-11368
> URL: https://issues.apache.org/jira/browse/IGNITE-11368
> Project: Ignite
> Issue Type: Task
> Components: jdbc, odbc, sql
>Reporter: Yury Gerzhedovich
>Assignee: Ilya Shishkov
>Priority: Major
> Labels: ise, newbie
> Attachments: indexes_sqlline.txt
>
>
> As of now indexes information for JDBC drivers get by another way then system
> SQL view INDEXES. Need to use single source of the information to have
> consistent picture.
> So, JDBC drivers should use the same source as SQL view INDEXES
> (org.apache.ignite.internal.processors.query.h2.sys.view.SqlSystemViewIndexes)
> Start point for JDBC index metadata is
> org.apache.ignite.internal.jdbc2.JdbcDatabaseMetadata#getIndexInfo
> Also order of result should be correspond Javadoc ('ordered by NON_UNIQUE,
> TYPE, INDEX_NAME, and ORDINAL_POSITION') - at present it is not so.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17456) After node restart, there are duplicate messages about WAL segment compression.
[
https://issues.apache.org/jira/browse/IGNITE-17456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-17456:
--
Ignite Flags: (was: Docs Required,Release Notes Required)
> After node restart, there are duplicate messages about WAL segment
> compression.
> ---
>
> Key: IGNITE-17456
> URL: https://issues.apache.org/jira/browse/IGNITE-17456
> Project: Ignite
> Issue Type: Bug
>Reporter: Pavel Pereslegin
>Assignee: Pavel Pereslegin
>Priority: Minor
>
> If you enable compression of WAL segments and at the same time set the WAL
> archive size too small, then compression will not actually occur, but after
> each node restart, "fake" notifications about segment compression are
> sequentially written to the log from the beginning (see log example below).
> {noformat}
> [2022-08-02T14:46:33,757][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Resolved write ahead log work directory:
> /home/xtern/src/java/ignite/work/db/wal/node00-63fb6fa2-fcea-42aa-a3c8-b36cd330ba7c
> [2022-08-02T14:46:33,757][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Resolved write ahead log archive directory:
> /home/xtern/src/java/ignite/work/db/wal/archive/node00-63fb6fa2-fcea-42aa-a3c8-b36cd330ba7c
> [2022-08-02T14:46:33,759][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Enqueuing segment for compression [idx=0]
> [2022-08-02T14:46:33,759][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Enqueuing segment for compression [idx=1]
> [2022-08-02T14:46:33,759][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Enqueuing segment for compression [idx=2]
> [2022-08-02T14:46:33,759][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Enqueuing segment for compression [idx=3]
> ...
> [2022-08-02T14:46:33,761][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
> Enqueuing segment for compression [idx=49]
> [2022-08-02T14:46:33,761][INFO
> ][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileHandleManagerImpl]
> Initialized write-ahead log manager [mode=LOG_ONLY]
> ...
> [2022-08-02T14:46:34,084][INFO
> ][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-1-#133%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
> Segment compressed notification [idx=0]
> [2022-08-02T14:46:34,084][INFO
> ][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
> Segment compressed notification [idx=1]
> [2022-08-02T14:46:34,084][INFO
> ][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
> Segment compressed notification [idx=2]
> [2022-08-02T14:46:34,084][INFO
> ][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
> Segment compressed notification [idx=3]
> ...
> [2022-08-02T14:46:34,092][INFO
> ][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
> Segment compressed notification [idx=49]
> [2022-08-02T14:46:34,093][INFO
> ][exchange-worker-#127%wal.WalCompactionAfterRestartTest0%][GridCacheProcessor]
> Finished recovery for cache [cache=ignite-sys-cache, grp=ignite-sys-cache,
> startVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
> {noformat}
> Reproducer:
> {code:java}
> private final ListeningTestLogger logger = new ListeningTestLogger(log);
> /** {@inheritDoc} */
> @Override protected IgniteConfiguration getConfiguration(String name)
> throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(name);
> cfg.setGridLogger(logger)
> .setDataStorageConfiguration(new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(new
> DataRegionConfiguration()
> .setPersistenceEnabled(true))
> .setWalSegmentSize(512 * 1024)
> .setMaxWalArchiveSize(512 * 1024)
> .setWalCompactionEnabled(true)
> );
> return cfg;
> }
> @Test
> public void testCompactionDuplicationNotificationsOnRestart() throws
> Exception {
> String msg = "Segment compressed notification [idx=0]";
> LogListener exactOnceMsg = LogListener.matches(msg).times(1).build();
> LogListener duplicationMsgCheck =
> LogListener.matches(msg).times(0).build();
> logger.registerListener(exactOnceMsg);
> IgniteEx ignite
[jira] [Created] (IGNITE-17456) After node restart, there are duplicate messages about WAL segment compression.
Pavel Pereslegin created IGNITE-17456:
-
Summary: After node restart, there are duplicate messages about
WAL segment compression.
Key: IGNITE-17456
URL: https://issues.apache.org/jira/browse/IGNITE-17456
Project: Ignite
Issue Type: Bug
Reporter: Pavel Pereslegin
Assignee: Pavel Pereslegin
If you enable compression of WAL segments and at the same time set the WAL
archive size too small, then compression will not actually occur, but after
each node restart, "fake" notifications about segment compression are
sequentially written to the log from the beginning (see log example below).
{noformat}
[2022-08-02T14:46:33,757][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Resolved write ahead log work directory:
/home/xtern/src/java/ignite/work/db/wal/node00-63fb6fa2-fcea-42aa-a3c8-b36cd330ba7c
[2022-08-02T14:46:33,757][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Resolved write ahead log archive directory:
/home/xtern/src/java/ignite/work/db/wal/archive/node00-63fb6fa2-fcea-42aa-a3c8-b36cd330ba7c
[2022-08-02T14:46:33,759][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=0]
[2022-08-02T14:46:33,759][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=1]
[2022-08-02T14:46:33,759][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=2]
[2022-08-02T14:46:33,759][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=3]
...
[2022-08-02T14:46:33,761][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileWriteAheadLogManager]
Enqueuing segment for compression [idx=49]
[2022-08-02T14:46:33,761][INFO
][test-runner-#1%wal.WalCompactionAfterRestartTest%][FileHandleManagerImpl]
Initialized write-ahead log manager [mode=LOG_ONLY]
...
[2022-08-02T14:46:34,084][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-1-#133%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=0]
[2022-08-02T14:46:34,084][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=1]
[2022-08-02T14:46:34,084][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=2]
[2022-08-02T14:46:34,084][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=3]
...
[2022-08-02T14:46:34,092][INFO
][wal-file-compressor-%wal.WalCompactionAfterRestartTest0%-0-#131%wal.WalCompactionAfterRestartTest0%][FileWriteAheadLogManager]
Segment compressed notification [idx=49]
[2022-08-02T14:46:34,093][INFO
][exchange-worker-#127%wal.WalCompactionAfterRestartTest0%][GridCacheProcessor]
Finished recovery for cache [cache=ignite-sys-cache, grp=ignite-sys-cache,
startVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
{noformat}
Reproducer:
{code:java}
private final ListeningTestLogger logger = new ListeningTestLogger(log);
/** {@inheritDoc} */
@Override protected IgniteConfiguration getConfiguration(String name)
throws Exception {
IgniteConfiguration cfg = super.getConfiguration(name);
cfg.setGridLogger(logger)
.setDataStorageConfiguration(new DataStorageConfiguration()
.setDefaultDataRegionConfiguration(new DataRegionConfiguration()
.setPersistenceEnabled(true))
.setWalSegmentSize(512 * 1024)
.setMaxWalArchiveSize(512 * 1024)
.setWalCompactionEnabled(true)
);
return cfg;
}
@Test
public void testCompactionDuplicationNotificationsOnRestart() throws
Exception {
String msg = "Segment compressed notification [idx=0]";
LogListener exactOnceMsg = LogListener.matches(msg).times(1).build();
LogListener duplicationMsgCheck =
LogListener.matches(msg).times(0).build();
logger.registerListener(exactOnceMsg);
IgniteEx ignite = startGrid(0);
ignite.cluster().state(ClusterState.ACTIVE);
for (int i = 0; i < 1_000; i++)
ignite.getOrCreateCache(DEFAULT_CACHE_NAME).put(i, i);
assertTrue(ignite.context().cache().context().wal().lastCompactedSegment() > 0);
assertTrue(exactOnceMsg.check());
// Restart grid.
stopGrid(0);
logger.registerListener(duplicationMsgCheck);
ignite = startGrid(0);
[jira] [Commented] (IGNITE-17398) Opencensus metrics do not work well with Prometheus since it do not have tags
[
https://issues.apache.org/jira/browse/IGNITE-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574186#comment-17574186
]
Andrey N. Gura commented on IGNITE-17398:
-
Tags/labels convenience is not free. Prometheus developers warn about it: [Do
not overuse
labels|https://prometheus.io/docs/practices/instrumentation/#do-not-overuse-labels].
Moreover, the OpenCensus exporter produces a lot of garbage due to a text
nature of metrics format. Adding tags will significantly increase GC pressure
and size of exported data.
So, from my point of view, we should not add tags/labels. It looks like
convenience vs Ignite stability and performance.
> Opencensus metrics do not work well with Prometheus since it do not have tags
> -
>
> Key: IGNITE-17398
> URL: https://issues.apache.org/jira/browse/IGNITE-17398
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.9.1
>Reporter: Sergey Kadaner
>Priority: Major
> Labels: metrics
> Attachments: image-2022-07-20-17-51-07-217.png
>
>
> The metrics created by the [new metrics
> system|https://ignite.apache.org/docs/latest/monitoring-metrics/new-metrics]
> are very inconvenient to use with Prometheus since it does not use tags.
> For example, Spring metric generated for the same cache looks like the
> following in Prometheus and is very convenient to use:
>
> {noformat}
> cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
> 1387.0{noformat}
>
> The native Ignite metric looks like the following:
>
> {noformat}
> cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
> The Spring reported statistics can be easily filtered by cache name and other
> attributes, while the build-in Ignite metrics do not provide an easy way to
> access cache names. The only option seems to be parsing the
> "cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by
> Grafana.
>
> For example with tags it is very easy to get a graph in Grafana with a cache
> hit percentage that includes every existing cache. It automatically extracts
> the cache name from the "cache" tag. Unfortunately, it is not possible if
> there are no tags.
> !image-2022-07-20-17-51-07-217.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-16559) Node's log contains "Failed to refresh a leader" messages.
[
https://issues.apache.org/jira/browse/IGNITE-16559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574173#comment-17574173
]
Sergey Uttsel commented on IGNITE-16559:
The first option was implemented. [~maliev], look at the PR please.
> Node's log contains "Failed to refresh a leader" messages.
> ---
>
> Key: IGNITE-16559
> URL: https://issues.apache.org/jira/browse/IGNITE-16559
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Assignee: Sergey Uttsel
>Priority: Major
> Labels: ignite-3
>
> We noticed that when we run
> {{ItMixedQueriesTest.testIgniteSchemaAwaresAlterTableCommand}} on TC it is
> possible that log contain such messages:
> {noformat}
> 2022-02-15 12:36:43:568 +0300
> [ERROR][%ItMixedQueriesTest_null_1%Raft-Group-Client-0][RaftGroupServiceImpl]
> Failed to refresh a leader
> [groupId=8e71fc5e-6b24-4b69-ba5a-6eae4c2165cf_part_16]
> java.util.concurrent.CompletionException:
> java.util.concurrent.TimeoutException
> at
> java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
> at
> java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)
> at
> java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:632)
> at
> java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
> at
> java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
> at
> org.apache.ignite.raft.jraft.rpc.impl.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:502)
> at
> org.apache.ignite.raft.jraft.rpc.impl.RaftGroupServiceImpl$1.lambda$accept$1(RaftGroupServiceImpl.java:544)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.util.concurrent.TimeoutException
> {noformat}
> Possible root cause:
> Seems, that we get TimeoutException when we try to get a leader from a client
> for a group, for which leader has not been elected yet. If you check the
> logs, you can see, that we get timeout exception and after that leader for
> the corresponding group has been elected.
> Note that we have only one node and 10 partitions for a table in the test,
> but raft leaders are elected sequentially on a node, so electing 10 leaders
> for raft groups on one node might take a little bit longer.
> Possible solution:
> Increase timeout for a client to get a leader for the first time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-14487) Table view bulk methods refactoring
[ https://issues.apache.org/jira/browse/IGNITE-14487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-14487: -- Fix Version/s: 3.0.0-alpha6 > Table view bulk methods refactoring > --- > > Key: IGNITE-14487 > URL: https://issues.apache.org/jira/browse/IGNITE-14487 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > Fix For: 3.0.0-alpha6 > > Original Estimate: 120h > Time Spent: 20m > Remaining Estimate: 119h 40m > > org.apache.ignite.internal.table.RecordBinaryViewImpl#wrap use streams that > have performance impact. Let's rewrite it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-14487) Table view bulk methods refactoring
[ https://issues.apache.org/jira/browse/IGNITE-14487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574166#comment-17574166 ] Andrey Mashenkov commented on IGNITE-14487: --- [~zstan], see modules/table/src/main/java/org/apache/ignite/internal/table/RecordBinaryViewImpl.java > Table view bulk methods refactoring > --- > > Key: IGNITE-14487 > URL: https://issues.apache.org/jira/browse/IGNITE-14487 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > Original Estimate: 120h > Time Spent: 10m > Remaining Estimate: 119h 50m > > org.apache.ignite.internal.table.RecordBinaryViewImpl#wrap use streams that > have performance impact. Let's rewrite it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17365) Move internal classes used by H2 to an appropriate package
[ https://issues.apache.org/jira/browse/IGNITE-17365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich updated IGNITE-17365: --- Component/s: sql > Move internal classes used by H2 to an appropriate package > -- > > Key: IGNITE-17365 > URL: https://issues.apache.org/jira/browse/IGNITE-17365 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Major > Labels: ignite-osgi > Fix For: 2.14 > > Time Spent: 20m > Remaining Estimate: 0h > > We should avoid intersectipon of package definition between different > sub-modules in the Ignite Project. This is required to build OSGi bundle. > The follwoing classes must be moved: > - GridCacheTwoStepQuery > - QueryTable > - RegisteredQueryCursor > - SqlQueryMXBean > - SqlQueryMXBeanImpl -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-17441) Revise SQL related resolved tickets but still presented mentions in code AI3
[ https://issues.apache.org/jira/browse/IGNITE-17441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574103#comment-17574103 ] Andrey Mashenkov edited comment on IGNITE-17441 at 8/2/22 8:18 AM: --- [~jooger], LGTM, merged to main. was (Author: amashenkov): [~jooger], LGTM, merged to master. > Revise SQL related resolved tickets but still presented mentions in code AI3 > > > Key: IGNITE-17441 > URL: https://issues.apache.org/jira/browse/IGNITE-17441 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Yury Gerzhedovich >Assignee: Yury Gerzhedovich >Priority: Major > Labels: ignite-3 > Time Spent: 10m > Remaining Estimate: 0h > > We have a bunch of resolved tickes which still present as TODO in AI3 code. > Let's get it right. > List of issues: > https://issues.apache.org/jira/browse/IGNITE-14636 > https://issues.apache.org/jira/browse/IGNITE-14730 > https://issues.apache.org/jira/browse/IGNITE-15031 > https://issues.apache.org/jira/browse/IGNITE-15412 > https://issues.apache.org/jira/browse/IGNITE-15584 > https://issues.apache.org/jira/browse/IGNITE-1 > https://issues.apache.org/jira/browse/IGNITE-16468 > https://issues.apache.org/jira/browse/IGNITE-16967 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17260) Enrich IgniteTransactions and Transaction interfaces with RO related methods
[
https://issues.apache.org/jira/browse/IGNITE-17260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17260:
-
Description:
It's required to explicitly register an intent of starting readOnly
transaction, so that overloaded versions
{code:java}
Transaction begin(HybridTimestamp timestamp){code}
and
{code:java}
CompletableFuture beginAsync(HybridTimestamp timestamp){code}
should be added to IgniteTransactions together with corresponding
implementation.
Besides that Transaction interface should be extended with
{code:java}
boolean isReadOnly();{code}
and
{code:java}
long timestamp();{code}
methods.
was:
It's required to explicitly register an intent of starting readOnly
transaction, so that overloaded versions
{code:java}
Transaction begin(long timestamp){code}
and
{code:java}
CompletableFuture beginAsync(long timestamp){code}
should be added to IgniteTransactions together with corresponding
implementation.
Besides that Transaction interface should be extended with
{code:java}
boolean isReadOnly();{code}
and
{code:java}
long timestamp();{code}
methods.
> Enrich IgniteTransactions and Transaction interfaces with RO related methods
>
>
> Key: IGNITE-17260
> URL: https://issues.apache.org/jira/browse/IGNITE-17260
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction3_ro
>
> It's required to explicitly register an intent of starting readOnly
> transaction, so that overloaded versions
> {code:java}
> Transaction begin(HybridTimestamp timestamp){code}
> and
> {code:java}
> CompletableFuture beginAsync(HybridTimestamp timestamp){code}
> should be added to IgniteTransactions together with corresponding
> implementation.
> Besides that Transaction interface should be extended with
> {code:java}
> boolean isReadOnly();{code}
> and
> {code:java}
> long timestamp();{code}
> methods.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-17260) Enrich IgniteTransactions and Transaction interfaces with RO related methods
[
https://issues.apache.org/jira/browse/IGNITE-17260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin reassigned IGNITE-17260:
Assignee: (was: Alexander Lapin)
> Enrich IgniteTransactions and Transaction interfaces with RO related methods
>
>
> Key: IGNITE-17260
> URL: https://issues.apache.org/jira/browse/IGNITE-17260
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction3_ro
>
> It's required to explicitly register an intent of starting readOnly
> transaction, so that overloaded versions
> {code:java}
> Transaction begin(long timestamp){code}
> and
> {code:java}
> CompletableFuture beginAsync(long timestamp){code}
> should be added to IgniteTransactions together with corresponding
> implementation.
> Besides that Transaction interface should be extended with
> {code:java}
> boolean isReadOnly();{code}
> and
>
> {code:java}
> long timestamp();{code}
>
> methods.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-17260) Enrich IgniteTransactions and Transaction interfaces with RO related methods
[
https://issues.apache.org/jira/browse/IGNITE-17260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin reassigned IGNITE-17260:
Assignee: Alexander Lapin
> Enrich IgniteTransactions and Transaction interfaces with RO related methods
>
>
> Key: IGNITE-17260
> URL: https://issues.apache.org/jira/browse/IGNITE-17260
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction3_ro
>
> It's required to explicitly register an intent of starting readOnly
> transaction, so that overloaded versions
> {code:java}
> Transaction begin(long timestamp){code}
> and
> {code:java}
> CompletableFuture beginAsync(long timestamp){code}
> should be added to IgniteTransactions together with corresponding
> implementation.
> Besides that Transaction interface should be extended with
> {code:java}
> boolean isReadOnly();{code}
> and
>
> {code:java}
> long timestamp();{code}
>
> methods.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
