[jira] [Commented] (IGNITE-13051) Optimized affinity switch on baseline node left is broken for client topology and MVCC
[ https://issues.apache.org/jira/browse/IGNITE-13051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113765#comment-17113765 ] Alexey Scherbakov commented on IGNITE-13051: [~avinogradov] I suppose [1] should be set to true if a client topology is created on a node. [1] org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl#mvccEnabled > Optimized affinity switch on baseline node left is broken for client topology > and MVCC > -- > > Key: IGNITE-13051 > URL: https://issues.apache.org/jira/browse/IGNITE-13051 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Scherbakov >Priority: Critical > Fix For: 2.9 > > > If a node contains only client cache topology with MVCC enabled, PME will > hang after changes introduced in IGNITE-12617. > Reproduced by > CachePartitionLossWithRestartsTest.testPartitionLossDetectionOnClientTopology[0 > false false -1] and enabled MVCC. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13057) Do not print /tmp path if pds file is not stored in tmp folder.
Semyon Danilov created IGNITE-13057: --- Summary: Do not print /tmp path if pds file is not stored in tmp folder. Key: IGNITE-13057 URL: https://issues.apache.org/jira/browse/IGNITE-13057 Project: Ignite Issue Type: Improvement Reporter: Semyon Danilov Assignee: Semyon Danilov Temporary folder for pds warning can be printed even if pds is not located there -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13003) Improve data loss handling
[ https://issues.apache.org/jira/browse/IGNITE-13003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113549#comment-17113549 ] Vyacheslav Koptilin commented on IGNITE-13003: -- Hello [~ascherbakov], The improvement looks good to me. > Improve data loss handling > -- > > Key: IGNITE-13003 > URL: https://issues.apache.org/jira/browse/IGNITE-13003 > Project: Ignite > Issue Type: Improvement >Affects Versions: 2.8 >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Major > Fix For: 2.9 > > > Current implementation of data loss handling in Ignite has pitfalls which > have to be fixed. > Details can be found by reading a dev-list discussion [1] > [1] > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Data-loss-handling-improvements-td47310.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-6499) Compact NULL fields binary representation
[ https://issues.apache.org/jira/browse/IGNITE-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Hostettler reassigned IGNITE-6499: Assignee: Steve Hostettler > Compact NULL fields binary representation > - > > Key: IGNITE-6499 > URL: https://issues.apache.org/jira/browse/IGNITE-6499 > Project: Ignite > Issue Type: Improvement > Components: binary >Affects Versions: 2.1 >Reporter: Alexandr Kuramshin >Assignee: Steve Hostettler >Priority: Major > Labels: iep-2 > > Current compact footer implementation writes offset for the every field in > schema. Depending on serialized size of an object offset may be 1, 2 or 4 > bytes. > Imagine an object with some 100 fields are null. It takes from 100 to 400 > bytes overhead. For middle-sized objects (about 260 bytes) it doubles the > memory usage. For a small-sized objects (about 40 bytes) the memory usage > increased by factor 3 or 4. > Proposed two optimizations, the both should be implemented, the most optimal > implementation should be selected dynamically upon object marshalling. > 1. Write field index and offset for the only non-null fields in footer. > Should be used for objects with a few non-null fields. > 2. Write footer header then field offsets for the only non-null fields as > follows > [0] bit mask for first 8 fields, 0 - field is null, 1 - field is non-null > [1] cumulative sum of "1" bits > [2] bit mask for the next 8 fields > [3] cumulative sum of "1" bits > ... and so on > [N1...N2] offset of first non-null field > [N3...N4] offset of next non-null field > ... and so on > If we want to read fields from 0 to 7, then we read first footer byte, step > through bits and find the offset index for non-null field or find that field > is null. > If we want to read fields from 8, then we read two footer bytes, take start > offset index from the first byte, and then step through bits and find the > offset index for non-null field or find that field is null. > This supports up to 255 non-null fields per binary object. > Overhead would be only 24 bytes per 100 null fields instead of 200 bytes for > the middle-sized object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-10959) Memory leaks in continuous query handlers
[ https://issues.apache.org/jira/browse/IGNITE-10959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113502#comment-17113502 ] chin commented on IGNITE-10959: --- I just want to say that we ran into OOM with ContinuousQuery constantly, likely because of this bug, with no way around it, so in the end we were forced to abandon Ignite and moved to Hazelcast instead, which is a shame. > Memory leaks in continuous query handlers > - > > Key: IGNITE-10959 > URL: https://issues.apache.org/jira/browse/IGNITE-10959 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.7 >Reporter: Denis Mekhanikov >Assignee: Maxim Muzafarov >Priority: Critical > Attachments: CacheContinuousQueryMemoryUsageTest.java, > CacheContinuousQueryMemoryUsageTest.result, > CacheContinuousQueryMemoryUsageTest2.java, continuousquery_leak_profile.png > > > Continuous query handlers don't clear internal data structures after cache > events are processed. > A test, that reproduces the problem, is attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-10959) Memory leaks in continuous query handlers
[ https://issues.apache.org/jira/browse/IGNITE-10959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-10959: - Priority: Critical (was: Major) > Memory leaks in continuous query handlers > - > > Key: IGNITE-10959 > URL: https://issues.apache.org/jira/browse/IGNITE-10959 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.7 >Reporter: Denis Mekhanikov >Assignee: Maxim Muzafarov >Priority: Critical > Attachments: CacheContinuousQueryMemoryUsageTest.java, > CacheContinuousQueryMemoryUsageTest.result, > CacheContinuousQueryMemoryUsageTest2.java, continuousquery_leak_profile.png > > > Continuous query handlers don't clear internal data structures after cache > events are processed. > A test, that reproduces the problem, is attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-10959) Memory leaks in continuous query handlers
[ https://issues.apache.org/jira/browse/IGNITE-10959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-10959: - Fix Version/s: (was: 2.9) > Memory leaks in continuous query handlers > - > > Key: IGNITE-10959 > URL: https://issues.apache.org/jira/browse/IGNITE-10959 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.7 >Reporter: Denis Mekhanikov >Assignee: Maxim Muzafarov >Priority: Major > Attachments: CacheContinuousQueryMemoryUsageTest.java, > CacheContinuousQueryMemoryUsageTest.result, > CacheContinuousQueryMemoryUsageTest2.java, continuousquery_leak_profile.png > > > Continuous query handlers don't clear internal data structures after cache > events are processed. > A test, that reproduces the problem, is attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-10959) Memory leaks in continuous query handlers
[ https://issues.apache.org/jira/browse/IGNITE-10959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov reassigned IGNITE-10959: Assignee: Maxim Muzafarov > Memory leaks in continuous query handlers > - > > Key: IGNITE-10959 > URL: https://issues.apache.org/jira/browse/IGNITE-10959 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.7 >Reporter: Denis Mekhanikov >Assignee: Maxim Muzafarov >Priority: Major > Fix For: 2.9 > > Attachments: CacheContinuousQueryMemoryUsageTest.java, > CacheContinuousQueryMemoryUsageTest.result, > CacheContinuousQueryMemoryUsageTest2.java, continuousquery_leak_profile.png > > > Continuous query handlers don't clear internal data structures after cache > events are processed. > A test, that reproduces the problem, is attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13056) SchemaManager refactoring
Nikolay Izhikov created IGNITE-13056: Summary: SchemaManager refactoring Key: IGNITE-13056 URL: https://issues.apache.org/jira/browse/IGNITE-13056 Project: Ignite Issue Type: New Feature Components: sql Affects Versions: 2.8 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Since Ignite wants to leverage from several SQL engines we need to make SchemaManager independent from H2. We also should consider moving it to the core module to make it available from any module that wants to use it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13055) ExecutorServices must handle AssertionErrors and log uncaught exceptions
[
https://issues.apache.org/jira/browse/IGNITE-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-13055:
-
Ignite Flags: (was: Docs Required,Release Notes Required)
> ExecutorServices must handle AssertionErrors and log uncaught exceptions
>
>
> Key: IGNITE-13055
> URL: https://issues.apache.org/jira/browse/IGNITE-13055
> Project: Ignite
> Issue Type: Improvement
>Reporter: Maxim Muzafarov
>Priority: Major
>
> Currently, most of ExecuterServices (e.g. {{sysExecSvc}}) handles only
> OutOfMemoryException. Services must also handle AssertionErrors and log any
> uncaught exceptions.
> The most important list of executor services configured with OOM handler only:
> - execSvc
> - svcExecSvc
> - sysExecSvc
> - p2pExecSvc
> - restExecSvc
> - utilityCacheExecSvc
> - affExecSvc
> - qryExecSvc
> See the dev-list discussion for details.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13002) Supports user's java object by JDBC thin client
[
https://issues.apache.org/jira/browse/IGNITE-13002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113481#comment-17113481
]
Ignite TC Bot commented on IGNITE-13002:
{panel:title=Branch: [pull/7812/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5316493buildTypeId=IgniteTests24Java8_RunAll]
> Supports user's java object by JDBC thin client
> ---
>
> Key: IGNITE-13002
> URL: https://issues.apache.org/jira/browse/IGNITE-13002
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Konstantin Orlov
>Assignee: Konstantin Orlov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Now JDBC thin client doesn't support no-SQL types.
> e.g.:
> {{CREATE TABLE TEST (ID INT PRIMARY KEY, VAL OTHER)}}
> {code}
> PreparedStatement stmt = conn.prepareStatement("INSERT INTO TEST VALUES(?,
> ?)")
> stmt.setInteger(1, 0);
> stmt.setObject(2, new MyClass());
> {code}
> We have to support {{GridBinaryMarshaller}} for the JDBC thin client.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13055) ExecutorServices must handle AssertionErrors and log uncaught exceptions
Maxim Muzafarov created IGNITE-13055:
Summary: ExecutorServices must handle AssertionErrors and log
uncaught exceptions
Key: IGNITE-13055
URL: https://issues.apache.org/jira/browse/IGNITE-13055
Project: Ignite
Issue Type: Improvement
Reporter: Maxim Muzafarov
Currently, most of ExecuterServices (e.g. {{sysExecSvc}}) handles only
OutOfMemoryException. Services must also handle AssertionErrors and log any
uncaught exceptions.
The most important list of executor services configured with OOM handler only:
- execSvc
- svcExecSvc
- sysExecSvc
- p2pExecSvc
- restExecSvc
- utilityCacheExecSvc
- affExecSvc
- qryExecSvc
See the dev-list discussion for details.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13012) Make node connection checking rely on the configuration. Simplify node ping routine.
[
https://issues.apache.org/jira/browse/IGNITE-13012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Steshin updated IGNITE-13012:
--
Description:
Node-to-next-node connection checking has several drawbacks which go together.
We should fix the following :
1) First thing firts, make connection check interval predictable and dependable
on the failureDetectionTimeout or similar params. Current value is a constant:
{code:java}static int ServerImpls.CON_CHECK_INTERVAL = 500{code}
2) Make connection check interval rely on common time of any last sent message.
Current ping is bound to own time:
{code:java}ServerImpl. RingMessageWorker.lastTimeConnCheckMsgSent{code}
This is weird because any discovery message does actually check connection. And
TpDiscoveryConnectionCheckMessage is just an addition when message queue is
empty for a long time.
3) Remove additional, randomly appearing quickened connection checking. Once
we do #1, this will become even more useless.
Despite we have a period of connection checking (see #1), we can also send ping
before the period exhausts. This premature node ping relies on the time of any
sent or even received message. Imagine: if node 2 receives no message from node
1 within some time, it decides to do extra ping node 3 not waiting for regular
ping. This happens quite randomly. Such behavior makes confusion and gives no
benefits.
4) Do not worry user with “Node disconnected” when everything is OK. Once we do
#1, this will become even more useless.
If #3 happens, node writes in the log on INFO: “Local node seems to be
disconnected from topology …” whereas it is not actually disconnected at all.
User can see this unexpected and worrying message if he typed
failureDetectionTimeout < 500ms.
was:
Node-to-next-node connection checking has several drawbacks. we shoul fix:
1) Make connection check interval dependable on failureDetectionTimeout or
similar params. Current value is a constant:
{code:java}static int ServerImpls.CON_CHECK_INTERVAL = 500;{code}
2) Connection checking isn't based on time of the last message sent.
TcpDiscoveryConnectionCheckMessage is bound to own time (ServerImpl.
RingMessageWorker.lastTimeConnCheckMsgSent). This is weird because any
discovery message actually checks connection. And
TpDiscoveryConnectionCheckMessage is just an addition when message queue is
empty for a long time.
3) Period of Node-to-Node connection checking can be sometimes shortened for
strange reason: if no sent or received message appears within
failureDetectionTimeout. Here, despite we have minimal period of connection
checking (ServerImpls.CON_CHECK_INTERVAL), we can also send
TpDiscoveryConnectionCheckMessage before this period exhausted. Moreover, this
premature node ping relies also on the time of last received message. Imagine:
if node 2 receives no message from node 1 within some time it decides to do
extra ping node 3 not waiting for regular ping interval. Such behavior makes
confusion and gives no additional guaranties.
4) If #3 happens, node writes in the log on INFO: “Local node seems to be
disconnected from topology …” whereas it is not actually disconnected. User can
see this message if he typed failureDetectionTimeout < 500ms. I wouldn’t like
seeing INFO in a log saying a node is might be disconnected. This sounds like
some troubles raised in network. But not as everything is OK.
Suggestions:
2) Make connection check interval rely on common time of last sent message.
Not on dedicated time.
3) Remove additional, random, quickened connection checking.
4) Do not worry user with “Node disconnected” when everything is OK.
> Make node connection checking rely on the configuration. Simplify node ping
> routine.
>
>
> Key: IGNITE-13012
> URL: https://issues.apache.org/jira/browse/IGNITE-13012
> Project: Ignite
> Issue Type: Improvement
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Major
> Labels: iep-45
>
> Node-to-next-node connection checking has several drawbacks which go
> together. We should fix the following :
> 1) First thing firts, make connection check interval predictable and
> dependable on the failureDetectionTimeout or similar params. Current value is
> a constant:
> {code:java}static int ServerImpls.CON_CHECK_INTERVAL = 500{code}
> 2) Make connection check interval rely on common time of any last sent
> message. Current ping is bound to own time:
> {code:java}ServerImpl. RingMessageWorker.lastTimeConnCheckMsgSent{code}
> This is weird because any discovery message does actually check connection.
> And TpDiscoveryConnectionCheckMessage is just an addition when message queue
> is empty for a long time.
> 3) Remove additional,
[jira] [Commented] (IGNITE-13003) Improve data loss handling
[
https://issues.apache.org/jira/browse/IGNITE-13003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113409#comment-17113409
]
Ignite TC Bot commented on IGNITE-13003:
{panel:title=Branch: [pull/7800/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5322051buildTypeId=IgniteTests24Java8_RunAll]
> Improve data loss handling
> --
>
> Key: IGNITE-13003
> URL: https://issues.apache.org/jira/browse/IGNITE-13003
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.8
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
> Fix For: 2.9
>
>
> Current implementation of data loss handling in Ignite has pitfalls which
> have to be fixed.
> Details can be found by reading a dev-list discussion [1]
> [1]
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Data-loss-handling-improvements-td47310.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13012) Make node connection checking rely on the configuration. Simplify node ping routine.
[
https://issues.apache.org/jira/browse/IGNITE-13012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Steshin updated IGNITE-13012:
--
Description:
Node-to-next-node connection checking has several drawbacks. we shoul fix:
1) Make connection check interval dependable on failureDetectionTimeout or
similar params. Current value is a constant:
{code:java}static int ServerImpls.CON_CHECK_INTERVAL = 500;{code}
2) Connection checking isn't based on time of the last message sent.
TcpDiscoveryConnectionCheckMessage is bound to own time (ServerImpl.
RingMessageWorker.lastTimeConnCheckMsgSent). This is weird because any
discovery message actually checks connection. And
TpDiscoveryConnectionCheckMessage is just an addition when message queue is
empty for a long time.
3) Period of Node-to-Node connection checking can be sometimes shortened for
strange reason: if no sent or received message appears within
failureDetectionTimeout. Here, despite we have minimal period of connection
checking (ServerImpls.CON_CHECK_INTERVAL), we can also send
TpDiscoveryConnectionCheckMessage before this period exhausted. Moreover, this
premature node ping relies also on the time of last received message. Imagine:
if node 2 receives no message from node 1 within some time it decides to do
extra ping node 3 not waiting for regular ping interval. Such behavior makes
confusion and gives no additional guaranties.
4) If #3 happens, node writes in the log on INFO: “Local node seems to be
disconnected from topology …” whereas it is not actually disconnected. User can
see this message if he typed failureDetectionTimeout < 500ms. I wouldn’t like
seeing INFO in a log saying a node is might be disconnected. This sounds like
some troubles raised in network. But not as everything is OK.
Suggestions:
2) Make connection check interval rely on common time of last sent message.
Not on dedicated time.
3) Remove additional, random, quickened connection checking.
4) Do not worry user with “Node disconnected” when everything is OK.
was:
Current noted-to-node connection checking has several drawbacks:
1) Minimal connection checking interval is not bound to failure detection
parameters:
static int ServerImpls.CON_CHECK_INTERVAL = 500;
2) Connection checking isn't based on time of the last message sent.
TcpDiscoveryConnectionCheckMessage is bound to own time (ServerImpl.
RingMessageWorker.lastTimeConnCheckMsgSent). This is weird because any
discovery message actually checks connection. And
TpDiscoveryConnectionCheckMessage is just an addition when message queue is
empty for a long time.
3) Period of Node-to-Node connection checking can be sometimes shortened
for strange reason: if no sent or received message appears within
failureDetectionTimeout. Here, despite we have minimal period of connection
checking (ServerImpls.CON_CHECK_INTERVAL), we can also send
TpDiscoveryConnectionCheckMessage before this period exhausted. Moreover, this
premature node ping relies also on the time of last received message. Imagine:
if node 2 receives no message from node 1 within some time it decides to do
extra ping node 3 not waiting for regular ping interval. Such behavior makes
confusion and gives no additional guaranties.
4) If #3 happens, node writes in the log on INFO: “Local node seems to be
disconnected from topology …” whereas it is not actually disconnected. User can
see this message if he typed failureDetectionTimeout < 500ms. I wouldn’t like
seeing INFO in a log saying a node is might be disconnected. This sounds like
some troubles raised in network. But not as everything is OK.
Suggestions:
1) Make connection check interval be based on failureDetectionTimeout or
similar params.
2) Make connection check interval rely on common time of last sent
message. Not on dedicated time.
3) Remove additional, random, quickened connection checking.
4) Do not worry user with “Node disconnected” when everything is OK.
> Make node connection checking rely on the configuration. Simplify node ping
> routine.
>
>
> Key: IGNITE-13012
> URL: https://issues.apache.org/jira/browse/IGNITE-13012
> Project: Ignite
> Issue Type: Improvement
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Major
> Labels: iep-45
>
> Node-to-next-node connection checking has several drawbacks. we shoul fix:
> 1) Make connection check interval dependable on failureDetectionTimeout or
> similar params. Current value is a constant:
> {code:java}static int ServerImpls.CON_CHECK_INTERVAL = 500;{code}
> 2) Connection checking isn't based on time of the last message sent.
> TcpDiscoveryConnectionCheckMessage is bound to own time (ServerImpl.
[jira] [Issue Comment Deleted] (IGNITE-13046) Flaky test SqlSystemViewsSelfTest.testTableViewDuringRebuilding
[
https://issues.apache.org/jira/browse/IGNITE-13046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey N. Gura updated IGNITE-13046:
Comment: was deleted
(was: {panel:title=Branch: [pull/7825/head] Base: [master] : Possible Blockers
(2)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Cache 5{color} [[tests
2|https://ci.ignite.apache.org/viewLog.html?buildId=5326640]]
* IgniteCacheWithIndexingTestSuite:
CacheGroupMetricsWithIndexTest.testIndexRebuildCountPartitionsLeft - Test has
low fail rate in base branch 0,0% and is not flaky
* IgniteCacheWithIndexingTestSuite:
CacheGroupMetricsWithIndexBuildFailTest.testIndexRebuildCountPartitionsLeft -
Test has low fail rate in base branch 0,0% and is not flaky
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5325287buildTypeId=IgniteTests24Java8_RunAll])
> Flaky test SqlSystemViewsSelfTest.testTableViewDuringRebuilding
> ---
>
> Key: IGNITE-13046
> URL: https://issues.apache.org/jira/browse/IGNITE-13046
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey N. Gura
>Assignee: Andrey N. Gura
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Test {{SqlSystemViewsSelfTest.testTableViewDuringRebuilding}} fails
> sporadically. The reason is changes introduced due to the IGNITE-12684 issue.
> There is race between runner thread and thread which completes future in
> {{IgniteH2Indexing#rebuildIndexesFromHash()}} method. Runner thread is
> unblocked due to the future completion and it tries to assert conditions
> while thread which completed the future changes state of index rebuild
> process.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13046) Flaky test SqlSystemViewsSelfTest.testTableViewDuringRebuilding
[
https://issues.apache.org/jira/browse/IGNITE-13046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113396#comment-17113396
]
Ignite TC Bot commented on IGNITE-13046:
{panel:title=Branch: [pull/7825/head] Base: [master] : Possible Blockers
(2)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Cache 5{color} [[tests
2|https://ci.ignite.apache.org/viewLog.html?buildId=5326640]]
* IgniteCacheWithIndexingTestSuite:
CacheGroupMetricsWithIndexTest.testIndexRebuildCountPartitionsLeft - Test has
low fail rate in base branch 0,0% and is not flaky
* IgniteCacheWithIndexingTestSuite:
CacheGroupMetricsWithIndexBuildFailTest.testIndexRebuildCountPartitionsLeft -
Test has low fail rate in base branch 0,0% and is not flaky
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5325287buildTypeId=IgniteTests24Java8_RunAll]
> Flaky test SqlSystemViewsSelfTest.testTableViewDuringRebuilding
> ---
>
> Key: IGNITE-13046
> URL: https://issues.apache.org/jira/browse/IGNITE-13046
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey N. Gura
>Assignee: Andrey N. Gura
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Test {{SqlSystemViewsSelfTest.testTableViewDuringRebuilding}} fails
> sporadically. The reason is changes introduced due to the IGNITE-12684 issue.
> There is race between runner thread and thread which completes future in
> {{IgniteH2Indexing#rebuildIndexesFromHash()}} method. Runner thread is
> unblocked due to the future completion and it tries to assert conditions
> while thread which completed the future changes state of index rebuild
> process.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-6785) Affinity field name forced to be upper-case
[
https://issues.apache.org/jira/browse/IGNITE-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Petrov reassigned IGNITE-6785:
--
Assignee: Mikhail Petrov
> Affinity field name forced to be upper-case
> ---
>
> Key: IGNITE-6785
> URL: https://issues.apache.org/jira/browse/IGNITE-6785
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.1, 2.8
>Reporter: Denis A. Magda
>Assignee: Mikhail Petrov
>Priority: Major
> Labels: newbie, usability
> Fix For: 2.9
>
> Attachments: SqlAffinityKeyTest.java, sql_bug.zip
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> If an SQL schema and cache is created with CREATE TABLE command and a user
> wants to use key-value APIs creating its own custom key class, then (at
> least) the key class's affinity field forced to be written in upper-case.
> Steps to reproduce using the project attached:
> * start a node with {{./ignite.sh ../examples/config/example-ignite.xml}}.
> * create {{City}} table using {{ignite_world.sql}}. SQLline is one of the
> quickest ways: https://apacheignite-sql.readme.io/docs/sqlline
> * Run {{KeyValueDataProcessing}} to catch the exception below
> {noformat}
> Exception in thread "main" class
> org.apache.ignite.binary.BinaryObjectException: Binary type has different
> affinity key fields [typeName=demo.model.CityKey,
> affKeyFieldName1=COUNTRYCODE, affKeyFieldName2=countryCode]
> at
> org.apache.ignite.internal.binary.BinaryUtils.mergeMetadata(BinaryUtils.java:987)
> {noformat}
> If fact {{CityKey}} names the affinity field in the same way as in CREATE
> TABLE - {{countryCode}}.
> Next, run {{KeyValueBinaryDataProcessing}} to spot another weird thing:
> * BinaryObject key accepts `countryCode` as the affinity field name.
> * If to print our a binary object value then all the fields are in the
> upper-case (they were not defined this way in CREATE TABLE):
> {noformat}
> demo.model.City [idHash=1613627715, hash=-1386587499, DISTRICT=Noord-Holland,
> POPULATION=711200, NAME=Amsterdam]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-6785) Affinity field name forced to be upper-case
[
https://issues.apache.org/jira/browse/IGNITE-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Petrov updated IGNITE-6785:
---
Attachment: SqlAffinityKeyTest.java
> Affinity field name forced to be upper-case
> ---
>
> Key: IGNITE-6785
> URL: https://issues.apache.org/jira/browse/IGNITE-6785
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.1, 2.8
>Reporter: Denis A. Magda
>Priority: Major
> Labels: newbie, usability
> Fix For: 2.9
>
> Attachments: SqlAffinityKeyTest.java, sql_bug.zip
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> If an SQL schema and cache is created with CREATE TABLE command and a user
> wants to use key-value APIs creating its own custom key class, then (at
> least) the key class's affinity field forced to be written in upper-case.
> Steps to reproduce using the project attached:
> * start a node with {{./ignite.sh ../examples/config/example-ignite.xml}}.
> * create {{City}} table using {{ignite_world.sql}}. SQLline is one of the
> quickest ways: https://apacheignite-sql.readme.io/docs/sqlline
> * Run {{KeyValueDataProcessing}} to catch the exception below
> {noformat}
> Exception in thread "main" class
> org.apache.ignite.binary.BinaryObjectException: Binary type has different
> affinity key fields [typeName=demo.model.CityKey,
> affKeyFieldName1=COUNTRYCODE, affKeyFieldName2=countryCode]
> at
> org.apache.ignite.internal.binary.BinaryUtils.mergeMetadata(BinaryUtils.java:987)
> {noformat}
> If fact {{CityKey}} names the affinity field in the same way as in CREATE
> TABLE - {{countryCode}}.
> Next, run {{KeyValueBinaryDataProcessing}} to spot another weird thing:
> * BinaryObject key accepts `countryCode` as the affinity field name.
> * If to print our a binary object value then all the fields are in the
> upper-case (they were not defined this way in CREATE TABLE):
> {noformat}
> demo.model.City [idHash=1613627715, hash=-1386587499, DISTRICT=Noord-Holland,
> POPULATION=711200, NAME=Amsterdam]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13034) Append additional yardstick test for page replacements test purpose.
[ https://issues.apache.org/jira/browse/IGNITE-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113348#comment-17113348 ] Maxim Muzafarov commented on IGNITE-13034: -- [~zstan] LGTM, thank you for your contribution. > Append additional yardstick test for page replacements test purpose. > > > Key: IGNITE-13034 > URL: https://issues.apache.org/jira/browse/IGNITE-13034 > Project: Ignite > Issue Type: Improvement > Components: yardstick >Affects Versions: 2.8 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Fix For: 2.9 > > Time Spent: 20m > Remaining Estimate: 0h > > There is a need in additional yardstick benchmarks, which will measure > throughput with page replacement already in action. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-12633) Calcite integration. Query cancel purpose.
[ https://issues.apache.org/jira/browse/IGNITE-12633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravil Galeyev reassigned IGNITE-12633: -- Assignee: Ravil Galeyev > Calcite integration. Query cancel purpose. > -- > > Key: IGNITE-12633 > URL: https://issues.apache.org/jira/browse/IGNITE-12633 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Igor Seliverstov >Assignee: Ravil Galeyev >Priority: Major > > Currently on a query participant node left whole query is cancelled, but the > end user doesn't know the purpose. We should pass causing cancel exception to > end user. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12823) .NET: Service method with user type array parameter can't be found
[
https://issues.apache.org/jira/browse/IGNITE-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113346#comment-17113346
]
Ignite TC Bot commented on IGNITE-12823:
{panel:title=Branch: [pull/7788/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5327287buildTypeId=IgniteTests24Java8_RunAll]
> .NET: Service method with user type array parameter can't be found
> --
>
> Key: IGNITE-12823
> URL: https://issues.apache.org/jira/browse/IGNITE-12823
> Project: Ignite
> Issue Type: Bug
> Components: platforms
>Reporter: Alexey Kukushkin
>Assignee: Nikolai Kulagin
>Priority: Major
> Labels: .NET, sbcf
> Attachments: ignite-12823-vs-2.8.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *+Setup+*
> * Ignite Java service with a method having an array of user types as a
> parameters, for example, caclulate(Parameter[] params)
> *+Actions+*
> * .NET client calls the Ignite Java service, for example,
> ignite.GetServices().GetServiceProxy().calculate(new[] \{new
> Parameter()});
> *+Expected+*
> * The service method is called
> *+Actual+*
> * Exception "Could not find proxy method 'calculate' in class ICalculator"
> *+Workaround+*
> * Replace array of user types with array of objects in the service methods
> signatures, for example, caclulate(Object[] params)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12961) Start snapshot operation via control.sh
[
https://issues.apache.org/jira/browse/IGNITE-12961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113329#comment-17113329
]
Maxim Muzafarov commented on IGNITE-12961:
--
[~NSAmelchev]
Merged to the master branch. Thank you for the review.
> Start snapshot operation via control.sh
> ---
>
> Key: IGNITE-12961
> URL: https://issues.apache.org/jira/browse/IGNITE-12961
> Project: Ignite
> Issue Type: Improvement
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Labels: iep-43
> Fix For: 2.9
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Add the ability to start snapshot operation via {{control.sh}} command line.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13054) Prevent nodes with stale data joining the active topology.
Alexey Scherbakov created IGNITE-13054: -- Summary: Prevent nodes with stale data joining the active topology. Key: IGNITE-13054 URL: https://issues.apache.org/jira/browse/IGNITE-13054 Project: Ignite Issue Type: Bug Reporter: Alexey Scherbakov Fix For: 2.9 After IGNITE-13003 LOST state is preserved then nodes with lost data are retuned to topology after failure. If resetting is performed on lesser topology, it's possible to get into sutiation where a node with most recent data is returned to active topology where some key already could be modified. This should not be allowed because brings conflicting data into grid. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-10100) Add public Java API to call Ignite.NET services
[
https://issues.apache.org/jira/browse/IGNITE-10100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113292#comment-17113292
]
Aleksey Plekhanov commented on IGNITE-10100:
[~ivandasch], I've looked at your patch and it looks good to me.
[~daradurvs], actually services lifecycle is not affected by this patch, only
GridServiceProxy work is affected. If you have no time I think we can merge
this patch without your visa. WDYT?
> Add public Java API to call Ignite.NET services
> ---
>
> Key: IGNITE-10100
> URL: https://issues.apache.org/jira/browse/IGNITE-10100
> Project: Ignite
> Issue Type: Improvement
> Components: platforms
>Affects Versions: 2.6
>Reporter: Alexey Kukushkin
>Assignee: Ivan Daschinskiy
>Priority: Major
> Labels: .NET, sbcf
> Fix For: 2.9
>
> Attachments: ignite-10100-vs-2.8.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Ignite wraps .NET services in PlatformDotNetServiceImpl implementing
> PlatformService interface.
> PlatformService is defined in internal Ignite package
> apache.ignite.internal.processors.platform.services. It exposes
> {{ invokeMethod(methodName, Object[] params): Object}}
> to call any service method dynamically. Right now there is no Ignite public
> API to call a PlatformService using static typing.
> We need to develop a public API to call PlatformDotNetServiceImpl using
> static typing in Java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13051) Optimized affinity switch on baseline node left is broken for client topology and MVCC
[ https://issues.apache.org/jira/browse/IGNITE-13051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113282#comment-17113282 ] Anton Vinogradov commented on IGNITE-13051: --- [~ascherbakov], Seems, ctx.kernalContext().coordinators().mvccEnabled() provides wrong result in this case. Do we able to gain mvcc state on client topology somehow? > Optimized affinity switch on baseline node left is broken for client topology > and MVCC > -- > > Key: IGNITE-13051 > URL: https://issues.apache.org/jira/browse/IGNITE-13051 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Scherbakov >Priority: Critical > Fix For: 2.9 > > > If a node contains only client cache topology with MVCC enabled, PME will > hang after changes introduced in IGNITE-12617. > Reproduced by > CachePartitionLossWithRestartsTest.testPartitionLossDetectionOnClientTopology[0 > false false -1] and enabled MVCC. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13053) resetLostPartitions is not working if invoked from a node where a cache not started
[ https://issues.apache.org/jira/browse/IGNITE-13053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-13053: --- Description: Reproduced by CachePartitionLossWithRestartsTest and an attemp to reset lost policies using node with index=nonAffIdx > resetLostPartitions is not working if invoked from a node where a cache not > started > --- > > Key: IGNITE-13053 > URL: https://issues.apache.org/jira/browse/IGNITE-13053 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Scherbakov >Priority: Major > Fix For: 2.9 > > > Reproduced by CachePartitionLossWithRestartsTest and an attemp to reset lost > policies using node with index=nonAffIdx -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13053) resetLostPartitions is not working if invoked from a node where a cache not started
Alexey Scherbakov created IGNITE-13053: -- Summary: resetLostPartitions is not working if invoked from a node where a cache not started Key: IGNITE-13053 URL: https://issues.apache.org/jira/browse/IGNITE-13053 Project: Ignite Issue Type: Bug Reporter: Alexey Scherbakov Fix For: 2.9 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
[ https://issues.apache.org/jira/browse/IGNITE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-13052: Description: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changes its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should also extract WAL pointer reservation and filtering small partitions from reserveHistoryForExchange(), but this shouldn't be a problem. Another point for optimization: searchPartitionCounter() and searchCheckpointEntry() are executed for each (grpId, partId). That means we'll perform O(number of partitions) linear lookups in history. This should be optimized as well: we can perform one lookup for all (grpId, partId) pairs. This is especially critical for reserveHistoryForPreloading() method complexity: it's executed from exchange thread. Memory overhead of storing described map on heap is insignificant. Its size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. P.S. Possibly, instead of storing map, we can keep earlistCheckpoint right in GridDhtLocalPartition. It may simplify implementation. was: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changes its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should also extract WAL pointer reservation and filtering small partitions from reserveHistoryForExchange(), but this shouldn't be a problem. Another point: possibly, instead of storing map, we can keep earlistCheckpoint right in GridDhtLocalPartition. It may simplify implementation. Memory overhead of storing described map on heap is insignificant. Its size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. > Calculate result of reserveHistoryForExchange in advance > > > Key: IGNITE-13052 > URL: https://issues.apache.org/jira/browse/IGNITE-13052 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Rakov >Priority: Major > Original Estimate: 80h > Remaining Estimate: 80h > > Method reserveHistoryForExchange() is called on every partition map exchange. > It's an expensive call: it requires iteration over the whole checkpoint > history with possible retrieve of GroupState from WAL (it's stored on heap > with SoftReference). On some deployments this operation can take several > minutes. > The idea of optimization is to calculate its result only on first PME > (ideally, even before first PME, on recovery stage), keep resulting map > (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. > From the first glance, the map should be updated: > 1) On checkpoint. If a new partition appears on local node, it should be > registered in the map with current checkpoint. If a partition is evicted from > local node, or changes its state to
[jira] [Updated] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
[ https://issues.apache.org/jira/browse/IGNITE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-13052: Description: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changes its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should also extract WAL pointer reservation and filtering small partitions from reserveHistoryForExchange(), but this shouldn't be a problem. Another point for optimization: searchPartitionCounter() and searchCheckpointEntry() are executed for each (grpId, partId). That means we'll perform O(number of partitions) linear lookups in history. This should be optimized as well: we can perform one lookup for all (grpId, partId) pairs. This is especially critical for reserveHistoryForPreloading() method complexity: it's executed from exchange thread. Memory overhead of storing described map on heap is insignificant. Its size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. P.S. Possibly, instead of storing map, we can keep earliestCheckpoint right in GridDhtLocalPartition. It may simplify implementation. was: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changes its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should also extract WAL pointer reservation and filtering small partitions from reserveHistoryForExchange(), but this shouldn't be a problem. Another point for optimization: searchPartitionCounter() and searchCheckpointEntry() are executed for each (grpId, partId). That means we'll perform O(number of partitions) linear lookups in history. This should be optimized as well: we can perform one lookup for all (grpId, partId) pairs. This is especially critical for reserveHistoryForPreloading() method complexity: it's executed from exchange thread. Memory overhead of storing described map on heap is insignificant. Its size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. P.S. Possibly, instead of storing map, we can keep earlistCheckpoint right in GridDhtLocalPartition. It may simplify implementation. > Calculate result of reserveHistoryForExchange in advance > > > Key: IGNITE-13052 > URL: https://issues.apache.org/jira/browse/IGNITE-13052 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Rakov >Priority: Major > Original Estimate: 80h > Remaining Estimate: 80h > > Method reserveHistoryForExchange() is called on every partition map exchange. > It's an expensive call: it requires iteration over the whole checkpoint > history with possible retrieve of GroupState from WAL (it's stored on heap > with SoftReference). On some deployments this operation can take several > minutes. > The idea of optimization is to calculate its result only on
[jira] [Commented] (IGNITE-10100) Add public Java API to call Ignite.NET services
[
https://issues.apache.org/jira/browse/IGNITE-10100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113263#comment-17113263
]
Ignite TC Bot commented on IGNITE-10100:
{panel:title=Branch: [pull/7751/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *- Run :: .NET*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5327305buildTypeId=IgniteTests24Java8_RunAllNet]
> Add public Java API to call Ignite.NET services
> ---
>
> Key: IGNITE-10100
> URL: https://issues.apache.org/jira/browse/IGNITE-10100
> Project: Ignite
> Issue Type: Improvement
> Components: platforms
>Affects Versions: 2.6
>Reporter: Alexey Kukushkin
>Assignee: Ivan Daschinskiy
>Priority: Major
> Labels: .NET, sbcf
> Fix For: 2.9
>
> Attachments: ignite-10100-vs-2.8.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Ignite wraps .NET services in PlatformDotNetServiceImpl implementing
> PlatformService interface.
> PlatformService is defined in internal Ignite package
> apache.ignite.internal.processors.platform.services. It exposes
> {{ invokeMethod(methodName, Object[] params): Object}}
> to call any service method dynamically. Right now there is no Ignite public
> API to call a PlatformService using static typing.
> We need to develop a public API to call PlatformDotNetServiceImpl using
> static typing in Java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12823) .NET: Service method with user type array parameter can't be found
[
https://issues.apache.org/jira/browse/IGNITE-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113256#comment-17113256
]
Nikolai Kulagin commented on IGNITE-12823:
--
PR: https://github.com/apache/ignite/pull/7788
> .NET: Service method with user type array parameter can't be found
> --
>
> Key: IGNITE-12823
> URL: https://issues.apache.org/jira/browse/IGNITE-12823
> Project: Ignite
> Issue Type: Bug
> Components: platforms
>Reporter: Alexey Kukushkin
>Assignee: Nikolai Kulagin
>Priority: Major
> Labels: .NET, sbcf
> Attachments: ignite-12823-vs-2.8.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *+Setup+*
> * Ignite Java service with a method having an array of user types as a
> parameters, for example, caclulate(Parameter[] params)
> *+Actions+*
> * .NET client calls the Ignite Java service, for example,
> ignite.GetServices().GetServiceProxy().calculate(new[] \{new
> Parameter()});
> *+Expected+*
> * The service method is called
> *+Actual+*
> * Exception "Could not find proxy method 'calculate' in class ICalculator"
> *+Workaround+*
> * Replace array of user types with array of objects in the service methods
> signatures, for example, caclulate(Object[] params)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
[ https://issues.apache.org/jira/browse/IGNITE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-13052: Description: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changes its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should also extract WAL pointer reservation and filtering small partitions from reserveHistoryForExchange(), but this shouldn't be a problem. Another point: possibly, instead of storing map, we can keep earlistCheckpoint right in GridDhtLocalPartition. It may simplify implementation. Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. was: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changed its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should also extract WAL pointer reservation and filtering small partitions from reserveHistoryForExchange(), but this shouldn't be a problem. Another point: possibly, instead of storing map, we can keep earlistCheckpoint right in GridDhtLocalPartition. It may simplify implementation. Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. > Calculate result of reserveHistoryForExchange in advance > > > Key: IGNITE-13052 > URL: https://issues.apache.org/jira/browse/IGNITE-13052 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Rakov >Priority: Major > Original Estimate: 80h > Remaining Estimate: 80h > > Method reserveHistoryForExchange() is called on every partition map exchange. > It's an expensive call: it requires iteration over the whole checkpoint > history with possible retrieve of GroupState from WAL (it's stored on heap > with SoftReference). On some deployments this operation can take several > minutes. > The idea of optimization is to calculate its result only on first PME > (ideally, even before first PME, on recovery stage), keep resulting map > (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. > From the first glance, the map should be updated: > 1) On checkpoint. If a new partition appears on local node, it should be > registered in the map with current checkpoint. If a partition is evicted from > local node, or changes its state to non-OWNING, it should be removed from the > map. If checkpoint is marked as inapplicable for a certain group, the whole > group should be removed from the map. > 2) On checkpoint history cleanup. For every (grpId, partId), previous > earliest checkpoint should be changed with setIfGreater to new earliest > checkpoint. > We should also extract WAL pointer reservation and filtering small partitions >
[jira] [Updated] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
[ https://issues.apache.org/jira/browse/IGNITE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-13052: Description: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changed its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should also extract WAL pointer reservation and filtering small partitions from reserveHistoryForExchange(), but this shouldn't be a problem. Another point: possibly, instead of storing map, we can keep earlistCheckpoint right in GridDhtLocalPartition. It may simplify implementation. Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. was: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changed its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should extract WAL pointer reservation Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. > Calculate result of reserveHistoryForExchange in advance > > > Key: IGNITE-13052 > URL: https://issues.apache.org/jira/browse/IGNITE-13052 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Rakov >Priority: Major > Original Estimate: 80h > Remaining Estimate: 80h > > Method reserveHistoryForExchange() is called on every partition map exchange. > It's an expensive call: it requires iteration over the whole checkpoint > history with possible retrieve of GroupState from WAL (it's stored on heap > with SoftReference). On some deployments this operation can take several > minutes. > The idea of optimization is to calculate its result only on first PME > (ideally, even before first PME, on recovery stage), keep resulting map > (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. > From the first glance, the map should be updated: > 1) On checkpoint. If a new partition appears on local node, it should be > registered in the map with current checkpoint. If a partition is evicted from > local node, or changed its state to non-OWNING, it should be removed from the > map. If checkpoint is marked as inapplicable for a certain group, the whole > group should be removed from the map. > 2) On checkpoint history cleanup. For every (grpId, partId), previous > earliest checkpoint should be changed with setIfGreater to new earliest > checkpoint. > We should also extract WAL pointer reservation and filtering small partitions > from reserveHistoryForExchange(), but this shouldn't be a problem. > Another point: possibly, instead of storing map, we can keep > earlistCheckpoint right in GridDhtLocalPartition. It may simplify > implementation. > Memory overhead of storing
[jira] [Updated] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
[ https://issues.apache.org/jira/browse/IGNITE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-13052: Description: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changes its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should also extract WAL pointer reservation and filtering small partitions from reserveHistoryForExchange(), but this shouldn't be a problem. Another point: possibly, instead of storing map, we can keep earlistCheckpoint right in GridDhtLocalPartition. It may simplify implementation. Memory overhead of storing described map on heap is insignificant. Its size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. was: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changes its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should also extract WAL pointer reservation and filtering small partitions from reserveHistoryForExchange(), but this shouldn't be a problem. Another point: possibly, instead of storing map, we can keep earlistCheckpoint right in GridDhtLocalPartition. It may simplify implementation. Memory overhead of storing described map on heap in significant. Its size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. > Calculate result of reserveHistoryForExchange in advance > > > Key: IGNITE-13052 > URL: https://issues.apache.org/jira/browse/IGNITE-13052 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Rakov >Priority: Major > Original Estimate: 80h > Remaining Estimate: 80h > > Method reserveHistoryForExchange() is called on every partition map exchange. > It's an expensive call: it requires iteration over the whole checkpoint > history with possible retrieve of GroupState from WAL (it's stored on heap > with SoftReference). On some deployments this operation can take several > minutes. > The idea of optimization is to calculate its result only on first PME > (ideally, even before first PME, on recovery stage), keep resulting map > (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. > From the first glance, the map should be updated: > 1) On checkpoint. If a new partition appears on local node, it should be > registered in the map with current checkpoint. If a partition is evicted from > local node, or changes its state to non-OWNING, it should be removed from the > map. If checkpoint is marked as inapplicable for a certain group, the whole > group should be removed from the map. > 2) On checkpoint history cleanup. For every (grpId, partId), previous > earliest checkpoint should be changed with setIfGreater to new earliest > checkpoint. > We should also extract WAL pointer reservation and filtering small partitions >
[jira] [Updated] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
[ https://issues.apache.org/jira/browse/IGNITE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-13052: Description: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changes its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should also extract WAL pointer reservation and filtering small partitions from reserveHistoryForExchange(), but this shouldn't be a problem. Another point: possibly, instead of storing map, we can keep earlistCheckpoint right in GridDhtLocalPartition. It may simplify implementation. Memory overhead of storing described map on heap in significant. Its size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. was: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changes its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should also extract WAL pointer reservation and filtering small partitions from reserveHistoryForExchange(), but this shouldn't be a problem. Another point: possibly, instead of storing map, we can keep earlistCheckpoint right in GridDhtLocalPartition. It may simplify implementation. Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. > Calculate result of reserveHistoryForExchange in advance > > > Key: IGNITE-13052 > URL: https://issues.apache.org/jira/browse/IGNITE-13052 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Rakov >Priority: Major > Original Estimate: 80h > Remaining Estimate: 80h > > Method reserveHistoryForExchange() is called on every partition map exchange. > It's an expensive call: it requires iteration over the whole checkpoint > history with possible retrieve of GroupState from WAL (it's stored on heap > with SoftReference). On some deployments this operation can take several > minutes. > The idea of optimization is to calculate its result only on first PME > (ideally, even before first PME, on recovery stage), keep resulting map > (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. > From the first glance, the map should be updated: > 1) On checkpoint. If a new partition appears on local node, it should be > registered in the map with current checkpoint. If a partition is evicted from > local node, or changes its state to non-OWNING, it should be removed from the > map. If checkpoint is marked as inapplicable for a certain group, the whole > group should be removed from the map. > 2) On checkpoint history cleanup. For every (grpId, partId), previous > earliest checkpoint should be changed with setIfGreater to new earliest > checkpoint. > We should also extract WAL pointer reservation and filtering small partitions >
[jira] [Updated] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
[ https://issues.apache.org/jira/browse/IGNITE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-13052: Description: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changed its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. We should extract WAL pointer reservation Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. was: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changed its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. > Calculate result of reserveHistoryForExchange in advance > > > Key: IGNITE-13052 > URL: https://issues.apache.org/jira/browse/IGNITE-13052 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Rakov >Priority: Major > Original Estimate: 80h > Remaining Estimate: 80h > > Method reserveHistoryForExchange() is called on every partition map exchange. > It's an expensive call: it requires iteration over the whole checkpoint > history with possible retrieve of GroupState from WAL (it's stored on heap > with SoftReference). On some deployments this operation can take several > minutes. > The idea of optimization is to calculate its result only on first PME > (ideally, even before first PME, on recovery stage), keep resulting map > (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. > From the first glance, the map should be updated: > 1) On checkpoint. If a new partition appears on local node, it should be > registered in the map with current checkpoint. If a partition is evicted from > local node, or changed its state to non-OWNING, it should be removed from the > map. If checkpoint is marked as inapplicable for a certain group, the whole > group should be removed from the map. > 2) On checkpoint history cleanup. For every (grpId, partId), previous > earliest checkpoint should be changed with setIfGreater to new earliest > checkpoint. > We should extract WAL pointer reservation > Memory overhead of storing described map on heap in significant. It's size > isn't greater than size of map returned from reserveHistoryForExchange(). > Described fix should be much simpler than IGNITE-12429. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
[ https://issues.apache.org/jira/browse/IGNITE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-13052: Description: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changed its state to non-OWNING, it should be removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. was: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changed its state to non-OWNING, it should removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. > Calculate result of reserveHistoryForExchange in advance > > > Key: IGNITE-13052 > URL: https://issues.apache.org/jira/browse/IGNITE-13052 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Rakov >Priority: Major > Original Estimate: 80h > Remaining Estimate: 80h > > Method reserveHistoryForExchange() is called on every partition map exchange. > It's an expensive call: it requires iteration over the whole checkpoint > history with possible retrieve of GroupState from WAL (it's stored on heap > with SoftReference). On some deployments this operation can take several > minutes. > The idea of optimization is to calculate its result only on first PME > (ideally, even before first PME, on recovery stage), keep resulting map > (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. > From the first glance, the map should be updated: > 1) On checkpoint. If a new partition appears on local node, it should be > registered in the map with current checkpoint. If a partition is evicted from > local node, or changed its state to non-OWNING, it should be removed from the > map. If checkpoint is marked as inapplicable for a certain group, the whole > group should be removed from the map. > 2) On checkpoint history cleanup. For every (grpId, partId), previous > earliest checkpoint should be changed with setIfGreater to new earliest > checkpoint. > Memory overhead of storing described map on heap in significant. It's size > isn't greater than size of map returned from reserveHistoryForExchange(). > Described fix should be much simpler than IGNITE-12429. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
[
https://issues.apache.org/jira/browse/IGNITE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Rakov updated IGNITE-13052:
Description:
Method reserveHistoryForExchange() is called on every partition map exchange.
It's an expensive call: it requires iteration over the whole checkpoint history
with possible retrieve of GroupState from WAL (it's stored on heap with
SoftReference). On some deployments this operation can take several minutes.
The idea of optimization is to calculate it's result only on first PME
(ideally, even before first PME, on recovery stage), keep resulting map (grpId,
partId -> earlisetCheckpoint) on heap and update it if necessary. From the
first glance, map should be updated:
1) On checkpoint. If a new partition appears on local node, it should be
registered in the map with current checkpoint. If a partition is evicted from
local node, or changed its state to non-OWNING, it should removed from the map.
If checkpoint is marked as inapplicable for a certain group, the whole group
should be removed from the map.
2) On checkpoint history cleanup. For every (grpId, partId), previous earliest
checkpoint should be changed with setIfGreater to new earliest checkpoint.
Memory overhead of storing described map on heap in significant. It's size
isn't greater than size of map returned from reserveHistoryForExchange().
Described fix should be much simpler than IGNITE-12429.
was:
Method reserveHistoryForExchange() is called on every partition map exchange.
It's an expensive call: it requires iteration over the whole checkpoint history
with possible retrieve of GroupState from WAL (it's stored on heap with
SoftReference). On some deployments this operation can take several minutes.
The idea of optimization is to calculate it's result only on first PME
(ideally, even before first PME, on recovery stage), keep resulting map {grpId,
partId -> earlisetCheckpoint} on heap and update it if necessary. From the
first glance, map should be updated:
1) On checkpoint. If a new partition appears on local node, it should be
registered in the map with current checkpoint. If a partition is evicted from
local node, or changed its state to non-OWNING, it should removed from the map.
If checkpoint is marked as inapplicable for a certain group, the whole group
should be removed from the map.
2) On checkpoint history cleanup. For every (grpId, partId), previous earliest
checkpoint should be changed with setIfGreater to new earliest checkpoint.
Memory overhead of storing described map on heap in significant. It's size
isn't greater than size of map returned from reserveHistoryForExchange().
Described fix should be much simpler than IGNITE-12429.
> Calculate result of reserveHistoryForExchange in advance
>
>
> Key: IGNITE-13052
> URL: https://issues.apache.org/jira/browse/IGNITE-13052
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Rakov
>Priority: Major
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Method reserveHistoryForExchange() is called on every partition map exchange.
> It's an expensive call: it requires iteration over the whole checkpoint
> history with possible retrieve of GroupState from WAL (it's stored on heap
> with SoftReference). On some deployments this operation can take several
> minutes.
> The idea of optimization is to calculate it's result only on first PME
> (ideally, even before first PME, on recovery stage), keep resulting map
> (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary.
> From the first glance, map should be updated:
> 1) On checkpoint. If a new partition appears on local node, it should be
> registered in the map with current checkpoint. If a partition is evicted from
> local node, or changed its state to non-OWNING, it should removed from the
> map. If checkpoint is marked as inapplicable for a certain group, the whole
> group should be removed from the map.
> 2) On checkpoint history cleanup. For every (grpId, partId), previous
> earliest checkpoint should be changed with setIfGreater to new earliest
> checkpoint.
> Memory overhead of storing described map on heap in significant. It's size
> isn't greater than size of map returned from reserveHistoryForExchange().
> Described fix should be much simpler than IGNITE-12429.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
[
https://issues.apache.org/jira/browse/IGNITE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Rakov updated IGNITE-13052:
Remaining Estimate: 80h (was: 240h)
Original Estimate: 80h (was: 240h)
> Calculate result of reserveHistoryForExchange in advance
>
>
> Key: IGNITE-13052
> URL: https://issues.apache.org/jira/browse/IGNITE-13052
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Rakov
>Priority: Major
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Method reserveHistoryForExchange() is called on every partition map exchange.
> It's an expensive call: it requires iteration over the whole checkpoint
> history with possible retrieve of GroupState from WAL (it's stored on heap
> with SoftReference). On some deployments this operation can take several
> minutes.
> The idea of optimization is to calculate it's result only on first PME
> (ideally, even before first PME, on recovery stage), keep resulting map
> {grpId, partId -> earlisetCheckpoint} on heap and update it if necessary.
> From the first glance, map should be updated:
> 1) On checkpoint. If a new partition appears on local node, it should be
> registered in the map with current checkpoint. If a partition is evicted from
> local node, or changed its state to non-OWNING, it should removed from the
> map. If checkpoint is marked as inapplicable for a certain group, the whole
> group should be removed from the map.
> 2) On checkpoint history cleanup. For every (grpId, partId), previous
> earliest checkpoint should be changed with setIfGreater to new earliest
> checkpoint.
> Memory overhead of storing described map on heap in significant. It's size
> isn't greater than size of map returned from reserveHistoryForExchange().
> Described fix should be much simpler than IGNITE-12429.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
[ https://issues.apache.org/jira/browse/IGNITE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-13052: Description: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, the map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changed its state to non-OWNING, it should removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. was: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changed its state to non-OWNING, it should removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. > Calculate result of reserveHistoryForExchange in advance > > > Key: IGNITE-13052 > URL: https://issues.apache.org/jira/browse/IGNITE-13052 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Rakov >Priority: Major > Original Estimate: 80h > Remaining Estimate: 80h > > Method reserveHistoryForExchange() is called on every partition map exchange. > It's an expensive call: it requires iteration over the whole checkpoint > history with possible retrieve of GroupState from WAL (it's stored on heap > with SoftReference). On some deployments this operation can take several > minutes. > The idea of optimization is to calculate its result only on first PME > (ideally, even before first PME, on recovery stage), keep resulting map > (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. > From the first glance, the map should be updated: > 1) On checkpoint. If a new partition appears on local node, it should be > registered in the map with current checkpoint. If a partition is evicted from > local node, or changed its state to non-OWNING, it should removed from the > map. If checkpoint is marked as inapplicable for a certain group, the whole > group should be removed from the map. > 2) On checkpoint history cleanup. For every (grpId, partId), previous > earliest checkpoint should be changed with setIfGreater to new earliest > checkpoint. > Memory overhead of storing described map on heap in significant. It's size > isn't greater than size of map returned from reserveHistoryForExchange(). > Described fix should be much simpler than IGNITE-12429. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
[ https://issues.apache.org/jira/browse/IGNITE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-13052: Description: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate its result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changed its state to non-OWNING, it should removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. was: Method reserveHistoryForExchange() is called on every partition map exchange. It's an expensive call: it requires iteration over the whole checkpoint history with possible retrieve of GroupState from WAL (it's stored on heap with SoftReference). On some deployments this operation can take several minutes. The idea of optimization is to calculate it's result only on first PME (ideally, even before first PME, on recovery stage), keep resulting map (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. From the first glance, map should be updated: 1) On checkpoint. If a new partition appears on local node, it should be registered in the map with current checkpoint. If a partition is evicted from local node, or changed its state to non-OWNING, it should removed from the map. If checkpoint is marked as inapplicable for a certain group, the whole group should be removed from the map. 2) On checkpoint history cleanup. For every (grpId, partId), previous earliest checkpoint should be changed with setIfGreater to new earliest checkpoint. Memory overhead of storing described map on heap in significant. It's size isn't greater than size of map returned from reserveHistoryForExchange(). Described fix should be much simpler than IGNITE-12429. > Calculate result of reserveHistoryForExchange in advance > > > Key: IGNITE-13052 > URL: https://issues.apache.org/jira/browse/IGNITE-13052 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Rakov >Priority: Major > Original Estimate: 80h > Remaining Estimate: 80h > > Method reserveHistoryForExchange() is called on every partition map exchange. > It's an expensive call: it requires iteration over the whole checkpoint > history with possible retrieve of GroupState from WAL (it's stored on heap > with SoftReference). On some deployments this operation can take several > minutes. > The idea of optimization is to calculate its result only on first PME > (ideally, even before first PME, on recovery stage), keep resulting map > (grpId, partId -> earlisetCheckpoint) on heap and update it if necessary. > From the first glance, map should be updated: > 1) On checkpoint. If a new partition appears on local node, it should be > registered in the map with current checkpoint. If a partition is evicted from > local node, or changed its state to non-OWNING, it should removed from the > map. If checkpoint is marked as inapplicable for a certain group, the whole > group should be removed from the map. > 2) On checkpoint history cleanup. For every (grpId, partId), previous > earliest checkpoint should be changed with setIfGreater to new earliest > checkpoint. > Memory overhead of storing described map on heap in significant. It's size > isn't greater than size of map returned from reserveHistoryForExchange(). > Described fix should be much simpler than IGNITE-12429. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13051) Optimized affinity switch on baseline node left is broken for client topology and MVCC
[ https://issues.apache.org/jira/browse/IGNITE-13051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-13051: --- Description: If a node contains only client cache topology with MVCC enabled, PME will hang after changes introduced in IGNITE-12617. Reproduced by CachePartitionLossWithRestartsTest.testPartitionLossDetectionOnClientTopology[0 false false -1] and enabled MVCC. was: If a node contains only client cache topology with MVCC enabled, PME will hang. Reproduced by CachePartitionLossWithRestartsTest.testPartitionLossDetectionOnClientTopology[0 false false -1] and enabled MVCC. > Optimized affinity switch on baseline node left is broken for client topology > and MVCC > -- > > Key: IGNITE-13051 > URL: https://issues.apache.org/jira/browse/IGNITE-13051 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Scherbakov >Priority: Major > Fix For: 2.9 > > > If a node contains only client cache topology with MVCC enabled, PME will > hang after changes introduced in IGNITE-12617. > Reproduced by > CachePartitionLossWithRestartsTest.testPartitionLossDetectionOnClientTopology[0 > false false -1] and enabled MVCC. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13051) Optimized affinity switch on baseline node left is broken for client topology and MVCC
[ https://issues.apache.org/jira/browse/IGNITE-13051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-13051: --- Priority: Critical (was: Major) > Optimized affinity switch on baseline node left is broken for client topology > and MVCC > -- > > Key: IGNITE-13051 > URL: https://issues.apache.org/jira/browse/IGNITE-13051 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Scherbakov >Priority: Critical > Fix For: 2.9 > > > If a node contains only client cache topology with MVCC enabled, PME will > hang after changes introduced in IGNITE-12617. > Reproduced by > CachePartitionLossWithRestartsTest.testPartitionLossDetectionOnClientTopology[0 > false false -1] and enabled MVCC. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13052) Calculate result of reserveHistoryForExchange in advance
Ivan Rakov created IGNITE-13052:
---
Summary: Calculate result of reserveHistoryForExchange in advance
Key: IGNITE-13052
URL: https://issues.apache.org/jira/browse/IGNITE-13052
Project: Ignite
Issue Type: Improvement
Reporter: Ivan Rakov
Method reserveHistoryForExchange() is called on every partition map exchange.
It's an expensive call: it requires iteration over the whole checkpoint history
with possible retrieve of GroupState from WAL (it's stored on heap with
SoftReference). On some deployments this operation can take several minutes.
The idea of optimization is to calculate it's result only on first PME
(ideally, even before first PME, on recovery stage), keep resulting map {grpId,
partId -> earlisetCheckpoint} on heap and update it if necessary. From the
first glance, map should be updated:
1) On checkpoint. If a new partition appears on local node, it should be
registered in the map with current checkpoint. If a partition is evicted from
local node, or changed its state to non-OWNING, it should removed from the map.
If checkpoint is marked as inapplicable for a certain group, the whole group
should be removed from the map.
2) On checkpoint history cleanup. For every (grpId, partId), previous earliest
checkpoint should be changed with setIfGreater to new earliest checkpoint.
Memory overhead of storing described map on heap in significant. It's size
isn't greater than size of map returned from reserveHistoryForExchange().
Described fix should be much simpler than IGNITE-12429.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13051) Optimized affinity switch on baseline node left is broken for client topology and MVCC
[ https://issues.apache.org/jira/browse/IGNITE-13051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-13051: --- Description: If a node contains only client cache topology with MVCC enabled, PME will hang. Reproduced by CachePartitionLossWithRestartsTest.testPartitionLossDetectionOnClientTopology[0 false false -1] and enabled MVCC. Summary: Optimized affinity switch on baseline node left is broken for client topology and MVCC (was: Optimized affinity switch on baseline node left is broken for client topology) > Optimized affinity switch on baseline node left is broken for client topology > and MVCC > -- > > Key: IGNITE-13051 > URL: https://issues.apache.org/jira/browse/IGNITE-13051 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Scherbakov >Priority: Major > Fix For: 2.9 > > > If a node contains only client cache topology with MVCC enabled, PME will > hang. > Reproduced by > CachePartitionLossWithRestartsTest.testPartitionLossDetectionOnClientTopology[0 > false false -1] and enabled MVCC. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13051) Optimized affinity switch on baseline node left is broken for client topology
Alexey Scherbakov created IGNITE-13051: -- Summary: Optimized affinity switch on baseline node left is broken for client topology Key: IGNITE-13051 URL: https://issues.apache.org/jira/browse/IGNITE-13051 Project: Ignite Issue Type: Bug Reporter: Alexey Scherbakov Fix For: 2.9 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-10959) Memory leaks in continuous query handlers
[ https://issues.apache.org/jira/browse/IGNITE-10959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113213#comment-17113213 ] Cong Guo commented on IGNITE-10959: --- Hi, Is this issue still open? Is there any patch to fix this issue? I can reproduce this issue by restarting one node in a cluster. # Start a 3-node Ignite cluster, and each node has a continuous query to watch updates in cache. # Send updates to three nodes continuously. # Restart one Ignite node, and then memory leak can be seen on two other nodes. Based on the heap dump, the memory issue happens in entryBufs of CacheContinuousQueryHandler. I already close the continuous query when I stop the Ignite node. However, the pending buffer on two other nodes still increases when the node starts again. Is there any method to discard the pending buffer when one node (the master node of a continuous query) leaves the cluster? !ObjectRef.PNG! > Memory leaks in continuous query handlers > - > > Key: IGNITE-10959 > URL: https://issues.apache.org/jira/browse/IGNITE-10959 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.7 >Reporter: Denis Mekhanikov >Priority: Major > Fix For: 2.9 > > Attachments: CacheContinuousQueryMemoryUsageTest.java, > CacheContinuousQueryMemoryUsageTest.result, > CacheContinuousQueryMemoryUsageTest2.java, continuousquery_leak_profile.png > > > Continuous query handlers don't clear internal data structures after cache > events are processed. > A test, that reproduces the problem, is attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13034) Append additional yardstick test for page replacements test purpose.
[ https://issues.apache.org/jira/browse/IGNITE-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113138#comment-17113138 ] Stanilovsky Evgeny commented on IGNITE-13034: - [~mmuzaf] can u help me with review and merge ? > Append additional yardstick test for page replacements test purpose. > > > Key: IGNITE-13034 > URL: https://issues.apache.org/jira/browse/IGNITE-13034 > Project: Ignite > Issue Type: Improvement > Components: yardstick >Affects Versions: 2.8 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > There is a need in additional yardstick benchmarks, which will measure > throughput with page replacement already in action. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13034) Append additional yardstick test for page replacements test purpose.
[
https://issues.apache.org/jira/browse/IGNITE-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113120#comment-17113120
]
Ignite TC Bot commented on IGNITE-13034:
{panel:title=Branch: [pull/7828/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *~[Excluded] - Run :: IntelliJ IDEA Inspections*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5326904buildTypeId=IgniteTests24Java8_RunIntelliJIdeaInspections]
> Append additional yardstick test for page replacements test purpose.
>
>
> Key: IGNITE-13034
> URL: https://issues.apache.org/jira/browse/IGNITE-13034
> Project: Ignite
> Issue Type: Improvement
> Components: yardstick
>Affects Versions: 2.8
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There is a need in additional yardstick benchmarks, which will measure
> throughput with page replacement already in action.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13050) ClusterGroup that is recomputed on topology change
Ivan Bessonov created IGNITE-13050: -- Summary: ClusterGroup that is recomputed on topology change Key: IGNITE-13050 URL: https://issues.apache.org/jira/browse/IGNITE-13050 Project: Ignite Issue Type: Improvement Reporter: Ivan Bessonov Assignee: Ivan Bessonov Currently, ClusterGroup comes in two favors: One is a static set of UUIDs which will not change, second is predicate that is recomputed over ALL nodes on EVERY operation. This has bitten our client because recomputing of ClusterGroup happens in tcp-communication thread clogging it and delaying every operation in cluster. This is a major problem. It would be nice if there was a ClusterGroup with predicate which would recompute once per topology affinity change. Bonus points if it precisely tracks current topology with zero delay or overrun. Would be nice to upgrade firstNode/lastNode predicates to that mechanism since now they are static - topology changes but firstNode/lastNode projections don't, they may point to absent node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13048) WAL FSYNC mode doesn't work with disabled archiver
[
https://issues.apache.org/jira/browse/IGNITE-13048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov updated IGNITE-13048:
---
Description:
Assumption that work directory always contains configured number of segments is
inapplicable to "archiveless" WAL.
{noformat}
Caused by:
org.apache.ignite.internal.processors.cache.persistence.StorageException:
Failed to initialize WAL log segment (WAL segment size change is not supported
in 'DEFAULT' WAL mode)
[filePath=/home/vsisko/gridgain/backend-work/work/db/wal/web_console_data/0001.wal,
fileSize=24313258, configSize=10]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.checkFiles(FileWriteAheadLogManager.java:2427)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.checkOrPrepareFiles(FileWriteAheadLogManager.java:1404)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.start0(FileWriteAheadLogManager.java:435)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.start(GridCacheSharedManagerAdapter.java:60)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.start(GridCacheProcessor.java:841)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1717)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1020)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2039)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1731)
~[ignite-core-8.7.5.jar:8.7.5]
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1157)
~[ignite-core-8.7.5.jar:8.7.5]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:677)
~[ignite-core-8.7.5.jar:8.7.5]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:602)
~[ignite-core-8.7.5.jar:8.7.5]
at org.apache.ignite.Ignition.start(Ignition.java:322)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.console.config.GridConfiguration.igniteInstance(GridConfiguration.java:38)
~[classes/:?]
at
org.apache.ignite.console.config.GridConfiguration$$EnhancerBySpringCGLIB$$b50da981.CGLIB$igniteInstance$0()
~[classes/:?]
at
org.apache.ignite.console.config.GridConfiguration$$EnhancerBySpringCGLIB$$b50da981$$FastClassBySpringCGLIB$$d486ae88.invoke()
~[classes/:?]
at
org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
~[spring-core-4.3.23.RELEASE.jar:4.3.23.RELEASE]
at
org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358)
~[spring-context-4.3.23.RELEASE.jar:4.3.23.RELEASE]
at
org.apache.ignite.console.config.GridConfiguration$$EnhancerBySpringCGLIB$$b50da981.igniteInstance()
~[classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_222]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_222]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_222]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_222]
at
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
~[spring-beans-4.3.23.RELEASE.jar:4.3.23.RELEASE]
at
org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588)
~[spring-beans-4.3.23.RELEASE.jar:4.3.23.RELEASE]
... 83 more{noformat}
was:
{noformat}
Caused by:
org.apache.ignite.internal.processors.cache.persistence.StorageException:
Failed to initialize WAL log segment (WAL segment size change is not supported
in 'DEFAULT' WAL mode)
[filePath=/home/vsisko/gridgain/backend-work/work/db/wal/web_console_data/0001.wal,
fileSize=24313258, configSize=10]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.checkFiles(FileWriteAheadLogManager.java:2427)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.checkOrPrepareFiles(FileWriteAheadLogManager.java:1404)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.start0(FileWriteAheadLogManager.java:435)
~[ignite-core-8.7.5.jar:8.7.5]
at
[jira] [Updated] (IGNITE-13049) AssertionError with JmxExporterSpi on cache stop. Test testCacheConfigSingleLineOutputFormatTwoNodeManyCaches
[
https://issues.apache.org/jira/browse/IGNITE-13049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-13049:
-
Issue Type: Bug (was: Test)
> AssertionError with JmxExporterSpi on cache stop. Test
> testCacheConfigSingleLineOutputFormatTwoNodeManyCaches
> -
>
> Key: IGNITE-13049
> URL: https://issues.apache.org/jira/browse/IGNITE-13049
> Project: Ignite
> Issue Type: Bug
>Reporter: Maxim Muzafarov
>Priority: Major
> Attachments: Ignite_Tests_2.4_Java_8_9_10_11_Basic_3_19337.log
>
>
> The test throws AssertionError if {{JmxExporterSpi}} configured
> {{GridCommandHandlerClusterByClassWithSSLTest#testCacheConfigSingleLineOutputFormatTwoNodeManyCaches}}.
> Reproduced locally time to time.
> 1. Configure JmxExporterSpi for default {{getConfiguration}} method (see
> {{org.apache.ignite.util.GridCommandHandlerAbstractTest#getConfiguration}})
> 2. Run with SSL testCacheConfigSingleLineOutputFormatTwoNodeManyCaches
> {code}
> [12:17:28]W: [org.apache.ignite:ignite-core] [2020-05-21
> 12:17:28,500][ERROR][exchange-worker-#16994%gridCommandHandlerTest0%][IgniteTestResources]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.AssertionError]]
> [12:17:28]W: [org.apache.ignite:ignite-core]
> java.lang.AssertionError
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.spi.metric.jmx.JmxMetricExporterSpi.unregister(JmxMetricExporterSpi.java:110)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.util.IgniteUtils.notifyListeners(IgniteUtils.java:11694)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.metric.GridMetricManager.remove(GridMetricManager.java:393)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.CacheGroupContext.removeIOStatistic(CacheGroupContext.java:1321)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.cleanup(GridCacheProcessor.java:513)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCacheGroup(GridCacheProcessor.java:2900)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCacheGroup(GridCacheProcessor.java:2888)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.processCacheStopRequestOnExchangeDone(GridCacheProcessor.java:2780)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.onExchangeDone(GridCacheProcessor.java:2877)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:2404)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.finishExchangeOnCoordinator(GridDhtPartitionsExchangeFuture.java:3822)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onAllReceived(GridDhtPartitionsExchangeFuture.java:3590)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processSingleMessage(GridDhtPartitionsExchangeFuture.java:3180)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$200(GridDhtPartitionsExchangeFuture.java:155)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2967)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2955)
>
[jira] [Updated] (IGNITE-13049) AssertionError with JmxExporterSpi on cache stop. Test testCacheConfigSingleLineOutputFormatTwoNodeManyCaches
[
https://issues.apache.org/jira/browse/IGNITE-13049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-13049:
-
Attachment: Ignite_Tests_2.4_Java_8_9_10_11_Basic_3_19337.log
> AssertionError with JmxExporterSpi on cache stop. Test
> testCacheConfigSingleLineOutputFormatTwoNodeManyCaches
> -
>
> Key: IGNITE-13049
> URL: https://issues.apache.org/jira/browse/IGNITE-13049
> Project: Ignite
> Issue Type: Test
>Reporter: Maxim Muzafarov
>Priority: Major
> Attachments: Ignite_Tests_2.4_Java_8_9_10_11_Basic_3_19337.log
>
>
> The test throws AssertionError if {{JmxExporterSpi}} configured
> {{GridCommandHandlerClusterByClassWithSSLTest#testCacheConfigSingleLineOutputFormatTwoNodeManyCaches}}.
> Reproduced locally time to time.
> 1. Configure JmxExporterSpi for default {{getConfiguration}} method (see
> {{org.apache.ignite.util.GridCommandHandlerAbstractTest#getConfiguration}})
> 2. Run with SSL testCacheConfigSingleLineOutputFormatTwoNodeManyCaches
> {code}
> [12:17:28]W: [org.apache.ignite:ignite-core] [2020-05-21
> 12:17:28,500][ERROR][exchange-worker-#16994%gridCommandHandlerTest0%][IgniteTestResources]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.AssertionError]]
> [12:17:28]W: [org.apache.ignite:ignite-core]
> java.lang.AssertionError
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.spi.metric.jmx.JmxMetricExporterSpi.unregister(JmxMetricExporterSpi.java:110)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.util.IgniteUtils.notifyListeners(IgniteUtils.java:11694)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.metric.GridMetricManager.remove(GridMetricManager.java:393)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.CacheGroupContext.removeIOStatistic(CacheGroupContext.java:1321)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.cleanup(GridCacheProcessor.java:513)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCacheGroup(GridCacheProcessor.java:2900)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCacheGroup(GridCacheProcessor.java:2888)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.processCacheStopRequestOnExchangeDone(GridCacheProcessor.java:2780)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.onExchangeDone(GridCacheProcessor.java:2877)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:2404)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.finishExchangeOnCoordinator(GridDhtPartitionsExchangeFuture.java:3822)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onAllReceived(GridDhtPartitionsExchangeFuture.java:3590)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processSingleMessage(GridDhtPartitionsExchangeFuture.java:3180)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$200(GridDhtPartitionsExchangeFuture.java:155)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2967)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
>
[jira] [Created] (IGNITE-13049) AssertionError with JmxExporterSpi on cache stop. Test testCacheConfigSingleLineOutputFormatTwoNodeManyCaches
Maxim Muzafarov created IGNITE-13049:
Summary: AssertionError with JmxExporterSpi on cache stop. Test
testCacheConfigSingleLineOutputFormatTwoNodeManyCaches
Key: IGNITE-13049
URL: https://issues.apache.org/jira/browse/IGNITE-13049
Project: Ignite
Issue Type: Test
Reporter: Maxim Muzafarov
The test throws AssertionError if {{JmxExporterSpi}} configured
{{GridCommandHandlerClusterByClassWithSSLTest#testCacheConfigSingleLineOutputFormatTwoNodeManyCaches}}.
Reproduced locally time to time.
1. Configure JmxExporterSpi for default {{getConfiguration}} method (see
{{org.apache.ignite.util.GridCommandHandlerAbstractTest#getConfiguration}})
2. Run with SSL testCacheConfigSingleLineOutputFormatTwoNodeManyCaches
{code}
[12:17:28]W: [org.apache.ignite:ignite-core] [2020-05-21
12:17:28,500][ERROR][exchange-worker-#16994%gridCommandHandlerTest0%][IgniteTestResources]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext
[type=SYSTEM_WORKER_TERMINATION, err=java.lang.AssertionError]]
[12:17:28]W: [org.apache.ignite:ignite-core]
java.lang.AssertionError
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.spi.metric.jmx.JmxMetricExporterSpi.unregister(JmxMetricExporterSpi.java:110)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.util.IgniteUtils.notifyListeners(IgniteUtils.java:11694)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.metric.GridMetricManager.remove(GridMetricManager.java:393)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.CacheGroupContext.removeIOStatistic(CacheGroupContext.java:1321)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.cleanup(GridCacheProcessor.java:513)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCacheGroup(GridCacheProcessor.java:2900)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCacheGroup(GridCacheProcessor.java:2888)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.processCacheStopRequestOnExchangeDone(GridCacheProcessor.java:2780)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.onExchangeDone(GridCacheProcessor.java:2877)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:2404)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.finishExchangeOnCoordinator(GridDhtPartitionsExchangeFuture.java:3822)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onAllReceived(GridDhtPartitionsExchangeFuture.java:3590)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processSingleMessage(GridDhtPartitionsExchangeFuture.java:3180)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$200(GridDhtPartitionsExchangeFuture.java:155)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2967)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2955)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:399)
[12:17:28]W: [org.apache.ignite:ignite-core]at
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:354)
[12:17:28]W: [org.apache.ignite:ignite-core]at
[jira] [Updated] (IGNITE-13049) AssertionError with JmxExporterSpi on cache stop. Test testCacheConfigSingleLineOutputFormatTwoNodeManyCaches
[
https://issues.apache.org/jira/browse/IGNITE-13049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-13049:
-
Ignite Flags: (was: Docs Required,Release Notes Required)
> AssertionError with JmxExporterSpi on cache stop. Test
> testCacheConfigSingleLineOutputFormatTwoNodeManyCaches
> -
>
> Key: IGNITE-13049
> URL: https://issues.apache.org/jira/browse/IGNITE-13049
> Project: Ignite
> Issue Type: Test
>Reporter: Maxim Muzafarov
>Priority: Major
>
> The test throws AssertionError if {{JmxExporterSpi}} configured
> {{GridCommandHandlerClusterByClassWithSSLTest#testCacheConfigSingleLineOutputFormatTwoNodeManyCaches}}.
> Reproduced locally time to time.
> 1. Configure JmxExporterSpi for default {{getConfiguration}} method (see
> {{org.apache.ignite.util.GridCommandHandlerAbstractTest#getConfiguration}})
> 2. Run with SSL testCacheConfigSingleLineOutputFormatTwoNodeManyCaches
> {code}
> [12:17:28]W: [org.apache.ignite:ignite-core] [2020-05-21
> 12:17:28,500][ERROR][exchange-worker-#16994%gridCommandHandlerTest0%][IgniteTestResources]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.AssertionError]]
> [12:17:28]W: [org.apache.ignite:ignite-core]
> java.lang.AssertionError
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.spi.metric.jmx.JmxMetricExporterSpi.unregister(JmxMetricExporterSpi.java:110)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.util.IgniteUtils.notifyListeners(IgniteUtils.java:11694)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.metric.GridMetricManager.remove(GridMetricManager.java:393)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.CacheGroupContext.removeIOStatistic(CacheGroupContext.java:1321)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.cleanup(GridCacheProcessor.java:513)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCacheGroup(GridCacheProcessor.java:2900)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCacheGroup(GridCacheProcessor.java:2888)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.processCacheStopRequestOnExchangeDone(GridCacheProcessor.java:2780)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.onExchangeDone(GridCacheProcessor.java:2877)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:2404)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.finishExchangeOnCoordinator(GridDhtPartitionsExchangeFuture.java:3822)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onAllReceived(GridDhtPartitionsExchangeFuture.java:3590)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processSingleMessage(GridDhtPartitionsExchangeFuture.java:3180)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$200(GridDhtPartitionsExchangeFuture.java:155)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2967)
> [12:17:28]W: [org.apache.ignite:ignite-core]at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2955)
> [12:17:28]W:
[jira] [Created] (IGNITE-13048) WAL FSYNC mode doesn't work with disabled archiver
Ivan Bessonov created IGNITE-13048:
--
Summary: WAL FSYNC mode doesn't work with disabled archiver
Key: IGNITE-13048
URL: https://issues.apache.org/jira/browse/IGNITE-13048
Project: Ignite
Issue Type: Bug
Reporter: Ivan Bessonov
Assignee: Ivan Bessonov
{noformat}
Caused by:
org.apache.ignite.internal.processors.cache.persistence.StorageException:
Failed to initialize WAL log segment (WAL segment size change is not supported
in 'DEFAULT' WAL mode)
[filePath=/home/vsisko/gridgain/backend-work/work/db/wal/web_console_data/0001.wal,
fileSize=24313258, configSize=10]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.checkFiles(FileWriteAheadLogManager.java:2427)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.checkOrPrepareFiles(FileWriteAheadLogManager.java:1404)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.start0(FileWriteAheadLogManager.java:435)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.start(GridCacheSharedManagerAdapter.java:60)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.start(GridCacheProcessor.java:841)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1717)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1020)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2039)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1731)
~[ignite-core-8.7.5.jar:8.7.5]
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1157)
~[ignite-core-8.7.5.jar:8.7.5]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:677)
~[ignite-core-8.7.5.jar:8.7.5]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:602)
~[ignite-core-8.7.5.jar:8.7.5]
at org.apache.ignite.Ignition.start(Ignition.java:322)
~[ignite-core-8.7.5.jar:8.7.5]
at
org.apache.ignite.console.config.GridConfiguration.igniteInstance(GridConfiguration.java:38)
~[classes/:?]
at
org.apache.ignite.console.config.GridConfiguration$$EnhancerBySpringCGLIB$$b50da981.CGLIB$igniteInstance$0()
~[classes/:?]
at
org.apache.ignite.console.config.GridConfiguration$$EnhancerBySpringCGLIB$$b50da981$$FastClassBySpringCGLIB$$d486ae88.invoke()
~[classes/:?]
at
org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
~[spring-core-4.3.23.RELEASE.jar:4.3.23.RELEASE]
at
org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358)
~[spring-context-4.3.23.RELEASE.jar:4.3.23.RELEASE]
at
org.apache.ignite.console.config.GridConfiguration$$EnhancerBySpringCGLIB$$b50da981.igniteInstance()
~[classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_222]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_222]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_222]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_222]
at
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
~[spring-beans-4.3.23.RELEASE.jar:4.3.23.RELEASE]
at
org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588)
~[spring-beans-4.3.23.RELEASE.jar:4.3.23.RELEASE]
... 83 more{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13039) Get rid of possibility to change final static fields through reflection for test purpose.
[ https://issues.apache.org/jira/browse/IGNITE-13039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113065#comment-17113065 ] Stanilovsky Evgeny commented on IGNITE-13039: - [~agoncharuk] visa ok, can u check plz ? > Get rid of possibility to change final static fields through reflection for > test purpose. > - > > Key: IGNITE-13039 > URL: https://issues.apache.org/jira/browse/IGNITE-13039 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.8 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Rare for tests purpose there is a need for changing some private fields > through reflection. > Changing *static final* fields in this manner could bring more tears, check > for example [1] and [2]. > [1] > https://stackoverflow.com/questions/3301635/change-private-static-final-field-using-java-reflection > [2] http://docs.oracle.com/javase/specs/jls/se7/html/jls-15.html#jls-15.28 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13039) Get rid of possibility to change final static fields through reflection for test purpose.
[
https://issues.apache.org/jira/browse/IGNITE-13039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113062#comment-17113062
]
Ignite TC Bot commented on IGNITE-13039:
{panel:title=Branch: [pull/7819/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5322391buildTypeId=IgniteTests24Java8_RunAll]
> Get rid of possibility to change final static fields through reflection for
> test purpose.
> -
>
> Key: IGNITE-13039
> URL: https://issues.apache.org/jira/browse/IGNITE-13039
> Project: Ignite
> Issue Type: Improvement
> Components: general
>Affects Versions: 2.8
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Rare for tests purpose there is a need for changing some private fields
> through reflection.
> Changing *static final* fields in this manner could bring more tears, check
> for example [1] and [2].
> [1]
> https://stackoverflow.com/questions/3301635/change-private-static-final-field-using-java-reflection
> [2] http://docs.oracle.com/javase/specs/jls/se7/html/jls-15.html#jls-15.28
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13043) Fix compilation error in Ignite C++, when boost version is greater than 1.70
[ https://issues.apache.org/jira/browse/IGNITE-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Sapego updated IGNITE-13043: - Fix Version/s: 2.9 > Fix compilation error in Ignite C++, when boost version is greater than 1.70 > - > > Key: IGNITE-13043 > URL: https://issues.apache.org/jira/browse/IGNITE-13043 > Project: Ignite > Issue Type: Bug > Components: platforms >Reporter: Ivan Daschinskiy >Assignee: Ivan Daschinskiy >Priority: Trivial > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > Fix compilation issue when libboost greater than 1.71 in > TeamcityBoostLogFormatter -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13043) Fix compilation error in Ignite C++, when boost version is greater than 1.70
[ https://issues.apache.org/jira/browse/IGNITE-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113023#comment-17113023 ] Igor Sapego commented on IGNITE-13043: -- [~ivandasch] Looks good. Merged to master. Please, close PR. > Fix compilation error in Ignite C++, when boost version is greater than 1.70 > - > > Key: IGNITE-13043 > URL: https://issues.apache.org/jira/browse/IGNITE-13043 > Project: Ignite > Issue Type: Bug > Components: platforms >Reporter: Ivan Daschinskiy >Assignee: Ivan Daschinskiy >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > > Fix compilation issue when libboost greater than 1.71 in > TeamcityBoostLogFormatter -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-10970) ATOMICITY table creation error message should include TRANSACTIONAL_SNAPSHOT
[
https://issues.apache.org/jira/browse/IGNITE-10970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112991#comment-17112991
]
Maksim Timonin commented on IGNITE-10970:
-
[~ilyak] hi! Please check my [PR|https://github.com/apache/ignite/pull/7822]
[MTCGA
report|https://mtcga.gridgain.com/pr.html?serverId=apache=IgniteTests24Java8_RunAll=pull/7822/head=Latest]
> ATOMICITY table creation error message should include TRANSACTIONAL_SNAPSHOT
>
>
> Key: IGNITE-10970
> URL: https://issues.apache.org/jira/browse/IGNITE-10970
> Project: Ignite
> Issue Type: Task
> Components: mvcc, sql
>Affects Versions: 2.7
>Reporter: Ilya Kasnacheev
>Assignee: Maksim Timonin
>Priority: Minor
> Labels: newbie, usability
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> 0: jdbc:ignite:thin://localhost> CREATE TABLE city2(id LONG PRIMARY KEY, name
> VARCHAR,name1 VARCHAR) WITH "atomicity=mvcc";
> Error: Invalid value of "ATOMICITY" parameter (should be either TRANSACTIONAL
> or ATOMIC): mvcc (state=42000,code=1001)
> {code}
> This error message should also suggest TRANSACTIONAL_SNAPSHOT to activate
> MVCC, which totally works.
> Docs update request sent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13036) Add event triggered while page replacement start.
[ https://issues.apache.org/jira/browse/IGNITE-13036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112988#comment-17112988 ] Alexey Goncharuk commented on IGNITE-13036: --- Looks good, thanks! Merged to master. > Add event triggered while page replacement start. > - > > Key: IGNITE-13036 > URL: https://issues.apache.org/jira/browse/IGNITE-13036 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.8 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Page replacement between in memory pages and disk stored pages will be > triggered if no DataRegion free space available for the new payload. There is > a need for checking such events. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13036) Add event triggered while page replacement start.
[ https://issues.apache.org/jira/browse/IGNITE-13036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Goncharuk updated IGNITE-13036: -- Fix Version/s: 2.9 > Add event triggered while page replacement start. > - > > Key: IGNITE-13036 > URL: https://issues.apache.org/jira/browse/IGNITE-13036 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.8 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Fix For: 2.9 > > Time Spent: 20m > Remaining Estimate: 0h > > Page replacement between in memory pages and disk stored pages will be > triggered if no DataRegion free space available for the new payload. There is > a need for checking such events. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-10100) Add public Java API to call Ignite.NET services
[
https://issues.apache.org/jira/browse/IGNITE-10100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112856#comment-17112856
]
Ignite TC Bot commented on IGNITE-10100:
{panel:title=Branch: [pull/7751/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5323506buildTypeId=IgniteTests24Java8_RunAll]
> Add public Java API to call Ignite.NET services
> ---
>
> Key: IGNITE-10100
> URL: https://issues.apache.org/jira/browse/IGNITE-10100
> Project: Ignite
> Issue Type: Improvement
> Components: platforms
>Affects Versions: 2.6
>Reporter: Alexey Kukushkin
>Assignee: Ivan Daschinskiy
>Priority: Major
> Labels: .NET, sbcf
> Fix For: 2.9
>
> Attachments: ignite-10100-vs-2.8.patch
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Ignite wraps .NET services in PlatformDotNetServiceImpl implementing
> PlatformService interface.
> PlatformService is defined in internal Ignite package
> apache.ignite.internal.processors.platform.services. It exposes
> {{ invokeMethod(methodName, Object[] params): Object}}
> to call any service method dynamically. Right now there is no Ignite public
> API to call a PlatformService using static typing.
> We need to develop a public API to call PlatformDotNetServiceImpl using
> static typing in Java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-10077) P2P class loading shouldn't be applied to continuous queries with autoUnsubscribe=false
[
https://issues.apache.org/jira/browse/IGNITE-10077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denis Mekhanikov reassigned IGNITE-10077:
-
Assignee: (was: Denis Mekhanikov)
> P2P class loading shouldn't be applied to continuous queries with
> autoUnsubscribe=false
> ---
>
> Key: IGNITE-10077
> URL: https://issues.apache.org/jira/browse/IGNITE-10077
> Project: Ignite
> Issue Type: Improvement
>Reporter: Denis Mekhanikov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{ContinuousQuery#autoUnsubscribe}} flag makes it possible to deploy a
> continuous query, which won't be undeployed, when the subscriber node leaves.
> P2P class loading is not applicable in these settings, because when the
> subscriber leaves topology, it becomes impossible to load any classes from it
> to new nodes.
> So, P2P class loading should be disabled for continuous queries with
> {{autoUnsubscribe=false}}
> Dev list discussion:
> [http://apache-ignite-developers.2346864.n4.nabble.com/Continuous-query-deployment-td36732.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-8010) Validate marshaller mappings on join
[ https://issues.apache.org/jira/browse/IGNITE-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Mekhanikov reassigned IGNITE-8010: Assignee: (was: Denis Mekhanikov) > Validate marshaller mappings on join > > > Key: IGNITE-8010 > URL: https://issues.apache.org/jira/browse/IGNITE-8010 > Project: Ignite > Issue Type: Improvement >Reporter: Denis Mekhanikov >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When a new node joins topology, it sends its marshaller mappings in a > discovery join message. > Additional validation should be implemented to make nodes with conflicting > marshaller mappings be rejected from connecting to the cluster. > Such procedure is already implemented for binary metadata. Marshaller > mappings should be validated in a similar way. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13044) Additional possibility to check for high contending keys generated by the transaction payload.
[
https://issues.apache.org/jira/browse/IGNITE-13044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112851#comment-17112851
]
Ignite TC Bot commented on IGNITE-13044:
{panel:title=Branch: [pull/7824/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5323752buildTypeId=IgniteTests24Java8_RunAll]
> Additional possibility to check for high contending keys generated by the
> transaction payload.
> --
>
> Key: IGNITE-13044
> URL: https://issues.apache.org/jira/browse/IGNITE-13044
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.8
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In process of limited keys highload updates there are possible to obtain high
> queue of locking candidates. Want to be able for monitoring such situations,
> jmx seems to be correct place for it. Any additional info would be append
> here after pr approval and resolution also appropriate documentation ticket,
> of course, need to be linked.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12085) ThreadPool metrics register after all components start
[
https://issues.apache.org/jira/browse/IGNITE-12085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112837#comment-17112837
]
Ignite TC Bot commented on IGNITE-12085:
{panel:title=Branch: [pull/7818/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5324222buildTypeId=IgniteTests24Java8_RunAll]
> ThreadPool metrics register after all components start
> --
>
> Key: IGNITE-12085
> URL: https://issues.apache.org/jira/browse/IGNITE-12085
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Mikhail Petrov
>Priority: Major
> Labels: IEP-35, await
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For now, thread pool metrics register after all {{GridComponent}} starts.
> But there are specific scenarios when some component blocks {{onKernalStart}}
> execution for a long time. {{GridCacheProcessor}} can be taken as an example.
> This leads to the situation when some metric info is lost.
> Seems, we can register thread pool metrics right after only **required**
> components are started and don't wait for all components.
> {code:java}
> // Callbacks.
> for (GridComponent comp : ctx) {
> comp.onKernalStart(active);
> }
> // Start plugins.
> for (PluginProvider provider : ctx.plugins().allProviders())
> provider.onIgniteStart();
> ctx.metric().registerThreadPools(utilityCachePool, execSvc,
> svcExecSvc, sysExecSvc, stripedExecSvc,
> p2pExecSvc, mgmtExecSvc, igfsExecSvc, dataStreamExecSvc,
> restExecSvc, affExecSvc, idxExecSvc,
> callbackExecSvc, qryExecSvc, schemaExecSvc, rebalanceExecSvc,
> rebalanceStripedExecSvc, customExecSvcs);
> // Register MBeans.
> mBeansMgr.registerAllMBeans(utilityCachePool, execSvc,
> svcExecSvc, sysExecSvc, stripedExecSvc, p2pExecSvc,
> mgmtExecSvc, igfsExecSvc, dataStreamExecSvc, restExecSvc,
> affExecSvc, idxExecSvc, callbackExecSvc,
> qryExecSvc, schemaExecSvc, rebalanceExecSvc,
> rebalanceStripedExecSvc, customExecSvcs, ctx.workersRegistry());
> {code}
> {code:java}
> public class GridCacheProcessor {
> @Override public void onKernalStart(boolean active) throws
> IgniteCheckedException {
> //.
> final List syncFuts = new
> ArrayList<>(caches.size());
> sharedCtx.forAllCaches(new CIX1() {
> @Override public void applyx(GridCacheContext cctx) {
> CacheConfiguration cfg = cctx.config();
> if (cctx.affinityNode() &&
> cfg.getRebalanceMode() == SYNC &&
> startTopVer.equals(cctx.startTopologyVersion())) {
> CacheMode cacheMode = cfg.getCacheMode();
> if (cacheMode == REPLICATED || (cacheMode == PARTITIONED
> && cfg.getRebalanceDelay() >= 0))
> // Need to wait outside to avoid a deadlock
> syncFuts.add(cctx.preloader().syncFuture());
> }
> }
> });
> for (int i = 0, size = syncFuts.size(); i < size; i++)
> syncFuts.get(i).get();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
