[jira] [Updated] (IGNITE-16907) Add ability to use Raft log as storage WAL within the scope of local recovery
[
https://issues.apache.org/jira/browse/IGNITE-16907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirill Tkalenko updated IGNITE-16907:
-
Fix Version/s: 3.0.0-alpha6
> Add ability to use Raft log as storage WAL within the scope of local recovery
> -
>
> Key: IGNITE-16907
> URL: https://issues.apache.org/jira/browse/IGNITE-16907
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Ivan Bessonov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h4. Problem
> From the birds eye view raft-to-storage flow looks similar to
> #
> {code:java}
> RaftGroupService#run(writeCommand());{code}
> # Inner raft replication logic, when replicated on majority adjust
> raft.commitedIndex.
> # Propagate command to RaftGroupListener (raft state machine).
> {code:java}
> RaftGroupListener#onWrite(closure(writeCommand()));{code}
> # Within state machine insert data from writeCommand to underneath storage:
> {code:java}
> var insertRes = storage.insert(cmd.getRow(), cmd.getTimestamp());{code}
> # ack that data was applied successfully
> {code:java}
> clo.result(insertRes);{code}
> # move raft.appliedIndex to corresponding value, meaning that the data for
> this index is applied to the state machine.
> The most interesting part, especially for given ticket, relates to step 4.
> In real world storage doesn't flush every mutator on disk, instead it buffers
> some amount of such mutators and flush them all-together as a part of some
> checkpointing process. Thus, if node fails before mutatorsBuffer.flush() it
> might lost some data because raft will apply data starting from appliedIndex
> + 1 on recovery.
> h4. Possible solutions:
> There are several possibilities to solve this issue:
> # In-storage WAL. Bad solution, because there's already raft log that can be
> used as a WAL. Such duplication is redundant.
> # Local recovery starting from appliedIndex - mutatorsBuffer.size. Bad
> solution. Won't work for not-idempotent operations. Exposes inner storage
> details such as mutatorBuffer.size.
> # proposedIndex propagation + checkpointIndex synchonization. Seems fine.
> More details below:
> * First off all, in order to coordinate raft replicator and storage,
> proposedIndex should be propagated to raftGroupListener and storage.
> * On every checkpoint, storage will persist corresponding proposed index as
> checkpointIndex.
> ** In case of storage inner checkpoints, storage won't notify raft
> replicator about new checkpointIndex. This kind of notification is an
> optimization that does not affect the correctness of the protocol.
> ** In case of outer checkpoint intention, e.g. raft snapshotting for the
> purposes of raft log truncation, corresponding checkpointIndex will be
> propagated to raft replicator within a callback "onShapshotDone".
> * During local recovery raft will apply raft log entries from the very
> begging. If checkpointIndex occurred to be bigger than proposedIndex on an
> another raft log entity it fails the proposed closure with
> IndexMismatchException(checkpointIndex) that leads to proposedIndex shift and
> optional async raft log truncation.
> Let's consider following example:
> ] checkpointBuffer = 3. [P] - perisisted entities, [!P] - not perisisted/in
> memory one.
> # raft.put(k1,v1)
> ## -> raftlog[cmd(k1,v1, index:1)]
> ## -> storage[(k1,v1), index:1]
> ## -> appliedIndex:1
> # raft.put(k2,v2)
> ## -> raftlog[cmd(k1,v1, index:1), \\{*}cmd(k2,v2, index:2)\\{*}]
> ## -> storage[(k1,v1), \\{*}(k2,v2)\\{*}, ** index:\\{*}2\\{*}]
> ## -> appliedIndex:{*}2{*}
> # raft.put(k3,v3)
> ## -> raftlog[cmd(k1,v1, index:1), cmd(k2,v2, index:2), \\{*}cmd(k3,v3,
> index:3)\\{*}]
> ## -> storage[(k1,v1), (k2,v2), \\{*}(k3,v3)\\{*}, index:\\{*}3\\{*}]
> ## -> appliedIndex:{*}3{*}
> ## *inner storage checkpoint*
> ### raftlog[cmd(k1,v1, index:1), cmd(k2,v2, index:2), cmd(k3,v3, index:3)]
> ### storage[(k1,v1, proposedIndex:1), (k2,v2, proposedIndex:2), (k3,v3,
> proposedIndex:3)]
> ### {*}checkpointedData[(k1,v1),* *(k2,v2),* \\{*}(k3,v3),
> checkpointIndex:3\\{*}{*}\\{*}{*}]{*}{*}{{*}}
> # raft.put(k4,v4)
> ## -> raftlog[cmd(k1,v1, index:1), cmd(k2,v2, index:2), cmd(k3,v3,
> index:3), \\{*}cmd(k4,v4, index:4)\\{*}]
> ## -> storage[(k1,v1), (k2,v2), (k3,v3), *(k4,v4)* index:\\{*}4\\{*}]
> ## -> checkpointedData[(k1,v1), (k2,v2), (k3,v3), checkpointIndex:3]
> ## -> appliedIndex:{*}4{*}
> # Node failure
> # Node restart
> ## StorageRecovery: storage.apply(checkpointedData)
> ## raft-to-storage data application starting from index: 1 // raft doesn't
> know checkpointedIndex at this point.
> ### ->
[jira] [Commented] (IGNITE-17220) YCSB benchmark run for ignite2 vs ignite3
[ https://issues.apache.org/jira/browse/IGNITE-17220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566649#comment-17566649 ] Evgeny Stanilovsky commented on IGNITE-17220: - Sorry but i can`t understand this table results ... > YCSB benchmark run for ignite2 vs ignite3 > - > > Key: IGNITE-17220 > URL: https://issues.apache.org/jira/browse/IGNITE-17220 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Assignee: Alexander Belyak >Priority: Major > Labels: ignite-3 > Attachments: results.zip > > > For further investigation of ignite3 performance issues, we need to run the > following benchmarks to compare ignite2 vs ignite3 performance: > * Usual ycsb benchmark with mixed load patterns > * Insert-only ycsb benchmark > For ignite2 and ignite3 in the following configurations: > * 3 ignite nodes setup (so, table must have 1 partition and 3 replicas) > * 1 ignite node setup (so, table must have 1 partitoin and 1 replica) > Also, please provide: > * Hardware configuration of the environment, where benchmark was executed > * JFRs for every node in every run. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-16337) Update lucene version up to 8.11.2 or above
[ https://issues.apache.org/jira/browse/IGNITE-16337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566641#comment-17566641 ] Ilya Korol commented on IGNITE-16337: - [~aonikolaev] sorry for delay. I've rebased ticket branch to latest master and updated lucene version to 8.11.2. I also decided to run TC tests for this branch just in case, so if tests will be OK, I will merge it. > Update lucene version up to 8.11.2 or above > --- > > Key: IGNITE-16337 > URL: https://issues.apache.org/jira/browse/IGNITE-16337 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.11 >Reporter: Ilya Korol >Assignee: Ilya Korol >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Our Lucene version is quite outdated right now (7.4), while there are 2 major > releases available: 8.x.x with latest 8.11.1 and 9.0.0. So it would be good > to update this library because there were dozens of bugfixes, improvements > and optimizations since 7.4.x. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17334) Basic volatile RAFT log storage
[ https://issues.apache.org/jira/browse/IGNITE-17334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566456#comment-17566456 ] Roman Puchkovskiy commented on IGNITE-17334: It is decided to drop 'rejection' policy. We will always try to spill-out to disk when in-memory buffer overflows > Basic volatile RAFT log storage > --- > > Key: IGNITE-17334 > URL: https://issues.apache.org/jira/browse/IGNITE-17334 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > > The storage should contain the following components: > # An elastic (but with possible hard-limit) in-memory storage for log > entries. On this iteration it will be on-heap data structure (for instance, a > skip list). Later, we'll build something better (like an off-heap > implementation, maybe using pagemem). > # An algorithm for determining when the in-memory storage cannot accept new > entries. On this iteration, the only supported algorithm will be simply > limiting the store by the number of stored entries. Later we will add > something smarter, like the total size of the stored entries and adaptive > algorithms. > # A policy for what to do if the store cannot accept a record. On this > iteration, it will be just to reject the entry. Later, we can add support for > spilling out to disk. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-17334) Basic volatile RAFT log storage
[ https://issues.apache.org/jira/browse/IGNITE-17334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566456#comment-17566456 ] Roman Puchkovskiy edited comment on IGNITE-17334 at 7/13/22 6:19 PM: - It is decided to drop 'rejection' policy. We will always try to spill-out to disk when ithe n-memory buffer overflows. was (Author: rpuch): It is decided to drop 'rejection' policy. We will always try to spill-out to disk when in-memory buffer overflows > Basic volatile RAFT log storage > --- > > Key: IGNITE-17334 > URL: https://issues.apache.org/jira/browse/IGNITE-17334 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > > The storage should contain the following components: > # An elastic (but with possible hard-limit) in-memory storage for log > entries. On this iteration it will be on-heap data structure (for instance, a > skip list). Later, we'll build something better (like an off-heap > implementation, maybe using pagemem). > # An algorithm for determining when the in-memory storage cannot accept new > entries. On this iteration, the only supported algorithm will be simply > limiting the store by the number of stored entries. Later we will add > something smarter, like the total size of the stored entries and adaptive > algorithms. > # A policy for what to do if the store cannot accept a record. On this > iteration, it will be just to reject the entry. Later, we can add support for > spilling out to disk. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17365) Move internal classes used by H2 to an appropriate package
[ https://issues.apache.org/jira/browse/IGNITE-17365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-17365: - Issue Type: Improvement (was: Task) > Move internal classes used by H2 to an appropriate package > -- > > Key: IGNITE-17365 > URL: https://issues.apache.org/jira/browse/IGNITE-17365 > Project: Ignite > Issue Type: Improvement >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Major > Labels: ignite-osgi > Fix For: 2.14 > > Time Spent: 20m > Remaining Estimate: 0h > > We should avoid intersectipon of package definition between different > sub-modules in the Ignite Project. This is required to build OSGi bundle. > The follwoing classes must be moved: > - GridCacheTwoStepQuery > - QueryTable > - RegisteredQueryCursor > - SqlQueryMXBean > - SqlQueryMXBeanImpl -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-6352) ignite-indexing is not compatible to OSGI
[ https://issues.apache.org/jira/browse/IGNITE-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-6352: Issue Type: Bug (was: Improvement) > ignite-indexing is not compatible to OSGI > - > > Key: IGNITE-6352 > URL: https://issues.apache.org/jira/browse/IGNITE-6352 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 2.1 >Reporter: Mikhail Cherkasov >Priority: Major > Labels: ignite-osgi > > the issue is reported by user, there's his message: > When trying to start Ignite in an OSGi context I get the following exception: > Caused by: java.lang.NoClassDefFoundError: org/h2/server/Service > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.ignite.internal.IgniteComponentType.inClassPath(IgniteComponentType.java:153) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1832) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1648) > at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1076) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:506) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:482) > at org.apache.ignite.Ignition.start(Ignition.java:304) > That is because the h2 bundle (jar) is properly osgified, but does NOT > export the package org.h2.server, so it isn't visible to my code's classloader -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-6352) ignite-indexing is not compatible to OSGI
[ https://issues.apache.org/jira/browse/IGNITE-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-6352: Labels: ignite-osgi (was: ) > ignite-indexing is not compatible to OSGI > - > > Key: IGNITE-6352 > URL: https://issues.apache.org/jira/browse/IGNITE-6352 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 2.1 >Reporter: Mikhail Cherkasov >Priority: Major > Labels: ignite-osgi > > the issue is reported by user, there's his message: > When trying to start Ignite in an OSGi context I get the following exception: > Caused by: java.lang.NoClassDefFoundError: org/h2/server/Service > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.ignite.internal.IgniteComponentType.inClassPath(IgniteComponentType.java:153) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1832) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1648) > at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1076) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:506) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:482) > at org.apache.ignite.Ignition.start(Ignition.java:304) > That is because the h2 bundle (jar) is properly osgified, but does NOT > export the package org.h2.server, so it isn't visible to my code's classloader -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-7670) Errors Installing Ignite OSGi Karaf features on JBoss fuse
[
https://issues.apache.org/jira/browse/IGNITE-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-7670:
Labels: ignite-osgi (was: )
> Errors Installing Ignite OSGi Karaf features on JBoss fuse
> --
>
> Key: IGNITE-7670
> URL: https://issues.apache.org/jira/browse/IGNITE-7670
> Project: Ignite
> Issue Type: Bug
> Components: osgi
>Affects Versions: 2.3
> Environment: jboss-fuse-6.2.1.redhat-159
>Reporter: Stefano Violi
>Priority: Critical
> Labels: ignite-osgi
>
> Installing Ignite's features repository on JBoss Fuse generates the following
> error:
> *Karaf shell command:*
> JBossFuse:karaf@root18007> features:addurl
> mvn:org.apache.ignite/ignite-osgi-karaf/2.3.0/xml/features
> *Error:*
> Error executing command: Unable to add repositories:
> Unrecognized root element:
> \{http://karaf.apache.org/xmlns/features/v1.3.0}features
>
> I also generated a new feature repository compliant with Fuse, adding the
> ignite-core and ignite-indexing features: in this way i installed them on
> JBoss Fuse.
> By the way, when i use these features from my component and I start Ignite
> with the following Java code
> Ignition.start();
> I get the following error:
> Caused by: java.lang.NoClassDefFoundError: org/h2/server/Service
> at java.lang.Class.forName0(Native Method)[:1.8.0_112]
> at java.lang.Class.forName(Class.java:264)[:1.8.0_112]
> at
> org.apache.ignite.internal.IgniteComponentType.inClassPath(IgniteComponentType.java:153)
>
> I think there is some OSGi package import/export problem on Ignite features
> and boundles.
> Below the feature:
>
> feature name="ignite-core" version="2.3.0" description="Apache Ignite ::
> Core">
>
> dependency="true">mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.javax-cache-api/1.0.0_1
> dependency="true">mvn:org.apache.ignite/ignite-osgi/2.5.0-SNAPSHOT
> start="true">mvn:org.apache.ignite/ignite-core/2.5.0-SNAPSHOT
>
>
>
>
>
>
> dependency="true">mvn:org.osgi/org.osgi.enterprise/5.0.0
> dependency="true">mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.lucene/5.5.2_1
> dependency="true">mvn:com.h2database/h2/1.4.195
> start="false">mvn:org.apache.ignite/ignite-indexing/2.3.0
>
>
> Thanks
> Kind regards
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-11668) OSGI: Self-imported package causes failure upon Karaf restart
[
https://issues.apache.org/jira/browse/IGNITE-11668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11668:
-
Labels: ignite-osgi (was: )
> OSGI: Self-imported package causes failure upon Karaf restart
> -
>
> Key: IGNITE-11668
> URL: https://issues.apache.org/jira/browse/IGNITE-11668
> Project: Ignite
> Issue Type: Bug
> Components: osgi
>Affects Versions: 2.7
> Environment: Ignite 2.7
> Apache Karaf 4.2.0
>Reporter: Oleksii Mohylin
>Assignee: Oleksii Mohylin
>Priority: Major
> Labels: ignite-osgi
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I've got problem using Ignite 2.7 in Apache Karaf 4.2.0. The problem is
> caused by strange bundle meta of ignite-osgi. It exports package
> org.apache.ignite.osgi.classloaders and imports it at the same time. Here's
> extract from METADATA.MF:
> {noformat}
> Import-Package: org.apache.ignite;version="[2.7,3)",org.apache.ignite.
> configuration;version="[2.7,3)",org.apache.ignite.internal.util;versi
> on="[2.7,3)",org.apache.ignite.internal.util.tostring;version="[2.7,3
> )",org.apache.ignite.internal.util.typedef.internal;version="[2.7,3)"
> ,org.apache.ignite.osgi.classloaders,org.osgi.framework;version="[1.7
> ,2)"
> Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))"
> Fragment-Host: org.apache.ignite.ignite-core
> Export-Package: org.apache.ignite.osgi.classloaders;uses:="org.apache.
> ignite.internal.util.tostring,org.osgi.framework";version="2.7.0",org
> .apache.ignite.osgi;uses:="org.apache.ignite,org.apache.ignite.config
> uration,org.apache.ignite.osgi.classloaders,org.osgi.framework";versi
> on="2.7.0"
> {noformat}
> There is no problem with initial installation of my application into Karaf,
> although after Karaf ignite-osgi dependency is not resolved, and this
> exception in log:
> {noformat}
> org.osgi.framework.BundleException: Unable to resolve graphql-core [399](R
> 399.0): missing requirement [graphql-core [399](R 399.0)]
> osgi.wiring.package;
> (&(osgi.wiring.package=org.apache.ignite.osgi.classloaders)(version>=2.7.0)(!(version>=3.0.0)))
> [caused by: Unable to resolve org.apache.ignite.ignite-osgi [432](R 432.0):
> missing requirement [org.apache.ignite.ignite-osgi [432](R 432.0)]
> osgi.wiring.host;
> (&(osgi.wiring.host=org.apache.ignite.ignite-core)(bundle-version>=0.0.0))]
> Unresolved requirements: [[graphql-core [399](R 399.0)] osgi.wiring.package;
> (&(osgi.wiring.package=org.apache.ignite.osgi.classloaders)(version>=2.7.0)(!(version>=3.0.0)))]
> {noformat}
> *Proposed solution:*
> remove the self-import by adding instruction to bundle plugin configuration
> in modules/osgi/pom.xml
> {code:java}
>
> org.apache.felix
> maven-bundle-plugin
>
>
> org.apache.ignite.ignite-core
>
> !org.apache.ignite.osgi.classloaders,*
>
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17367) Java thin: client hangs on startup with TLSv1.1
[
https://issues.apache.org/jira/browse/IGNITE-17367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566423#comment-17566423
]
Ignite TC Bot commented on IGNITE-17367:
{panel:title=Branch: [pull/10158/head] Base: [master] : Possible Blockers
(29)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Cassandra Store{color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=6680814]]
{color:#d04437}Compute (Affinity Run){color} [[tests
8|https://ci.ignite.apache.org/viewLog.html?buildId=6680817]]
* org.apache.ignite.testsuites.IgniteCacheAffinityRunTestSuite:
org.apache.ignite.internal.processors.cache.IgniteCacheLockPartitionOnAffinityRunTxCacheOpTest.
- History for base branch is absent.
* IgniteCacheAffinityRunTestSuite:
IgniteBaselineLockPartitionOnAffinityRunAtomicCacheTest.testReservedPartitionCacheOp
- Test has low fail rate in base branch 0,0% and is not flaky
* IgniteCacheAffinityRunTestSuite:
IgniteCacheLockPartitionOnAffinityRunTest.testReleasePartitionJobMasterLeave -
Test has low fail rate in base branch 0,0% and is not flaky
* IgniteCacheAffinityRunTestSuite:
IgniteCacheLockPartitionOnAffinityRunTest.testReleasePartitionJobUnmarshalingFails
- Test has low fail rate in base branch 0,0% and is not flaky
* IgniteCacheAffinityRunTestSuite:
IgniteCacheLockPartitionOnAffinityRunTest.testReleasePartitionJobImplementMasterLeave
- Test has low fail rate in base branch 0,0% and is not flaky
* IgniteCacheAffinityRunTestSuite:
IgniteBaselineLockPartitionOnAffinityRunAtomicCacheTest.testNotReservedAtomicCacheOp
- Test has low fail rate in base branch 0,0% and is not flaky
* org.apache.ignite.testsuites.IgniteCacheAffinityRunTestSuite:
org.apache.ignite.internal.processors.database.baseline.IgniteBaselineLockPartitionOnAffinityRunAtomicCacheTest.
- History for base branch is absent.
* org.apache.ignite.testsuites.IgniteCacheAffinityRunTestSuite:
org.apache.ignite.internal.processors.database.baseline.IgniteBaselineLockPartitionOnAffinityRunTxCacheTest.
- History for base branch is absent.
{color:#d04437}Queries 3{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6680866]]
* IgniteBinaryCacheQueryTestSuite3:
CacheEventsCdcTest.testCreateDropSQLTable[persistence=true] - Test has low fail
rate in base branch 0,0% and is not flaky
{color:#d04437}Thin Client: Java{color} [[tests 0 Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=6680882]]
{color:#d04437}Queries 2 (lazy=true){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6680865]]
* IgniteBinaryCacheQueryLazyTestSuite2:
IgniteCacheQueryNodeRestartDistributedJoinSelfTest.testRestarts - Test has low
fail rate in base branch 0,0% and is not flaky
{color:#d04437}Client Nodes{color} [[tests
7|https://ci.ignite.apache.org/viewLog.html?buildId=6680816]]
* IgniteClientReconnectTestSuite:
IgniteClientReconnectCollectionsTest.testQueueReconnectRemoved - Test has low
fail rate in base branch 0,0% and is not flaky
* IgniteClientReconnectTestSuite:
IgniteClientReconnectCollectionsTest.testServerReconnect - Test has low fail
rate in base branch 0,0% and is not flaky
* IgniteClientReconnectTestSuite:
IgniteClientReconnectCollectionsTest.testSetReconnectRemoved - Test has low
fail rate in base branch 0,0% and is not flaky
* IgniteClientReconnectTestSuite:
IgniteClientReconnectCollectionsTest.testCollectionsReconnectClusterRestart -
Test has low fail rate in base branch 0,0% and is not flaky
* IgniteClientNodesTestSuite:
IgniteCacheManyClientsTest.testManyClientsClientDiscovery - Test has low fail
rate in base branch 0,0% and is not flaky
* IgniteClientNodesTestSuite:
IgniteCacheManyClientsTest.testManyClientsSequentiallyClientDiscovery - Test
has low fail rate in base branch 0,0% and is not flaky
* IgniteClientNodesTestSuite:
IgniteCacheManyClientsTest.testManyClientsForceServerMode - Test has low fail
rate in base branch 0,0% and is not flaky
{color:#d04437}Queries 1 (lazy=true){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6680863]]
* IgniteBinaryCacheQueryLazyTestSuite:
DynamicIndexServerCoordinatorBasicSelfTest.testCreateIndexWithParallelismPartitionedTransactional
- Test has low fail rate in base branch 0,0% and is not flaky
{color:#d04437}Queries 4{color} [[tests
9|https://ci.ignite.apache.org/viewLog.html?buildId=6680868]]
* IgniteBinaryCacheQueryTestSuite4:
SqlDataTypesCoverageTests.testDecimalDataType[atomicityMode=ATOMIC,
cacheMode=PARTITIONED, ttlFactory=null, backups=2, evictionFactory=null,
onheapCacheEnabled=true, writeSyncMode=FULL_SYNC, persistenceEnabled=false,
useBinaryArrays=false] - Test has low fail rate in base branch 0,0% and is not
flaky
* IgniteBinaryCacheQueryTestSuite4:
SqlDataTypesCoverageTests.testDecimalDataType[atomicityMode=ATOMIC,
cacheMode=PARTITIONED, ttlFactory=EternalExpiryPolicy, backups=2,
[jira] [Assigned] (IGNITE-16183) Raft heartbeat coalescing prototype
[ https://issues.apache.org/jira/browse/IGNITE-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mirza Aliev reassigned IGNITE-16183: Assignee: (was: Mirza Aliev) > Raft heartbeat coalescing prototype > --- > > Key: IGNITE-16183 > URL: https://issues.apache.org/jira/browse/IGNITE-16183 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Priority: Major > Labels: ignite-3 > > Currently, the dedicated raft group is created for every partition. This > approach leads to creating a huge number of raft groups which is expensive. > In order to optimise a multi raft group architecture, our competitors uses > some techniques, one technique is to merge several heartbeat from different > raft groups from one node and send it as one. We need to prototype such > approach and check it. > Some different optimisations can be found here > https://tikv.org/deep-dive/scalability/multi-raft/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17370) Support empty values of varlen types as default
Konstantin Orlov created IGNITE-17370: - Summary: Support empty values of varlen types as default Key: IGNITE-17370 URL: https://issues.apache.org/jira/browse/IGNITE-17370 Project: Ignite Issue Type: Bug Components: sql Reporter: Konstantin Orlov Currently, an empty string is stored in configuration instead of null value. This leads to inability to determine the real value stored as default, hence both null and empty string are desirialized as null. Need to rework this in order to be able to configure and restore all sort of values as defaults. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17365) Move internal classes used by H2 to an appropriate package
[ https://issues.apache.org/jira/browse/IGNITE-17365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566373#comment-17566373 ] Maxim Muzafarov commented on IGNITE-17365: -- Merged to the master branch. > Move internal classes used by H2 to an appropriate package > -- > > Key: IGNITE-17365 > URL: https://issues.apache.org/jira/browse/IGNITE-17365 > Project: Ignite > Issue Type: Task >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Major > Labels: ignite-osgi > Fix For: 2.14 > > Time Spent: 20m > Remaining Estimate: 0h > > We should avoid intersectipon of package definition between different > sub-modules in the Ignite Project. This is required to build OSGi bundle. > The follwoing classes must be moved: > - GridCacheTwoStepQuery > - QueryTable > - RegisteredQueryCursor > - SqlQueryMXBean > - SqlQueryMXBeanImpl -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17364) Move control utility index tasks to the core module
[
https://issues.apache.org/jira/browse/IGNITE-17364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566368#comment-17566368
]
Ignite TC Bot commented on IGNITE-17364:
{panel:title=Branch: [pull/10155/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10155/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6680127buildTypeId=IgniteTests24Java8_RunAll]
> Move control utility index tasks to the core module
> ---
>
> Key: IGNITE-17364
> URL: https://issues.apache.org/jira/browse/IGNITE-17364
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: ignite-osgi
> Time Spent: 20m
> Remaining Estimate: 0h
>
> There are some control utility tasks (indexes list, index validate, index
> rebuild), which are depended on H2 and located in indexing module. But
> instead of H2 module classes core module classes having almost the same
> functionality can be used.
> We should rewrite these tasks to use core module classes to make tasks
> workable without H2.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-17219) Revise types support in schema module
[ https://issues.apache.org/jira/browse/IGNITE-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Orlov reassigned IGNITE-17219: - Assignee: Konstantin Orlov (was: Evgeny Stanilovsky) > Revise types support in schema module > - > > Key: IGNITE-17219 > URL: https://issues.apache.org/jira/browse/IGNITE-17219 > Project: Ignite > Issue Type: Bug > Components: general >Reporter: Konstantin Orlov >Assignee: Konstantin Orlov >Priority: Major > Labels: ignite-3 > Time Spent: 3h 40m > Remaining Estimate: 0h > > Looks like not all types are properly integrated into the system: some of > them can't be serialised within SchemaDescriptor (temporal types), for some > of them default value is improperly converted from schemaDefinition to a > changer (byte array). > Lets extend test coverage in order to find out all such missed places and fix > them. Cases to consider: > * (de-)serialisation of all supported types and theirs default > * conversion from schemaDefinition to changer -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17369) Snapshot is inconsistent under streamed loading.
[
https://issues.apache.org/jira/browse/IGNITE-17369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Steshin updated IGNITE-17369:
--
Attachment: IgniteClusterShanpshotStreamerTest.java
> Snapshot is inconsistent under streamed loading.
>
>
> Key: IGNITE-17369
> URL: https://issues.apache.org/jira/browse/IGNITE-17369
> Project: Ignite
> Issue Type: Improvement
>Reporter: Vladimir Steshin
>Priority: Major
> Attachments: IgniteClusterShanpshotStreamerTest.java
>
>
> Ignite fails to restore snapshot created under streamed load:
> {code:java}
> Conflict partition: PartitionKeyV2 [grpId=109386747,
> grpName=SQL_PUBLIC_TEST_TBL1, partId=148]
> Partition instances: [PartitionHashRecordV2 [isPrimary=false,
> consistentId=snapshot.IgniteClusterShanpshotStreamerTest0, updateCntr=29,
> partitionState=OWNING, size=29, partHash=827765854], PartitionHashRecordV2
> [isPrimary=false, consistentId=snapshot.IgniteClusterShanpshotStreamerTest1,
> updateCntr=9, partitionState=OWNING, size=9, partHash=-1515069105]]
> Conflict partition: PartitionKeyV2 [grpId=109386747,
> grpName=SQL_PUBLIC_TEST_TBL1, partId=146]
> Partition instances: [PartitionHashRecordV2 [isPrimary=false,
> consistentId=snapshot.IgniteClusterShanpshotStreamerTest0, updateCntr=28,
> partitionState=OWNING, size=28, partHash=1497908810], PartitionHashRecordV2
> [isPrimary=false, consistentId=snapshot.IgniteClusterShanpshotStreamerTest1,
> updateCntr=5, partitionState=OWNING, size=5, partHash=821195757]]
> {code}
> Test (attached):
> {code:java}
> public void testClusterSnapshotConsistencyWithStreamer() throws Exception
> {
> int grids = 2;
> CountDownLatch loadNumberBeforeSnapshot = new CountDownLatch(60_000);
> AtomicBoolean stopLoading = new AtomicBoolean(false);
> dfltCacheCfg = null;
> Class.forName("org.apache.ignite.IgniteJdbcDriver");
> String tableName = "TEST_TBL1";
> startGrids(grids);
> grid(0).cluster().state(ACTIVE);
> IgniteInternalFuture load1 = runLoad(tableName, false, 1, true,
> stopLoading, loadNumberBeforeSnapshot);
> loadNumberBeforeSnapshot.await();
> grid(0).snapshot().createSnapshot(SNAPSHOT_NAME).get();
> stopLoading.set(true);
> load1.get();
> grid(0).cache("SQL_PUBLIC_" + tableName).destroy();
> grid(0).snapshot().restoreSnapshot(SNAPSHOT_NAME,
> F.asList("SQL_PUBLIC_TEST_TBL1")).get();
> }
> /** */
> private IgniteInternalFuture runLoad(String tblName, boolean useCache,
> int backups, boolean streaming, AtomicBoolean stop,
> CountDownLatch startSnp) {
> return GridTestUtils.runMultiThreadedAsync(() -> {
> if(useCache) {
> String cacheName = "SQL_PUBLIC_" + tblName.toUpperCase();
> IgniteCache cache = grid(0)
> .createCache(new CacheConfiguration Object>(cacheName).setBackups(backups)
> .setCacheMode(CacheMode.REPLICATED));
> try (IgniteDataStreamer ds =
> grid(0).dataStreamer(cacheName)) {
> for (int i = 0; !stop.get(); ++i) {
> if (streaming)
> ds.addData(i, new Account(i, i - 1));
> else
> cache.put(i, new Account(i, i - 1));
> if (startSnp.getCount() > 0)
> startSnp.countDown();
> Thread.yield();
> }
> }
> } else {
> try (Connection conn =
> DriverManager.getConnection("jdbc:ignite:thin://127.0.0.1/")) {
> createTable(conn, tblName, backups);
> try (PreparedStatement stmt =
> conn.prepareStatement("INSERT INTO " + tblName +
> "(id, name, orgid, dep) VALUES(?, ?, ?, ?)")) {
> if (streaming)
> conn.prepareStatement("SET STREAMING
> ON;").execute();
> int leftLimit = 97; // letter 'a'
> int rightLimit = 122; // letter'z'
> int targetStringLength = 15;
> Random rand = new Random();
> //
> for (int i = 0; !stop.get(); ++i) {
> int orgid = rand.ints(1, 0,
> 5).findFirst().getAsInt();
> String val = rand.ints(leftLimit, rightLimit +
> 1).limit(targetStringLength)
> .collect(StringBuilder::new,
> StringBuilder::appendCodePoint, StringBuilder::append)
> .toString();
> stmt.setInt(1, i);
>
[jira] [Created] (IGNITE-17369) Snapshot is inconsistent under streamed loading.
Vladimir Steshin created IGNITE-17369:
-
Summary: Snapshot is inconsistent under streamed loading.
Key: IGNITE-17369
URL: https://issues.apache.org/jira/browse/IGNITE-17369
Project: Ignite
Issue Type: Improvement
Reporter: Vladimir Steshin
Attachments: IgniteClusterShanpshotStreamerTest.java
Ignite fails to restore snapshot created under streamed load:
{code:java}
Conflict partition: PartitionKeyV2 [grpId=109386747,
grpName=SQL_PUBLIC_TEST_TBL1, partId=148]
Partition instances: [PartitionHashRecordV2 [isPrimary=false,
consistentId=snapshot.IgniteClusterShanpshotStreamerTest0, updateCntr=29,
partitionState=OWNING, size=29, partHash=827765854], PartitionHashRecordV2
[isPrimary=false, consistentId=snapshot.IgniteClusterShanpshotStreamerTest1,
updateCntr=9, partitionState=OWNING, size=9, partHash=-1515069105]]
Conflict partition: PartitionKeyV2 [grpId=109386747,
grpName=SQL_PUBLIC_TEST_TBL1, partId=146]
Partition instances: [PartitionHashRecordV2 [isPrimary=false,
consistentId=snapshot.IgniteClusterShanpshotStreamerTest0, updateCntr=28,
partitionState=OWNING, size=28, partHash=1497908810], PartitionHashRecordV2
[isPrimary=false, consistentId=snapshot.IgniteClusterShanpshotStreamerTest1,
updateCntr=5, partitionState=OWNING, size=5, partHash=821195757]]
{code}
Test (attached):
{code:java}
public void testClusterSnapshotConsistencyWithStreamer() throws Exception {
int grids = 2;
CountDownLatch loadNumberBeforeSnapshot = new CountDownLatch(60_000);
AtomicBoolean stopLoading = new AtomicBoolean(false);
dfltCacheCfg = null;
Class.forName("org.apache.ignite.IgniteJdbcDriver");
String tableName = "TEST_TBL1";
startGrids(grids);
grid(0).cluster().state(ACTIVE);
IgniteInternalFuture load1 = runLoad(tableName, false, 1, true,
stopLoading, loadNumberBeforeSnapshot);
loadNumberBeforeSnapshot.await();
grid(0).snapshot().createSnapshot(SNAPSHOT_NAME).get();
stopLoading.set(true);
load1.get();
grid(0).cache("SQL_PUBLIC_" + tableName).destroy();
grid(0).snapshot().restoreSnapshot(SNAPSHOT_NAME,
F.asList("SQL_PUBLIC_TEST_TBL1")).get();
}
/** */
private IgniteInternalFuture runLoad(String tblName, boolean useCache,
int backups, boolean streaming, AtomicBoolean stop,
CountDownLatch startSnp) {
return GridTestUtils.runMultiThreadedAsync(() -> {
if(useCache) {
String cacheName = "SQL_PUBLIC_" + tblName.toUpperCase();
IgniteCache cache = grid(0)
.createCache(new CacheConfiguration(cacheName).setBackups(backups)
.setCacheMode(CacheMode.REPLICATED));
try (IgniteDataStreamer ds =
grid(0).dataStreamer(cacheName)) {
for (int i = 0; !stop.get(); ++i) {
if (streaming)
ds.addData(i, new Account(i, i - 1));
else
cache.put(i, new Account(i, i - 1));
if (startSnp.getCount() > 0)
startSnp.countDown();
Thread.yield();
}
}
} else {
try (Connection conn =
DriverManager.getConnection("jdbc:ignite:thin://127.0.0.1/")) {
createTable(conn, tblName, backups);
try (PreparedStatement stmt = conn.prepareStatement("INSERT
INTO " + tblName +
"(id, name, orgid, dep) VALUES(?, ?, ?, ?)")) {
if (streaming)
conn.prepareStatement("SET STREAMING
ON;").execute();
int leftLimit = 97; // letter 'a'
int rightLimit = 122; // letter'z'
int targetStringLength = 15;
Random rand = new Random();
//
for (int i = 0; !stop.get(); ++i) {
int orgid = rand.ints(1, 0,
5).findFirst().getAsInt();
String val = rand.ints(leftLimit, rightLimit +
1).limit(targetStringLength)
.collect(StringBuilder::new,
StringBuilder::appendCodePoint, StringBuilder::append)
.toString();
stmt.setInt(1, i);
stmt.setString(2, val);
stmt.setInt(3, orgid);
stmt.setInt(4, 0);
stmt.executeUpdate();
if (startSnp.getCount() > 0)
startSnp.countDown();
Thread.yield();
}
}
}
catch
[jira] [Created] (IGNITE-17368) [IEP-80] Removal of GridClient based JDBC driver
Aleksey Plekhanov created IGNITE-17368:
--
Summary: [IEP-80] Removal of GridClient based JDBC driver
Key: IGNITE-17368
URL: https://issues.apache.org/jira/browse/IGNITE-17368
Project: Ignite
Issue Type: Improvement
Reporter: Aleksey Plekhanov
{{GridClient}} based JDBC driver deprecated since 2015, we should remove it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17367) Java thin: client hangs on startup with TLSv1.1
[ https://issues.apache.org/jira/browse/IGNITE-17367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-17367: Description: Java thin client hangs on startup when using deprecated *TLSv1.1*. An exception is thrown in *GridNioSslFilter.onSessionOpened*, but *GridNioClientConnectionMultiplexer* waits on *sesFut* forever. * Only reproduces on updated systems where TLSv1.1 is considered deprecated and causes an exception. * Reproduces in *SsslParametersTest#testOneCommonProtocol*. was:Java thin client hangs on startup when using deprecated TLSv1.1. An exception is thrown in *GridNioSslFilter.onSessionOpened*, but *GridNioClientConnectionMultiplexer* waits on *sesFut* forever. > Java thin: client hangs on startup with TLSv1.1 > --- > > Key: IGNITE-17367 > URL: https://issues.apache.org/jira/browse/IGNITE-17367 > Project: Ignite > Issue Type: Bug > Components: thin client >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Fix For: 2.14 > > > Java thin client hangs on startup when using deprecated *TLSv1.1*. An > exception is thrown in *GridNioSslFilter.onSessionOpened*, but > *GridNioClientConnectionMultiplexer* waits on *sesFut* forever. > * Only reproduces on updated systems where TLSv1.1 is considered deprecated > and causes an exception. > * Reproduces in *SsslParametersTest#testOneCommonProtocol*. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17367) Java thin: client hangs on startup with TLSv1.1
[
https://issues.apache.org/jira/browse/IGNITE-17367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Tupitsyn updated IGNITE-17367:
Description: Java thin client hangs on startup when using deprecated
TLSv1.1. An exception is thrown in *GridNioSslFilter.onSessionOpened*, but
*GridNioClientConnectionMultiplexer* waits on *sesFut* forever. (was: Java
thin client hangs on startup when using deprecated TLSv1.1. An exception is
thrown in {{GridNioSslFilter.onSessionOpened}}.)
> Java thin: client hangs on startup with TLSv1.1
> ---
>
> Key: IGNITE-17367
> URL: https://issues.apache.org/jira/browse/IGNITE-17367
> Project: Ignite
> Issue Type: Bug
> Components: thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
> Fix For: 2.14
>
>
> Java thin client hangs on startup when using deprecated TLSv1.1. An exception
> is thrown in *GridNioSslFilter.onSessionOpened*, but
> *GridNioClientConnectionMultiplexer* waits on *sesFut* forever.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17367) Java thin: client hangs on startup with TLSv1.1
Pavel Tupitsyn created IGNITE-17367:
---
Summary: Java thin: client hangs on startup with TLSv1.1
Key: IGNITE-17367
URL: https://issues.apache.org/jira/browse/IGNITE-17367
Project: Ignite
Issue Type: Bug
Components: thin client
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
Fix For: 2.14
Java thin client hangs on startup when using deprecated TLSv1.1. An exception
is thrown in {{GridNioSslFilter.onSessionOpened}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17317) Test IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata is flaky on TC
[
https://issues.apache.org/jira/browse/IGNITE-17317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566319#comment-17566319
]
Roman Puchkovskiy commented on IGNITE-17317:
The patch looks good to me
> Test
> IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata
> is flaky on TC
> -
>
> Key: IGNITE-17317
> URL: https://issues.apache.org/jira/browse/IGNITE-17317
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
> Fix For: 2.14
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Test history is available here: [TC
> link|https://ci.ignite.apache.org/test/-1717009344468901983?currentProjectId=IgniteTests24Java8=%3Cdefault%3E]
> Test fails on a cache put operation before a condition being verified is
> reached:
> {code:java}
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
> (retry update if possible).: [2]
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
> at
> org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
> {code}
> There is also a stack trace in logs indicating there is some problem when
> writing to WAL:
> {code:java}
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
> (retry update if possible).: [2]
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
> at
> org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2428)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: class
> org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException:
> Failed to update keys (retry update if possible).: [2]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:401)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:248)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateResponse(GridDhtAtomicCache.java:3213)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$500(GridDhtAtomicCache.java:145)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:305)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:300)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1151)
> at
>
[jira] [Updated] (IGNITE-17317) Test IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata is flaky on TC
[
https://issues.apache.org/jira/browse/IGNITE-17317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Puchkovskiy updated IGNITE-17317:
---
Reviewer: Roman Puchkovskiy
> Test
> IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata
> is flaky on TC
> -
>
> Key: IGNITE-17317
> URL: https://issues.apache.org/jira/browse/IGNITE-17317
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
> Fix For: 2.14
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Test history is available here: [TC
> link|https://ci.ignite.apache.org/test/-1717009344468901983?currentProjectId=IgniteTests24Java8=%3Cdefault%3E]
> Test fails on a cache put operation before a condition being verified is
> reached:
> {code:java}
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
> (retry update if possible).: [2]
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
> at
> org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
> {code}
> There is also a stack trace in logs indicating there is some problem when
> writing to WAL:
> {code:java}
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
> (retry update if possible).: [2]
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
> at
> org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2428)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: class
> org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException:
> Failed to update keys (retry update if possible).: [2]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:401)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:248)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateResponse(GridDhtAtomicCache.java:3213)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$500(GridDhtAtomicCache.java:145)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:305)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:300)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1151)
> at
>
[jira] [Updated] (IGNITE-17220) YCSB benchmark run for ignite2 vs ignite3
[ https://issues.apache.org/jira/browse/IGNITE-17220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-17220: Attachment: (was: ycsb_2022-07-11_11-56-18.summary.json) > YCSB benchmark run for ignite2 vs ignite3 > - > > Key: IGNITE-17220 > URL: https://issues.apache.org/jira/browse/IGNITE-17220 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Assignee: Alexander Belyak >Priority: Major > Labels: ignite-3 > Attachments: results.zip > > > For further investigation of ignite3 performance issues, we need to run the > following benchmarks to compare ignite2 vs ignite3 performance: > * Usual ycsb benchmark with mixed load patterns > * Insert-only ycsb benchmark > For ignite2 and ignite3 in the following configurations: > * 3 ignite nodes setup (so, table must have 1 partition and 3 replicas) > * 1 ignite node setup (so, table must have 1 partitoin and 1 replica) > Also, please provide: > * Hardware configuration of the environment, where benchmark was executed > * JFRs for every node in every run. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17220) YCSB benchmark run for ignite2 vs ignite3
[ https://issues.apache.org/jira/browse/IGNITE-17220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-17220: Attachment: ycsb_2022-07-11_11-56-18.summary.json > YCSB benchmark run for ignite2 vs ignite3 > - > > Key: IGNITE-17220 > URL: https://issues.apache.org/jira/browse/IGNITE-17220 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Assignee: Alexander Belyak >Priority: Major > Labels: ignite-3 > Attachments: results.zip > > > For further investigation of ignite3 performance issues, we need to run the > following benchmarks to compare ignite2 vs ignite3 performance: > * Usual ycsb benchmark with mixed load patterns > * Insert-only ycsb benchmark > For ignite2 and ignite3 in the following configurations: > * 3 ignite nodes setup (so, table must have 1 partition and 3 replicas) > * 1 ignite node setup (so, table must have 1 partitoin and 1 replica) > Also, please provide: > * Hardware configuration of the environment, where benchmark was executed > * JFRs for every node in every run. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17366) Document the service call context and interceptors.
Pavel Pereslegin created IGNITE-17366: - Summary: Document the service call context and interceptors. Key: IGNITE-17366 URL: https://issues.apache.org/jira/browse/IGNITE-17366 Project: Ignite Issue Type: Sub-task Components: documentation Reporter: Pavel Pereslegin Assignee: Pavel Pereslegin Document the service call context and interceptors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17365) Move internal classes used by H2 to an appropriate package
[ https://issues.apache.org/jira/browse/IGNITE-17365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-17365: - Description: We should avoid intersectipon of package definition between different sub-modules in the Ignite Project. This is required to build OSGi bundle. The follwoing classes must be moved: - GridCacheTwoStepQuery - QueryTable - RegisteredQueryCursor - SqlQueryMXBean - SqlQueryMXBeanImpl was: The follwoing classes must be moved: - GridCacheTwoStepQuery - QueryTable - RegisteredQueryCursor - SqlQueryMXBean - SqlQueryMXBeanImpl > Move internal classes used by H2 to an appropriate package > -- > > Key: IGNITE-17365 > URL: https://issues.apache.org/jira/browse/IGNITE-17365 > Project: Ignite > Issue Type: Task >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Major > Labels: ignite-osgi > Fix For: 2.14 > > Time Spent: 10m > Remaining Estimate: 0h > > We should avoid intersectipon of package definition between different > sub-modules in the Ignite Project. This is required to build OSGi bundle. > The follwoing classes must be moved: > - GridCacheTwoStepQuery > - QueryTable > - RegisteredQueryCursor > - SqlQueryMXBean > - SqlQueryMXBeanImpl -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17365) Move internal classes used by H2 to an appropriate package
[ https://issues.apache.org/jira/browse/IGNITE-17365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-17365: - Summary: Move internal classes used by H2 to an appropriate package (was: Move H2 internal classes to an appropriate package) > Move internal classes used by H2 to an appropriate package > -- > > Key: IGNITE-17365 > URL: https://issues.apache.org/jira/browse/IGNITE-17365 > Project: Ignite > Issue Type: Task >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Major > Labels: ignite-osgi > Fix For: 2.14 > > > The follwoing classes must be moved: > - GridCacheTwoStepQuery > - QueryTable > - RegisteredQueryCursor > - SqlQueryMXBean > - SqlQueryMXBeanImpl -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16907) Add ability to use Raft log as storage WAL within the scope of local recovery
[
https://issues.apache.org/jira/browse/IGNITE-16907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov updated IGNITE-16907:
---
Reviewer: Kirill Tkalenko
> Add ability to use Raft log as storage WAL within the scope of local recovery
> -
>
> Key: IGNITE-16907
> URL: https://issues.apache.org/jira/browse/IGNITE-16907
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Ivan Bessonov
>Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h4. Problem
> From the birds eye view raft-to-storage flow looks similar to
> #
> {code:java}
> RaftGroupService#run(writeCommand());{code}
> # Inner raft replication logic, when replicated on majority adjust
> raft.commitedIndex.
> # Propagate command to RaftGroupListener (raft state machine).
> {code:java}
> RaftGroupListener#onWrite(closure(writeCommand()));{code}
> # Within state machine insert data from writeCommand to underneath storage:
> {code:java}
> var insertRes = storage.insert(cmd.getRow(), cmd.getTimestamp());{code}
> # ack that data was applied successfully
> {code:java}
> clo.result(insertRes);{code}
> # move raft.appliedIndex to corresponding value, meaning that the data for
> this index is applied to the state machine.
> The most interesting part, especially for given ticket, relates to step 4.
> In real world storage doesn't flush every mutator on disk, instead it buffers
> some amount of such mutators and flush them all-together as a part of some
> checkpointing process. Thus, if node fails before mutatorsBuffer.flush() it
> might lost some data because raft will apply data starting from appliedIndex
> + 1 on recovery.
> h4. Possible solutions:
> There are several possibilities to solve this issue:
> # In-storage WAL. Bad solution, because there's already raft log that can be
> used as a WAL. Such duplication is redundant.
> # Local recovery starting from appliedIndex - mutatorsBuffer.size. Bad
> solution. Won't work for not-idempotent operations. Exposes inner storage
> details such as mutatorBuffer.size.
> # proposedIndex propagation + checkpointIndex synchonization. Seems fine.
> More details below:
> * First off all, in order to coordinate raft replicator and storage,
> proposedIndex should be propagated to raftGroupListener and storage.
> * On every checkpoint, storage will persist corresponding proposed index as
> checkpointIndex.
> ** In case of storage inner checkpoints, storage won't notify raft
> replicator about new checkpointIndex. This kind of notification is an
> optimization that does not affect the correctness of the protocol.
> ** In case of outer checkpoint intention, e.g. raft snapshotting for the
> purposes of raft log truncation, corresponding checkpointIndex will be
> propagated to raft replicator within a callback "onShapshotDone".
> * During local recovery raft will apply raft log entries from the very
> begging. If checkpointIndex occurred to be bigger than proposedIndex on an
> another raft log entity it fails the proposed closure with
> IndexMismatchException(checkpointIndex) that leads to proposedIndex shift and
> optional async raft log truncation.
> Let's consider following example:
> ] checkpointBuffer = 3. [P] - perisisted entities, [!P] - not perisisted/in
> memory one.
> # raft.put(k1,v1)
> ## -> raftlog[cmd(k1,v1, index:1)]
> ## -> storage[(k1,v1), index:1]
> ## -> appliedIndex:1
> # raft.put(k2,v2)
> ## -> raftlog[cmd(k1,v1, index:1), \\{*}cmd(k2,v2, index:2)\\{*}]
> ## -> storage[(k1,v1), \\{*}(k2,v2)\\{*}, ** index:\\{*}2\\{*}]
> ## -> appliedIndex:{*}2{*}
> # raft.put(k3,v3)
> ## -> raftlog[cmd(k1,v1, index:1), cmd(k2,v2, index:2), \\{*}cmd(k3,v3,
> index:3)\\{*}]
> ## -> storage[(k1,v1), (k2,v2), \\{*}(k3,v3)\\{*}, index:\\{*}3\\{*}]
> ## -> appliedIndex:{*}3{*}
> ## *inner storage checkpoint*
> ### raftlog[cmd(k1,v1, index:1), cmd(k2,v2, index:2), cmd(k3,v3, index:3)]
> ### storage[(k1,v1, proposedIndex:1), (k2,v2, proposedIndex:2), (k3,v3,
> proposedIndex:3)]
> ### {*}checkpointedData[(k1,v1),* *(k2,v2),* \\{*}(k3,v3),
> checkpointIndex:3\\{*}{*}\\{*}{*}]{*}{*}{{*}}
> # raft.put(k4,v4)
> ## -> raftlog[cmd(k1,v1, index:1), cmd(k2,v2, index:2), cmd(k3,v3,
> index:3), \\{*}cmd(k4,v4, index:4)\\{*}]
> ## -> storage[(k1,v1), (k2,v2), (k3,v3), *(k4,v4)* index:\\{*}4\\{*}]
> ## -> checkpointedData[(k1,v1), (k2,v2), (k3,v3), checkpointIndex:3]
> ## -> appliedIndex:{*}4{*}
> # Node failure
> # Node restart
> ## StorageRecovery: storage.apply(checkpointedData)
> ## raft-to-storage data application starting from index: 1 // raft doesn't
> know checkpointedIndex at this point.
> ### -> storageResponse::IndexMismatchException(3)
>
[jira] [Updated] (IGNITE-17365) Move H2 internal classes to an appropriate package
[ https://issues.apache.org/jira/browse/IGNITE-17365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-17365: - Ignite Flags: (was: Docs Required,Release Notes Required) > Move H2 internal classes to an appropriate package > -- > > Key: IGNITE-17365 > URL: https://issues.apache.org/jira/browse/IGNITE-17365 > Project: Ignite > Issue Type: Task >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Major > Labels: ignite-osgi > Fix For: 2.14 > > > The follwoing classes must be moved: > - GridCacheTwoStepQuery > - QueryTable > - RegisteredQueryCursor > - SqlQueryMXBean > - SqlQueryMXBeanImpl -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17365) Move H2 internal classes to an appropriate package
Maxim Muzafarov created IGNITE-17365: Summary: Move H2 internal classes to an appropriate package Key: IGNITE-17365 URL: https://issues.apache.org/jira/browse/IGNITE-17365 Project: Ignite Issue Type: Task Reporter: Maxim Muzafarov Assignee: Maxim Muzafarov Fix For: 2.14 The follwoing classes must be moved: - GridCacheTwoStepQuery - QueryTable - RegisteredQueryCursor - SqlQueryMXBean - SqlQueryMXBeanImpl -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17184) Ignite OSGi Karaf 2.13.0 doesn't exist in maven central
[ https://issues.apache.org/jira/browse/IGNITE-17184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-17184: - Labels: ignite-osgi (was: ) > Ignite OSGi Karaf 2.13.0 doesn't exist in maven central > --- > > Key: IGNITE-17184 > URL: https://issues.apache.org/jira/browse/IGNITE-17184 > Project: Ignite > Issue Type: Bug > Components: osgi >Affects Versions: 2.13 >Reporter: Valeria Borodina >Priority: Major > Labels: ignite-osgi > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17216) Ignite modules in maven central aren't "bundle".
[ https://issues.apache.org/jira/browse/IGNITE-17216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-17216: - Labels: ignite-osgi (was: ) > Ignite modules in maven central aren't "bundle". > > > Key: IGNITE-17216 > URL: https://issues.apache.org/jira/browse/IGNITE-17216 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.12, 2.13 >Reporter: Valeria Borodina >Priority: Major > Labels: ignite-osgi > > In osgi we use bundle not just jar. Ignite-osgi-karaf also installation how > bundle. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16335) Ignite 2.12.0 jars are not OSGi compatible
[ https://issues.apache.org/jira/browse/IGNITE-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-16335: - Labels: ignite-osgi (was: ) > Ignite 2.12.0 jars are not OSGi compatible > -- > > Key: IGNITE-16335 > URL: https://issues.apache.org/jira/browse/IGNITE-16335 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.12 >Reporter: Richard Bryan >Priority: Blocker > Labels: ignite-osgi > > The ignite jars for 2.12.0 published to maven central are not > OSGi-compatible. For instance, this is the 2.12.0 MANIFEST for ignite-osgi: > Manifest-Version: 1.0 > Implementation-Title: ignite-osgi > Implementation-Version: 2.12.0 > Specification-Vendor: The Apache Software Foundation > Specification-Title: ignite-osgi > Build-Jdk-Spec: 1.8 > Created-By: Maven Jar Plugin 3.2.0 > Specification-Version: 2.12 > Implementation-Vendor: The Apache Software Foundation > > And this is the MANIFEST from 2.11.1: > > Manifest-Version: 1.0 > Bundle-License: http://www.apache.org/licenses/LICENSE-2.0.txt > Bundle-SymbolicName: org.apache.ignite.ignite-osgi > Archiver-Version: Plexus Archiver > Built-By: teamcity > Bnd-LastModified: 1639998601403 > Specification-Title: ignite-osgi > Implementation-Vendor-Id: org.apache.ignite > Bundle-DocURL: http://ignite.apache.org > Import-Package: org.apache.ignite;version="[2.11,3)",org.apache.ignite > .configuration;version="[2.11,3)",org.apache.ignite.internal.util;ver > sion="[2.11,3)",org.apache.ignite.internal.util.tostring;version="[2. > 11,3)",org.apache.ignite.internal.util.typedef.internal;version="[2.1 > 1,3)",org.apache.ignite.osgi.classloaders,org.osgi.framework;version= > "[1.7,2)" > Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))" > Fragment-Host: org.apache.ignite.ignite-core > Export-Package: org.apache.ignite.osgi;uses:="org.apache.ignite,org.ap > ache.ignite.configuration,org.apache.ignite.osgi.classloaders,org.osg > i.framework";version="2.11.1",org.apache.ignite.osgi.classloaders;use > s:="org.apache.ignite.internal.util.tostring,org.osgi.framework";vers > ion="2.11.1" > Bundle-Name: ignite-osgi > Implementation-Title: ignite-osgi > Bundle-Description: Java-based middleware for in-memory processing of > big data in a distributed environment. > Implementation-Version: 2.11.1 > Embed-Directory: lib > Specification-Vendor: The Apache Software Foundation > Bundle-ManifestVersion: 2 > Bundle-Vendor: The Apache Software Foundation > Tool: Bnd-3.5.0.201709291849 > Implementation-Vendor: The Apache Software Foundation > Bundle-Version: 2.11.1 > Embed-Transitive: false > Created-By: Apache Maven Bundle Plugin > Build-Jdk: 1.8.0_261 > Specification-Version: 2.11.1 > > > > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17364) Move control utility index tasks to the core module
[ https://issues.apache.org/jira/browse/IGNITE-17364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-17364: - Labels: ignite-osgi (was: ) > Move control utility index tasks to the core module > --- > > Key: IGNITE-17364 > URL: https://issues.apache.org/jira/browse/IGNITE-17364 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: ignite-osgi > Time Spent: 20m > Remaining Estimate: 0h > > There are some control utility tasks (indexes list, index validate, index > rebuild), which are depended on H2 and located in indexing module. But > instead of H2 module classes core module classes having almost the same > functionality can be used. > We should rewrite these tasks to use core module classes to make tasks > workable without H2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17046) Move H2RowComparator logic to the core module
[
https://issues.apache.org/jira/browse/IGNITE-17046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-17046:
-
Labels: ignite-osgi (was: )
> Move H2RowComparator logic to the core module
> -
>
> Key: IGNITE-17046
> URL: https://issues.apache.org/jira/browse/IGNITE-17046
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: ignite-osgi
> Fix For: 2.14
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> {{H2RowComparator}} depends on H2 classes and can't be used without
> ignite-indexing module. We should move the logic from this comparator to the
> core module providing backward compatibility, to be able to use indexes
> without H2.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-13570) Migrate OSGI module to ignite-extensions
[ https://issues.apache.org/jira/browse/IGNITE-13570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-13570: - Labels: ignite-osgi (was: ) > Migrate OSGI module to ignite-extensions > > > Key: IGNITE-13570 > URL: https://issues.apache.org/jira/browse/IGNITE-13570 > Project: Ignite > Issue Type: Sub-task > Components: extensions >Reporter: Alexey Goncharuk >Assignee: Maxim Muzafarov >Priority: Major > Labels: ignite-osgi > Fix For: 2.14 > > Time Spent: 20m > Remaining Estimate: 0h > > Migrate OSGI module to ignite-extensions > https://github.com/apache/ignite-extensions > Details: > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-15424) Calcite engine. Move schema management infrastructure to the core module
[
https://issues.apache.org/jira/browse/IGNITE-15424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-15424:
-
Labels: calcite2-required ignite-osgi osgi (was: calcite2-required osgi)
> Calcite engine. Move schema management infrastructure to the core module
> -
>
> Key: IGNITE-15424
> URL: https://issues.apache.org/jira/browse/IGNITE-15424
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite2-required, ignite-osgi, osgi
>
> Currently, schema management commands initiated in the {{ignite-core}}
> module, but all processing implemented in the {{ignite-indexing}} module.
> Ignite Calciite SQL engine use {{SchemaChangeListener}} to maintain own
> structures.
> To get rid of {{ignite-indexing}} (and H2) dependency from the
> {{ignite-calcite}} module we should move schema management implementation to
> the core module.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17190) Calcite engine. Unbind statistics from H2
[ https://issues.apache.org/jira/browse/IGNITE-17190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-17190: - Labels: calcite calcite2-required ignite-osgi (was: calcite calcite2-required) > Calcite engine. Unbind statistics from H2 > - > > Key: IGNITE-17190 > URL: https://issues.apache.org/jira/browse/IGNITE-17190 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksey Plekhanov >Assignee: Ivan Daschinsky >Priority: Major > Labels: calcite, calcite2-required, ignite-osgi > > Currently, table statistics in Ignite uses some H2 classes (Value for > example). We should unbind statistics from H2 and move statistics to the core > module to be able to use it in calcite module without dependency to H2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-15424) Calcite engine. Move schema management infrastructure to the core module
[
https://issues.apache.org/jira/browse/IGNITE-15424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-15424:
-
Labels: calcite2-required osgi (was: calcite2-required)
> Calcite engine. Move schema management infrastructure to the core module
> -
>
> Key: IGNITE-15424
> URL: https://issues.apache.org/jira/browse/IGNITE-15424
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite2-required, osgi
>
> Currently, schema management commands initiated in the {{ignite-core}}
> module, but all processing implemented in the {{ignite-indexing}} module.
> Ignite Calciite SQL engine use {{SchemaChangeListener}} to maintain own
> structures.
> To get rid of {{ignite-indexing}} (and H2) dependency from the
> {{ignite-calcite}} module we should move schema management implementation to
> the core module.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17317) Test IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata is flaky on TC
[
https://issues.apache.org/jira/browse/IGNITE-17317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566254#comment-17566254
]
Ignite TC Bot commented on IGNITE-17317:
{panel:title=Branch: [pull/10151/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10151/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6677268buildTypeId=IgniteTests24Java8_RunAll]
> Test
> IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata
> is flaky on TC
> -
>
> Key: IGNITE-17317
> URL: https://issues.apache.org/jira/browse/IGNITE-17317
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
> Fix For: 2.14
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Test history is available here: [TC
> link|https://ci.ignite.apache.org/test/-1717009344468901983?currentProjectId=IgniteTests24Java8=%3Cdefault%3E]
> Test fails on a cache put operation before a condition being verified is
> reached:
> {code:java}
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
> (retry update if possible).: [2]
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
> at
> org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
> {code}
> There is also a stack trace in logs indicating there is some problem when
> writing to WAL:
> {code:java}
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
> (retry update if possible).: [2]
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
> at
> org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2428)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: class
> org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException:
> Failed to update keys (retry update if possible).: [2]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:401)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:248)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateResponse(GridDhtAtomicCache.java:3213)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$500(GridDhtAtomicCache.java:145)
> at
>
[jira] [Updated] (IGNITE-15247) JDBC driver for 3.0: Error handling.
[ https://issues.apache.org/jira/browse/IGNITE-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-15247: -- Description: *Motivation:* JDBC driver must throws exceptions of java.sql.SQLException kind with a SQLState code (XOPEN type or the SQL:2003 type). Standard says SQLState is implementation-defined thing and define the SQLState format, some predefined categories and ability to specify vendor-code of 'int' type. Vendor codes must be well-documented. Let's map Ignite exceptions/error codes to proper SQLException with SQLState. 1. As we always know at what phase the query execution (in JDBC driver or in SQL engine) was failed, we can use proper standard SQLState category. 2. We use IGN-XXX-nnn format (where XXX is a group name and nnn is an unique error code within a group) [1], we can't map Ignite error code to SQL vendor code directly. So, we should convert non-SQL origins to IGN-SQL-nnn somehow(?) and use 'nnn' as vendor code(?). https://issues.apache.org/jira/browse/IGNITE-14931 was: *Motivation:* JDBC driver must throws exceptions of java.sql.SQLException kind with a SQLState code (XOPEN type or the SQL:2003 type). Standard says SQLState is implementation-defined thing and define the SQLState format, some predefined categories and ability to specify vendor-code. Vendor codes must be well-documented. Let's map Ignite exceptions/error codes to proper SQLException with SQLState. > JDBC driver for 3.0: Error handling. > > > Key: IGNITE-15247 > URL: https://issues.apache.org/jira/browse/IGNITE-15247 > Project: Ignite > Issue Type: New Feature > Components: jdbc >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > *Motivation:* > JDBC driver must throws exceptions of java.sql.SQLException kind with a > SQLState code (XOPEN type or the SQL:2003 type). > Standard says SQLState is implementation-defined thing and define the > SQLState format, some predefined categories and ability to specify > vendor-code of 'int' type. > Vendor codes must be well-documented. > Let's map Ignite exceptions/error codes to proper SQLException with SQLState. > 1. As we always know at what phase the query execution (in JDBC driver or in > SQL engine) was failed, we can use proper standard SQLState category. > 2. We use IGN-XXX-nnn format (where XXX is a group name and nnn is an unique > error code within a group) [1], we can't map Ignite error code to SQL vendor > code directly. > So, we should convert non-SQL origins to IGN-SQL-nnn somehow(?) and use 'nnn' > as vendor code(?). > https://issues.apache.org/jira/browse/IGNITE-14931 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-15247) JDBC driver for 3.0: Error handling.
[ https://issues.apache.org/jira/browse/IGNITE-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-15247: -- Description: *Motivation:* JDBC driver must throws exceptions of java.sql.SQLException kind with a SQLState code (XOPEN type or the SQL:2003 type). Standard says SQLState is implementation-defined thing and define the SQLState format, some predefined categories and ability to specify vendor-code. Vendor codes must be well-documented. Let's map Ignite exceptions/error codes to proper SQLException with SQLState. was: *Motivation:* JDBC driver must throws exceptions of type java.sql.SQLException with mandatory SQLState codes (vendor code and standard, if exists). These codes must be documented. Let's * Create list of supported SQLState codes. * Map existed Ignite exceptions/error codes to proper SQLState > JDBC driver for 3.0: Error handling. > > > Key: IGNITE-15247 > URL: https://issues.apache.org/jira/browse/IGNITE-15247 > Project: Ignite > Issue Type: New Feature > Components: jdbc >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > *Motivation:* > JDBC driver must throws exceptions of java.sql.SQLException kind with a > SQLState code (XOPEN type or the SQL:2003 type). > Standard says SQLState is implementation-defined thing and define the > SQLState format, some predefined categories and ability to specify > vendor-code. > Vendor codes must be well-documented. > Let's map Ignite exceptions/error codes to proper SQLException with SQLState. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-15247) JDBC driver for 3.0: Error handling.
[ https://issues.apache.org/jira/browse/IGNITE-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-15247: -- Description: *Motivation:* JDBC driver must throws exceptions of type java.sql.SQLException with mandatory SQLState codes (vendor code and standard, if exists). These codes must be documented. Let's * Create list of supported SQLState codes. * Map existed Ignite exceptions/error codes to proper SQLState was: *Motivation:* JDBC driver must throws exceptions of type java.sql.SQLException with mandatory SQLState codes (vendor code and standard, if exists). These codes must be documented. Let's use proper error codes in exceptions for failure cases and add tests. > JDBC driver for 3.0: Error handling. > > > Key: IGNITE-15247 > URL: https://issues.apache.org/jira/browse/IGNITE-15247 > Project: Ignite > Issue Type: New Feature > Components: jdbc >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > *Motivation:* > JDBC driver must throws exceptions of type java.sql.SQLException with > mandatory SQLState codes (vendor code and standard, if exists). > These codes must be documented. > Let's > * Create list of supported SQLState codes. > * Map existed Ignite exceptions/error codes to proper SQLState > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-15247) JDBC driver for 3.0: Error handling.
[ https://issues.apache.org/jira/browse/IGNITE-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-15247: -- Description: *Motivation:* JDBC driver must throws exceptions of type java.sql.SQLException with mandatory SQLState codes (vendor code and standard, if exists). These codes must be documented. Let's use proper error codes in exceptions for failure cases and add tests. was:Let's use proper error codes in exception for failure cases and add tests. > JDBC driver for 3.0: Error handling. > > > Key: IGNITE-15247 > URL: https://issues.apache.org/jira/browse/IGNITE-15247 > Project: Ignite > Issue Type: New Feature > Components: jdbc >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > *Motivation:* > JDBC driver must throws exceptions of type java.sql.SQLException with > mandatory SQLState codes (vendor code and standard, if exists). > These codes must be documented. > Let's use proper error codes in exceptions for failure cases and add tests. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17364) Move control utility index tasks to the core module
Aleksey Plekhanov created IGNITE-17364: -- Summary: Move control utility index tasks to the core module Key: IGNITE-17364 URL: https://issues.apache.org/jira/browse/IGNITE-17364 Project: Ignite Issue Type: Improvement Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov There are some control utility tasks (indexes list, index validate, index rebuild), which are depended on H2 and located in indexing module. But instead of H2 module classes core module classes having almost the same functionality can be used. We should rewrite these tasks to use core module classes to make tasks workable without H2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17355) CLI management of metrics
[ https://issues.apache.org/jira/browse/IGNITE-17355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey N. Gura updated IGNITE-17355: Description: It should be possible to enable or disable metrics sources by metrics source name and to get list of all metrics sources names via Ignite Shell. It should be possible per node basis as well as cluster wide Possible syntax: ignite metrics enable ignite metrics disable ignite metrics list CLI must return error in case of specified name of metrics source doesn't exists. CLI should not return any error in cases of state of metrics source wasn't changed (e.g. metrics source is allready enabled) was: It should be possible to enable or disable metrics sources by name via Ignite Shell. Possible syntax: ignite metrics enable ignite metrics disable > CLI management of metrics > - > > Key: IGNITE-17355 > URL: https://issues.apache.org/jira/browse/IGNITE-17355 > Project: Ignite > Issue Type: New Feature >Reporter: Denis Chudov >Priority: Major > Labels: ignite-3 > > It should be possible to enable or disable metrics sources by metrics source > name and to get list of all metrics sources names via Ignite Shell. > It should be possible per node basis as well as cluster wide > Possible syntax: > ignite metrics enable > ignite metrics disable > ignite metrics list > CLI must return error in case of specified name of metrics source doesn't > exists. > CLI should not return any error in cases of state of metrics source wasn't > changed (e.g. metrics source is allready enabled) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17354) Metrics framework
[ https://issues.apache.org/jira/browse/IGNITE-17354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey N. Gura updated IGNITE-17354: Description: *Metrics types* Metrics framework should provide the following metrics types: - Gauge - is an instantaneous measurement of a value provided by some existing component. Gauge should support primitive types: int, long, double - Metric - is just a wrapper on a numeric value which could be increased or decreased to some value. Metric should support primitive types: int, long, double. - Hit Rate - accumulates approximate hit rate statistics based on hits in the last time interval. - Distribution - distributes values by buckets where each bucket represent some numeric interval (Histogram in AI 2). Internal type - primitive long (should be enough). *Concurrency characteristics* For scalar numeric metrics it is enough to have atomic number (e.g. AtomicInteger) and striped number (e.g. LongAdder). Such approaches affects memory footprint and performance differently. *Design* Metrics should have the same life cycle as well as component that produces these metrics. So metrics related to some particular component should be tied together in MetricsSet. the only purpose of metrics set is provide access to metrics values from exporters. Metrics instances itself placed in MetricsSource - an entity which keeps instances of metrics and provides access to the metrics through an interface that is specific for each metrics source. A component that produces metrics must control metrics source life cycle (create it and register in metrics registry, see below). All metrics sources (it is not important, enabled or disabled metrics for particular metrics source) must be registered in metrics registry on component start and removed on component stop. MetricsSource itself produces an instance of MetricsSet which should be registered in metrics registry on event "metrics enabled" and unregistered on event "metrics disabled". Metrics registry provide an access to all metrics sets from exporters side. It is possible that metrics registry is overloaded by functionality (manage by metrics sources and metrics sets), so, probably, special component is need for these purposes (e.g. metrics manager). Each instance of metric has a name (local in some metric set) and description. So the full metric name it is a concatenation of metrics source name and metric name separated by dot. For composite metrics like distribution we should treat each metrics inside (e.g. each range) as separate metric. So the full name for each internal metric will be metrics source + dot + metric instance name + dot + range as string (e.g. 0_100). Metrics set must be immutable in order to meet the requirements described in the epic. Data structure (likely map) that is responsible for keeping enabled metrics set should be modified using copy-on-write semantics in order to avoid data races between main functionality (metrics enabling\disabling) and exporters. was: POC for metrics framework is here: [https://github.com/apache/ignite-3/tree/metrics] It is needed to be finished. There should be implemented: - metric registry - metric manager - metric sources - classes for metric types (existing + hit rate + histogram/bucket) > Metrics framework > -- > > Key: IGNITE-17354 > URL: https://issues.apache.org/jira/browse/IGNITE-17354 > Project: Ignite > Issue Type: New Feature >Reporter: Denis Chudov >Priority: Major > Labels: ignite-3 > > *Metrics types* > Metrics framework should provide the following metrics types: > - Gauge - is an instantaneous measurement of a value provided by some > existing component. Gauge should support primitive types: int, long, double > - Metric - is just a wrapper on a numeric value which could be increased or > decreased to some value. Metric should support primitive types: int, long, > double. > - Hit Rate - accumulates approximate hit rate statistics based on hits in the > last time interval. > - Distribution - distributes values by buckets where each bucket represent > some numeric interval (Histogram in AI 2). Internal type - primitive long > (should be enough). > *Concurrency characteristics* > For scalar numeric metrics it is enough to have atomic number (e.g. > AtomicInteger) and striped number (e.g. LongAdder). Such approaches affects > memory footprint and performance differently. > *Design* > Metrics should have the same life cycle as well as component that produces > these metrics. So metrics related to some particular component should be tied > together in MetricsSet. the only purpose of metrics set is provide access to > metrics values from exporters. Metrics instances itself placed in > MetricsSource - an entity which keeps instances of metrics
[jira] [Updated] (IGNITE-17353) Ignite 3 Metrics
[ https://issues.apache.org/jira/browse/IGNITE-17353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey N. Gura updated IGNITE-17353: Description: Design and implement metrics framework for Apache Ignite 3 internal purposes. *Requirements:* # Ease of maintaining existing metrics and adding new ones (related metrics should be placed together, compile time checking where possible (methods instead of text identifiers)) # Avoidance of data races (related metrics life cycle should be tied with corresponding component) # Possibility to enable\disable some subset of metrics at runtime # Metrics consistency (for similar components must be the same sets of related metrics independently of component configuration) # Reasonable performance characteristics (avoid of boxed types, concurrency characteristics accordingly to use cases, etc) On this stage metrics will be exposed through JMX interface only (JMX exporter) and metrics management will be possible through CLI (enable/disable/list) was:Design and implement metrics framework for Apache Ignite 3 internal purposes. > Ignite 3 Metrics > > > Key: IGNITE-17353 > URL: https://issues.apache.org/jira/browse/IGNITE-17353 > Project: Ignite > Issue Type: Epic >Reporter: Denis Chudov >Priority: Major > Labels: ignite-3 > > Design and implement metrics framework for Apache Ignite 3 internal purposes. > *Requirements:* > # Ease of maintaining existing metrics and adding new ones (related metrics > should be placed together, compile time checking where possible (methods > instead of text identifiers)) > # Avoidance of data races (related metrics life cycle should be tied with > corresponding component) > # Possibility to enable\disable some subset of metrics at runtime > # Metrics consistency (for similar components must be the same sets of > related metrics independently of component configuration) > # Reasonable performance characteristics (avoid of boxed types, concurrency > characteristics accordingly to use cases, etc) > On this stage metrics will be exposed through JMX interface only (JMX > exporter) and metrics management will be possible through CLI > (enable/disable/list) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17354) Metrics framework
[ https://issues.apache.org/jira/browse/IGNITE-17354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey N. Gura updated IGNITE-17354: Summary: Metrics framework (was: Metric framework ) > Metrics framework > -- > > Key: IGNITE-17354 > URL: https://issues.apache.org/jira/browse/IGNITE-17354 > Project: Ignite > Issue Type: New Feature >Reporter: Denis Chudov >Priority: Major > Labels: ignite-3 > > POC for metrics framework is here: > [https://github.com/apache/ignite-3/tree/metrics] > It is needed to be finished. There should be implemented: > - metric registry > - metric manager > - metric sources > - classes for metric types (existing + hit rate + histogram/bucket) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17353) Ignite 3 Metrics
[ https://issues.apache.org/jira/browse/IGNITE-17353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey N. Gura updated IGNITE-17353: Description: Design and implement metrics framework for Apache Ignite 3 internal purposes. > Ignite 3 Metrics > > > Key: IGNITE-17353 > URL: https://issues.apache.org/jira/browse/IGNITE-17353 > Project: Ignite > Issue Type: Epic >Reporter: Denis Chudov >Priority: Major > Labels: ignite-3 > > Design and implement metrics framework for Apache Ignite 3 internal purposes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17354) Metric framework
[ https://issues.apache.org/jira/browse/IGNITE-17354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey N. Gura updated IGNITE-17354: Description: POC for metrics framework is here: [https://github.com/apache/ignite-3/tree/metrics] It is needed to be finished. There should be implemented: - metric registry - metric manager - metric sources - classes for metric types (existing + hit rate + histogram/bucket) was: POC for metric framework is here: https://github.com/apache/ignite-3/tree/metrics It is needed to be finished. There should be implemented: - metric registry - metric manager - metric sources - classes for metric types (existing + hit rate + histogram/bucket) > Metric framework > - > > Key: IGNITE-17354 > URL: https://issues.apache.org/jira/browse/IGNITE-17354 > Project: Ignite > Issue Type: New Feature >Reporter: Denis Chudov >Priority: Major > Labels: ignite-3 > > POC for metrics framework is here: > [https://github.com/apache/ignite-3/tree/metrics] > It is needed to be finished. There should be implemented: > - metric registry > - metric manager > - metric sources > - classes for metric types (existing + hit rate + histogram/bucket) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17363) Update Ignite dependency: Maven Surefire Common
Aleksandr created IGNITE-17363: -- Summary: Update Ignite dependency: Maven Surefire Common Key: IGNITE-17363 URL: https://issues.apache.org/jira/browse/IGNITE-17363 Project: Ignite Issue Type: Improvement Reporter: Aleksandr Assignee: Aleksandr Update Ignite dependency: Maven Surefire Common 3.0.0-M4 to 3.0.0-M7 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16337) Update lucene version up to 8.11.2 or above
[ https://issues.apache.org/jira/browse/IGNITE-16337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Korol updated IGNITE-16337: Release Note: Lucene library version was updated to 8.12.0 (was: Lucene library version was updated to 8.11.0) > Update lucene version up to 8.11.2 or above > --- > > Key: IGNITE-16337 > URL: https://issues.apache.org/jira/browse/IGNITE-16337 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.11 >Reporter: Ilya Korol >Assignee: Ilya Korol >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Our Lucene version is quite outdated right now (7.4), while there are 2 major > releases available: 8.x.x with latest 8.11.1 and 9.0.0. So it would be good > to update this library because there were dozens of bugfixes, improvements > and optimizations since 7.4.x. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16337) Update lucene version up to 8.11.2 or above
[ https://issues.apache.org/jira/browse/IGNITE-16337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Korol updated IGNITE-16337: Summary: Update lucene version up to 8.11.2 or above (was: Update lucene version up to 8.11.1 or above) > Update lucene version up to 8.11.2 or above > --- > > Key: IGNITE-16337 > URL: https://issues.apache.org/jira/browse/IGNITE-16337 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.11 >Reporter: Ilya Korol >Assignee: Ilya Korol >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Our Lucene version is quite outdated right now (7.4), while there are 2 major > releases available: 8.x.x with latest 8.11.1 and 9.0.0. So it would be good > to update this library because there were dozens of bugfixes, improvements > and optimizations since 7.4.x. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17362) GridLuceneFile filename remains in the GridLuceneDirectory.fileMap after IndexOutput was closed
[
https://issues.apache.org/jira/browse/IGNITE-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Korol updated IGNITE-17362:
Description:
Found during work for IGNITE-16337
I'm not sure whether this is a bug or not, need someone who has good
understanding of lucene related machinery.
{code:java}
package org.apache.ignite.internal.processors.query.h2.opt;
// imports
public class GridLuceneDirectoryTest extends GridCommonAbstractTest {
private GridLuceneDirectory dir;
@Before
public void setUp() throws Exception {
dir = new GridLuceneDirectory(new GridUnsafeMemory(0));
}
@Test
public void testReleaseRefShouldRemoveFileNameFromDirMap() throws
IOException {
final String fileName = "foo";
assertNoFilenames();
IndexOutput indexOutput = dir.createOutput(fileName, IOContext.DEFAULT);
assertTrue("Missing created file: " + fileName,
dirFileNames().contains(fileName));
indexOutput.close();
// Now underlying file ref count is 0 that triggers
GridLuceneFile.deferredDelete()
// however dir would still hold a reference to this file in
GridLuceneDirectory.fileMap
assertNoFilenames();
assertNoPendingDeletions();
}
private List dirFileNames() {
return Arrays.asList(dir.listAll());
}
private void assertNoFilenames() {
assertTrue("Lucene Directory is not empty", dirFileNames().isEmpty());
}
private void assertNoPendingDeletions() throws IOException {
assertTrue("Pending deletions is not empty",
dir.getPendingDeletions().isEmpty());
}
}
{code}
was:
I'm not sure whether this is a bug or not, need someone who has good
understanding of lucene related machinery.
{code:java}
package org.apache.ignite.internal.processors.query.h2.opt;
// imports
public class GridLuceneDirectoryTest extends GridCommonAbstractTest {
private GridLuceneDirectory dir;
@Before
public void setUp() throws Exception {
dir = new GridLuceneDirectory(new GridUnsafeMemory(0));
}
@Test
public void testReleaseRefShouldRemoveFileNameFromDirMap() throws
IOException {
final String fileName = "foo";
assertNoFilenames();
IndexOutput indexOutput = dir.createOutput(fileName, IOContext.DEFAULT);
assertTrue("Missing created file: " + fileName,
dirFileNames().contains(fileName));
indexOutput.close();
// Now underlying file ref count is 0 that triggers
GridLuceneFile.deferredDelete()
// however dir would still hold a reference to this file in
GridLuceneDirectory.fileMap
assertNoFilenames();
assertNoPendingDeletions();
}
private List dirFileNames() {
return Arrays.asList(dir.listAll());
}
private void assertNoFilenames() {
assertTrue("Lucene Directory is not empty", dirFileNames().isEmpty());
}
private void assertNoPendingDeletions() throws IOException {
assertTrue("Pending deletions is not empty",
dir.getPendingDeletions().isEmpty());
}
}
{code}
> GridLuceneFile filename remains in the GridLuceneDirectory.fileMap after
> IndexOutput was closed
> ---
>
> Key: IGNITE-17362
> URL: https://issues.apache.org/jira/browse/IGNITE-17362
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.13
>Reporter: Ilya Korol
>Priority: Minor
>
> Found during work for IGNITE-16337
> I'm not sure whether this is a bug or not, need someone who has good
> understanding of lucene related machinery.
> {code:java}
> package org.apache.ignite.internal.processors.query.h2.opt;
> // imports
> public class GridLuceneDirectoryTest extends GridCommonAbstractTest {
> private GridLuceneDirectory dir;
> @Before
> public void setUp() throws Exception {
> dir = new GridLuceneDirectory(new GridUnsafeMemory(0));
> }
> @Test
> public void testReleaseRefShouldRemoveFileNameFromDirMap() throws
> IOException {
> final String fileName = "foo";
> assertNoFilenames();
> IndexOutput indexOutput = dir.createOutput(fileName,
> IOContext.DEFAULT);
> assertTrue("Missing created file: " + fileName,
> dirFileNames().contains(fileName));
> indexOutput.close();
> // Now underlying file ref count is 0 that triggers
> GridLuceneFile.deferredDelete()
> // however dir would still hold a reference to this file in
> GridLuceneDirectory.fileMap
> assertNoFilenames();
> assertNoPendingDeletions();
> }
> private List dirFileNames() {
> return Arrays.asList(dir.listAll());
> }
> private void assertNoFilenames() {
> assertTrue("Lucene Directory is not empty", dirFileNames().isEmpty());
> }
>
[jira] [Created] (IGNITE-17362) GridLuceneFile filename remains in the GridLuceneDirectory.fileMap after IndexOutput was closed
Ilya Korol created IGNITE-17362:
---
Summary: GridLuceneFile filename remains in the
GridLuceneDirectory.fileMap after IndexOutput was closed
Key: IGNITE-17362
URL: https://issues.apache.org/jira/browse/IGNITE-17362
Project: Ignite
Issue Type: Bug
Affects Versions: 2.13
Reporter: Ilya Korol
I'm not sure whether this is a bug or not, need someone who has good
understanding of lucene related machinery.
{code:java}
package org.apache.ignite.internal.processors.query.h2.opt;
// imports
public class GridLuceneDirectoryTest extends GridCommonAbstractTest {
private GridLuceneDirectory dir;
@Before
public void setUp() throws Exception {
dir = new GridLuceneDirectory(new GridUnsafeMemory(0));
}
@Test
public void testReleaseRefShouldRemoveFileNameFromDirMap() throws
IOException {
final String fileName = "foo";
assertNoFilenames();
IndexOutput indexOutput = dir.createOutput(fileName, IOContext.DEFAULT);
assertTrue("Missing created file: " + fileName,
dirFileNames().contains(fileName));
indexOutput.close();
// Now underlying file ref count is 0 that triggers
GridLuceneFile.deferredDelete()
// however dir would still hold a reference to this file in
GridLuceneDirectory.fileMap
assertNoFilenames();
assertNoPendingDeletions();
}
private List dirFileNames() {
return Arrays.asList(dir.listAll());
}
private void assertNoFilenames() {
assertTrue("Lucene Directory is not empty", dirFileNames().isEmpty());
}
private void assertNoPendingDeletions() throws IOException {
assertTrue("Pending deletions is not empty",
dir.getPendingDeletions().isEmpty());
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
