[jira] [Created] (IGNITE-17657) Partition data loss after rolling restart
Alexey Kukushkin created IGNITE-17657:
-
Summary: Partition data loss after rolling restart
Key: IGNITE-17657
URL: https://issues.apache.org/jira/browse/IGNITE-17657
Project: Ignite
Issue Type: Bug
Affects Versions: 2.13
Reporter: Alexey Kukushkin
*Setup*
An active 3+ node Apache Ignite 2.13 cluster with a cache with 1 backup,
enabled persistence and default partition loss policy.
*Actions*
# An application is continuously writing data to the cache
# The nodes are sequentially restarted one after another while the data is
being written to the cache. The next node is restarted only after the data
rebalancing is complete. Using the {{KeysToRebalanceLeft}} metric to monitor
rebalancing (see the
[documentation|https://ignite.apache.org/docs/latest/monitoring-metrics/metrics#monitoring-rebalancing]
for more details)
# The application reads some of the data after restarting all the nodes.
*Expected*
No data is lost since there is 1 backup and the nodes are restarted
sequentially after rebalancing is complete.
*Actual*
Sometimes (in our case in more than 50% of cases) there is a "partition data
has been lost" exception on the attempt to read the data.
*Notes*
Tried to create a JUnit reproducer (all nodes within the same JVM) for the
above scenario - no success so far.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17276) CompletableFuture#supplyAsync never completes with Ignite ExecutorService
Alexey Kukushkin created IGNITE-17276:
-
Summary: CompletableFuture#supplyAsync never completes with Ignite
ExecutorService
Key: IGNITE-17276
URL: https://issues.apache.org/jira/browse/IGNITE-17276
Project: Ignite
Issue Type: Bug
Components: compute
Affects Versions: 2.13
Reporter: Alexey Kukushkin
h3. Scenario: CompletableFuture#supplyAsync with Ignite ExecutorService
Given a cluster of two Ignite nodes
And Ignite {{ExecutorService}} for a remote node
When a task is executed using CompletableFuture#supplyAsync with the Ignite's
{{ExecutorService}}
Then the task eventually completes
Actual result: the task never completes. Waiting on the task's future blocks
forever.
Note; the scenario works fine if the task is executed on the local Ignite node.
{code:java}
public class Reproducer {
@Test
public void supplyIgniteExecutorToCompletableFuture() throws
InterruptedException, ExecutionException {
Function igniteCfgFactory = igniteName ->
new IgniteConfiguration()
.setIgniteInstanceName(igniteName)
.setDiscoverySpi(
new TcpDiscoverySpi()
.setIpFinder(new
TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
);
try (var ignored = Ignition.start(igniteCfgFactory.apply("server"));
final var ignite = Ignition.start(igniteCfgFactory.apply("app"))) {
final var igniteExecutor =
ignite.executorService(ignite.cluster().forRemotes());
final var future = CompletableFuture.supplyAsync(new Task(),
igniteExecutor);
final var result = future.get();
assertNotNull(result);
}
}
private static class Task implements Supplier {
@Override public String get() {
return UUID.randomUUID().toString();
}
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17189) Java 17: Untrusted Ignite node never exits after failure to connect to the cluster
Alexey Kukushkin created IGNITE-17189:
-
Summary: Java 17: Untrusted Ignite node never exits after failure
to connect to the cluster
Key: IGNITE-17189
URL: https://issues.apache.org/jira/browse/IGNITE-17189
Project: Ignite
Issue Type: Bug
Components: security
Affects Versions: 2.13
Reporter: Alexey Kukushkin
*Steps to reproduce*
* Start an Ignite server node with SSL enabled
* Start another Ignite node with SSL enabled and SSL keys signed by wrong CA
authority
*Expected*
*Actual*
*Reproducer*
{code:java}
@Test
public void untrustedServerNeverExitsOnFailureToConnectToCluster() {
final Function igniteConfigurationFactory
= name ->
new IgniteConfiguration()
.setIgniteInstanceName(name)
.setDiscoverySpi(
new TcpDiscoverySpi()
.setIpFinder(
new
TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500..47501"))
)
);
final var serverSslContextFactory = new SslContextFactory();
serverSslContextFactory.setProtocols("TLSv1.2");
serverSslContextFactory.setKeyStoreFilePath("server-keys.jks");
serverSslContextFactory.setKeyStorePassword("123456".toCharArray());
serverSslContextFactory.setTrustStoreFilePath("server-trust.jks");
serverSslContextFactory.setTrustStorePassword("123456".toCharArray());
final var tmpSslContextFactory = new SslContextFactory();
tmpSslContextFactory.setProtocols("TLSv1.2");
tmpSslContextFactory.setKeyStoreFilePath("tmp-keys.jks");
tmpSslContextFactory.setKeyStorePassword("123456".toCharArray());
tmpSslContextFactory.setTrustStoreFilePath("tmp-trust.jks");
tmpSslContextFactory.setTrustStorePassword("123456".toCharArray());
final var igniteConfiguration =
igniteConfigurationFactory.apply("server1")
.setSslContextFactory(tmpSslContextFactory);
final var untrustedIgniteConfiguration =
igniteConfigurationFactory.apply("server2")
.setSslContextFactory(tmpSslContextFactory);
try (final var unused = Ignition.start(igniteConfiguration)) {
try (final var unused2 =
Ignition.start(untrustedIgniteConfiguration)) {
}
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin resolved IGNITE-15241.
---
Resolution: Won't Fix
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: Ignite-H2-Vulnerabilities.png
>
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197. Black Duck SCA detects these [security
> vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
> in H2:
> !Ignite-H2-Vulnerabilities.png!
> We did preliminary real impact analysis considering how Ignite uses H2:
> * [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> store data in H2 and thus there can be no H2 backups in Ignite.
> * [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> support the {{CREATE ALIAS}} statement
> * [CVE-2021-23463|https://www.cvedetails.com/cve/CVE-2021-23463/]
> This vulnerability is not applicable to H2 in Ignite since Ignite uses H2
> version 1.4.197 and the vulnerability is applicable to H2 version 1.4.198 and
> up to 2.0.202.
> * [CVE-2022-23221|https://www.cvedetails.com/cve/CVE-2022-23221/]
> This vulnerability is not applicable to H2 in Ignite since Ignite runs H2
> in embedded mode. H2 cannot be externally exposed in embedded mode. The
> vulnerability could be exploited on the local machine where Ignite is
> running. However, this limits the severity a lot.
> * [CVE-2021-42392|https://www.cvedetails.com/cve/CVE-2021-42392/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> use and does not expose the {{org.h2.util.JdbcUtils.getConnection}} method.
>
> We realize all those vulnerabilities are not applicable to H2 in Apache
> Ignite. However, our security policies are very formal and require somehow
> addressing the security vulnerabilities anyway.
> We believe there are lots of other enterprises having the same issue. For
> example, there is another issue IGNITE-14381 referencing the same problem.
> The latest H2 1.4.200 has no vulnerabilities.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15241:
--
Description:
Upgrade H2 dependency of the ignite-indexing module to the latest version
1.4.200.
Apache Ignite SQL (module {{{}ignite-indexing{}}}) depends on H2 database
version 1.4.197. Black Duck SCA detects these [security
vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
in H2:
!Ignite-H2-Vulnerabilities.png!
We did preliminary real impact analysis considering how Ignite uses H2:
* [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
store data in H2 and thus there can be no H2 backups in Ignite.
* [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
support the {{CREATE ALIAS}} statement
* [CVE-2021-23463|https://www.cvedetails.com/cve/CVE-2021-23463/]
This vulnerability is not applicable to H2 in Ignite since Ignite uses H2
version 1.4.197 and the vulnerability is applicable to H2 version 1.4.198 and
up to 2.0.202.
* [CVE-2022-23221|https://www.cvedetails.com/cve/CVE-2022-23221/]
This vulnerability is not applicable to H2 in Ignite since Ignite runs H2 in
embedded mode. H2 cannot be externally exposed in embedded mode. The
vulnerability could be exploited on the local machine where Ignite is running.
However, this limits the severity a lot.
* [CVE-2021-42392|https://www.cvedetails.com/cve/CVE-2021-42392/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not use
and does not expose the {{org.h2.util.JdbcUtils.getConnection}} method.
We realize all those vulnerabilities are not applicable to H2 in Apache Ignite.
However, our security policies are very formal and require somehow addressing
the security vulnerabilities anyway.
We believe there are lots of other enterprises having the same issue. For
example, there is another issue IGNITE-14381 referencing the same problem.
was:
Upgrade H2 dependency of the ignite-indexing module to the latest version
1.4.200.
Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
1.4.197. Black Duck SCA detects these [security
vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
in H2:
!Ignite-H2-Vulnerabilities.png!
We did preliminary real impact analysis considering how Ignite uses H2:
* [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
store data in H2 and thus there can be no H2 backups in Ignite.
* [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
support the {{CREATE ALIAS}} statement
* [CVE-2021-23463|https://www.cvedetails.com/cve/CVE-2021-23463/]
This vulnerability is not applicable to H2 in Ignite since Ignite uses H2
version 1.4.197 and the vulnerability is applicable to H2 version 1.4.198 and
up to 2.0.202.
* [CVE-2022-23221|https://www.cvedetails.com/cve/CVE-2022-23221/]
This vulnerability is not applicable to H2 in Ignite since Ignite runs H2 in
embedded mode. H2 cannot be externally exposed in embedded mode. The
vulnerability could be exploited on the local machine where Ignite is running.
However, this limits the severity a lot.
* [CVE-2021-42392|https://www.cvedetails.com/cve/CVE-2021-42392/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
use and does not expose the {{org.h2.util.JdbcUtils.getConnection}} method.
We realize all those vulnerabilities are not applicable to H2 in Apache Ignite.
However, our security policies are very formal and require somehow addressing
the security vulnerabilities anyway.
We believe there are lots of other enterprises having the same issue. For
example, there is another issue IGNITE-14381 referencing the same problem.
The latest H2 1.4.200 has no vulnerabilities.
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: Ignite-H2-Vulnerabilities.png
>
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{{}ignite-indexing{}}}) depends on H2 database
> version 1.4.197. Black Duck SCA detects these [security
>
[jira] [Closed] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin closed IGNITE-15241.
-
Ignite Flags: (was: Docs Required,Release Notes Required)
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: Ignite-H2-Vulnerabilities.png
>
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197. Black Duck SCA detects these [security
> vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
> in H2:
> !Ignite-H2-Vulnerabilities.png!
> We did preliminary real impact analysis considering how Ignite uses H2:
> * [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> store data in H2 and thus there can be no H2 backups in Ignite.
> * [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> support the {{CREATE ALIAS}} statement
> * [CVE-2021-23463|https://www.cvedetails.com/cve/CVE-2021-23463/]
> This vulnerability is not applicable to H2 in Ignite since Ignite uses H2
> version 1.4.197 and the vulnerability is applicable to H2 version 1.4.198 and
> up to 2.0.202.
> * [CVE-2022-23221|https://www.cvedetails.com/cve/CVE-2022-23221/]
> This vulnerability is not applicable to H2 in Ignite since Ignite runs H2
> in embedded mode. H2 cannot be externally exposed in embedded mode. The
> vulnerability could be exploited on the local machine where Ignite is
> running. However, this limits the severity a lot.
> * [CVE-2021-42392|https://www.cvedetails.com/cve/CVE-2021-42392/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> use and does not expose the {{org.h2.util.JdbcUtils.getConnection}} method.
>
> We realize all those vulnerabilities are not applicable to H2 in Apache
> Ignite. However, our security policies are very formal and require somehow
> addressing the security vulnerabilities anyway.
> We believe there are lots of other enterprises having the same issue. For
> example, there is another issue IGNITE-14381 referencing the same problem.
> The latest H2 1.4.200 has no vulnerabilities.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17542992#comment-17542992
]
Alexey Kukushkin commented on IGNITE-15241:
---
The [H2's PR 2227|https://github.com/h2database/h2database/pull/2227] really
makes it impossible to upgrade to a newer H2 version where all the
vulnerabilities are addressed.
I see the following options to address the problem:
# None of the vulnerabilities is really applicable to H2 in Apache Ignite due
to specifics of how Ignite uses H2. See the impact analysis in the description
of this JIRA.
This can be used as a justification for appropriate team (DevOps, security) in
the organization to add the H2 modules used by Ignite to the list of exceptions
of the security vulnerabilities scanner.
# H2 module shading: rename the H2 module group or name and replace the
default H2 with the new one. This could be done manually or using the [Apache
Maven Shade Plugin|https://maven.apache.org/plugins/maven-shade-plugin/]. There
is not guarantee that specific security vulnerabilities scanner will not detect
such a trick but most likely it will not and the scan would be clean.
# [Calcite-based SQL
Engine|https://ignite.apache.org/docs/latest/SQL/sql-calcite] was added in
Ignite 2.13. This is an alternative to H2 and H2 could be excluded if the
Calcite-based engine is configured. The Calcite engine is in beta in release
2.13 and the community wants to announce it as production ready in release 2.14
or 2.15. However, there is no guarantee about that and the release dates are
not known at the moment of writing this comment.
However, some application development teams may consider trying the Calcite
engine and it may prove to be stable enough for them, allowing to get rid of
the H2 dependency.
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: Ignite-H2-Vulnerabilities.png
>
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197. Black Duck SCA detects these [security
> vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
> in H2:
> !Ignite-H2-Vulnerabilities.png!
> We did preliminary real impact analysis considering how Ignite uses H2:
> * [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> store data in H2 and thus there can be no H2 backups in Ignite.
> * [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> support the {{CREATE ALIAS}} statement
> * [CVE-2021-23463|https://www.cvedetails.com/cve/CVE-2021-23463/]
> This vulnerability is not applicable to H2 in Ignite since Ignite uses H2
> version 1.4.197 and the vulnerability is applicable to H2 version 1.4.198 and
> up to 2.0.202.
> * [CVE-2022-23221|https://www.cvedetails.com/cve/CVE-2022-23221/]
> This vulnerability is not applicable to H2 in Ignite since Ignite runs H2
> in embedded mode. H2 cannot be externally exposed in embedded mode. The
> vulnerability could be exploited on the local machine where Ignite is
> running. However, this limits the severity a lot.
> * [CVE-2021-42392|https://www.cvedetails.com/cve/CVE-2021-42392/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> use and does not expose the {{org.h2.util.JdbcUtils.getConnection}} method.
>
> We realize all those vulnerabilities are not applicable to H2 in Apache
> Ignite. However, our security policies are very formal and require somehow
> addressing the security vulnerabilities anyway.
> We believe there are lots of other enterprises having the same issue. For
> example, there is another issue IGNITE-14381 referencing the same problem.
> The latest H2 1.4.200 has no vulnerabilities.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15241:
--
Description:
Upgrade H2 dependency of the ignite-indexing module to the latest version
1.4.200.
Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
1.4.197. Black Duck SCA detects these [security
vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
in H2:
!Ignite-H2-Vulnerabilities.png!
We did preliminary real impact analysis considering how Ignite uses H2:
* [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
store data in H2 and thus there can be no H2 backups in Ignite.
* [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
support the {{CREATE ALIAS}} statement
* [CVE-2021-23463|https://www.cvedetails.com/cve/CVE-2021-23463/]
This vulnerability is not applicable to H2 in Ignite since Ignite uses H2
version 1.4.197 and the vulnerability is applicable to H2 version 1.4.198 and
up to 2.0.202.
* [CVE-2022-23221|https://www.cvedetails.com/cve/CVE-2022-23221/]
This vulnerability is not applicable to H2 in Ignite since Ignite runs H2 in
embedded mode. H2 cannot be externally exposed in embedded mode. The
vulnerability could be exploited on the local machine where Ignite is running.
However, this limits the severity a lot.
* [CVE-2021-42392|https://www.cvedetails.com/cve/CVE-2021-42392/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
use and does not expose the {{org.h2.util.JdbcUtils.getConnection}} method.
We realize all those vulnerabilities are not applicable to H2 in Apache Ignite.
However, our security policies are very formal and require somehow addressing
the security vulnerabilities anyway.
We believe there are lots of other enterprises having the same issue. For
example, there is another issue IGNITE-14381 referencing the same problem.
The latest H2 1.4.200 has no vulnerabilities.
was:
Upgrade H2 dependency of the ignite-indexing module to the latest version
1.4.200.
Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
1.4.197. Black Duck SCA detects these [security
vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
in H2:
!Ignite-H2-Vulnerabilities.png!
We did preliminary real impact analysis considering how Ignite uses H2:
* [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
store data in H2 and thus there can be no H2 backups in Ignite.
* [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
support the {{CREATE ALIAS}} statement
* [CVE-2021-23463|https://www.cvedetails.com/cve/CVE-2021-23463/]
This vulnerability is not applicable to H2 in Ignite since Ignite uses H2
version 1.4.197 and the vulnerability is applicable to H2 version 1.4.198 and
up to 2.0.202.
* [CVE-2022-23221|https://www.cvedetails.com/cve/CVE-2022-23221/]
We realize all those vulnerabilities are not applicable to H2 in Apache Ignite.
However, our security policies are very formal and require somehow addressing
the security vulnerabilities anyway.
We believe there are lots of other enterprises having the same issue. For
example, there is another issue IGNITE-14381 referencing the same problem.
The latest H2 1.4.200 has no vulnerabilities.
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: Ignite-H2-Vulnerabilities.png
>
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197. Black Duck SCA detects these [security
> vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
> in H2:
> !Ignite-H2-Vulnerabilities.png!
> We did preliminary real impact analysis considering how Ignite uses H2:
> * [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> store data in H2 and thus there can be no H2 backups in Ignite.
> *
[jira] [Updated] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15241:
--
Description:
Upgrade H2 dependency of the ignite-indexing module to the latest version
1.4.200.
Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
1.4.197. Black Duck SCA detects these [security
vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
in H2:
!Ignite-H2-Vulnerabilities.png!
We did preliminary real impact analysis considering how Ignite uses H2:
* [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
store data in H2 and thus there can be no H2 backups in Ignite.
* [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
support the {{CREATE ALIAS}} statement
* [CVE-2021-23463|https://www.cvedetails.com/cve/CVE-2021-23463/]
This vulnerability is not applicable to H2 in Ignite since Ignite uses H2
version 1.4.197 and the vulnerability is applicable to H2 version 1.4.198 and
up to 2.0.202.
* [CVE-2022-23221|https://www.cvedetails.com/cve/CVE-2022-23221/]
We realize all those vulnerabilities are not applicable to H2 in Apache Ignite.
However, our security policies are very formal and require somehow addressing
the security vulnerabilities anyway.
We believe there are lots of other enterprises having the same issue. For
example, there is another issue IGNITE-14381 referencing the same problem.
The latest H2 1.4.200 has no vulnerabilities.
was:
Upgrade H2 dependency of the ignite-indexing module to the latest version
1.4.200.
Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
1.4.197. Black Duck SCA detects these [security
vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
in H2:
!Ignite-H2-Vulnerabilities.png!
We did preliminary *real* impact analysis considering how Ignite uses H2:
* [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
store data in H2 and thus there can be no H2 backups in Ignite.
* [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
support the {{CREATE ALIAS}} statement
* [CVE-2021-23463|https://www.cvedetails.com/cve/CVE-2021-23463/]
This vulnerability is not applicable to H2 in Ignite since Ignite uses H2
version 1.4.197 and the vulnerability is applicable to H2 version 1.4.198 and
up to 2.0.202.
*
We realize all those vulnerabilities are not applicable to H2 in Apache Ignite.
However, our security policies are very formal and require somehow addressing
the security vulnerabilities anyway.
We believe there are lots of other enterprises having the same issue. For
example, there is another issue IGNITE-14381 referencing the same problem.
The latest H2 1.4.200 has no vulnerabilities.
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: Ignite-H2-Vulnerabilities.png
>
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197. Black Duck SCA detects these [security
> vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
> in H2:
> !Ignite-H2-Vulnerabilities.png!
> We did preliminary real impact analysis considering how Ignite uses H2:
> * [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> store data in H2 and thus there can be no H2 backups in Ignite.
> * [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> support the {{CREATE ALIAS}} statement
> * [CVE-2021-23463|https://www.cvedetails.com/cve/CVE-2021-23463/]
> This vulnerability is not applicable to H2 in Ignite since Ignite uses H2
> version 1.4.197 and the vulnerability is applicable to H2 version 1.4.198 and
> up to 2.0.202.
> * [CVE-2022-23221|https://www.cvedetails.com/cve/CVE-2022-23221/]
>
> We realize all those vulnerabilities are not applicable to H2 in Apache
> Ignite. However,
[jira] [Updated] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15241:
--
Description:
Upgrade H2 dependency of the ignite-indexing module to the latest version
1.4.200.
Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
1.4.197. Black Duck SCA detects these [security
vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
in H2:
!Ignite-H2-Vulnerabilities.png!
We did preliminary *real* impact analysis considering how Ignite uses H2:
* [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
store data in H2 and thus there can be no H2 backups in Ignite.
* [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
support the {{CREATE ALIAS}} statement
* [CVE-2021-23463|https://www.cvedetails.com/cve/CVE-2021-23463/]
This vulnerability is not applicable to H2 in Ignite since Ignite uses H2
version 1.4.197 and the vulnerability is applicable to H2 version 1.4.198 and
up to 2.0.202.
*
We realize all those vulnerabilities are not applicable to H2 in Apache Ignite.
However, our security policies are very formal and require somehow addressing
the security vulnerabilities anyway.
We believe there are lots of other enterprises having the same issue. For
example, there is another issue IGNITE-14381 referencing the same problem.
The latest H2 1.4.200 has no vulnerabilities.
was:
Upgrade H2 dependency of the ignite-indexing module to the latest version
1.4.200.
Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
1.4.197. Black Duck SCA detects these [security
vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
in H2:
!Ignite-H2-Vulnerabilities.png!
We did preliminary *real* impact analysis considering how Ignite uses H2:
* [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
store data in H2 and thus there can be no H2 backups in Ignite.
* [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
support the {{CREATE ALIAS}} statement
We realize all those vulnerabilities are not applicable to H2 in Apache Ignite.
However, our security policies are very formal and require somehow addressing
the security vulnerabilities anyway.
We believe there are lots of other enterprises having the same issue. For
example, there is another issue IGNITE-14381 referencing the same problem.
The latest H2 1.4.200 has no vulnerabilities.
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: Ignite-H2-Vulnerabilities.png
>
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197. Black Duck SCA detects these [security
> vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
> in H2:
> !Ignite-H2-Vulnerabilities.png!
> We did preliminary *real* impact analysis considering how Ignite uses H2:
> * [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> store data in H2 and thus there can be no H2 backups in Ignite.
> * [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> support the {{CREATE ALIAS}} statement
> * [CVE-2021-23463|https://www.cvedetails.com/cve/CVE-2021-23463/]
> This vulnerability is not applicable to H2 in Ignite since Ignite uses H2
> version 1.4.197 and the vulnerability is applicable to H2 version 1.4.198 and
> up to 2.0.202.
> *
> We realize all those vulnerabilities are not applicable to H2 in Apache
> Ignite. However, our security policies are very formal and require somehow
> addressing the security vulnerabilities anyway.
> We believe there are lots of other enterprises having the same issue. For
> example, there is another issue IGNITE-14381 referencing the same problem.
> The latest H2 1.4.200 has no vulnerabilities.
--
This message was sent by Atlassian Jira
[jira] [Updated] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15241:
--
Description:
Upgrade H2 dependency of the ignite-indexing module to the latest version
1.4.200.
Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
1.4.197. Black Duck SCA detects these [security
vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
in H2:
!Ignite-H2-Vulnerabilities.png!
We did preliminary *real* impact analysis considering how Ignite uses H2:
* [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
store data in H2 and thus there can be no H2 backups in Ignite.
* [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
This vulnerability is not applicable to H2 in Ignite since Ignite does not
support the {{CREATE ALIAS}} statement
We realize all those vulnerabilities are not applicable to H2 in Apache Ignite.
However, our security policies are very formal and require somehow addressing
the security vulnerabilities anyway.
We believe there are lots of other enterprises having the same issue. For
example, there is another issue IGNITE-14381 referencing the same problem.
The latest H2 1.4.200 has no vulnerabilities.
was:
Upgrade H2 dependency of the ignite-indexing module to the latest version
1.4.200.
Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
1.4.197. Black Duck SCA detects these [security
vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
in H2 (see !Ignite-H2-Vulnerabilities.png! :
* *Critical* [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
*Impact Analysis*; This vulnerability is not applicable to the H2 in Ignite
since Ignite does not store data in H2 and thus there can be no H2 backups in
Ignite.
* d !Ignite-H2-Vulnerabilities.png!
We realize all those vulnerabilities are not applicable to H2 in Apache Ignite.
However, our security policies are very formal and require somehow addressing
the security vulnerabilities anyway.
We believe there are lots of other enterprises having the same issue. For
example, there is another issue IGNITE-14381 referencing the same problem.
The latest H2 1.4.200 has no vulnerabilities.
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: Ignite-H2-Vulnerabilities.png
>
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197. Black Duck SCA detects these [security
> vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
> in H2:
> !Ignite-H2-Vulnerabilities.png!
> We did preliminary *real* impact analysis considering how Ignite uses H2:
> * [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> store data in H2 and thus there can be no H2 backups in Ignite.
> * [CVE-2018-10054|https://www.cvedetails.com/cve/CVE-2018-10054/]
> This vulnerability is not applicable to H2 in Ignite since Ignite does not
> support the {{CREATE ALIAS}} statement
> We realize all those vulnerabilities are not applicable to H2 in Apache
> Ignite. However, our security policies are very formal and require somehow
> addressing the security vulnerabilities anyway.
> We believe there are lots of other enterprises having the same issue. For
> example, there is another issue IGNITE-14381 referencing the same problem.
> The latest H2 1.4.200 has no vulnerabilities.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15241:
--
Description:
Upgrade H2 dependency of the ignite-indexing module to the latest version
1.4.200.
Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
1.4.197. Black Duck SCA detects these [security
vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
in H2 (see !Ignite-H2-Vulnerabilities.png! :
* *Critical* [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
*Impact Analysis*; This vulnerability is not applicable to the H2 in Ignite
since Ignite does not store data in H2 and thus there can be no H2 backups in
Ignite.
* d !Ignite-H2-Vulnerabilities.png!
We realize all those vulnerabilities are not applicable to H2 in Apache Ignite.
However, our security policies are very formal and require somehow addressing
the security vulnerabilities anyway.
We believe there are lots of other enterprises having the same issue. For
example, there is another issue IGNITE-14381 referencing the same problem.
The latest H2 1.4.200 has no vulnerabilities.
was:
Upgrade H2 dependency of the ignite-indexing module to the latest version
1.4.200.
Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
1.4.197, which has these two [security
vulnerabilities|https://www.cvedetails.com/vulnerability-list/vendor_id-17893/product_id-45580/year-2018/H2database-H2.html]
[CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/] is regarded as
a critical vulnerability by our analyzer (Black Duck SCA) and makes it
impossible to use Ignite SQL due to security policies. We realize this
vulnerability is probably not even applicable to the H2 in Ignite since there
is no H2 database or H2 backups in Ignite. Still the security policies are very
formal and do not allow that anyway.
We believe there are lots of other enterprises having the same issue. For
example, there is another issue IGNITE-14381 referencing the same problem.
The latest H2 1.4.200 has no vulnerabilities.
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: Ignite-H2-Vulnerabilities.png
>
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197. Black Duck SCA detects these [security
> vulnerabilities|https://www.cvedetails.com/product/45580/H2database-H2.html?vendor_id=17893]
> in H2 (see !Ignite-H2-Vulnerabilities.png! :
> * *Critical* [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/]
> *Impact Analysis*; This vulnerability is not applicable to the H2 in Ignite
> since Ignite does not store data in H2 and thus there can be no H2 backups in
> Ignite.
> * d !Ignite-H2-Vulnerabilities.png!
> We realize all those vulnerabilities are not applicable to H2 in Apache
> Ignite. However, our security policies are very formal and require somehow
> addressing the security vulnerabilities anyway.
> We believe there are lots of other enterprises having the same issue. For
> example, there is another issue IGNITE-14381 referencing the same problem.
> The latest H2 1.4.200 has no vulnerabilities.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15241:
--
Attachment: Ignite-H2-Vulnerabilities.png
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: Ignite-H2-Vulnerabilities.png
>
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197, which has these two [security
> vulnerabilities|https://www.cvedetails.com/vulnerability-list/vendor_id-17893/product_id-45580/year-2018/H2database-H2.html]
> [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/] is regarded
> as a critical vulnerability by our analyzer (Black Duck SCA) and makes it
> impossible to use Ignite SQL due to security policies. We realize this
> vulnerability is probably not even applicable to the H2 in Ignite since there
> is no H2 database or H2 backups in Ignite. Still the security policies are
> very formal and do not allow that anyway.
> We believe there are lots of other enterprises having the same issue. For
> example, there is another issue IGNITE-14381 referencing the same problem.
> The latest H2 1.4.200 has no vulnerabilities.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15241:
--
Affects Version/s: 2.13
(was: 2.10)
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197, which has these two [security
> vulnerabilities|https://www.cvedetails.com/vulnerability-list/vendor_id-17893/product_id-45580/year-2018/H2database-H2.html]
> [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/] is regarded
> as a critical vulnerability by our analyzer (Black Duck SCA) and makes it
> impossible to use Ignite SQL due to security policies. We realize this
> vulnerability is probably not even applicable to the H2 in Ignite since there
> is no H2 database or H2 backups in Ignite. Still the security policies are
> very formal and do not allow that anyway.
> We believe there are lots of other enterprises having the same issue. For
> example, there is another issue IGNITE-14381 referencing the same problem.
> The latest H2 1.4.200 has no vulnerabilities.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Reopened] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin reopened IGNITE-15241:
---
I guess the issue was closed by mistake. I do not see how the H2 pull request
mentioned in the previous comment addresses this problem
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.10
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197, which has these two [security
> vulnerabilities|https://www.cvedetails.com/vulnerability-list/vendor_id-17893/product_id-45580/year-2018/H2database-H2.html]
> [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/] is regarded
> as a critical vulnerability by our analyzer (Black Duck SCA) and makes it
> impossible to use Ignite SQL due to security policies. We realize this
> vulnerability is probably not even applicable to the H2 in Ignite since there
> is no H2 database or H2 backups in Ignite. Still the security policies are
> very formal and do not allow that anyway.
> We believe there are lots of other enterprises having the same issue. For
> example, there is another issue IGNITE-14381 referencing the same problem.
> The latest H2 1.4.200 has no vulnerabilities.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16272) IgniteOutOfMemoryException is not a critical failure
[
https://issues.apache.org/jira/browse/IGNITE-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16272:
--
Remaining Estimate: 48h (was: 40h)
Original Estimate: 48h (was: 40h)
> IgniteOutOfMemoryException is not a critical failure
>
>
> Key: IGNITE-16272
> URL: https://issues.apache.org/jira/browse/IGNITE-16272
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> h3. The Problem
> Ignite raises the
> [IgniteOutOfMemoryException|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/internal/mem/IgniteOutOfMemoryException.java]
> if a data region size is exceeded when trying to add more data to a cache.
> Ignite considers the {{IgniteOutOfMemoryException}} as a critical failure.
> This causes shutting down the Ignite server with the default failure handler.
> However, reaching the data region capacity does not seem to be such a
> critical problem requiring the server shutdown or restart. For example, in
> our application we just want to report this problem back to the users and
> notify the DevOps without applying the critical failure handler. To achieve
> that, we had to define a custom {{FailureHandler}} that detects and ignores
> the {{IgniteOutOfMemoryException}} and all the exception caused by the
> {{{}IgniteOutOfMemoryException{}}}, allowing the final exception to reach the
> application. This solution is clumsy and unreliable since it uses the
> internal {{IgniteOutOfMemoryException}} definition and relies on a complex
> secondary exception structure trying to find the
> {{IgniteOutOfMemoryException}} among the suppressed exception and causes.
> Ignite out-of-the-box failure handlers have the {{ignoredFailure}} property
> that allows filtering out some kinds of failures. However, the
> {{IgniteOutOfMemoryException}} is not among the
> [FailureType|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/failure/FailureType.java]
> that can be ignored.
> h3. The Proposal
> # Does anyone really want to treat the "data region capacity exceeded"
> problem as a critical failure and stop or restart the server?
> ** Consider never treating this condition as a critical failure. This change
> is not backward compatible.
> ** Or add another item to the {{FailureType}} enumeration to optionally
> allow the users not to have that treated as a critical failure. This is
> backward-compatible.
> # Make the {{IgniteOutOfMemoryException}} a public API (now it is in the
> {{internal}} package)
> # Consider renaming {{IgniteOutOfMemoryException}} (for example, to
> something like {{{}NotEnoughStorageException{}}}) since the current name is
> similar to a really critical and usually unrecoverable Java's
> {{OutOfMemoryError}} although the {{IgniteOutOfMemoryException}} is not that
> critical.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-15336) TLSv1.3 and Java 11
[
https://issues.apache.org/jira/browse/IGNITE-15336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15336:
--
Remaining Estimate: 64h (was: 48h)
Original Estimate: 64h (was: 48h)
> TLSv1.3 and Java 11
> ---
>
> Key: IGNITE-15336
> URL: https://issues.apache.org/jira/browse/IGNITE-15336
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.10
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 64h
> Remaining Estimate: 64h
>
> Enhance Ignite to support TLSv1.3 with Java 11.
> Ignite currently does not support TLSv1.3 with Java 11. This is
> [documented|https://ignite.apache.org/docs/latest/setup#running-ignite-with-java-11-or-later]:
>
> _TLSv1.3, which is available in Java 11, is not supported at the moment.
> Consider adding {{‑Djdk.tls.client.protocols=TLSv1.2}} if SSL between nodes
> is used._
> h3. Reproducer
> The problem can be reproduced by trying to connect an Ignite Java thin client
> to Ignite server over an secure connection.
> Add TLSv1.3 to the list of protocols in {{SSLContext}} and set the
> {{SSLContext}} in the {{ClientConnectorConfiguration}} on the server and
> {{ClientConfiguration}} on the client.
> An attempt to {{Ignition.startClient({sslContext: {protocols:
> ['TLSv1.3']}})}} fails with a {{ClientConnectionException}} saying _Channel
> is closed_.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-14326) Resetting Ignite cache entry TTL
[
https://issues.apache.org/jira/browse/IGNITE-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-14326:
--
Remaining Estimate: 64h (was: 56h)
Original Estimate: 64h (was: 56h)
> Resetting Ignite cache entry TTL
>
>
> Key: IGNITE-14326
> URL: https://issues.apache.org/jira/browse/IGNITE-14326
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.3
>Reporter: Stephen Darlington
>Assignee: Stephen Darlington
>Priority: Major
> Labels: cggg
> Original Estimate: 64h
> Remaining Estimate: 64h
>
> Ignite provides a way to specify an expiry policy on per entry level, but
> there is no way to refresh the TTL without first retrieving the record, which
> is slow and resource consuming if an entry is large.
> Provide a method to reset Ignite cache entry TTL.
> Suggested API (to be discussed with community):
> {{IgniteCache#touch(key)}}: resets the TTL using the latest TTL value or does
> nothing if no TTL was specified for the key.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16431) Entry expiration requires twice the entry size of heap
[
https://issues.apache.org/jira/browse/IGNITE-16431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16431:
--
Remaining Estimate: 64h (was: 56h)
Original Estimate: 64h (was: 56h)
> Entry expiration requires twice the entry size of heap
> --
>
> Key: IGNITE-16431
> URL: https://issues.apache.org/jira/browse/IGNITE-16431
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: 500MB-put-expiry-master.png
>
> Original Estimate: 64h
> Remaining Estimate: 64h
>
> Ignite takes twice the entry size off the heap to expire the entry when
> {{{}eagerTtl=true{}}}. See the attached heap memory usage diagram of putting
> and then expiring a 500MB entry in Ignite.
> This makes Ignite inefficient with handling large objects causing
> {{OutOfMemory}} errors.
> Do we really need loading entry's value on heap at all to expiry the entry?
> Please enhance Ignite cache entry expiration not to load the entry's value on
> heap even once or explain why it is not possible.
> !500MB-put-expiry-master.png!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16448) Optimize unacknowledged messages queue for FULL_SYNC caches
[
https://issues.apache.org/jira/browse/IGNITE-16448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16448:
--
Remaining Estimate: 80h (was: 56h)
Original Estimate: 80h (was: 56h)
> Optimize unacknowledged messages queue for FULL_SYNC caches
> ---
>
> Key: IGNITE-16448
> URL: https://issues.apache.org/jira/browse/IGNITE-16448
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: apache-ignite-msgReqs-heap-usage.png
>
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Apache Ignite queues all unacknowledged messages in the
> {{GridNioRecoveryDescriptor#msgReqs}} queue. Size of this queue is
> effectively controlled by the {{TcpCommunicationSpi.ackSendThreshold}}
> property, which is 32 by default.
> Queuing PUTs of large entries results in significant heap memory usage.
> Please check if we really need to queue PUTs into FULL_SYNC caches. It seems
> to me we should not since a failure to write to a backup would be sent back
> to the client right away without any retries. If that is the case then please
> optimize the heap usage by not queueing FULL_SYNC cache update messages.
> See the heap usage example attached: !apache-ignite-msgReqs-heap-usage.png!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-14326) Resetting Ignite cache entry TTL
[
https://issues.apache.org/jira/browse/IGNITE-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-14326:
--
Remaining Estimate: 56h (was: 40h)
Original Estimate: 56h (was: 40h)
> Resetting Ignite cache entry TTL
>
>
> Key: IGNITE-14326
> URL: https://issues.apache.org/jira/browse/IGNITE-14326
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.3
>Reporter: Stephen Darlington
>Assignee: Stephen Darlington
>Priority: Major
> Labels: cggg
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> Ignite provides a way to specify an expiry policy on per entry level, but
> there is no way to refresh the TTL without first retrieving the record, which
> is slow and resource consuming if an entry is large.
> Provide a method to reset Ignite cache entry TTL.
> Suggested API (to be discussed with community):
> {{IgniteCache#touch(key)}}: resets the TTL using the latest TTL value or does
> nothing if no TTL was specified for the key.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16137) ContainsKey operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-16137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16137:
--
Remaining Estimate: 64h (was: 40h)
Original Estimate: 64h (was: 40h)
> ContainsKey operation fetches entry value into heap
> ---
>
> Key: IGNITE-16137
> URL: https://issues.apache.org/jira/browse/IGNITE-16137
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 64h
> Remaining Estimate: 64h
>
> [See similar problem for the remove()
> operation|https://issues.apache.org/jira/browse/IGNITE-15959]
> {{IgniteCache#containsKey(key)}} operation fetches full entry into heap
> memory. This is inefficient when working with large objects: our application
> running with limited heap memory fails with {{java.lang.OutOfMemoryError:
> Java heap space}} when trying to check if a key exists.
> It seems wrong that Ignite needs to fetch the full entry on heap to check if
> the key exists. Please enhance Ignite to not be doing that or explain why
> Ignite must do that.
> h2. Reproducer
> h3. Steps
> Create a Gradle project with the below class and run it as
> {{./gradlew test --tests apache.ignite.issues.ContainsOperationHeapUsage}}
> {{build.gradle}}
> {code:groovy}
> test {
> minHeapSize = "512m"
> maxHeapSize = "512m"
> }
> {code}
> {{ContainsOperationHeapUsage.java}}
> {code:java}
> public class ContainsOperationHeapUsage {
> /** Run with -Xmx512m -Xms512m */
> @Test
> public void containsOperationFetchesValueOnHeap() {
> var igniteCfg = new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
> )
> .setCacheConfiguration(new CacheConfiguration<>("blobs"));
> try (var ignite = Ignition.start(igniteCfg)) {
> Cache cache = ignite.cache("blobs");
> // Put a BLOB having size of 35% of free memory to the cache
> Runtime.getRuntime().gc();
> var freeMemory = Runtime.getRuntime().freeMemory();
> var blobSize = (int)(freeMemory * 0.35);
> putBlob(cache, blobSize);
> // Use 70% of the free heap
> Runtime.getRuntime().gc();
> var unused = new byte[2 * blobSize];
> // Check if the blob exists in the cache.
> // This throws "OutOfMemoryError: Java heap space" since Ignite
> retrieves full entry to the heap.
> // Why does Ignite retrieve entry value to check if the key
> exists?
> cache.containsKey(1);
> }
> }
> private static void putBlob(Cache cache, int blobSize) {
> var blob = new byte[blobSize];
> cache.put(1, blob);
> }
> }
> {code}
> h3. Expected
> The test passes
> h3. Actual
> The {{cache.containsKey}} operatoin fails with:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
> at
> org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetOne.found(BPlusTree.java:3156)
> at
>
[jira] [Updated] (IGNITE-16135) Expiry policy is not applied to Ignite cache after serialization
[
https://issues.apache.org/jira/browse/IGNITE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16135:
--
Remaining Estimate: 56h (was: 40h)
Original Estimate: 56h (was: 40h)
> Expiry policy is not applied to Ignite cache after serialization
>
>
> Key: IGNITE-16135
> URL: https://issues.apache.org/jira/browse/IGNITE-16135
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> Expiry Policy specified in Ignite cache operation context is not applied to
> the cache after the cache serialization/deserialization.
> h3. Steps to Reproduce
> # Run an Ignite server with a cache without expiry policy
> # A client node joins the server, gets the cache proxy and applies expiry
> policy to the cache proxy
> # The client executes a Compute task taking the cache proxy as a parameter
> on the server.
> # The Compute task puts an entry to the cache using the cache proxy received
> from the client.
> # The client sleeps for a time period longer than the TTL specified for the
> expiry policy
> # The client checks if the entry that the Compute task added still exists
> h3. Expected
> The entry does not exist since it must expire
> h3. Actual
> The entry still exists in the cache.
> h3. Reproducer
> {code:java}
> public class IgniteCacheSerializationTest {
> /** Expiry Policy specified in Ignite cache operation context is enabled
> after the cache serialization. */
> @Test
> public void igniteCacheWithExpiryPolicySerialization() throws
> InterruptedException {
> var cacheName = "cache1";
> var ttl = 1000;
> Supplier igniteCfgFactory = () ->
> new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(
> new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500"))
> )
> )
> .setCacheConfiguration(new CacheConfiguration<>(cacheName));
> try (var ignored =
> Ignition.start(igniteCfgFactory.get().setIgniteInstanceName("server"))) {
> try (var ignite = Ignition.start(
>
> igniteCfgFactory.get().setIgniteInstanceName("test").setClientMode(true))
> ) {
> // Apply a 1-second expiry policy to the cache
> var expiryPolicy = TouchedExpiryPolicy.factoryOf(new
> Duration(TimeUnit.MILLISECONDS, ttl)).create();
> var cache = ignite. String>cache(cacheName).withExpiryPolicy(expiryPolicy);
> // Run a Compute Task that serializes the cache with the
> 1-second expiry policy applied.
> // The task adds a key 1 to the cache
> ignite.compute().affinityRun(cacheName, 1, new Put1(cache));
> // Sleep for a time twice longer than the TTL
> Thread.sleep(ttl * 2);
> // The key 1 must expiry by this time
> assertFalse(cache.containsKey(1), "The entry still exists in
> the cache");
> }
> }
> }
> private static final class Put1 implements IgniteRunnable {
> private final IgniteCache cache;
> public Put1(IgniteCache cache) {
> this.cache = cache;
> }
> @Override public void run() {
> cache.put(1, "ignored");
> }
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16715) Ignite should always use IgniteLogger
[
https://issues.apache.org/jira/browse/IGNITE-16715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16715:
--
Remaining Estimate: 56h (was: 40h)
Original Estimate: 56h (was: 40h)
> Ignite should always use IgniteLogger
> -
>
> Key: IGNITE-16715
> URL: https://issues.apache.org/jira/browse/IGNITE-16715
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> Ignite sometimes logs directly to stdout (e.g. some startup output). This
> makes it hard to setup custom logging.
> Please make Ignite always use {{IgniteLogger}} or explain why it cannot be
> done.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16716) Ignite JUL Logger should not change the configuration
[
https://issues.apache.org/jira/browse/IGNITE-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16716:
--
Remaining Estimate: 32h (was: 24h)
Original Estimate: 32h (was: 24h)
> Ignite JUL Logger should not change the configuration
> -
>
> Key: IGNITE-16716
> URL: https://issues.apache.org/jira/browse/IGNITE-16716
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 32h
> Remaining Estimate: 32h
>
> Ignite's {{JavaLogger}} forcefully re-configures logging by adding/removing
> appenders and changing severity based on some system properties and
> environment variables.
> This makes it hard or even impossible to consistently switch between
> different loggers keeping the same level of output.
> Please enhance the {{JavaLogger}} to no mess with the configuration or
> explain why that cannot be done
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16922) Getting an entry with expiry policy causes IgniteOutOfMemoryException
[
https://issues.apache.org/jira/browse/IGNITE-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16922:
--
Remaining Estimate: 64h (was: 40h)
Original Estimate: 64h (was: 40h)
> Getting an entry with expiry policy causes IgniteOutOfMemoryException
> -
>
> Key: IGNITE-16922
> URL: https://issues.apache.org/jira/browse/IGNITE-16922
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 64h
> Remaining Estimate: 64h
>
> {{IgniteCache#get(key)}} operation causes {{IgniteOutOfMemoryException}} if
> {{AccessedExpiryPolicy}} or {{TouchedExpiryPolicy}} is enabled for the
> {{key}} and Ignite has not enough storage for another entry of the same or
> bigger size.
> This happens because:
> # Ignite needs to update TTL
> # TTL is part of the entry and Ignite overwrites full entry to update the TTL
> # The problem is Ignite runs common code that checks if Ignite has enough
> storage to write the entry with updated TTL back. The check fails causing the
> {{IgniteCache#get(key)}} operation to throw {{IgniteOutOfMemoryException}}.
> # This behavior is very confusing for Ignite users: why would a "read"
> operation throw Ignite OOM?
> Can we update the TTL atomically and skip the storage size check?
> Please enhance Ignite not to throw Ignite OOM on {{get}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16431) Entry expiration requires twice the entry size of heap
[
https://issues.apache.org/jira/browse/IGNITE-16431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16431:
--
Remaining Estimate: 56h (was: 40h)
Original Estimate: 56h (was: 40h)
> Entry expiration requires twice the entry size of heap
> --
>
> Key: IGNITE-16431
> URL: https://issues.apache.org/jira/browse/IGNITE-16431
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: 500MB-put-expiry-master.png
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> Ignite takes twice the entry size off the heap to expire the entry when
> {{{}eagerTtl=true{}}}. See the attached heap memory usage diagram of putting
> and then expiring a 500MB entry in Ignite.
> This makes Ignite inefficient with handling large objects causing
> {{OutOfMemory}} errors.
> Do we really need loading entry's value on heap at all to expiry the entry?
> Please enhance Ignite cache entry expiration not to load the entry's value on
> heap even once or explain why it is not possible.
> !500MB-put-expiry-master.png!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16448) Optimize unacknowledged messages queue for FULL_SYNC caches
[
https://issues.apache.org/jira/browse/IGNITE-16448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16448:
--
Remaining Estimate: 56h (was: 40h)
Original Estimate: 56h (was: 40h)
> Optimize unacknowledged messages queue for FULL_SYNC caches
> ---
>
> Key: IGNITE-16448
> URL: https://issues.apache.org/jira/browse/IGNITE-16448
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: apache-ignite-msgReqs-heap-usage.png
>
> Original Estimate: 56h
> Remaining Estimate: 56h
>
> Apache Ignite queues all unacknowledged messages in the
> {{GridNioRecoveryDescriptor#msgReqs}} queue. Size of this queue is
> effectively controlled by the {{TcpCommunicationSpi.ackSendThreshold}}
> property, which is 32 by default.
> Queuing PUTs of large entries results in significant heap memory usage.
> Please check if we really need to queue PUTs into FULL_SYNC caches. It seems
> to me we should not since a failure to write to a backup would be sent back
> to the client right away without any retries. If that is the case then please
> optimize the heap usage by not queueing FULL_SYNC cache update messages.
> See the heap usage example attached: !apache-ignite-msgReqs-heap-usage.png!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15241:
--
Remaining Estimate: 80h (was: 336h)
Original Estimate: 80h (was: 336h)
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.10
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 80h
> Remaining Estimate: 80h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197, which has these two [security
> vulnerabilities|https://www.cvedetails.com/vulnerability-list/vendor_id-17893/product_id-45580/year-2018/H2database-H2.html]
> [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/] is regarded
> as a critical vulnerability by our analyzer (Black Duck SCA) and makes it
> impossible to use Ignite SQL due to security policies. We realize this
> vulnerability is probably not even applicable to the H2 in Ignite since there
> is no H2 database or H2 backups in Ignite. Still the security policies are
> very formal and do not allow that anyway.
> We believe there are lots of other enterprises having the same issue. For
> example, there is another issue IGNITE-14381 referencing the same problem.
> The latest H2 1.4.200 has no vulnerabilities.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16137) ContainsKey operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-16137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16137:
--
Remaining Estimate: 40h (was: 168h)
Original Estimate: 40h (was: 168h)
> ContainsKey operation fetches entry value into heap
> ---
>
> Key: IGNITE-16137
> URL: https://issues.apache.org/jira/browse/IGNITE-16137
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 40h
> Remaining Estimate: 40h
>
> [See similar problem for the remove()
> operation|https://issues.apache.org/jira/browse/IGNITE-15959]
> {{IgniteCache#containsKey(key)}} operation fetches full entry into heap
> memory. This is inefficient when working with large objects: our application
> running with limited heap memory fails with {{java.lang.OutOfMemoryError:
> Java heap space}} when trying to check if a key exists.
> It seems wrong that Ignite needs to fetch the full entry on heap to check if
> the key exists. Please enhance Ignite to not be doing that or explain why
> Ignite must do that.
> h2. Reproducer
> h3. Steps
> Create a Gradle project with the below class and run it as
> {{./gradlew test --tests apache.ignite.issues.ContainsOperationHeapUsage}}
> {{build.gradle}}
> {code:groovy}
> test {
> minHeapSize = "512m"
> maxHeapSize = "512m"
> }
> {code}
> {{ContainsOperationHeapUsage.java}}
> {code:java}
> public class ContainsOperationHeapUsage {
> /** Run with -Xmx512m -Xms512m */
> @Test
> public void containsOperationFetchesValueOnHeap() {
> var igniteCfg = new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
> )
> .setCacheConfiguration(new CacheConfiguration<>("blobs"));
> try (var ignite = Ignition.start(igniteCfg)) {
> Cache cache = ignite.cache("blobs");
> // Put a BLOB having size of 35% of free memory to the cache
> Runtime.getRuntime().gc();
> var freeMemory = Runtime.getRuntime().freeMemory();
> var blobSize = (int)(freeMemory * 0.35);
> putBlob(cache, blobSize);
> // Use 70% of the free heap
> Runtime.getRuntime().gc();
> var unused = new byte[2 * blobSize];
> // Check if the blob exists in the cache.
> // This throws "OutOfMemoryError: Java heap space" since Ignite
> retrieves full entry to the heap.
> // Why does Ignite retrieve entry value to check if the key
> exists?
> cache.containsKey(1);
> }
> }
> private static void putBlob(Cache cache, int blobSize) {
> var blob = new byte[blobSize];
> cache.put(1, blob);
> }
> }
> {code}
> h3. Expected
> The test passes
> h3. Actual
> The {{cache.containsKey}} operatoin fails with:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
> at
> org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetOne.found(BPlusTree.java:3156)
> at
>
[jira] [Updated] (IGNITE-15959) Remove operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15959:
--
Remaining Estimate: 40h (was: 168h)
Original Estimate: 40h (was: 168h)
> Remove operation fetches entry value into heap
> --
>
> Key: IGNITE-15959
> URL: https://issues.apache.org/jira/browse/IGNITE-15959
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 40h
> Remaining Estimate: 40h
>
> [See similar problem for the containsKey()
> operation|https://issues.apache.org/jira/browse/IGNITE-16137]
> {{IgniteCache#remove(key)}} operation fetches full entry into heap memory.
> This is inefficient when working with large objects: our application running
> with limited heap memory fails with {{java.lang.OutOfMemoryError: Java heap
> space}} when trying to remove an entry by key.
> It seems wrong that Ignite needs to fetch the full entry on heap to remove
> the entry. Please enhance Ignite to not be doing that or explain why Ignite
> must do that.
> h2. Reproducer
> h3. Steps
> Create a Gradle project with the below class and run it as
> {{./gradlew test --tests apache.ignite.issues.RemoveOperationHeapUsage}}
> {{build.gradle}}
> {code:groovy}
> test {
> minHeapSize = "512m"
> maxHeapSize = "512m"
> }
> {code}
> {{RemoveOperationHeapUsage.java}}
> {code:java}
> public class RemoveOperationHeapUsage {
> /** Run the test with -Xmx512m -Xms512m */
> @Test
> public void removeOperationFetchesValueOnHeap() {
> var igniteCfg = new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
> )
> .setCacheConfiguration(new CacheConfiguration<>("blobs"));
> try (var ignite = Ignition.start(igniteCfg)) {
> Cache cache = ignite.cache("blobs");
> // Put a BLOB having size of 35% of free memory to the cache
> Runtime.getRuntime().gc();
> var freeMemory = Runtime.getRuntime().freeMemory();
> var blobSize = (int)(freeMemory * 0.35);
> putBlob(cache, blobSize);
> // Use 70% of the free heap
> Runtime.getRuntime().gc();
> var unused = new byte[2 * blobSize];
> // Remove the blob from the cache.
> // This throws "OutOfMemoryError: Java heap space" since Ignite
> retrieves full entry to the heap.
> // Why does Ignite retrieve entry value to delete the entry?
> cache.remove(1);
> }
> }
> private static void putBlob(Cache cache, int blobSize) {
> var blob = new byte[blobSize];
> cache.put(1, blob);
> }
> }
> {code}
> h3. Expected
> The test passes
> h3. Actual
> The {{cache.remove}} operatoin fails with:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
> at
> org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:3987)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
> at
>
[jira] [Updated] (IGNITE-16135) Expiry policy is not applied to Ignite cache after serialization
[
https://issues.apache.org/jira/browse/IGNITE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16135:
--
Remaining Estimate: 40h (was: 168h)
Original Estimate: 40h (was: 168h)
> Expiry policy is not applied to Ignite cache after serialization
>
>
> Key: IGNITE-16135
> URL: https://issues.apache.org/jira/browse/IGNITE-16135
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 40h
> Remaining Estimate: 40h
>
> Expiry Policy specified in Ignite cache operation context is not applied to
> the cache after the cache serialization/deserialization.
> h3. Steps to Reproduce
> # Run an Ignite server with a cache without expiry policy
> # A client node joins the server, gets the cache proxy and applies expiry
> policy to the cache proxy
> # The client executes a Compute task taking the cache proxy as a parameter
> on the server.
> # The Compute task puts an entry to the cache using the cache proxy received
> from the client.
> # The client sleeps for a time period longer than the TTL specified for the
> expiry policy
> # The client checks if the entry that the Compute task added still exists
> h3. Expected
> The entry does not exist since it must expire
> h3. Actual
> The entry still exists in the cache.
> h3. Reproducer
> {code:java}
> public class IgniteCacheSerializationTest {
> /** Expiry Policy specified in Ignite cache operation context is enabled
> after the cache serialization. */
> @Test
> public void igniteCacheWithExpiryPolicySerialization() throws
> InterruptedException {
> var cacheName = "cache1";
> var ttl = 1000;
> Supplier igniteCfgFactory = () ->
> new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(
> new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500"))
> )
> )
> .setCacheConfiguration(new CacheConfiguration<>(cacheName));
> try (var ignored =
> Ignition.start(igniteCfgFactory.get().setIgniteInstanceName("server"))) {
> try (var ignite = Ignition.start(
>
> igniteCfgFactory.get().setIgniteInstanceName("test").setClientMode(true))
> ) {
> // Apply a 1-second expiry policy to the cache
> var expiryPolicy = TouchedExpiryPolicy.factoryOf(new
> Duration(TimeUnit.MILLISECONDS, ttl)).create();
> var cache = ignite. String>cache(cacheName).withExpiryPolicy(expiryPolicy);
> // Run a Compute Task that serializes the cache with the
> 1-second expiry policy applied.
> // The task adds a key 1 to the cache
> ignite.compute().affinityRun(cacheName, 1, new Put1(cache));
> // Sleep for a time twice longer than the TTL
> Thread.sleep(ttl * 2);
> // The key 1 must expiry by this time
> assertFalse(cache.containsKey(1), "The entry still exists in
> the cache");
> }
> }
> }
> private static final class Put1 implements IgniteRunnable {
> private final IgniteCache cache;
> public Put1(IgniteCache cache) {
> this.cache = cache;
> }
> @Override public void run() {
> cache.put(1, "ignored");
> }
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-14326) Resetting Ignite cache entry TTL
[
https://issues.apache.org/jira/browse/IGNITE-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-14326:
--
Remaining Estimate: 40h (was: 168h)
Original Estimate: 40h (was: 168h)
> Resetting Ignite cache entry TTL
>
>
> Key: IGNITE-14326
> URL: https://issues.apache.org/jira/browse/IGNITE-14326
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.3
>Reporter: Stephen Darlington
>Assignee: Stephen Darlington
>Priority: Major
> Labels: cggg
> Original Estimate: 40h
> Remaining Estimate: 40h
>
> Ignite provides a way to specify an expiry policy on per entry level, but
> there is no way to refresh the TTL without first retrieving the record, which
> is slow and resource consuming if an entry is large.
> Provide a method to reset Ignite cache entry TTL.
> Suggested API (to be discussed with community):
> {{IgniteCache#touch(key)}}: resets the TTL using the latest TTL value or does
> nothing if no TTL was specified for the key.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16496) SSLException: closing inbound before receiving peer's close_notify
[
https://issues.apache.org/jira/browse/IGNITE-16496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16496:
--
Remaining Estimate: 40h (was: 168h)
Original Estimate: 40h (was: 168h)
> SSLException: closing inbound before receiving peer's close_notify
> --
>
> Key: IGNITE-16496
> URL: https://issues.apache.org/jira/browse/IGNITE-16496
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 40h
> Remaining Estimate: 40h
>
> Ignite nodes output the warning below on startup when TLS protocol v1.2 is
> used:
> {noformat}
> 2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
> o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
> inbound before receiving peer's close_notify
> javax.net.ssl.SSLException: closing inbound before receiving peer's
> close_notify
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
> ~[na:na]
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
> ~[na:na]
>at
> org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
> ~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
> To reproduce the problem just start two server nodes with TLS v1.3 enabled
> and the warnings will be printed in the log before the cluster is formed.
> h3. Analysis
> The problem _probably_ happens due to [this
> code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
> calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
> alert, which TLS 1.2 is expecting (see [RFC
> 8446|https://datatracker.ietf.org/doc/html/rfc8446#section-6]). I guess the
> right approach to close an SSL socket is just calling {{Socke#close}}, which
> should properly wait/send a {{close_notify}}
> Some references to consider:
> [JDK-8215102 Closing connection to Mysql database results in
> exception|https://bugs.openjdk.java.net/browse/JDK-8215102]
> [Fix for #93590 - ignore javax.net.ssl.SSLException: closing inbound before
> receiving peer's close_notify on
> java11+|https://github.com/mysql/mysql-connector-j/pull/32]
> [JDK-8251553 Socket closure issues in migrating from JDK 8 to JDK
> 11|https://bugs.openjdk.java.net/browse/JDK-8251553]
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16448) Optimize unacknowledged messages queue for FULL_SYNC caches
[
https://issues.apache.org/jira/browse/IGNITE-16448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16448:
--
Remaining Estimate: 40h (was: 168h)
Original Estimate: 40h (was: 168h)
> Optimize unacknowledged messages queue for FULL_SYNC caches
> ---
>
> Key: IGNITE-16448
> URL: https://issues.apache.org/jira/browse/IGNITE-16448
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: apache-ignite-msgReqs-heap-usage.png
>
> Original Estimate: 40h
> Remaining Estimate: 40h
>
> Apache Ignite queues all unacknowledged messages in the
> {{GridNioRecoveryDescriptor#msgReqs}} queue. Size of this queue is
> effectively controlled by the {{TcpCommunicationSpi.ackSendThreshold}}
> property, which is 32 by default.
> Queuing PUTs of large entries results in significant heap memory usage.
> Please check if we really need to queue PUTs into FULL_SYNC caches. It seems
> to me we should not since a failure to write to a backup would be sent back
> to the client right away without any retries. If that is the case then please
> optimize the heap usage by not queueing FULL_SYNC cache update messages.
> See the heap usage example attached: !apache-ignite-msgReqs-heap-usage.png!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16272) IgniteOutOfMemoryException is not a critical failure
[
https://issues.apache.org/jira/browse/IGNITE-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16272:
--
Remaining Estimate: 40h (was: 168h)
Original Estimate: 40h (was: 168h)
> IgniteOutOfMemoryException is not a critical failure
>
>
> Key: IGNITE-16272
> URL: https://issues.apache.org/jira/browse/IGNITE-16272
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 40h
> Remaining Estimate: 40h
>
> h3. The Problem
> Ignite raises the
> [IgniteOutOfMemoryException|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/internal/mem/IgniteOutOfMemoryException.java]
> if a data region size is exceeded when trying to add more data to a cache.
> Ignite considers the {{IgniteOutOfMemoryException}} as a critical failure.
> This causes shutting down the Ignite server with the default failure handler.
> However, reaching the data region capacity does not seem to be such a
> critical problem requiring the server shutdown or restart. For example, in
> our application we just want to report this problem back to the users and
> notify the DevOps without applying the critical failure handler. To achieve
> that, we had to define a custom {{FailureHandler}} that detects and ignores
> the {{IgniteOutOfMemoryException}} and all the exception caused by the
> {{{}IgniteOutOfMemoryException{}}}, allowing the final exception to reach the
> application. This solution is clumsy and unreliable since it uses the
> internal {{IgniteOutOfMemoryException}} definition and relies on a complex
> secondary exception structure trying to find the
> {{IgniteOutOfMemoryException}} among the suppressed exception and causes.
> Ignite out-of-the-box failure handlers have the {{ignoredFailure}} property
> that allows filtering out some kinds of failures. However, the
> {{IgniteOutOfMemoryException}} is not among the
> [FailureType|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/failure/FailureType.java]
> that can be ignored.
> h3. The Proposal
> # Does anyone really want to treat the "data region capacity exceeded"
> problem as a critical failure and stop or restart the server?
> ** Consider never treating this condition as a critical failure. This change
> is not backward compatible.
> ** Or add another item to the {{FailureType}} enumeration to optionally
> allow the users not to have that treated as a critical failure. This is
> backward-compatible.
> # Make the {{IgniteOutOfMemoryException}} a public API (now it is in the
> {{internal}} package)
> # Consider renaming {{IgniteOutOfMemoryException}} (for example, to
> something like {{{}NotEnoughStorageException{}}}) since the current name is
> similar to a really critical and usually unrecoverable Java's
> {{OutOfMemoryError}} although the {{IgniteOutOfMemoryException}} is not that
> critical.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16715) Ignite should always use IgniteLogger
[
https://issues.apache.org/jira/browse/IGNITE-16715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16715:
--
Remaining Estimate: 40h (was: 72h)
Original Estimate: 40h (was: 72h)
> Ignite should always use IgniteLogger
> -
>
> Key: IGNITE-16715
> URL: https://issues.apache.org/jira/browse/IGNITE-16715
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 40h
> Remaining Estimate: 40h
>
> Ignite sometimes logs directly to stdout (e.g. some startup output). This
> makes it hard to setup custom logging.
> Please make Ignite always use {{IgniteLogger}} or explain why it cannot be
> done.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16431) Entry expiration requires twice the entry size of heap
[
https://issues.apache.org/jira/browse/IGNITE-16431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16431:
--
Remaining Estimate: 40h (was: 168h)
Original Estimate: 40h (was: 168h)
> Entry expiration requires twice the entry size of heap
> --
>
> Key: IGNITE-16431
> URL: https://issues.apache.org/jira/browse/IGNITE-16431
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: 500MB-put-expiry-master.png
>
> Original Estimate: 40h
> Remaining Estimate: 40h
>
> Ignite takes twice the entry size off the heap to expire the entry when
> {{{}eagerTtl=true{}}}. See the attached heap memory usage diagram of putting
> and then expiring a 500MB entry in Ignite.
> This makes Ignite inefficient with handling large objects causing
> {{OutOfMemory}} errors.
> Do we really need loading entry's value on heap at all to expiry the entry?
> Please enhance Ignite cache entry expiration not to load the entry's value on
> heap even once or explain why it is not possible.
> !500MB-put-expiry-master.png!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16889) Error-prone X#searchForCause() implementation
[
https://issues.apache.org/jira/browse/IGNITE-16889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16889:
--
Remaining Estimate: 16h (was: 24h)
Original Estimate: 16h (was: 24h)
> Error-prone X#searchForCause() implementation
> -
>
> Key: IGNITE-16889
> URL: https://issues.apache.org/jira/browse/IGNITE-16889
> Project: Ignite
> Issue Type: Bug
> Components: general
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 16h
> Remaining Estimate: 16h
>
> The {{X#seachForCause}} utility method fails with a {{NullPointerException}}
> if the analyzed {{Throwable}} has no {{suppressed}} exception due to the
> [missing null
> check|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/typedef/X.java#L570].
> I never encountered any issues with that with real usage of Ignite. However,
> writing a unit test that stabs the {{IgniteCache}} methods to throw an
> exception without initialized {{suppressed}} exceptions results in the
> stabbed method to return {{null}} instead of throwing an exception. The
> workaround is to properly initialize the {{suppressed}} list in the mocked
> exception.
> I suggest to make the generic {{X#seachForCause}} utility method handle the
> case when the {{suppressed}} list is not initialized. That would simplify the
> unit tests code.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16922) Getting an entry with expiry policy causes IgniteOutOfMemoryException
[
https://issues.apache.org/jira/browse/IGNITE-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16922:
--
Remaining Estimate: 40h (was: 72h)
Original Estimate: 40h (was: 72h)
> Getting an entry with expiry policy causes IgniteOutOfMemoryException
> -
>
> Key: IGNITE-16922
> URL: https://issues.apache.org/jira/browse/IGNITE-16922
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 40h
> Remaining Estimate: 40h
>
> {{IgniteCache#get(key)}} operation causes {{IgniteOutOfMemoryException}} if
> {{AccessedExpiryPolicy}} or {{TouchedExpiryPolicy}} is enabled for the
> {{key}} and Ignite has not enough storage for another entry of the same or
> bigger size.
> This happens because:
> # Ignite needs to update TTL
> # TTL is part of the entry and Ignite overwrites full entry to update the TTL
> # The problem is Ignite runs common code that checks if Ignite has enough
> storage to write the entry with updated TTL back. The check fails causing the
> {{IgniteCache#get(key)}} operation to throw {{IgniteOutOfMemoryException}}.
> # This behavior is very confusing for Ignite users: why would a "read"
> operation throw Ignite OOM?
> Can we update the TTL atomically and skip the storage size check?
> Please enhance Ignite not to throw Ignite OOM on {{get}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16716) Ignite JUL Logger should not change the configuration
[
https://issues.apache.org/jira/browse/IGNITE-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16716:
--
Remaining Estimate: 24h (was: 48h)
Original Estimate: 24h (was: 48h)
> Ignite JUL Logger should not change the configuration
> -
>
> Key: IGNITE-16716
> URL: https://issues.apache.org/jira/browse/IGNITE-16716
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Ignite's {{JavaLogger}} forcefully re-configures logging by adding/removing
> appenders and changing severity based on some system properties and
> environment variables.
> This makes it hard or even impossible to consistently switch between
> different loggers keeping the same level of output.
> Please enhance the {{JavaLogger}} to no mess with the configuration or
> explain why that cannot be done
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-15962) CacheEntryRemovedListener makes the PUT operation to create ENTRY CREATED events
[
https://issues.apache.org/jira/browse/IGNITE-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15962:
--
Remaining Estimate: 48h (was: 72h)
Original Estimate: 48h (was: 72h)
> CacheEntryRemovedListener makes the PUT operation to create ENTRY CREATED
> events
>
>
> Key: IGNITE-15962
> URL: https://issues.apache.org/jira/browse/IGNITE-15962
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: CacheContinuousQueryEntry-HeapDump.png,
> CacheContinuousQueryEntry-StackTrace.png, EventListenerHeapUsage.java,
> IgniteServer.java, build.gradle
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> {{CacheEntryRemovedListener}} configured with
> {{MutableCacheEntryListenerConfiguration.isOldValueRequired = false}} makes
> the {{Cache#put}} operation to create {{CREATED}} events on the primary
> server for a short period of time. This is inefficient when working with
> large objects: our application running with limited heap memory fails with
> {{java.lang.OutOfMemoryError: Java heap space}} after executing a {{PUT}}
> operation since the entry referencing the object is "stuck" on the heap.
> h2. Reproducer
> h3. Steps
> We need two Ignite server nodes to demonstrate the problem. A cache is
> deployed on both the nodes and both the nodes listen to the {{ENTRY REMOVED}}
> events. Both the nodes disable fetching the {{CacheEntryEvent#getOldValue}}
> by setting {{MutableCacheEntryListenerConfiguration.isOldValueRequired =
> false}}
> # Build and run the attached [^IgniteServer.java]. This server is configured
> like described above and runs until manually terminated.
> # Build the run the attached test [^EventListenerHeapUsage.java] . This test:
> ## Starts another server configured like described above
> ## Puts a large object taking 35% of the free memory. This is done in an
> isolated scope so the memory after the operation is garbage collected,
> ## Attempts to allocate 70% of the heap.
> h3. Expected
> The last "allocate 70% of the heap" operation should succeed.
> h3. Actual
> {noformat}
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> apache.ignite.issues.EventListenerHeapUsage.entryRemovedEventListenerFetchesValueOnHeap(EventListenerHeapUsage.java:33)
> {noformat}
> h2 Analysis
> For some reason configuring the event listener makes the {{IgniteCache#put}}
> operation to create and attach an instance of {{CacheContinuousQueryEntry}}
> with {{evtType=CREATED}} and {{newObject}} referencing the entry.
> The attached heap dump !CacheContinuousQueryEntry-HeapDump.png! collected on
> the {{OutOfMemoryError}} demonstrates the large object "stuck" on the heap.
> The attached {{CacheContinuousQueryEntry}} initialization
> !CacheContinuousQueryEntry-StackTrace.png! proves the {{ENTRY CREATED}} event
> is being created even though we did not configure it.
> It seems wrong that Ignite needs to attached the {{ENTRY CREATED}} event to
> the "update" message. Please enhance Ignite to not be doing that or explain
> why Ignite must do that.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-15336) TLSv1.3 and Java 11
[
https://issues.apache.org/jira/browse/IGNITE-15336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15336:
--
Remaining Estimate: 48h (was: 168h)
Original Estimate: 48h (was: 168h)
The issues is probably already fixed. The described note was removed from
Ignite documentation.
Need to confirm and close this without a fix if that is the case.
> TLSv1.3 and Java 11
> ---
>
> Key: IGNITE-15336
> URL: https://issues.apache.org/jira/browse/IGNITE-15336
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.10
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> Enhance Ignite to support TLSv1.3 with Java 11.
> Ignite currently does not support TLSv1.3 with Java 11. This is
> [documented|https://ignite.apache.org/docs/latest/setup#running-ignite-with-java-11-or-later]:
>
> _TLSv1.3, which is available in Java 11, is not supported at the moment.
> Consider adding {{‑Djdk.tls.client.protocols=TLSv1.2}} if SSL between nodes
> is used._
> h3. Reproducer
> The problem can be reproduced by trying to connect an Ignite Java thin client
> to Ignite server over an secure connection.
> Add TLSv1.3 to the list of protocols in {{SSLContext}} and set the
> {{SSLContext}} in the {{ClientConnectorConfiguration}} on the server and
> {{ClientConfiguration}} on the client.
> An attempt to {{Ignition.startClient({sslContext: {protocols:
> ['TLSv1.3']}})}} fails with a {{ClientConnectionException}} saying _Channel
> is closed_.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16715) Ignite should always use IgniteLogger
[
https://issues.apache.org/jira/browse/IGNITE-16715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16715:
--
Remaining Estimate: 72h (was: 168h)
Original Estimate: 72h (was: 168h)
> Ignite should always use IgniteLogger
> -
>
> Key: IGNITE-16715
> URL: https://issues.apache.org/jira/browse/IGNITE-16715
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> Ignite sometimes logs directly to stdout (e.g. some startup output). This
> makes it hard to setup custom logging.
> Please make Ignite always use {{IgniteLogger}} or explain why it cannot be
> done.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16716) Ignite JUL Logger should not change the configuration
[
https://issues.apache.org/jira/browse/IGNITE-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16716:
--
Remaining Estimate: 48h (was: 168h)
Original Estimate: 48h (was: 168h)
> Ignite JUL Logger should not change the configuration
> -
>
> Key: IGNITE-16716
> URL: https://issues.apache.org/jira/browse/IGNITE-16716
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> Ignite's {{JavaLogger}} forcefully re-configures logging by adding/removing
> appenders and changing severity based on some system properties and
> environment variables.
> This makes it hard or even impossible to consistently switch between
> different loggers keeping the same level of output.
> Please enhance the {{JavaLogger}} to no mess with the configuration or
> explain why that cannot be done
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16922) Getting an entry with expiry policy causes IgniteOutOfMemoryException
[
https://issues.apache.org/jira/browse/IGNITE-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16922:
--
Remaining Estimate: 72h (was: 168h)
Original Estimate: 72h (was: 168h)
> Getting an entry with expiry policy causes IgniteOutOfMemoryException
> -
>
> Key: IGNITE-16922
> URL: https://issues.apache.org/jira/browse/IGNITE-16922
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> {{IgniteCache#get(key)}} operation causes {{IgniteOutOfMemoryException}} if
> {{AccessedExpiryPolicy}} or {{TouchedExpiryPolicy}} is enabled for the
> {{key}} and Ignite has not enough storage for another entry of the same or
> bigger size.
> This happens because:
> # Ignite needs to update TTL
> # TTL is part of the entry and Ignite overwrites full entry to update the TTL
> # The problem is Ignite runs common code that checks if Ignite has enough
> storage to write the entry with updated TTL back. The check fails causing the
> {{IgniteCache#get(key)}} operation to throw {{IgniteOutOfMemoryException}}.
> # This behavior is very confusing for Ignite users: why would a "read"
> operation throw Ignite OOM?
> Can we update the TTL atomically and skip the storage size check?
> Please enhance Ignite not to throw Ignite OOM on {{get}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Closed] (IGNITE-7641) Add CacheEntry#ttl method
[
https://issues.apache.org/jira/browse/IGNITE-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin closed IGNITE-7641.
> Add CacheEntry#ttl method
> -
>
> Key: IGNITE-7641
> URL: https://issues.apache.org/jira/browse/IGNITE-7641
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Stephen Darlington
>Priority: Major
> Labels: cggg
> Time Spent: 50m
> Remaining Estimate: 168h
>
> Ignite provides a way to specify an expiry policy on per entry level, but
> there is no way to know the current TTL for a particular key.
> We can add {{CacheEntry#ttl()}} and/or {{IgniteCache#ttl(K key)}} method that
> will provide this information. Looks like it's already available via
> {{GridCacheMapEntry#ttl()}}, so we just need to properly expose it to public
> API.
> Here is the user forum discussion about this:
> http://apache-ignite-users.70518.x6.nabble.com/Get-TTL-of-the-specific-K-V-entry-td19817.html
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (IGNITE-7641) Add CacheEntry#ttl method
[
https://issues.apache.org/jira/browse/IGNITE-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin resolved IGNITE-7641.
--
Resolution: Duplicate
The requirement is described in IGNITE-14326. Closing this issue as a duplicate
> Add CacheEntry#ttl method
> -
>
> Key: IGNITE-7641
> URL: https://issues.apache.org/jira/browse/IGNITE-7641
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Stephen Darlington
>Priority: Major
> Labels: cggg
> Time Spent: 50m
> Remaining Estimate: 168h
>
> Ignite provides a way to specify an expiry policy on per entry level, but
> there is no way to know the current TTL for a particular key.
> We can add {{CacheEntry#ttl()}} and/or {{IgniteCache#ttl(K key)}} method that
> will provide this information. Looks like it's already available via
> {{GridCacheMapEntry#ttl()}}, so we just need to properly expose it to public
> API.
> Here is the user forum discussion about this:
> http://apache-ignite-users.70518.x6.nabble.com/Get-TTL-of-the-specific-K-V-entry-td19817.html
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-14326) Resetting Ignite cache entry TTL
[
https://issues.apache.org/jira/browse/IGNITE-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-14326:
--
Description:
Ignite provides a way to specify an expiry policy on per entry level, but there
is no way to refresh the TTL without first retrieving the record, which is slow
and resource consuming if an entry is large.
Provide a method to reset Ignite cache entry TTL.
Suggested API (to be discussed with community):
{{IgniteCache#touch(key)}}: resets the TTL using the latest TTL value or does
nothing if no TTL was specified for the key.
was:
Ignite provides a way to specify an expiry policy on per entry level, but there
is no way to know the current TTL for a particular key or refresh the TTL
without first retrieving the record.
Ticket IGNITE-7641 details the IgniteCache#ttl() method. This ticket is about
_setting_ the TTL.
The API for this is less well defined than the get TTL method. Suggest a number
of options:
* Update using the default TTL policy
* Update using a specified long
* Update using a specified Expiry policy
> Resetting Ignite cache entry TTL
>
>
> Key: IGNITE-14326
> URL: https://issues.apache.org/jira/browse/IGNITE-14326
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.3
>Reporter: Stephen Darlington
>Assignee: Stephen Darlington
>Priority: Major
> Labels: cggg
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Ignite provides a way to specify an expiry policy on per entry level, but
> there is no way to refresh the TTL without first retrieving the record, which
> is slow and resource consuming if an entry is large.
> Provide a method to reset Ignite cache entry TTL.
> Suggested API (to be discussed with community):
> {{IgniteCache#touch(key)}}: resets the TTL using the latest TTL value or does
> nothing if no TTL was specified for the key.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-14326) Resetting Ignite cache entry TTL
[ https://issues.apache.org/jira/browse/IGNITE-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kukushkin updated IGNITE-14326: -- Summary: Resetting Ignite cache entry TTL (was: Add CacheEntry#setTtl method) > Resetting Ignite cache entry TTL > > > Key: IGNITE-14326 > URL: https://issues.apache.org/jira/browse/IGNITE-14326 > Project: Ignite > Issue Type: Improvement > Components: cache >Affects Versions: 2.3 >Reporter: Stephen Darlington >Assignee: Stephen Darlington >Priority: Major > Labels: cggg > Original Estimate: 168h > Remaining Estimate: 168h > > Ignite provides a way to specify an expiry policy on per entry level, but > there is no way to know the current TTL for a particular key or refresh the > TTL without first retrieving the record. > Ticket IGNITE-7641 details the IgniteCache#ttl() method. This ticket is about > _setting_ the TTL. > The API for this is less well defined than the get TTL method. Suggest a > number of options: > * Update using the default TTL policy > * Update using a specified long > * Update using a specified Expiry policy -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (IGNITE-15336) TLSv1.3 and Java 11
[
https://issues.apache.org/jira/browse/IGNITE-15336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15336:
--
Remaining Estimate: 168h (was: 336h)
Original Estimate: 168h (was: 336h)
> TLSv1.3 and Java 11
> ---
>
> Key: IGNITE-15336
> URL: https://issues.apache.org/jira/browse/IGNITE-15336
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.10
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Enhance Ignite to support TLSv1.3 with Java 11.
> Ignite currently does not support TLSv1.3 with Java 11. This is
> [documented|https://ignite.apache.org/docs/latest/setup#running-ignite-with-java-11-or-later]:
>
> _TLSv1.3, which is available in Java 11, is not supported at the moment.
> Consider adding {{‑Djdk.tls.client.protocols=TLSv1.2}} if SSL between nodes
> is used._
> h3. Reproducer
> The problem can be reproduced by trying to connect an Ignite Java thin client
> to Ignite server over an secure connection.
> Add TLSv1.3 to the list of protocols in {{SSLContext}} and set the
> {{SSLContext}} in the {{ClientConnectorConfiguration}} on the server and
> {{ClientConfiguration}} on the client.
> An attempt to {{Ignition.startClient({sslContext: {protocols:
> ['TLSv1.3']}})}} fails with a {{ClientConnectionException}} saying _Channel
> is closed_.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-15336) TLSv1.3 and Java 11
[
https://issues.apache.org/jira/browse/IGNITE-15336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15336:
--
Remaining Estimate: 336h
Original Estimate: 336h
> TLSv1.3 and Java 11
> ---
>
> Key: IGNITE-15336
> URL: https://issues.apache.org/jira/browse/IGNITE-15336
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.10
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Enhance Ignite to support TLSv1.3 with Java 11.
> Ignite currently does not support TLSv1.3 with Java 11. This is
> [documented|https://ignite.apache.org/docs/latest/setup#running-ignite-with-java-11-or-later]:
>
> _TLSv1.3, which is available in Java 11, is not supported at the moment.
> Consider adding {{‑Djdk.tls.client.protocols=TLSv1.2}} if SSL between nodes
> is used._
> h3. Reproducer
> The problem can be reproduced by trying to connect an Ignite Java thin client
> to Ignite server over an secure connection.
> Add TLSv1.3 to the list of protocols in {{SSLContext}} and set the
> {{SSLContext}} in the {{ClientConnectorConfiguration}} on the server and
> {{ClientConfiguration}} on the client.
> An attempt to {{Ignition.startClient({sslContext: {protocols:
> ['TLSv1.3']}})}} fails with a {{ClientConnectionException}} saying _Channel
> is closed_.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-14326) Add CacheEntry#setTtl method
[ https://issues.apache.org/jira/browse/IGNITE-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kukushkin updated IGNITE-14326: -- Remaining Estimate: 168h Original Estimate: 168h > Add CacheEntry#setTtl method > > > Key: IGNITE-14326 > URL: https://issues.apache.org/jira/browse/IGNITE-14326 > Project: Ignite > Issue Type: Improvement > Components: cache >Affects Versions: 2.3 >Reporter: Stephen Darlington >Assignee: Stephen Darlington >Priority: Major > Labels: cggg > Original Estimate: 168h > Remaining Estimate: 168h > > Ignite provides a way to specify an expiry policy on per entry level, but > there is no way to know the current TTL for a particular key or refresh the > TTL without first retrieving the record. > Ticket IGNITE-7641 details the IgniteCache#ttl() method. This ticket is about > _setting_ the TTL. > The API for this is less well defined than the get TTL method. Suggest a > number of options: > * Update using the default TTL policy > * Update using a specified long > * Update using a specified Expiry policy -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15241:
--
Remaining Estimate: 336h
Original Estimate: 336h
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.10
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197, which has these two [security
> vulnerabilities|https://www.cvedetails.com/vulnerability-list/vendor_id-17893/product_id-45580/year-2018/H2database-H2.html]
> [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/] is regarded
> as a critical vulnerability by our analyzer (Black Duck SCA) and makes it
> impossible to use Ignite SQL due to security policies. We realize this
> vulnerability is probably not even applicable to the H2 in Ignite since there
> is no H2 database or H2 backups in Ignite. Still the security policies are
> very formal and do not allow that anyway.
> We believe there are lots of other enterprises having the same issue. For
> example, there is another issue IGNITE-14381 referencing the same problem.
> The latest H2 1.4.200 has no vulnerabilities.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-7641) Add CacheEntry#ttl method
[
https://issues.apache.org/jira/browse/IGNITE-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-7641:
-
Remaining Estimate: 168h (was: 0h)
> Add CacheEntry#ttl method
> -
>
> Key: IGNITE-7641
> URL: https://issues.apache.org/jira/browse/IGNITE-7641
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Stephen Darlington
>Priority: Major
> Labels: cggg
> Time Spent: 50m
> Remaining Estimate: 168h
>
> Ignite provides a way to specify an expiry policy on per entry level, but
> there is no way to know the current TTL for a particular key.
> We can add {{CacheEntry#ttl()}} and/or {{IgniteCache#ttl(K key)}} method that
> will provide this information. Looks like it's already available via
> {{GridCacheMapEntry#ttl()}}, so we just need to properly expose it to public
> API.
> Here is the user forum discussion about this:
> http://apache-ignite-users.70518.x6.nabble.com/Get-TTL-of-the-specific-K-V-entry-td19817.html
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-15959) Remove operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15959:
--
Remaining Estimate: 168h
Original Estimate: 168h
> Remove operation fetches entry value into heap
> --
>
> Key: IGNITE-15959
> URL: https://issues.apache.org/jira/browse/IGNITE-15959
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> [See similar problem for the containsKey()
> operation|https://issues.apache.org/jira/browse/IGNITE-16137]
> {{IgniteCache#remove(key)}} operation fetches full entry into heap memory.
> This is inefficient when working with large objects: our application running
> with limited heap memory fails with {{java.lang.OutOfMemoryError: Java heap
> space}} when trying to remove an entry by key.
> It seems wrong that Ignite needs to fetch the full entry on heap to remove
> the entry. Please enhance Ignite to not be doing that or explain why Ignite
> must do that.
> h2. Reproducer
> h3. Steps
> Create a Gradle project with the below class and run it as
> {{./gradlew test --tests apache.ignite.issues.RemoveOperationHeapUsage}}
> {{build.gradle}}
> {code:groovy}
> test {
> minHeapSize = "512m"
> maxHeapSize = "512m"
> }
> {code}
> {{RemoveOperationHeapUsage.java}}
> {code:java}
> public class RemoveOperationHeapUsage {
> /** Run the test with -Xmx512m -Xms512m */
> @Test
> public void removeOperationFetchesValueOnHeap() {
> var igniteCfg = new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
> )
> .setCacheConfiguration(new CacheConfiguration<>("blobs"));
> try (var ignite = Ignition.start(igniteCfg)) {
> Cache cache = ignite.cache("blobs");
> // Put a BLOB having size of 35% of free memory to the cache
> Runtime.getRuntime().gc();
> var freeMemory = Runtime.getRuntime().freeMemory();
> var blobSize = (int)(freeMemory * 0.35);
> putBlob(cache, blobSize);
> // Use 70% of the free heap
> Runtime.getRuntime().gc();
> var unused = new byte[2 * blobSize];
> // Remove the blob from the cache.
> // This throws "OutOfMemoryError: Java heap space" since Ignite
> retrieves full entry to the heap.
> // Why does Ignite retrieve entry value to delete the entry?
> cache.remove(1);
> }
> }
> private static void putBlob(Cache cache, int blobSize) {
> var blob = new byte[blobSize];
> cache.put(1, blob);
> }
> }
> {code}
> h3. Expected
> The test passes
> h3. Actual
> The {{cache.remove}} operatoin fails with:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
> at
> org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:3987)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
> at
>
[jira] [Updated] (IGNITE-15962) CacheEntryRemovedListener makes the PUT operation to create ENTRY CREATED events
[
https://issues.apache.org/jira/browse/IGNITE-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15962:
--
Remaining Estimate: 72h
Original Estimate: 72h
> CacheEntryRemovedListener makes the PUT operation to create ENTRY CREATED
> events
>
>
> Key: IGNITE-15962
> URL: https://issues.apache.org/jira/browse/IGNITE-15962
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: CacheContinuousQueryEntry-HeapDump.png,
> CacheContinuousQueryEntry-StackTrace.png, EventListenerHeapUsage.java,
> IgniteServer.java, build.gradle
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> {{CacheEntryRemovedListener}} configured with
> {{MutableCacheEntryListenerConfiguration.isOldValueRequired = false}} makes
> the {{Cache#put}} operation to create {{CREATED}} events on the primary
> server for a short period of time. This is inefficient when working with
> large objects: our application running with limited heap memory fails with
> {{java.lang.OutOfMemoryError: Java heap space}} after executing a {{PUT}}
> operation since the entry referencing the object is "stuck" on the heap.
> h2. Reproducer
> h3. Steps
> We need two Ignite server nodes to demonstrate the problem. A cache is
> deployed on both the nodes and both the nodes listen to the {{ENTRY REMOVED}}
> events. Both the nodes disable fetching the {{CacheEntryEvent#getOldValue}}
> by setting {{MutableCacheEntryListenerConfiguration.isOldValueRequired =
> false}}
> # Build and run the attached [^IgniteServer.java]. This server is configured
> like described above and runs until manually terminated.
> # Build the run the attached test [^EventListenerHeapUsage.java] . This test:
> ## Starts another server configured like described above
> ## Puts a large object taking 35% of the free memory. This is done in an
> isolated scope so the memory after the operation is garbage collected,
> ## Attempts to allocate 70% of the heap.
> h3. Expected
> The last "allocate 70% of the heap" operation should succeed.
> h3. Actual
> {noformat}
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> apache.ignite.issues.EventListenerHeapUsage.entryRemovedEventListenerFetchesValueOnHeap(EventListenerHeapUsage.java:33)
> {noformat}
> h2 Analysis
> For some reason configuring the event listener makes the {{IgniteCache#put}}
> operation to create and attach an instance of {{CacheContinuousQueryEntry}}
> with {{evtType=CREATED}} and {{newObject}} referencing the entry.
> The attached heap dump !CacheContinuousQueryEntry-HeapDump.png! collected on
> the {{OutOfMemoryError}} demonstrates the large object "stuck" on the heap.
> The attached {{CacheContinuousQueryEntry}} initialization
> !CacheContinuousQueryEntry-StackTrace.png! proves the {{ENTRY CREATED}} event
> is being created even though we did not configure it.
> It seems wrong that Ignite needs to attached the {{ENTRY CREATED}} event to
> the "update" message. Please enhance Ignite to not be doing that or explain
> why Ignite must do that.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16137) ContainsKey operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-16137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16137:
--
Remaining Estimate: 168h
Original Estimate: 168h
> ContainsKey operation fetches entry value into heap
> ---
>
> Key: IGNITE-16137
> URL: https://issues.apache.org/jira/browse/IGNITE-16137
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> [See similar problem for the remove()
> operation|https://issues.apache.org/jira/browse/IGNITE-15959]
> {{IgniteCache#containsKey(key)}} operation fetches full entry into heap
> memory. This is inefficient when working with large objects: our application
> running with limited heap memory fails with {{java.lang.OutOfMemoryError:
> Java heap space}} when trying to check if a key exists.
> It seems wrong that Ignite needs to fetch the full entry on heap to check if
> the key exists. Please enhance Ignite to not be doing that or explain why
> Ignite must do that.
> h2. Reproducer
> h3. Steps
> Create a Gradle project with the below class and run it as
> {{./gradlew test --tests apache.ignite.issues.ContainsOperationHeapUsage}}
> {{build.gradle}}
> {code:groovy}
> test {
> minHeapSize = "512m"
> maxHeapSize = "512m"
> }
> {code}
> {{ContainsOperationHeapUsage.java}}
> {code:java}
> public class ContainsOperationHeapUsage {
> /** Run with -Xmx512m -Xms512m */
> @Test
> public void containsOperationFetchesValueOnHeap() {
> var igniteCfg = new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
> )
> .setCacheConfiguration(new CacheConfiguration<>("blobs"));
> try (var ignite = Ignition.start(igniteCfg)) {
> Cache cache = ignite.cache("blobs");
> // Put a BLOB having size of 35% of free memory to the cache
> Runtime.getRuntime().gc();
> var freeMemory = Runtime.getRuntime().freeMemory();
> var blobSize = (int)(freeMemory * 0.35);
> putBlob(cache, blobSize);
> // Use 70% of the free heap
> Runtime.getRuntime().gc();
> var unused = new byte[2 * blobSize];
> // Check if the blob exists in the cache.
> // This throws "OutOfMemoryError: Java heap space" since Ignite
> retrieves full entry to the heap.
> // Why does Ignite retrieve entry value to check if the key
> exists?
> cache.containsKey(1);
> }
> }
> private static void putBlob(Cache cache, int blobSize) {
> var blob = new byte[blobSize];
> cache.put(1, blob);
> }
> }
> {code}
> h3. Expected
> The test passes
> h3. Actual
> The {{cache.containsKey}} operatoin fails with:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
> at
> org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetOne.found(BPlusTree.java:3156)
> at
>
[jira] [Updated] (IGNITE-16272) IgniteOutOfMemoryException is not a critical failure
[
https://issues.apache.org/jira/browse/IGNITE-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16272:
--
Remaining Estimate: 168h
Original Estimate: 168h
> IgniteOutOfMemoryException is not a critical failure
>
>
> Key: IGNITE-16272
> URL: https://issues.apache.org/jira/browse/IGNITE-16272
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> h3. The Problem
> Ignite raises the
> [IgniteOutOfMemoryException|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/internal/mem/IgniteOutOfMemoryException.java]
> if a data region size is exceeded when trying to add more data to a cache.
> Ignite considers the {{IgniteOutOfMemoryException}} as a critical failure.
> This causes shutting down the Ignite server with the default failure handler.
> However, reaching the data region capacity does not seem to be such a
> critical problem requiring the server shutdown or restart. For example, in
> our application we just want to report this problem back to the users and
> notify the DevOps without applying the critical failure handler. To achieve
> that, we had to define a custom {{FailureHandler}} that detects and ignores
> the {{IgniteOutOfMemoryException}} and all the exception caused by the
> {{{}IgniteOutOfMemoryException{}}}, allowing the final exception to reach the
> application. This solution is clumsy and unreliable since it uses the
> internal {{IgniteOutOfMemoryException}} definition and relies on a complex
> secondary exception structure trying to find the
> {{IgniteOutOfMemoryException}} among the suppressed exception and causes.
> Ignite out-of-the-box failure handlers have the {{ignoredFailure}} property
> that allows filtering out some kinds of failures. However, the
> {{IgniteOutOfMemoryException}} is not among the
> [FailureType|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/failure/FailureType.java]
> that can be ignored.
> h3. The Proposal
> # Does anyone really want to treat the "data region capacity exceeded"
> problem as a critical failure and stop or restart the server?
> ** Consider never treating this condition as a critical failure. This change
> is not backward compatible.
> ** Or add another item to the {{FailureType}} enumeration to optionally
> allow the users not to have that treated as a critical failure. This is
> backward-compatible.
> # Make the {{IgniteOutOfMemoryException}} a public API (now it is in the
> {{internal}} package)
> # Consider renaming {{IgniteOutOfMemoryException}} (for example, to
> something like {{{}NotEnoughStorageException{}}}) since the current name is
> similar to a really critical and usually unrecoverable Java's
> {{OutOfMemoryError}} although the {{IgniteOutOfMemoryException}} is not that
> critical.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16135) Expiry policy is not applied to Ignite cache after serialization
[
https://issues.apache.org/jira/browse/IGNITE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16135:
--
Remaining Estimate: 168h
Original Estimate: 168h
> Expiry policy is not applied to Ignite cache after serialization
>
>
> Key: IGNITE-16135
> URL: https://issues.apache.org/jira/browse/IGNITE-16135
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Expiry Policy specified in Ignite cache operation context is not applied to
> the cache after the cache serialization/deserialization.
> h3. Steps to Reproduce
> # Run an Ignite server with a cache without expiry policy
> # A client node joins the server, gets the cache proxy and applies expiry
> policy to the cache proxy
> # The client executes a Compute task taking the cache proxy as a parameter
> on the server.
> # The Compute task puts an entry to the cache using the cache proxy received
> from the client.
> # The client sleeps for a time period longer than the TTL specified for the
> expiry policy
> # The client checks if the entry that the Compute task added still exists
> h3. Expected
> The entry does not exist since it must expire
> h3. Actual
> The entry still exists in the cache.
> h3. Reproducer
> {code:java}
> public class IgniteCacheSerializationTest {
> /** Expiry Policy specified in Ignite cache operation context is enabled
> after the cache serialization. */
> @Test
> public void igniteCacheWithExpiryPolicySerialization() throws
> InterruptedException {
> var cacheName = "cache1";
> var ttl = 1000;
> Supplier igniteCfgFactory = () ->
> new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(
> new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500"))
> )
> )
> .setCacheConfiguration(new CacheConfiguration<>(cacheName));
> try (var ignored =
> Ignition.start(igniteCfgFactory.get().setIgniteInstanceName("server"))) {
> try (var ignite = Ignition.start(
>
> igniteCfgFactory.get().setIgniteInstanceName("test").setClientMode(true))
> ) {
> // Apply a 1-second expiry policy to the cache
> var expiryPolicy = TouchedExpiryPolicy.factoryOf(new
> Duration(TimeUnit.MILLISECONDS, ttl)).create();
> var cache = ignite. String>cache(cacheName).withExpiryPolicy(expiryPolicy);
> // Run a Compute Task that serializes the cache with the
> 1-second expiry policy applied.
> // The task adds a key 1 to the cache
> ignite.compute().affinityRun(cacheName, 1, new Put1(cache));
> // Sleep for a time twice longer than the TTL
> Thread.sleep(ttl * 2);
> // The key 1 must expiry by this time
> assertFalse(cache.containsKey(1), "The entry still exists in
> the cache");
> }
> }
> }
> private static final class Put1 implements IgniteRunnable {
> private final IgniteCache cache;
> public Put1(IgniteCache cache) {
> this.cache = cache;
> }
> @Override public void run() {
> cache.put(1, "ignored");
> }
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16448) Optimize unacknowledged messages queue for FULL_SYNC caches
[
https://issues.apache.org/jira/browse/IGNITE-16448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16448:
--
Remaining Estimate: 168h
Original Estimate: 168h
> Optimize unacknowledged messages queue for FULL_SYNC caches
> ---
>
> Key: IGNITE-16448
> URL: https://issues.apache.org/jira/browse/IGNITE-16448
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: apache-ignite-msgReqs-heap-usage.png
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Apache Ignite queues all unacknowledged messages in the
> {{GridNioRecoveryDescriptor#msgReqs}} queue. Size of this queue is
> effectively controlled by the {{TcpCommunicationSpi.ackSendThreshold}}
> property, which is 32 by default.
> Queuing PUTs of large entries results in significant heap memory usage.
> Please check if we really need to queue PUTs into FULL_SYNC caches. It seems
> to me we should not since a failure to write to a backup would be sent back
> to the client right away without any retries. If that is the case then please
> optimize the heap usage by not queueing FULL_SYNC cache update messages.
> See the heap usage example attached: !apache-ignite-msgReqs-heap-usage.png!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16496) SSLException: closing inbound before receiving peer's close_notify
[
https://issues.apache.org/jira/browse/IGNITE-16496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16496:
--
Remaining Estimate: 168h
Original Estimate: 168h
> SSLException: closing inbound before receiving peer's close_notify
> --
>
> Key: IGNITE-16496
> URL: https://issues.apache.org/jira/browse/IGNITE-16496
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Ignite nodes output the warning below on startup when TLS protocol v1.2 is
> used:
> {noformat}
> 2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
> o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
> inbound before receiving peer's close_notify
> javax.net.ssl.SSLException: closing inbound before receiving peer's
> close_notify
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
> ~[na:na]
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
> ~[na:na]
>at
> org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
> ~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
> To reproduce the problem just start two server nodes with TLS v1.3 enabled
> and the warnings will be printed in the log before the cluster is formed.
> h3. Analysis
> The problem _probably_ happens due to [this
> code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
> calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
> alert, which TLS 1.2 is expecting (see [RFC
> 8446|https://datatracker.ietf.org/doc/html/rfc8446#section-6]). I guess the
> right approach to close an SSL socket is just calling {{Socke#close}}, which
> should properly wait/send a {{close_notify}}
> Some references to consider:
> [JDK-8215102 Closing connection to Mysql database results in
> exception|https://bugs.openjdk.java.net/browse/JDK-8215102]
> [Fix for #93590 - ignore javax.net.ssl.SSLException: closing inbound before
> receiving peer's close_notify on
> java11+|https://github.com/mysql/mysql-connector-j/pull/32]
> [JDK-8251553 Socket closure issues in migrating from JDK 8 to JDK
> 11|https://bugs.openjdk.java.net/browse/JDK-8251553]
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16431) Entry expiration requires twice the entry size of heap
[
https://issues.apache.org/jira/browse/IGNITE-16431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16431:
--
Remaining Estimate: 168h
Original Estimate: 168h
> Entry expiration requires twice the entry size of heap
> --
>
> Key: IGNITE-16431
> URL: https://issues.apache.org/jira/browse/IGNITE-16431
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: 500MB-put-expiry-master.png
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Ignite takes twice the entry size off the heap to expire the entry when
> {{{}eagerTtl=true{}}}. See the attached heap memory usage diagram of putting
> and then expiring a 500MB entry in Ignite.
> This makes Ignite inefficient with handling large objects causing
> {{OutOfMemory}} errors.
> Do we really need loading entry's value on heap at all to expiry the entry?
> Please enhance Ignite cache entry expiration not to load the entry's value on
> heap even once or explain why it is not possible.
> !500MB-put-expiry-master.png!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16715) Ignite should always use IgniteLogger
[
https://issues.apache.org/jira/browse/IGNITE-16715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16715:
--
Remaining Estimate: 168h
Original Estimate: 168h
> Ignite should always use IgniteLogger
> -
>
> Key: IGNITE-16715
> URL: https://issues.apache.org/jira/browse/IGNITE-16715
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Ignite sometimes logs directly to stdout (e.g. some startup output). This
> makes it hard to setup custom logging.
> Please make Ignite always use {{IgniteLogger}} or explain why it cannot be
> done.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16889) Error-prone X#searchForCause() implementation
[
https://issues.apache.org/jira/browse/IGNITE-16889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16889:
--
Remaining Estimate: 24h
Original Estimate: 24h
> Error-prone X#searchForCause() implementation
> -
>
> Key: IGNITE-16889
> URL: https://issues.apache.org/jira/browse/IGNITE-16889
> Project: Ignite
> Issue Type: Bug
> Components: general
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The {{X#seachForCause}} utility method fails with a {{NullPointerException}}
> if the analyzed {{Throwable}} has no {{suppressed}} exception due to the
> [missing null
> check|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/typedef/X.java#L570].
> I never encountered any issues with that with real usage of Ignite. However,
> writing a unit test that stabs the {{IgniteCache}} methods to throw an
> exception without initialized {{suppressed}} exceptions results in the
> stabbed method to return {{null}} instead of throwing an exception. The
> workaround is to properly initialize the {{suppressed}} list in the mocked
> exception.
> I suggest to make the generic {{X#seachForCause}} utility method handle the
> case when the {{suppressed}} list is not initialized. That would simplify the
> unit tests code.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16716) Ignite JUL Logger should not change the configuration
[
https://issues.apache.org/jira/browse/IGNITE-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16716:
--
Remaining Estimate: 168h
Original Estimate: 168h
> Ignite JUL Logger should not change the configuration
> -
>
> Key: IGNITE-16716
> URL: https://issues.apache.org/jira/browse/IGNITE-16716
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Ignite's {{JavaLogger}} forcefully re-configures logging by adding/removing
> appenders and changing severity based on some system properties and
> environment variables.
> This makes it hard or even impossible to consistently switch between
> different loggers keeping the same level of output.
> Please enhance the {{JavaLogger}} to no mess with the configuration or
> explain why that cannot be done
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (IGNITE-16922) Getting an entry with expiry policy causes IgniteOutOfMemoryException
[
https://issues.apache.org/jira/browse/IGNITE-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16922:
--
Remaining Estimate: 168h
Original Estimate: 168h
> Getting an entry with expiry policy causes IgniteOutOfMemoryException
> -
>
> Key: IGNITE-16922
> URL: https://issues.apache.org/jira/browse/IGNITE-16922
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.13
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> {{IgniteCache#get(key)}} operation causes {{IgniteOutOfMemoryException}} if
> {{AccessedExpiryPolicy}} or {{TouchedExpiryPolicy}} is enabled for the
> {{key}} and Ignite has not enough storage for another entry of the same or
> bigger size.
> This happens because:
> # Ignite needs to update TTL
> # TTL is part of the entry and Ignite overwrites full entry to update the TTL
> # The problem is Ignite runs common code that checks if Ignite has enough
> storage to write the entry with updated TTL back. The check fails causing the
> {{IgniteCache#get(key)}} operation to throw {{IgniteOutOfMemoryException}}.
> # This behavior is very confusing for Ignite users: why would a "read"
> operation throw Ignite OOM?
> Can we update the TTL atomically and skip the storage size check?
> Please enhance Ignite not to throw Ignite OOM on {{get}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (IGNITE-16922) Getting an entry with expiry policy causes IgniteOutOfMemoryException
Alexey Kukushkin created IGNITE-16922:
-
Summary: Getting an entry with expiry policy causes
IgniteOutOfMemoryException
Key: IGNITE-16922
URL: https://issues.apache.org/jira/browse/IGNITE-16922
Project: Ignite
Issue Type: Bug
Affects Versions: 2.13
Reporter: Alexey Kukushkin
{{IgniteCache#get(key)}} operation causes {{IgniteOutOfMemoryException}} if
{{AccessedExpiryPolicy}} or {{TouchedExpiryPolicy}} is enabled for the {{key}}
and Ignite has not enough storage for another entry of the same or bigger size.
This happens because:
# Ignite needs to update TTL
# TTL is part of the entry and Ignite overwrites full entry to update the TTL
# The problem is Ignite runs common code that checks if Ignite has enough
storage to write the entry with updated TTL back. The check fails causing the
{{IgniteCache#get(key)}} operation to throw {{IgniteOutOfMemoryException}}.
# This behavior is very confusing for Ignite users: why would a "read"
operation throw Ignite OOM?
Can we update the TTL atomically and skip the storage size check?
Please enhance Ignite not to throw Ignite OOM on {{get}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (IGNITE-16889) Error-prone X#searchForCause() implementation
Alexey Kukushkin created IGNITE-16889:
-
Summary: Error-prone X#searchForCause() implementation
Key: IGNITE-16889
URL: https://issues.apache.org/jira/browse/IGNITE-16889
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 2.12
Reporter: Alexey Kukushkin
The {{X#seachForCause}} utility method fails with a {{NullPointerException}} if
the analyzed {{Throwable}} has no {{suppressed}} exception due to the [missing
null
check|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/typedef/X.java#L570].
I never encountered any issues with that with real usage of Ignite. However,
writing a unit test that stabs the {{IgniteCache}} methods to throw an
exception without initialized {{suppressed}} exceptions results in the stabbed
method to return {{null}} instead of throwing an exception. The workaround is
to properly initialize the {{suppressed}} list in the mocked exception.
I suggest to make the generic {{X#seachForCause}} utility method handle the
case when the {{suppressed}} list is not initialized. That would simplify the
unit tests code.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (IGNITE-16716) Ignite JUL Logger should not change the configuration
Alexey Kukushkin created IGNITE-16716:
-
Summary: Ignite JUL Logger should not change the configuration
Key: IGNITE-16716
URL: https://issues.apache.org/jira/browse/IGNITE-16716
Project: Ignite
Issue Type: Improvement
Reporter: Alexey Kukushkin
Ignite's {{JavaLogger}} forcefully re-configures logging by adding/removing
appenders and changing severity based on some system properties and environment
variables.
This makes it hard or even impossible to consistently switch between different
loggers keeping the same level of output.
Please enhance the {{JavaLogger}} to no mess with the configuration or explain
why that cannot be done
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (IGNITE-16715) Ignite should always use IgniteLogger
Alexey Kukushkin created IGNITE-16715:
-
Summary: Ignite should always use IgniteLogger
Key: IGNITE-16715
URL: https://issues.apache.org/jira/browse/IGNITE-16715
Project: Ignite
Issue Type: Improvement
Reporter: Alexey Kukushkin
Ignite sometimes logs directly to stdout (e.g. some startup output). This makes
it hard to setup custom logging.
Please make Ignite always use {{IgniteLogger}} or explain why it cannot be done.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-16496) SSLException: closing inbound before receiving peer's close_notify
[
https://issues.apache.org/jira/browse/IGNITE-16496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16496:
--
Description:
Ignite nodes output the warning below on startup when TLS protocol v1.2 is used:
{noformat}
2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
inbound before receiving peer's close_notify
javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
~[na:na]
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
~[na:na]
at org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
~[ignite-core-2.12.0.jar!/:2.12.0]
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
~[ignite-core-2.12.0.jar!/:2.12.0]
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
To reproduce the problem just start two server nodes with TLS v1.3 enabled and
the warnings will be printed in the log before the cluster is formed.
h3. Analysis
The problem _probably_ happens due to [this
code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
alert, which TLS 1.2 is expecting (see [RFC
8446|https://datatracker.ietf.org/doc/html/rfc8446#section-6]). I guess the
right approach to close an SSL socket is just calling {{Socke#close}}, which
should properly wait/send a {{close_notify}}
Some references to consider:
[JDK-8215102 Closing connection to Mysql database results in
exception|https://bugs.openjdk.java.net/browse/JDK-8215102]
[Fix for #93590 - ignore javax.net.ssl.SSLException: closing inbound before
receiving peer's close_notify on
java11+|https://github.com/mysql/mysql-connector-j/pull/32]
[JDK-8251553 Socket closure issues in migrating from JDK 8 to JDK
11|https://bugs.openjdk.java.net/browse/JDK-8251553]
was:
Ignite nodes output the warning below on startup when TLS protocol v1.2 is used:
{noformat}
2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
inbound before receiving peer's close_notify
javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
~[na:na]
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
~[na:na]
at org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
~[ignite-core-2.12.0.jar!/:2.12.0]
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
~[ignite-core-2.12.0.jar!/:2.12.0]
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
To reproduce the problem just start two server nodes with TLS v1.3 enabled and
the warnings will be printed in the log before the cluster is formed.
h3. h3. Analysis
The problem _probably_ happens due to [this
code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
alert, which TLS 1.2 is expecting (see [RFC
8446|https://datatracker.ietf.org/doc/html/rfc8446#section-6]). I guess the
right approach to close an SSL socket is just calling {{Socke#close}}, which
should properly wait/send a {{close_notify}}
> SSLException: closing inbound before receiving peer's close_notify
> --
>
> Key: IGNITE-16496
> URL: https://issues.apache.org/jira/browse/IGNITE-16496
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> Ignite nodes output the warning below on startup when TLS protocol v1.2 is
> used:
> {noformat}
> 2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
> o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
> inbound before receiving peer's close_notify
> javax.net.ssl.SSLException: closing inbound before receiving peer's
> close_notify
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
> ~[na:na]
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
> ~[na:na]
>at
> org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at
>
[jira] [Updated] (IGNITE-16496) SSLException: closing inbound before receiving peer's close_notify
[
https://issues.apache.org/jira/browse/IGNITE-16496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16496:
--
Summary: SSLException: closing inbound before receiving peer's close_notify
(was: SSLException: closing inbound before receiving peer's close_notify (TLS
1.2))
> SSLException: closing inbound before receiving peer's close_notify
> --
>
> Key: IGNITE-16496
> URL: https://issues.apache.org/jira/browse/IGNITE-16496
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> Ignite nodes output the warning below on startup when TLS protocol v1.2 is
> used:
> {noformat}
> 2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
> o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
> inbound before receiving peer's close_notify
> javax.net.ssl.SSLException: closing inbound before receiving peer's
> close_notify
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
> ~[na:na]
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
> ~[na:na]
>at
> org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
> ~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
> To reproduce the problem just start two server nodes with TLS v1.3 enabled
> and the warnings will be printed in the log before the cluster is formed.
> h3. h3. Analysis
> The problem _probably_ happens due to [this
> code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
> calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
> alert, which TLS 1.2 is expecting (see [RFC
> 8446|https://datatracker.ietf.org/doc/html/rfc8446#section-6]). I guess the
> right approach to close an SSL socket is just calling {{Socke#close}}, which
> should properly wait/send a {{close_notify}}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-16496) SSLException: closing inbound before receiving peer's close_notify (TLS 1.2)
[
https://issues.apache.org/jira/browse/IGNITE-16496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16496:
--
Description:
Ignite nodes output the warning below on startup when TLS protocol v1.2 is used:
{noformat}
2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
inbound before receiving peer's close_notify
javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
~[na:na]
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
~[na:na]
at org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
~[ignite-core-2.12.0.jar!/:2.12.0]
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
~[ignite-core-2.12.0.jar!/:2.12.0]
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
To reproduce the problem just start two server nodes with TLS v1.3 enabled and
the warnings will be printed in the log before the cluster is formed.
h3. h3. Analysis
The problem _probably_ happens due to [this
code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
alert, which TLS 1.2 is expecting (see [RFC
8446|https://datatracker.ietf.org/doc/html/rfc8446#section-6]). I guess the
right approach to close an SSL socket is just calling {{Socke#close}}, which
should properly wait/send a {{close_notify}}
was:
Ignite nodes output the warning below on startup when TLS protocol v1.2 is used:
{noformat}
2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
inbound before receiving peer's close_notify
javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
~[na:na]
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
~[na:na]
at org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
~[ignite-core-2.12.0.jar!/:2.12.0]
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
~[ignite-core-2.12.0.jar!/:2.12.0]
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
To reproduce the problem just start two server nodes with TLS v1.3 enabled and
the warnings will be printed in the log before the cluster is formed.
h3. h3. Analysis
The problem _probably_ happens due to [this
code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
alert, which TLS 1.2 is expecting (see [RFC
8446|https://datatracker.ietf.org/doc/html/rfc8446#section-6]. I guess the
right approach to close an SSL socket is just calling {{Socke#close}}, which
should properly wait/send a {{close_notify}}
> SSLException: closing inbound before receiving peer's close_notify (TLS 1.2)
>
>
> Key: IGNITE-16496
> URL: https://issues.apache.org/jira/browse/IGNITE-16496
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> Ignite nodes output the warning below on startup when TLS protocol v1.2 is
> used:
> {noformat}
> 2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
> o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
> inbound before receiving peer's close_notify
> javax.net.ssl.SSLException: closing inbound before receiving peer's
> close_notify
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
> ~[na:na]
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
> ~[na:na]
>at
> org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
> ~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
> To reproduce the problem just start two server nodes with TLS v1.3 enabled
> and the warnings will be printed in the log before the cluster is formed.
> h3. h3. Analysis
> The problem
[jira] [Updated] (IGNITE-16496) SSLException: closing inbound before receiving peer's close_notify (TLS 1.2)
[
https://issues.apache.org/jira/browse/IGNITE-16496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16496:
--
Description:
Ignite nodes output the warning below on startup when TLS protocol v1.2 is used:
{noformat}
2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
inbound before receiving peer's close_notify
javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
~[na:na]
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
~[na:na]
at org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
~[ignite-core-2.12.0.jar!/:2.12.0]
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
~[ignite-core-2.12.0.jar!/:2.12.0]
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
To reproduce the problem just start two server nodes with TLS v1.3 enabled and
the warnings will be printed in the log before the cluster is formed.
h3. h3. Analysis
The problem _probably_ happens due to [this
code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
alert, which TLS 1.2 is expecting (see [RFC
8446|https://datatracker.ietf.org/doc/html/rfc8446#section-6]. I guess the
right approach to close an SSL socket is just calling {{Socke#close}}, which
should properly wait/send a {{close_notify}}
was:
Ignite nodes output the warning below on startup when TLS protocol v1.2 is used:
{noformat}
2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
inbound before receiving peer's close_notify
javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
~[na:na]
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
~[na:na]
at org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
~[ignite-core-2.12.0.jar!/:2.12.0]
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
~[ignite-core-2.12.0.jar!/:2.12.0]
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
To reproduce the problem just start two server nodes with TLS v1.3 enabled and
the warnings will be printed in the log before the cluster is formed.
h3. h3. Analysis
The problem _probably_ happens due to [this
code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
alert, which TLS 1.2 is expecting. I guess the right approach to close an SSL
socket is just calling {{Socke#close}}, which should properly wait/send a
{{close_notify}}
> SSLException: closing inbound before receiving peer's close_notify (TLS 1.2)
>
>
> Key: IGNITE-16496
> URL: https://issues.apache.org/jira/browse/IGNITE-16496
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> Ignite nodes output the warning below on startup when TLS protocol v1.2 is
> used:
> {noformat}
> 2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
> o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
> inbound before receiving peer's close_notify
> javax.net.ssl.SSLException: closing inbound before receiving peer's
> close_notify
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
> ~[na:na]
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
> ~[na:na]
>at
> org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
> ~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
> To reproduce the problem just start two server nodes with TLS v1.3 enabled
> and the warnings will be printed in the log before the cluster is formed.
> h3. h3. Analysis
> The problem _probably_ happens due to [this
>
[jira] [Updated] (IGNITE-16496) SSLException: closing inbound before receiving peer's close_notify (TLS 1.2)
[
https://issues.apache.org/jira/browse/IGNITE-16496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16496:
--
Description:
Ignite nodes output the warning below on startup when TLS protocol v1.2 is used:
{noformat}
2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
inbound before receiving peer's close_notify
javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
~[na:na]
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
~[na:na]
at org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
~[ignite-core-2.12.0.jar!/:2.12.0]
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
~[ignite-core-2.12.0.jar!/:2.12.0]
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
To reproduce the problem just start two server nodes with TLS v1.3 enabled and
the warnings will be printed in the log before the cluster is formed.
h3. h3. Analysis
The problem _probably_ happens due to [this
code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
alert, which TLS 1.2 is expecting. I guess the right approach to close an SSL
socket is just calling {{Socke#close}}, which should properly wait/send a
{{close_notify}}
was:
Ignite nodes output the warning below on startup when TLS protocol v1.3 is used:
{noformat}
2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
inbound before receiving peer's close_notify
javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
~[na:na]
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
~[na:na]
at org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
~[ignite-core-2.12.0.jar!/:2.12.0]
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
~[ignite-core-2.12.0.jar!/:2.12.0]
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
To reproduce the problem just start two server nodes with TLS v1.3 enabled and
the warnings will be printed in the log before the cluster is formed.
h3. h3. Analysis
The problem _probably_ happens due to [this
code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
alert, which TLS 1.3 is expecting. I guess the right approach to close an SSL
socket is just calling {{Socke#close}}, which should properly wait/send a
{{close_notify}}
> SSLException: closing inbound before receiving peer's close_notify (TLS 1.2)
>
>
> Key: IGNITE-16496
> URL: https://issues.apache.org/jira/browse/IGNITE-16496
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> Ignite nodes output the warning below on startup when TLS protocol v1.2 is
> used:
> {noformat}
> 2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
> o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
> inbound before receiving peer's close_notify
> javax.net.ssl.SSLException: closing inbound before receiving peer's
> close_notify
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
> ~[na:na]
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
> ~[na:na]
>at
> org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
> ~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
> To reproduce the problem just start two server nodes with TLS v1.3 enabled
> and the warnings will be printed in the log before the cluster is formed.
> h3. h3. Analysis
> The problem _probably_ happens due to [this
>
[jira] [Updated] (IGNITE-16496) SSLException: closing inbound before receiving peer's close_notify (TLS 1.2)
[
https://issues.apache.org/jira/browse/IGNITE-16496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16496:
--
Summary: SSLException: closing inbound before receiving peer's close_notify
(TLS 1.2) (was: SSLException: closing inbound before receiving peer's
close_notify (TLS 1.3))
> SSLException: closing inbound before receiving peer's close_notify (TLS 1.2)
>
>
> Key: IGNITE-16496
> URL: https://issues.apache.org/jira/browse/IGNITE-16496
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.12
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> Ignite nodes output the warning below on startup when TLS protocol v1.3 is
> used:
> {noformat}
> 2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
> o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
> inbound before receiving peer's close_notify
> javax.net.ssl.SSLException: closing inbound before receiving peer's
> close_notify
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
> ~[na:na]
>at
> java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
> ~[na:na]
>at
> org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
> ~[ignite-core-2.12.0.jar!/:2.12.0]
>at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
> ~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
> To reproduce the problem just start two server nodes with TLS v1.3 enabled
> and the warnings will be printed in the log before the cluster is formed.
> h3. h3. Analysis
> The problem _probably_ happens due to [this
> code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
> calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
> alert, which TLS 1.3 is expecting. I guess the right approach to close an SSL
> socket is just calling {{Socke#close}}, which should properly wait/send a
> {{close_notify}}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (IGNITE-16496) SSLException: closing inbound before receiving peer's close_notify (TLS 1.3)
Alexey Kukushkin created IGNITE-16496:
-
Summary: SSLException: closing inbound before receiving peer's
close_notify (TLS 1.3)
Key: IGNITE-16496
URL: https://issues.apache.org/jira/browse/IGNITE-16496
Project: Ignite
Issue Type: Bug
Affects Versions: 2.12
Reporter: Alexey Kukushkin
Ignite nodes output the warning below on startup when TLS protocol v1.3 is used:
{noformat}
2022-02-08 11:53:05.705 WARN 19384 --- [1:62095]-#4-#51]
o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Failed to shutdown socket: closing
inbound before receiving peer's close_notify
javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:745)
~[na:na]
at
java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:724)
~[na:na]
at org.apache.ignite.internal.util.IgniteUtils.close(IgniteUtils.java:4249)
~[ignite-core-2.12.0.jar!/:2.12.0]
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:7370)
~[ignite-core-2.12.0.jar!/:2.12.0]
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
~[ignite-core-2.12.0.jar!/:2.12.0] {noformat}
To reproduce the problem just start two server nodes with TLS v1.3 enabled and
the warnings will be printed in the log before the cluster is formed.
h3. h3. Analysis
The problem _probably_ happens due to [this
code|https://github.com/apache/ignite/blob/2.12.0/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L4426]
calling {{Socket#shutdownInput()}} before receiving SSL {{close_notify}}
alert, which TLS 1.3 is expecting. I guess the right approach to close an SSL
socket is just calling {{Socke#close}}, which should properly wait/send a
{{close_notify}}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (IGNITE-16448) Optimize unacknowledged messages queue for FULL_SYNC caches
Alexey Kukushkin created IGNITE-16448:
-
Summary: Optimize unacknowledged messages queue for FULL_SYNC
caches
Key: IGNITE-16448
URL: https://issues.apache.org/jira/browse/IGNITE-16448
Project: Ignite
Issue Type: Improvement
Affects Versions: 2.12
Reporter: Alexey Kukushkin
Attachments: apache-ignite-msgReqs-heap-usage.png
Apache Ignite queues all unacknowledged messages in the
{{GridNioRecoveryDescriptor#msgReqs}} queue. Size of this queue is effectively
controlled by the {{TcpCommunicationSpi.ackSendThreshold}} property, which is
32 by default.
Queuing PUTs of large entries results in significant heap memory usage.
Please check if we really need to queue PUTs into FULL_SYNC caches. It seems to
me we should not since a failure to write to a backup would be sent back to the
client right away without any retries. If that is the case then please optimize
the heap usage by not queueing FULL_SYNC cache update messages.
See the heap usage example attached: !apache-ignite-msgReqs-heap-usage.png!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (IGNITE-16431) Entry expiration requires twice the entry size of heap
Alexey Kukushkin created IGNITE-16431:
-
Summary: Entry expiration requires twice the entry size of heap
Key: IGNITE-16431
URL: https://issues.apache.org/jira/browse/IGNITE-16431
Project: Ignite
Issue Type: Improvement
Affects Versions: 2.12
Reporter: Alexey Kukushkin
Attachments: 500MB-put-expiry-master.png
Ignite takes twice the entry size off the heap to expire the entry when
{{{}eagerTtl=true{}}}. See the attached heap memory usage diagram of putting
and then expiring a 500MB entry in Ignite.
This makes Ignite inefficient with handling large objects causing
{{OutOfMemory}} errors.
Do we really need loading entry's value on heap at all to expiry the entry?
Please enhance Ignite cache entry expiration not to load the entry's value on
heap even once or explain why it is not possible.
!500MB-put-expiry-master.png!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-16272) IgniteOutOfMemoryException is not a critical failure
[
https://issues.apache.org/jira/browse/IGNITE-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16272:
--
Labels: cggg (was: c# cggg)
> IgniteOutOfMemoryException is not a critical failure
>
>
> Key: IGNITE-16272
> URL: https://issues.apache.org/jira/browse/IGNITE-16272
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> h3. The Problem
> Ignite raises the
> [IgniteOutOfMemoryException|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/internal/mem/IgniteOutOfMemoryException.java]
> if a data region size is exceeded when trying to add more data to a cache.
> Ignite considers the {{IgniteOutOfMemoryException}} as a critical failure.
> This causes shutting down the Ignite server with the default failure handler.
> However, reaching the data region capacity does not seem to be such a
> critical problem requiring the server shutdown or restart. For example, in
> our application we just want to report this problem back to the users and
> notify the DevOps without applying the critical failure handler. To achieve
> that, we had to define a custom {{FailureHandler}} that detects and ignores
> the {{IgniteOutOfMemoryException}} and all the exception caused by the
> {{{}IgniteOutOfMemoryException{}}}, allowing the final exception to reach the
> application. This solution is clumsy and unreliable since it uses the
> internal {{IgniteOutOfMemoryException}} definition and relies on a complex
> secondary exception structure trying to find the
> {{IgniteOutOfMemoryException}} among the suppressed exception and causes.
> Ignite out-of-the-box failure handlers have the {{ignoredFailure}} property
> that allows filtering out some kinds of failures. However, the
> {{IgniteOutOfMemoryException}} is not among the
> [FailureType|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/failure/FailureType.java]
> that can be ignored.
> h3. The Proposal
> # Does anyone really want to treat the "data region capacity exceeded"
> problem as a critical failure and stop or restart the server?
> ** Consider never treating this condition as a critical failure. This change
> is not backward compatible.
> ** Or add another item to the {{FailureType}} enumeration to optionally
> allow the users not to have that treated as a critical failure. This is
> backward-compatible.
> # Make the {{IgniteOutOfMemoryException}} a public API (now it is in the
> {{internal}} package)
> # Consider renaming {{IgniteOutOfMemoryException}} (for example, to
> something like {{{}NotEnoughStorageException{}}}) since the current name is
> similar to a really critical and usually unrecoverable Java's
> {{OutOfMemoryError}} although the {{IgniteOutOfMemoryException}} is not that
> critical.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-16272) IgniteOutOfMemoryException is not a critical failure
[
https://issues.apache.org/jira/browse/IGNITE-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16272:
--
Labels: c# cggg (was: cggg)
> IgniteOutOfMemoryException is not a critical failure
>
>
> Key: IGNITE-16272
> URL: https://issues.apache.org/jira/browse/IGNITE-16272
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: c#, cggg
>
> h3. The Problem
> Ignite raises the
> [IgniteOutOfMemoryException|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/internal/mem/IgniteOutOfMemoryException.java]
> if a data region size is exceeded when trying to add more data to a cache.
> Ignite considers the {{IgniteOutOfMemoryException}} as a critical failure.
> This causes shutting down the Ignite server with the default failure handler.
> However, reaching the data region capacity does not seem to be such a
> critical problem requiring the server shutdown or restart. For example, in
> our application we just want to report this problem back to the users and
> notify the DevOps without applying the critical failure handler. To achieve
> that, we had to define a custom {{FailureHandler}} that detects and ignores
> the {{IgniteOutOfMemoryException}} and all the exception caused by the
> {{{}IgniteOutOfMemoryException{}}}, allowing the final exception to reach the
> application. This solution is clumsy and unreliable since it uses the
> internal {{IgniteOutOfMemoryException}} definition and relies on a complex
> secondary exception structure trying to find the
> {{IgniteOutOfMemoryException}} among the suppressed exception and causes.
> Ignite out-of-the-box failure handlers have the {{ignoredFailure}} property
> that allows filtering out some kinds of failures. However, the
> {{IgniteOutOfMemoryException}} is not among the
> [FailureType|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/failure/FailureType.java]
> that can be ignored.
> h3. The Proposal
> # Does anyone really want to treat the "data region capacity exceeded"
> problem as a critical failure and stop or restart the server?
> ** Consider never treating this condition as a critical failure. This change
> is not backward compatible.
> ** Or add another item to the {{FailureType}} enumeration to optionally
> allow the users not to have that treated as a critical failure. This is
> backward-compatible.
> # Make the {{IgniteOutOfMemoryException}} a public API (now it is in the
> {{internal}} package)
> # Consider renaming {{IgniteOutOfMemoryException}} (for example, to
> something like {{{}NotEnoughStorageException{}}}) since the current name is
> similar to a really critical and usually unrecoverable Java's
> {{OutOfMemoryError}} although the {{IgniteOutOfMemoryException}} is not that
> critical.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-15241) Ignite H2 Security Vulnerabilities
[
https://issues.apache.org/jira/browse/IGNITE-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15241:
--
Labels: cggg (was: )
> Ignite H2 Security Vulnerabilities
> --
>
> Key: IGNITE-15241
> URL: https://issues.apache.org/jira/browse/IGNITE-15241
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.10
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> Upgrade H2 dependency of the ignite-indexing module to the latest version
> 1.4.200.
> Apache Ignite SQL (module {{ignite-indexing}}) depends on H2 database version
> 1.4.197, which has these two [security
> vulnerabilities|https://www.cvedetails.com/vulnerability-list/vendor_id-17893/product_id-45580/year-2018/H2database-H2.html]
> [CVE-2018-14335|https://www.cvedetails.com/cve/CVE-2018-14335/] is regarded
> as a critical vulnerability by our analyzer (Black Duck SCA) and makes it
> impossible to use Ignite SQL due to security policies. We realize this
> vulnerability is probably not even applicable to the H2 in Ignite since there
> is no H2 database or H2 backups in Ignite. Still the security policies are
> very formal and do not allow that anyway.
> We believe there are lots of other enterprises having the same issue. For
> example, there is another issue IGNITE-14381 referencing the same problem.
> The latest H2 1.4.200 has no vulnerabilities.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-15336) TLSv1.3 and Java 11
[
https://issues.apache.org/jira/browse/IGNITE-15336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15336:
--
Labels: cggg (was: )
> TLSv1.3 and Java 11
> ---
>
> Key: IGNITE-15336
> URL: https://issues.apache.org/jira/browse/IGNITE-15336
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.10
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> Enhance Ignite to support TLSv1.3 with Java 11.
> Ignite currently does not support TLSv1.3 with Java 11. This is
> [documented|https://ignite.apache.org/docs/latest/setup#running-ignite-with-java-11-or-later]:
>
> _TLSv1.3, which is available in Java 11, is not supported at the moment.
> Consider adding {{‑Djdk.tls.client.protocols=TLSv1.2}} if SSL between nodes
> is used._
> h3. Reproducer
> The problem can be reproduced by trying to connect an Ignite Java thin client
> to Ignite server over an secure connection.
> Add TLSv1.3 to the list of protocols in {{SSLContext}} and set the
> {{SSLContext}} in the {{ClientConnectorConfiguration}} on the server and
> {{ClientConfiguration}} on the client.
> An attempt to {{Ignition.startClient({sslContext: {protocols:
> ['TLSv1.3']}})}} fails with a {{ClientConnectionException}} saying _Channel
> is closed_.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-15460) Incorrect "Set max direct memory size" suggestion
[
https://issues.apache.org/jira/browse/IGNITE-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15460:
--
Labels: cggg (was: )
> Incorrect "Set max direct memory size" suggestion
> -
>
> Key: IGNITE-15460
> URL: https://issues.apache.org/jira/browse/IGNITE-15460
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.10
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> Ignite always outputs this performance suggestion on startup:
> {{Set max direct memory size if getting 'OOME: Direct buffer memory' (add
> '-XX:MaxDirectMemorySize=[g|G|m|M|k|K]' to JVM options)}}
>
> That is incorrect since:
> * The Ignite [Memory and JVM
> Tuning|https://ignite.apache.org/docs/latest/perf-and-troubleshooting/memory-tuning#memory-and-jvm-tuning]
> page suggests NOT to explicitly set the {{MaxDirectMemorySize}} (unless
> native persistence is used but in this case set it to {{walSegmentSize *
> 4}}). Not setting {{MaxDirectMemorySize}} seems the right default option
> since in this case JVM would choose the most appropriate default size.
> * Increasing direct memory size when getting OOME for direct memory seems
> too obvious to always print this information.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-15416) False warnings from default Checkpoint and Collision SPIs
[
https://issues.apache.org/jira/browse/IGNITE-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15416:
--
Labels: cggg (was: )
> False warnings from default Checkpoint and Collision SPIs
> -
>
> Key: IGNITE-15416
> URL: https://issues.apache.org/jira/browse/IGNITE-15416
> Project: Ignite
> Issue Type: Improvement
> Components: compute
>Affects Versions: 2.10, 2.11
>Reporter: Alexey Kukushkin
>Assignee: Roman Puchkovskiy
>Priority: Major
> Labels: cggg
> Fix For: 2.12
>
> Attachments: NoopCheckpointSpi.java
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Single-JAR Apache Ignite (no other dependencies but {{ignite-core}}) displays
> the two below warnings from the Checkpoint and Collision SPIs:
> {noformat}
> WARNING: Checkpoints are disabled (to enable configure any GridCheckpointSpi
> implementation)
> WARNING: Collision resolution is disabled (all jobs will be activated upon
> arrival).
> {noformat}
> A "warning" must be something requiring attention from the Ignite users. That
> is not the case for those messages:
> * The {{Checkpoints are disabled}} message warns that the [save/load/remove
> compute job state
> API|https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/spi/checkpoint/CheckpointSpi.html]
> is not implemented and would not have any effect.
> Although that is useful information, not all the Ignite apps really use the
> checkpoint API and that means it is a false alarm for them since they have
> nothing to do with it.
> * The {{Collision resolution is disabled}} message says that the default
> "all jobs will be activated upon arrival" policy is applied. This does not
> seem like a problem at all since this policy might be OK for most users.
> Also, not all the apps use Ignite Compute API (although it might be used
> implicitly). Thus, this is also not a warning since most users have nothing
> to do with it.
> Enhance Ignite to display a warning only when the user has to act upon it:
> * Ignite might display the {{Checkpoints are disabled}} message only when
> the [save/load/remove compute job state
> API|https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/spi/checkpoint/CheckpointSpi.html]
> is really used like demonstrated in the attached enhanced
> [^NoopCheckpointSpi.java]
> * The The {{Collision resolution is disabled}} message is not a warning at
> all. Consider turning it to an INFO message or removing it and making sure
> the Ignite documentation clearly explains the default "all jobs will be
> activated upon arrival" policy.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-16135) Expiry policy is not applied to Ignite cache after serialization
[
https://issues.apache.org/jira/browse/IGNITE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16135:
--
Labels: cggg (was: )
> Expiry policy is not applied to Ignite cache after serialization
>
>
> Key: IGNITE-16135
> URL: https://issues.apache.org/jira/browse/IGNITE-16135
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> Expiry Policy specified in Ignite cache operation context is not applied to
> the cache after the cache serialization/deserialization.
> h3. Steps to Reproduce
> # Run an Ignite server with a cache without expiry policy
> # A client node joins the server, gets the cache proxy and applies expiry
> policy to the cache proxy
> # The client executes a Compute task taking the cache proxy as a parameter
> on the server.
> # The Compute task puts an entry to the cache using the cache proxy received
> from the client.
> # The client sleeps for a time period longer than the TTL specified for the
> expiry policy
> # The client checks if the entry that the Compute task added still exists
> h3. Expected
> The entry does not exist since it must expire
> h3. Actual
> The entry still exists in the cache.
> h3. Reproducer
> {code:java}
> public class IgniteCacheSerializationTest {
> /** Expiry Policy specified in Ignite cache operation context is enabled
> after the cache serialization. */
> @Test
> public void igniteCacheWithExpiryPolicySerialization() throws
> InterruptedException {
> var cacheName = "cache1";
> var ttl = 1000;
> Supplier igniteCfgFactory = () ->
> new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(
> new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500"))
> )
> )
> .setCacheConfiguration(new CacheConfiguration<>(cacheName));
> try (var ignored =
> Ignition.start(igniteCfgFactory.get().setIgniteInstanceName("server"))) {
> try (var ignite = Ignition.start(
>
> igniteCfgFactory.get().setIgniteInstanceName("test").setClientMode(true))
> ) {
> // Apply a 1-second expiry policy to the cache
> var expiryPolicy = TouchedExpiryPolicy.factoryOf(new
> Duration(TimeUnit.MILLISECONDS, ttl)).create();
> var cache = ignite. String>cache(cacheName).withExpiryPolicy(expiryPolicy);
> // Run a Compute Task that serializes the cache with the
> 1-second expiry policy applied.
> // The task adds a key 1 to the cache
> ignite.compute().affinityRun(cacheName, 1, new Put1(cache));
> // Sleep for a time twice longer than the TTL
> Thread.sleep(ttl * 2);
> // The key 1 must expiry by this time
> assertFalse(cache.containsKey(1), "The entry still exists in
> the cache");
> }
> }
> }
> private static final class Put1 implements IgniteRunnable {
> private final IgniteCache cache;
> public Put1(IgniteCache cache) {
> this.cache = cache;
> }
> @Override public void run() {
> cache.put(1, "ignored");
> }
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-16272) IgniteOutOfMemoryException is not a critical failure
[
https://issues.apache.org/jira/browse/IGNITE-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16272:
--
Labels: cggg (was: )
> IgniteOutOfMemoryException is not a critical failure
>
>
> Key: IGNITE-16272
> URL: https://issues.apache.org/jira/browse/IGNITE-16272
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> h3. The Problem
> Ignite raises the
> [IgniteOutOfMemoryException|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/internal/mem/IgniteOutOfMemoryException.java]
> if a data region size is exceeded when trying to add more data to a cache.
> Ignite considers the {{IgniteOutOfMemoryException}} as a critical failure.
> This causes shutting down the Ignite server with the default failure handler.
> However, reaching the data region capacity does not seem to be such a
> critical problem requiring the server shutdown or restart. For example, in
> our application we just want to report this problem back to the users and
> notify the DevOps without applying the critical failure handler. To achieve
> that, we had to define a custom {{FailureHandler}} that detects and ignores
> the {{IgniteOutOfMemoryException}} and all the exception caused by the
> {{{}IgniteOutOfMemoryException{}}}, allowing the final exception to reach the
> application. This solution is clumsy and unreliable since it uses the
> internal {{IgniteOutOfMemoryException}} definition and relies on a complex
> secondary exception structure trying to find the
> {{IgniteOutOfMemoryException}} among the suppressed exception and causes.
> Ignite out-of-the-box failure handlers have the {{ignoredFailure}} property
> that allows filtering out some kinds of failures. However, the
> {{IgniteOutOfMemoryException}} is not among the
> [FailureType|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/failure/FailureType.java]
> that can be ignored.
> h3. The Proposal
> # Does anyone really want to treat the "data region capacity exceeded"
> problem as a critical failure and stop or restart the server?
> ** Consider never treating this condition as a critical failure. This change
> is not backward compatible.
> ** Or add another item to the {{FailureType}} enumeration to optionally
> allow the users not to have that treated as a critical failure. This is
> backward-compatible.
> # Make the {{IgniteOutOfMemoryException}} a public API (now it is in the
> {{internal}} package)
> # Consider renaming {{IgniteOutOfMemoryException}} (for example, to
> something like {{{}NotEnoughStorageException{}}}) since the current name is
> similar to a really critical and usually unrecoverable Java's
> {{OutOfMemoryError}} although the {{IgniteOutOfMemoryException}} is not that
> critical.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-15959) Remove operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15959:
--
Labels: cggg (was: )
> Remove operation fetches entry value into heap
> --
>
> Key: IGNITE-15959
> URL: https://issues.apache.org/jira/browse/IGNITE-15959
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> [See similar problem for the containsKey()
> operation|https://issues.apache.org/jira/browse/IGNITE-16137]
> {{IgniteCache#remove(key)}} operation fetches full entry into heap memory.
> This is inefficient when working with large objects: our application running
> with limited heap memory fails with {{java.lang.OutOfMemoryError: Java heap
> space}} when trying to remove an entry by key.
> It seems wrong that Ignite needs to fetch the full entry on heap to remove
> the entry. Please enhance Ignite to not be doing that or explain why Ignite
> must do that.
> h2. Reproducer
> h3. Steps
> Create a Gradle project with the below class and run it as
> {{./gradlew test --tests apache.ignite.issues.RemoveOperationHeapUsage}}
> {{build.gradle}}
> {code:groovy}
> test {
> minHeapSize = "512m"
> maxHeapSize = "512m"
> }
> {code}
> {{RemoveOperationHeapUsage.java}}
> {code:java}
> public class RemoveOperationHeapUsage {
> /** Run the test with -Xmx512m -Xms512m */
> @Test
> public void removeOperationFetchesValueOnHeap() {
> var igniteCfg = new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
> )
> .setCacheConfiguration(new CacheConfiguration<>("blobs"));
> try (var ignite = Ignition.start(igniteCfg)) {
> Cache cache = ignite.cache("blobs");
> // Put a BLOB having size of 35% of free memory to the cache
> Runtime.getRuntime().gc();
> var freeMemory = Runtime.getRuntime().freeMemory();
> var blobSize = (int)(freeMemory * 0.35);
> putBlob(cache, blobSize);
> // Use 70% of the free heap
> Runtime.getRuntime().gc();
> var unused = new byte[2 * blobSize];
> // Remove the blob from the cache.
> // This throws "OutOfMemoryError: Java heap space" since Ignite
> retrieves full entry to the heap.
> // Why does Ignite retrieve entry value to delete the entry?
> cache.remove(1);
> }
> }
> private static void putBlob(Cache cache, int blobSize) {
> var blob = new byte[blobSize];
> cache.put(1, blob);
> }
> }
> {code}
> h3. Expected
> The test passes
> h3. Actual
> The {{cache.remove}} operatoin fails with:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
> at
> org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:3987)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5921)
>
[jira] [Updated] (IGNITE-15962) CacheEntryRemovedListener makes the PUT operation to create ENTRY CREATED events
[
https://issues.apache.org/jira/browse/IGNITE-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15962:
--
Labels: cggg (was: )
> CacheEntryRemovedListener makes the PUT operation to create ENTRY CREATED
> events
>
>
> Key: IGNITE-15962
> URL: https://issues.apache.org/jira/browse/IGNITE-15962
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Attachments: CacheContinuousQueryEntry-HeapDump.png,
> CacheContinuousQueryEntry-StackTrace.png, EventListenerHeapUsage.java,
> IgniteServer.java, build.gradle
>
>
> {{CacheEntryRemovedListener}} configured with
> {{MutableCacheEntryListenerConfiguration.isOldValueRequired = false}} makes
> the {{Cache#put}} operation to create {{CREATED}} events on the primary
> server for a short period of time. This is inefficient when working with
> large objects: our application running with limited heap memory fails with
> {{java.lang.OutOfMemoryError: Java heap space}} after executing a {{PUT}}
> operation since the entry referencing the object is "stuck" on the heap.
> h2. Reproducer
> h3. Steps
> We need two Ignite server nodes to demonstrate the problem. A cache is
> deployed on both the nodes and both the nodes listen to the {{ENTRY REMOVED}}
> events. Both the nodes disable fetching the {{CacheEntryEvent#getOldValue}}
> by setting {{MutableCacheEntryListenerConfiguration.isOldValueRequired =
> false}}
> # Build and run the attached [^IgniteServer.java]. This server is configured
> like described above and runs until manually terminated.
> # Build the run the attached test [^EventListenerHeapUsage.java] . This test:
> ## Starts another server configured like described above
> ## Puts a large object taking 35% of the free memory. This is done in an
> isolated scope so the memory after the operation is garbage collected,
> ## Attempts to allocate 70% of the heap.
> h3. Expected
> The last "allocate 70% of the heap" operation should succeed.
> h3. Actual
> {noformat}
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> apache.ignite.issues.EventListenerHeapUsage.entryRemovedEventListenerFetchesValueOnHeap(EventListenerHeapUsage.java:33)
> {noformat}
> h2 Analysis
> For some reason configuring the event listener makes the {{IgniteCache#put}}
> operation to create and attach an instance of {{CacheContinuousQueryEntry}}
> with {{evtType=CREATED}} and {{newObject}} referencing the entry.
> The attached heap dump !CacheContinuousQueryEntry-HeapDump.png! collected on
> the {{OutOfMemoryError}} demonstrates the large object "stuck" on the heap.
> The attached {{CacheContinuousQueryEntry}} initialization
> !CacheContinuousQueryEntry-StackTrace.png! proves the {{ENTRY CREATED}} event
> is being created even though we did not configure it.
> It seems wrong that Ignite needs to attached the {{ENTRY CREATED}} event to
> the "update" message. Please enhance Ignite to not be doing that or explain
> why Ignite must do that.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-16137) ContainsKey operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-16137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-16137:
--
Labels: cggg (was: )
> ContainsKey operation fetches entry value into heap
> ---
>
> Key: IGNITE-16137
> URL: https://issues.apache.org/jira/browse/IGNITE-16137
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> [See similar problem for the remove()
> operation|https://issues.apache.org/jira/browse/IGNITE-15959]
> {{IgniteCache#containsKey(key)}} operation fetches full entry into heap
> memory. This is inefficient when working with large objects: our application
> running with limited heap memory fails with {{java.lang.OutOfMemoryError:
> Java heap space}} when trying to check if a key exists.
> It seems wrong that Ignite needs to fetch the full entry on heap to check if
> the key exists. Please enhance Ignite to not be doing that or explain why
> Ignite must do that.
> h2. Reproducer
> h3. Steps
> Create a Gradle project with the below class and run it as
> {{./gradlew test --tests apache.ignite.issues.ContainsOperationHeapUsage}}
> {{build.gradle}}
> {code:groovy}
> test {
> minHeapSize = "512m"
> maxHeapSize = "512m"
> }
> {code}
> {{ContainsOperationHeapUsage.java}}
> {code:java}
> public class ContainsOperationHeapUsage {
> /** Run with -Xmx512m -Xms512m */
> @Test
> public void containsOperationFetchesValueOnHeap() {
> var igniteCfg = new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
> )
> .setCacheConfiguration(new CacheConfiguration<>("blobs"));
> try (var ignite = Ignition.start(igniteCfg)) {
> Cache cache = ignite.cache("blobs");
> // Put a BLOB having size of 35% of free memory to the cache
> Runtime.getRuntime().gc();
> var freeMemory = Runtime.getRuntime().freeMemory();
> var blobSize = (int)(freeMemory * 0.35);
> putBlob(cache, blobSize);
> // Use 70% of the free heap
> Runtime.getRuntime().gc();
> var unused = new byte[2 * blobSize];
> // Check if the blob exists in the cache.
> // This throws "OutOfMemoryError: Java heap space" since Ignite
> retrieves full entry to the heap.
> // Why does Ignite retrieve entry value to check if the key
> exists?
> cache.containsKey(1);
> }
> }
> private static void putBlob(Cache cache, int blobSize) {
> var blob = new byte[blobSize];
> cache.put(1, blob);
> }
> }
> {code}
> h3. Expected
> The test passes
> h3. Actual
> The {{cache.containsKey}} operatoin fails with:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
> at
> org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetOne.found(BPlusTree.java:3156)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
> at
>
[jira] [Updated] (IGNITE-7641) Add CacheEntry#ttl method
[
https://issues.apache.org/jira/browse/IGNITE-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-7641:
-
Labels: cggg (was: )
> Add CacheEntry#ttl method
> -
>
> Key: IGNITE-7641
> URL: https://issues.apache.org/jira/browse/IGNITE-7641
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Stephen Darlington
>Priority: Major
> Labels: cggg
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Ignite provides a way to specify an expiry policy on per entry level, but
> there is no way to know the current TTL for a particular key.
> We can add {{CacheEntry#ttl()}} and/or {{IgniteCache#ttl(K key)}} method that
> will provide this information. Looks like it's already available via
> {{GridCacheMapEntry#ttl()}}, so we just need to properly expose it to public
> API.
> Here is the user forum discussion about this:
> http://apache-ignite-users.70518.x6.nabble.com/Get-TTL-of-the-specific-K-V-entry-td19817.html
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-14326) Add CacheEntry#setTtl method
[ https://issues.apache.org/jira/browse/IGNITE-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kukushkin updated IGNITE-14326: -- Labels: cggg (was: ) > Add CacheEntry#setTtl method > > > Key: IGNITE-14326 > URL: https://issues.apache.org/jira/browse/IGNITE-14326 > Project: Ignite > Issue Type: Improvement > Components: cache >Affects Versions: 2.3 >Reporter: Stephen Darlington >Assignee: Stephen Darlington >Priority: Major > Labels: cggg > > Ignite provides a way to specify an expiry policy on per entry level, but > there is no way to know the current TTL for a particular key or refresh the > TTL without first retrieving the record. > Ticket IGNITE-7641 details the IgniteCache#ttl() method. This ticket is about > _setting_ the TTL. > The API for this is less well defined than the get TTL method. Suggest a > number of options: > * Update using the default TTL policy > * Update using a specified long > * Update using a specified Expiry policy -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15962) CacheEntryRemovedListener makes the PUT operation to create ENTRY CREATED events
[
https://issues.apache.org/jira/browse/IGNITE-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15962:
--
Description:
{{CacheEntryRemovedListener}} configured with
{{MutableCacheEntryListenerConfiguration.isOldValueRequired = false}} makes the
{{Cache#put}} operation to create {{CREATED}} events on the primary server for
a short period of time. This is inefficient when working with large objects:
our application running with limited heap memory fails with
{{java.lang.OutOfMemoryError: Java heap space}} after executing a {{PUT}}
operation since the entry referencing the object is "stuck" on the heap.
h2. Reproducer
h3. Steps
We need two Ignite server nodes to demonstrate the problem. A cache is deployed
on both the nodes and both the nodes listen to the {{ENTRY REMOVED}} events.
Both the nodes disable fetching the {{CacheEntryEvent#getOldValue}} by setting
{{MutableCacheEntryListenerConfiguration.isOldValueRequired = false}}
# Build and run the attached [^IgniteServer.java]. This server is configured
like described above and runs until manually terminated.
# Build the run the attached test [^EventListenerHeapUsage.java] . This test:
## Starts another server configured like described above
## Puts a large object taking 35% of the free memory. This is done in an
isolated scope so the memory after the operation is garbage collected,
## Attempts to allocate 70% of the heap.
h3. Expected
The last "allocate 70% of the heap" operation should succeed.
h3. Actual
{noformat}
Caused by: java.lang.OutOfMemoryError: Java heap space
at
apache.ignite.issues.EventListenerHeapUsage.entryRemovedEventListenerFetchesValueOnHeap(EventListenerHeapUsage.java:33)
{noformat}
h2 Analysis
For some reason configuring the event listener makes the {{IgniteCache#put}}
operation to create and attach an instance of {{CacheContinuousQueryEntry}}
with {{evtType=CREATED}} and {{newObject}} referencing the entry.
The attached heap dump !CacheContinuousQueryEntry-HeapDump.png! collected on
the {{OutOfMemoryError}} demonstrates the large object "stuck" on the heap.
The attached {{CacheContinuousQueryEntry}} initialization
!CacheContinuousQueryEntry-StackTrace.png! proves the {{ENTRY CREATED}} event
is being created even though we did not configure it.
It seems wrong that Ignite needs to attached the {{ENTRY CREATED}} event to the
"update" message. Please enhance Ignite to not be doing that or explain why
Ignite must do that.
was:
{{CacheEntryRemovedListener}} configured with
{{MutableCacheEntryListenerConfiguration.isOldValueRequired = false}} makes the
{{Cache#put}} operation to send {{CREATED}} events attached to the "Update"
message. This is inefficient when working with large objects: our application
running with limited heap memory fails with {{java.lang.OutOfMemoryError: Java
heap space}} after executing a {{PUT}} operation since the entry referencing
the object is "stuck" on the heap.
h2. Reproducer
h3. Steps
We need two Ignite server nodes to demonstrate the problem. A cache is deployed
on both the nodes and both the nodes listen to the {{ENTRY REMOVED}} events.
Both the nodes disable fetching the {{CacheEntryEvent#getOldValue}} by setting
{{MutableCacheEntryListenerConfiguration.isOldValueRequired = false}}
# Build and run the attached [^IgniteServer.java]. This server is configured
like described above and runs until manually terminated.
# Build the run the attached test [^EventListenerHeapUsage.java] . This test:
## Starts another server configured like described above
## Puts a large object taking 35% of the free memory. This is done in an
isolated scope so the memory after the operation is garbage collected,
## Attempts to allocate 70% of the heap.
h3. Expected
The last "allocate 70% of the heap" operation should succeed.
h3. Actual
{noformat}
Caused by: java.lang.OutOfMemoryError: Java heap space
at
apache.ignite.issues.EventListenerHeapUsage.entryRemovedEventListenerFetchesValueOnHeap(EventListenerHeapUsage.java:33)
{noformat}
h2 Analysis
For some reason configuring the event listener makes the {{IgniteCache#put}}
operation to create and attach an instance of {{CacheContinuousQueryEntry}}
with {{evtType=CREATED}} and {{newObject}} referencing the entry.
The attached heap dump !CacheContinuousQueryEntry-HeapDump.png! collected on
the {{OutOfMemoryError}} demonstrates the large object "stuck" on the heap.
The attached {{CacheContinuousQueryEntry}} initialization
!CacheContinuousQueryEntry-StackTrace.png! proves the {{ENTRY CREATED}} event
is being created even though we did not configure it.
It seems wrong that Ignite needs to attached the {{ENTRY CREATED}} event to the
"update" message. Please enhance Ignite to not be doing that or explain why
Ignite must do that.
> CacheEntryRemovedListener makes the PUT operation to create ENTRY CREATED
[jira] [Updated] (IGNITE-15962) CacheEntryRemovedListener makes the PUT operation to create ENTRY CREATED events
[
https://issues.apache.org/jira/browse/IGNITE-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15962:
--
Summary: CacheEntryRemovedListener makes the PUT operation to create ENTRY
CREATED events (was: CacheEntryRemovedListener makes the PUT operation to send
ENTRY CREATED events)
> CacheEntryRemovedListener makes the PUT operation to create ENTRY CREATED
> events
>
>
> Key: IGNITE-15962
> URL: https://issues.apache.org/jira/browse/IGNITE-15962
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Attachments: CacheContinuousQueryEntry-HeapDump.png,
> CacheContinuousQueryEntry-StackTrace.png, EventListenerHeapUsage.java,
> IgniteServer.java, build.gradle
>
>
> {{CacheEntryRemovedListener}} configured with
> {{MutableCacheEntryListenerConfiguration.isOldValueRequired = false}} makes
> the {{Cache#put}} operation to send {{CREATED}} events attached to the
> "Update" message. This is inefficient when working with large objects: our
> application running with limited heap memory fails with
> {{java.lang.OutOfMemoryError: Java heap space}} after executing a {{PUT}}
> operation since the entry referencing the object is "stuck" on the heap.
> h2. Reproducer
> h3. Steps
> We need two Ignite server nodes to demonstrate the problem. A cache is
> deployed on both the nodes and both the nodes listen to the {{ENTRY REMOVED}}
> events. Both the nodes disable fetching the {{CacheEntryEvent#getOldValue}}
> by setting {{MutableCacheEntryListenerConfiguration.isOldValueRequired =
> false}}
> # Build and run the attached [^IgniteServer.java]. This server is configured
> like described above and runs until manually terminated.
> # Build the run the attached test [^EventListenerHeapUsage.java] . This test:
> ## Starts another server configured like described above
> ## Puts a large object taking 35% of the free memory. This is done in an
> isolated scope so the memory after the operation is garbage collected,
> ## Attempts to allocate 70% of the heap.
> h3. Expected
> The last "allocate 70% of the heap" operation should succeed.
> h3. Actual
> {noformat}
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> apache.ignite.issues.EventListenerHeapUsage.entryRemovedEventListenerFetchesValueOnHeap(EventListenerHeapUsage.java:33)
> {noformat}
> h2 Analysis
> For some reason configuring the event listener makes the {{IgniteCache#put}}
> operation to create and attach an instance of {{CacheContinuousQueryEntry}}
> with {{evtType=CREATED}} and {{newObject}} referencing the entry.
> The attached heap dump !CacheContinuousQueryEntry-HeapDump.png! collected on
> the {{OutOfMemoryError}} demonstrates the large object "stuck" on the heap.
> The attached {{CacheContinuousQueryEntry}} initialization
> !CacheContinuousQueryEntry-StackTrace.png! proves the {{ENTRY CREATED}}
> event is being created even though we did not configure it.
> It seems wrong that Ignite needs to attached the {{ENTRY CREATED}} event to
> the "update" message. Please enhance Ignite to not be doing that or explain
> why Ignite must do that.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (IGNITE-16272) IgniteOutOfMemoryException is not a critical failure
Alexey Kukushkin created IGNITE-16272:
-
Summary: IgniteOutOfMemoryException is not a critical failure
Key: IGNITE-16272
URL: https://issues.apache.org/jira/browse/IGNITE-16272
Project: Ignite
Issue Type: Improvement
Reporter: Alexey Kukushkin
h3. The Problem
Ignite raises the
[IgniteOutOfMemoryException|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/internal/mem/IgniteOutOfMemoryException.java]
if a data region size is exceeded when trying to add more data to a cache.
Ignite considers the {{IgniteOutOfMemoryException}} as a critical failure. This
causes shutting down the Ignite server with the default failure handler.
However, reaching the data region capacity does not seem to be such a critical
problem requiring the server shutdown or restart. For example, in our
application we just want to report this problem back to the users and notify
the DevOps without applying the critical failure handler. To achieve that, we
had to define a custom {{FailureHandler}} that detects and ignores the
{{IgniteOutOfMemoryException}} and all the exception caused by the
{{{}IgniteOutOfMemoryException{}}}, allowing the final exception to reach the
application. This solution is clumsy and unreliable since it uses the internal
{{IgniteOutOfMemoryException}} definition and relies on a complex secondary
exception structure trying to find the {{IgniteOutOfMemoryException}} among the
suppressed exception and causes.
Ignite out-of-the-box failure handlers have the {{ignoredFailure}} property
that allows filtering out some kinds of failures. However, the
{{IgniteOutOfMemoryException}} is not among the
[FailureType|https://github.com/apache/ignite/blob/2.11.1/modules/core/src/main/java/org/apache/ignite/failure/FailureType.java]
that can be ignored.
h3. The Proposal
# Does anyone really want to treat the "data region capacity exceeded" problem
as a critical failure and stop or restart the server?
** Consider never treating this condition as a critical failure. This change
is not backward compatible.
** Or add another item to the {{FailureType}} enumeration to optionally allow
the users not to have that treated as a critical failure. This is
backward-compatible.
# Make the {{IgniteOutOfMemoryException}} a public API (now it is in the
{{internal}} package)
# Consider renaming {{IgniteOutOfMemoryException}} (for example, to something
like {{{}NotEnoughStorageException{}}}) since the current name is similar to a
really critical and usually unrecoverable Java's {{OutOfMemoryError}} although
the {{IgniteOutOfMemoryException}} is not that critical.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (IGNITE-12807) Key and Value fields with same name and SQL DML
[
https://issues.apache.org/jira/browse/IGNITE-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin closed IGNITE-12807.
-
Ignite Flags: (was: Docs Required,Release Notes Required)
> Key and Value fields with same name and SQL DML
> ---
>
> Key: IGNITE-12807
> URL: https://issues.apache.org/jira/browse/IGNITE-12807
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: sbcf
>
> Key/Value API allows both the Key and Value have fields with same name. This
> is a very popular arrangement since most users are ready to sacrifice extra
> memory footprint for the sake of having a self-sufficient value entity.
> Using SQL DML to update such an entry will update only the key field, leaving
> the value field unchanged. This is a huge usability issue for the mixed K/V
> and SQL API apps.
> h1. Proposal
> h2. Requirements
> h3. Example Data Model
> Consider a business domain entity *Person \{ id: int, passportNo: String,
> name: String }*
> Suppose an Ignite application development team decided to map the Person
> entity to Ignite data model as:
> * *PersonKey \{ id: int, passportNo: String }*
> * *Person \{ passportNo: String, name: String }*
> h3. Public API
> * *Cache API*: add new method {{setKeyValueFields(keyValueFields:
> Set): QueryEntity}} to class {{QueryEntity}}
> ** The method marks Cache API Key and Value fields that SQL API must
> initialize (on INSERT/MERGE) and update (on UPDATE/MERGE) together.
> ** It is still possible to use Cache API to initialize the fields marked
> with {{setKeyValueFields}} to different values. SQL SELECT statement returns
> value of such a field from the Key entity.
> ** The method accepts a set of field names and returns the declaring class
> instance for chaining.
> ** The method throws {{ArgumentException}} if the Key and Value types are
> available and the field types are different within the Key and Value entities.
> * *SQL API*: add {{KEY_VALUE_FIELDS}} parameter to {{CREATE TABLE}}
> statement's additional parameters list.
> ** The parameter's value is a space-separated list of field names with the
> semantics equivalent to that of the {{setKeyValueFields}} method described
> above.
> ** The parameter can be specified only if both the {{KEY_TYPE}} and
> {{VALUE_TYPE}} parameters are specified.
> h3. Use Cases
> h4. Inserting Into Key and Value Fields With Same Name Initializes Both
> Fields in QueryEntity-Defined Cache
> * GIVEN a Person cache from the example data model configured like this in
> Ignite:
> {code:java}
> new CacheConfiguration("CACHE")
> .setQueryEntities(Collections.singleton(
> new QueryEntity(PersonKey.class, Person.class)
> .addQueryField("id", int.class.getName(), null)
> .addQueryField("passportNo", String.class.getName(), null)
> .addQueryField("name", String.class.getName(), null)
> .setKeyFields(Collections.singleton("id"))
> .setKeyValueFields(Collections.singleton("passportNo"))
> ));
> {code}
> ** AND an entry is added to the cache with this SQL statement:
> {code:sql}
> INSERT INTO CACHE.Person (ID, PASSPORTNO, NAME) VALUES (1, '1', 'Name1')
> {code}
> * WHEN the user gets the entity using Cache API:
> {code:java}
> final PersonKey K = new PersonKey(1, "1");
> Person v = cache.get(K);
> {code}
> * THEN the *passportNo* field is initialized to the same value within the
> key and value entities:
> {code:java}
> assertEquals(K.passportNo, v.passportNo);
> {code}
> h4. Querying Key and Value Fields With Same Name and Different Values Returns
> Value from the Key in QueryEntity-Defined Cache
> * GIVEN a Person cache from the previous use case
> ** AND an entry with different passportNo is added to the cache with this
> Cache API:
> {code:java}
> final PersonKey K = new PersonKey(1, "1");
> final Person V = new Person("2", "Name1");
> cache.put(K, V);
> {code}
> * WHEN the user runs this SQL to get the enty:
> {code:sql}
> SELECT ID, PASSPORTNO, NAME FROM CACHE.Person
> {code}
> * THEN the retrieved PASSPORTNO is that of the Key: "1"
> h4. Inserting Into Key and Value Fields With Same Name Initializes Both
> Fields in SQL-Defined Cache
> * GIVEN a Person cache from the example data model configured like this in
> Ignite:
> {code:sql}
> CREATE TABLE Person (
> id int,
> passportNo varchar,
> name varchar,
> PRIMARY KEY(id, passportNo)
> ) WITH "key_type=PersonKey, value_type=Person, key_value_fields=passportNo"
> {code}
> ** AND an entry is added to the cache with this SQL statement:
> {code:sql}
> INSERT INTO CACHE.Person (ID,
[jira] [Resolved] (IGNITE-12807) Key and Value fields with same name and SQL DML
[
https://issues.apache.org/jira/browse/IGNITE-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin resolved IGNITE-12807.
---
Resolution: Won't Fix
Closing the ticket as not accepted by the Ignite Community
> Key and Value fields with same name and SQL DML
> ---
>
> Key: IGNITE-12807
> URL: https://issues.apache.org/jira/browse/IGNITE-12807
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: sbcf
>
> Key/Value API allows both the Key and Value have fields with same name. This
> is a very popular arrangement since most users are ready to sacrifice extra
> memory footprint for the sake of having a self-sufficient value entity.
> Using SQL DML to update such an entry will update only the key field, leaving
> the value field unchanged. This is a huge usability issue for the mixed K/V
> and SQL API apps.
> h1. Proposal
> h2. Requirements
> h3. Example Data Model
> Consider a business domain entity *Person \{ id: int, passportNo: String,
> name: String }*
> Suppose an Ignite application development team decided to map the Person
> entity to Ignite data model as:
> * *PersonKey \{ id: int, passportNo: String }*
> * *Person \{ passportNo: String, name: String }*
> h3. Public API
> * *Cache API*: add new method {{setKeyValueFields(keyValueFields:
> Set): QueryEntity}} to class {{QueryEntity}}
> ** The method marks Cache API Key and Value fields that SQL API must
> initialize (on INSERT/MERGE) and update (on UPDATE/MERGE) together.
> ** It is still possible to use Cache API to initialize the fields marked
> with {{setKeyValueFields}} to different values. SQL SELECT statement returns
> value of such a field from the Key entity.
> ** The method accepts a set of field names and returns the declaring class
> instance for chaining.
> ** The method throws {{ArgumentException}} if the Key and Value types are
> available and the field types are different within the Key and Value entities.
> * *SQL API*: add {{KEY_VALUE_FIELDS}} parameter to {{CREATE TABLE}}
> statement's additional parameters list.
> ** The parameter's value is a space-separated list of field names with the
> semantics equivalent to that of the {{setKeyValueFields}} method described
> above.
> ** The parameter can be specified only if both the {{KEY_TYPE}} and
> {{VALUE_TYPE}} parameters are specified.
> h3. Use Cases
> h4. Inserting Into Key and Value Fields With Same Name Initializes Both
> Fields in QueryEntity-Defined Cache
> * GIVEN a Person cache from the example data model configured like this in
> Ignite:
> {code:java}
> new CacheConfiguration("CACHE")
> .setQueryEntities(Collections.singleton(
> new QueryEntity(PersonKey.class, Person.class)
> .addQueryField("id", int.class.getName(), null)
> .addQueryField("passportNo", String.class.getName(), null)
> .addQueryField("name", String.class.getName(), null)
> .setKeyFields(Collections.singleton("id"))
> .setKeyValueFields(Collections.singleton("passportNo"))
> ));
> {code}
> ** AND an entry is added to the cache with this SQL statement:
> {code:sql}
> INSERT INTO CACHE.Person (ID, PASSPORTNO, NAME) VALUES (1, '1', 'Name1')
> {code}
> * WHEN the user gets the entity using Cache API:
> {code:java}
> final PersonKey K = new PersonKey(1, "1");
> Person v = cache.get(K);
> {code}
> * THEN the *passportNo* field is initialized to the same value within the
> key and value entities:
> {code:java}
> assertEquals(K.passportNo, v.passportNo);
> {code}
> h4. Querying Key and Value Fields With Same Name and Different Values Returns
> Value from the Key in QueryEntity-Defined Cache
> * GIVEN a Person cache from the previous use case
> ** AND an entry with different passportNo is added to the cache with this
> Cache API:
> {code:java}
> final PersonKey K = new PersonKey(1, "1");
> final Person V = new Person("2", "Name1");
> cache.put(K, V);
> {code}
> * WHEN the user runs this SQL to get the enty:
> {code:sql}
> SELECT ID, PASSPORTNO, NAME FROM CACHE.Person
> {code}
> * THEN the retrieved PASSPORTNO is that of the Key: "1"
> h4. Inserting Into Key and Value Fields With Same Name Initializes Both
> Fields in SQL-Defined Cache
> * GIVEN a Person cache from the example data model configured like this in
> Ignite:
> {code:sql}
> CREATE TABLE Person (
> id int,
> passportNo varchar,
> name varchar,
> PRIMARY KEY(id, passportNo)
> ) WITH "key_type=PersonKey, value_type=Person, key_value_fields=passportNo"
> {code}
> ** AND an entry is added to the cache with this SQL statement:
> {code:sql}
> INSERT
[jira] (IGNITE-12807) Key and Value fields with same name and SQL DML
[ https://issues.apache.org/jira/browse/IGNITE-12807 ]
Alexey Kukushkin deleted comment on IGNITE-12807:
---
was (Author: kukushal):
Closing the ticket as not accepted by the Ignite Community
> Key and Value fields with same name and SQL DML
> ---
>
> Key: IGNITE-12807
> URL: https://issues.apache.org/jira/browse/IGNITE-12807
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: sbcf
>
> Key/Value API allows both the Key and Value have fields with same name. This
> is a very popular arrangement since most users are ready to sacrifice extra
> memory footprint for the sake of having a self-sufficient value entity.
> Using SQL DML to update such an entry will update only the key field, leaving
> the value field unchanged. This is a huge usability issue for the mixed K/V
> and SQL API apps.
> h1. Proposal
> h2. Requirements
> h3. Example Data Model
> Consider a business domain entity *Person \{ id: int, passportNo: String,
> name: String }*
> Suppose an Ignite application development team decided to map the Person
> entity to Ignite data model as:
> * *PersonKey \{ id: int, passportNo: String }*
> * *Person \{ passportNo: String, name: String }*
> h3. Public API
> * *Cache API*: add new method {{setKeyValueFields(keyValueFields:
> Set): QueryEntity}} to class {{QueryEntity}}
> ** The method marks Cache API Key and Value fields that SQL API must
> initialize (on INSERT/MERGE) and update (on UPDATE/MERGE) together.
> ** It is still possible to use Cache API to initialize the fields marked
> with {{setKeyValueFields}} to different values. SQL SELECT statement returns
> value of such a field from the Key entity.
> ** The method accepts a set of field names and returns the declaring class
> instance for chaining.
> ** The method throws {{ArgumentException}} if the Key and Value types are
> available and the field types are different within the Key and Value entities.
> * *SQL API*: add {{KEY_VALUE_FIELDS}} parameter to {{CREATE TABLE}}
> statement's additional parameters list.
> ** The parameter's value is a space-separated list of field names with the
> semantics equivalent to that of the {{setKeyValueFields}} method described
> above.
> ** The parameter can be specified only if both the {{KEY_TYPE}} and
> {{VALUE_TYPE}} parameters are specified.
> h3. Use Cases
> h4. Inserting Into Key and Value Fields With Same Name Initializes Both
> Fields in QueryEntity-Defined Cache
> * GIVEN a Person cache from the example data model configured like this in
> Ignite:
> {code:java}
> new CacheConfiguration("CACHE")
> .setQueryEntities(Collections.singleton(
> new QueryEntity(PersonKey.class, Person.class)
> .addQueryField("id", int.class.getName(), null)
> .addQueryField("passportNo", String.class.getName(), null)
> .addQueryField("name", String.class.getName(), null)
> .setKeyFields(Collections.singleton("id"))
> .setKeyValueFields(Collections.singleton("passportNo"))
> ));
> {code}
> ** AND an entry is added to the cache with this SQL statement:
> {code:sql}
> INSERT INTO CACHE.Person (ID, PASSPORTNO, NAME) VALUES (1, '1', 'Name1')
> {code}
> * WHEN the user gets the entity using Cache API:
> {code:java}
> final PersonKey K = new PersonKey(1, "1");
> Person v = cache.get(K);
> {code}
> * THEN the *passportNo* field is initialized to the same value within the
> key and value entities:
> {code:java}
> assertEquals(K.passportNo, v.passportNo);
> {code}
> h4. Querying Key and Value Fields With Same Name and Different Values Returns
> Value from the Key in QueryEntity-Defined Cache
> * GIVEN a Person cache from the previous use case
> ** AND an entry with different passportNo is added to the cache with this
> Cache API:
> {code:java}
> final PersonKey K = new PersonKey(1, "1");
> final Person V = new Person("2", "Name1");
> cache.put(K, V);
> {code}
> * WHEN the user runs this SQL to get the enty:
> {code:sql}
> SELECT ID, PASSPORTNO, NAME FROM CACHE.Person
> {code}
> * THEN the retrieved PASSPORTNO is that of the Key: "1"
> h4. Inserting Into Key and Value Fields With Same Name Initializes Both
> Fields in SQL-Defined Cache
> * GIVEN a Person cache from the example data model configured like this in
> Ignite:
> {code:sql}
> CREATE TABLE Person (
> id int,
> passportNo varchar,
> name varchar,
> PRIMARY KEY(id, passportNo)
> ) WITH "key_type=PersonKey, value_type=Person, key_value_fields=passportNo"
> {code}
> ** AND an entry is added to the cache with this SQL statement:
> {code:sql}
> INSERT INTO CACHE.Person (ID, PASSPORTNO, NAME) VALUES
[jira] [Commented] (IGNITE-12807) Key and Value fields with same name and SQL DML
[
https://issues.apache.org/jira/browse/IGNITE-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17474242#comment-17474242
]
Alexey Kukushkin commented on IGNITE-12807:
---
Closing the ticket as not accepted by the Ignite Community
> Key and Value fields with same name and SQL DML
> ---
>
> Key: IGNITE-12807
> URL: https://issues.apache.org/jira/browse/IGNITE-12807
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Kukushkin
>Assignee: Alexey Kukushkin
>Priority: Major
> Labels: sbcf
>
> Key/Value API allows both the Key and Value have fields with same name. This
> is a very popular arrangement since most users are ready to sacrifice extra
> memory footprint for the sake of having a self-sufficient value entity.
> Using SQL DML to update such an entry will update only the key field, leaving
> the value field unchanged. This is a huge usability issue for the mixed K/V
> and SQL API apps.
> h1. Proposal
> h2. Requirements
> h3. Example Data Model
> Consider a business domain entity *Person \{ id: int, passportNo: String,
> name: String }*
> Suppose an Ignite application development team decided to map the Person
> entity to Ignite data model as:
> * *PersonKey \{ id: int, passportNo: String }*
> * *Person \{ passportNo: String, name: String }*
> h3. Public API
> * *Cache API*: add new method {{setKeyValueFields(keyValueFields:
> Set): QueryEntity}} to class {{QueryEntity}}
> ** The method marks Cache API Key and Value fields that SQL API must
> initialize (on INSERT/MERGE) and update (on UPDATE/MERGE) together.
> ** It is still possible to use Cache API to initialize the fields marked
> with {{setKeyValueFields}} to different values. SQL SELECT statement returns
> value of such a field from the Key entity.
> ** The method accepts a set of field names and returns the declaring class
> instance for chaining.
> ** The method throws {{ArgumentException}} if the Key and Value types are
> available and the field types are different within the Key and Value entities.
> * *SQL API*: add {{KEY_VALUE_FIELDS}} parameter to {{CREATE TABLE}}
> statement's additional parameters list.
> ** The parameter's value is a space-separated list of field names with the
> semantics equivalent to that of the {{setKeyValueFields}} method described
> above.
> ** The parameter can be specified only if both the {{KEY_TYPE}} and
> {{VALUE_TYPE}} parameters are specified.
> h3. Use Cases
> h4. Inserting Into Key and Value Fields With Same Name Initializes Both
> Fields in QueryEntity-Defined Cache
> * GIVEN a Person cache from the example data model configured like this in
> Ignite:
> {code:java}
> new CacheConfiguration("CACHE")
> .setQueryEntities(Collections.singleton(
> new QueryEntity(PersonKey.class, Person.class)
> .addQueryField("id", int.class.getName(), null)
> .addQueryField("passportNo", String.class.getName(), null)
> .addQueryField("name", String.class.getName(), null)
> .setKeyFields(Collections.singleton("id"))
> .setKeyValueFields(Collections.singleton("passportNo"))
> ));
> {code}
> ** AND an entry is added to the cache with this SQL statement:
> {code:sql}
> INSERT INTO CACHE.Person (ID, PASSPORTNO, NAME) VALUES (1, '1', 'Name1')
> {code}
> * WHEN the user gets the entity using Cache API:
> {code:java}
> final PersonKey K = new PersonKey(1, "1");
> Person v = cache.get(K);
> {code}
> * THEN the *passportNo* field is initialized to the same value within the
> key and value entities:
> {code:java}
> assertEquals(K.passportNo, v.passportNo);
> {code}
> h4. Querying Key and Value Fields With Same Name and Different Values Returns
> Value from the Key in QueryEntity-Defined Cache
> * GIVEN a Person cache from the previous use case
> ** AND an entry with different passportNo is added to the cache with this
> Cache API:
> {code:java}
> final PersonKey K = new PersonKey(1, "1");
> final Person V = new Person("2", "Name1");
> cache.put(K, V);
> {code}
> * WHEN the user runs this SQL to get the enty:
> {code:sql}
> SELECT ID, PASSPORTNO, NAME FROM CACHE.Person
> {code}
> * THEN the retrieved PASSPORTNO is that of the Key: "1"
> h4. Inserting Into Key and Value Fields With Same Name Initializes Both
> Fields in SQL-Defined Cache
> * GIVEN a Person cache from the example data model configured like this in
> Ignite:
> {code:sql}
> CREATE TABLE Person (
> id int,
> passportNo varchar,
> name varchar,
> PRIMARY KEY(id, passportNo)
> ) WITH "key_type=PersonKey, value_type=Person, key_value_fields=passportNo"
> {code}
> ** AND an entry is added to the cache with this SQL statement:
> {code:sql}
