[jira] [Comment Edited] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952260#comment-16952260 ] Pavel Tupitsyn edited comment on IGNITE-6804 at 10/15/19 8:14 PM: -- [~ilyak] if you add items to a new empty hashtable, the resulting order is going to be the same every time. And, I guess, this is the most common scenario (e.g. I have a list of Persons, I want to put them all to cache - I have to create and populate a Map). This also indicates a problem with the API. We waste CPU time populating HashMap or SortedMap, when Ignite only iterates over the result, throwing away all the hash-related stuff. Anyway, you can keep "will" if you like, but it is better to be technically correct (the best kind of correct). was (Author: ptupitsyn): [~ilyak] if you add items to a new empty hashtable, the resulting order is going to be the same every time. And, I guess, this is the most common scenario. This also indicates a problem with the API. We waste CPU time populating HashMap or SortedMap, when Ignite only iterates over the result, throwing away all the hash-related stuff. Anyway, you can keep "will" if you like, but it is better to be technically correct (the best kind of correct). > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 10m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952260#comment-16952260 ] Pavel Tupitsyn commented on IGNITE-6804: [~ilyak] if you add items to a new empty hashtable, the resulting order is going to be the same every time. And, I guess, this is the most common scenario. This also indicates a problem with the API. We waste CPU time populating HashMap or SortedMap, when Ignite only iterates over the result, throwing away all the hash-related stuff. Anyway, you can keep "will" if you like, but it is better to be technically correct (the best kind of correct). > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 10m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12293) Thin client: config serialization should use OpCodes
[ https://issues.apache.org/jira/browse/IGNITE-12293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-12293: Ignite Flags: (was: Docs Required,Release Notes Required) > Thin client: config serialization should use OpCodes > > > Key: IGNITE-12293 > URL: https://issues.apache.org/jira/browse/IGNITE-12293 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.7.6 >Reporter: Alexandr Shapkin >Assignee: Alexandr Shapkin >Priority: Major > Labels: platforms > Fix For: 2.9 > > > Thin client configuration could be partially serialized via > ClientCacheConfigurationSerializer.java > But this #OpCode based serialization is limited only to .NET -> Java > direction. > Let's use OpCodes as a default implementation for all supported platforms, > this will help us to add a new configuration section only for a specific > platform. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12293) .NET thin client: config serialization should use OpCodes
Alexandr Shapkin created IGNITE-12293: - Summary: .NET thin client: config serialization should use OpCodes Key: IGNITE-12293 URL: https://issues.apache.org/jira/browse/IGNITE-12293 Project: Ignite Issue Type: Improvement Components: thin client Affects Versions: 2.7.6 Reporter: Alexandr Shapkin Assignee: Alexandr Shapkin Fix For: 2.9 Thin client configuration could be partially serialized via ClientCacheConfigurationSerializer.java But this #OpCode based serialization is limited only to .NET -> Java direction. Let's use OpCodes as a default implementation for all supported platforms, this will help us to add a new configuration section only for a specific platform. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12293) Thin client: config serialization should use OpCodes
[ https://issues.apache.org/jira/browse/IGNITE-12293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandr Shapkin updated IGNITE-12293: -- Summary: Thin client: config serialization should use OpCodes (was: .NET thin client: config serialization should use OpCodes) > Thin client: config serialization should use OpCodes > > > Key: IGNITE-12293 > URL: https://issues.apache.org/jira/browse/IGNITE-12293 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.7.6 >Reporter: Alexandr Shapkin >Assignee: Alexandr Shapkin >Priority: Major > Labels: platforms > Fix For: 2.9 > > > Thin client configuration could be partially serialized via > ClientCacheConfigurationSerializer.java > But this #OpCode based serialization is limited only to .NET -> Java > direction. > Let's use OpCodes as a default implementation for all supported platforms, > this will help us to add a new configuration section only for a specific > platform. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12292) Java thin client: In some cases txId intersection (tx started on different nodes) leads to errors
[
https://issues.apache.org/jira/browse/IGNITE-12292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-12292:
---
Ignite Flags: (was: Docs Required,Release Notes Required)
> Java thin client: In some cases txId intersection (tx started on different
> nodes) leads to errors
> -
>
> Key: IGNITE-12292
> URL: https://issues.apache.org/jira/browse/IGNITE-12292
> Project: Ignite
> Issue Type: Bug
> Components: thin client
>Affects Versions: 2.8
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Fix For: 2.8
>
>
> Reproducer:
> {code:java}
> public void testTxWithIdIntersection() throws Exception {
> int CLUSTER_SIZE = 2;
> try (LocalIgniteCluster cluster = LocalIgniteCluster.start(CLUSTER_SIZE);
> IgniteClient client = Ignition.startClient(new ClientConfiguration()
> .setAddresses(cluster.clientAddresses().toArray(new
> String[CLUSTER_SIZE])))
> ) {
> ClientCache cache = client.createCache(new
> ClientCacheConfiguration().setName("cache")
> .setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL));
> CyclicBarrier barrier = new CyclicBarrier(2);
> GridTestUtils.runAsync(() -> {
> try {
> // Another thread starts transaction here.
> barrier.await(1, TimeUnit.SECONDS);
> for (int i = 0; i < CLUSTER_SIZE; i++)
> dropAllThinClientConnections(Ignition.allGrids().get(i));
> ClientTransaction tx = client.transactions().txStart();
> barrier.await(1, TimeUnit.SECONDS);
> // Another thread puts to cache here.
> barrier.await(1, TimeUnit.SECONDS);
> tx.commit();
> barrier.await(1, TimeUnit.SECONDS);
> }
> catch (Exception e) {
> log.error("Unexpected error", e);
> }
> });
> ClientTransaction tx = client.transactions().txStart();
> barrier.await(1, TimeUnit.SECONDS);
> // Another thread drops connections and create new transaction here,
> which started on another node with the
> // same transaction id as we started in this thread.
> barrier.await(1, TimeUnit.SECONDS);
> try {
> cache.put(0, 0);
> fail("Exception expected");
> }
> catch (ClientException expected) {
> // No-op.
> }
> tx.close();
> barrier.await(1, TimeUnit.SECONDS);
> // Another thread commit transaction here.
> barrier.await(1, TimeUnit.SECONDS);
> assertFalse(cache.containsKey(0));
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12292) Java thin client: In some cases txId intersection (tx started on different nodes) leads to errors
Aleksey Plekhanov created IGNITE-12292:
--
Summary: Java thin client: In some cases txId intersection (tx
started on different nodes) leads to errors
Key: IGNITE-12292
URL: https://issues.apache.org/jira/browse/IGNITE-12292
Project: Ignite
Issue Type: Bug
Components: thin client
Affects Versions: 2.8
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Fix For: 2.8
Reproducer:
{code:java}
public void testTxWithIdIntersection() throws Exception {
int CLUSTER_SIZE = 2;
try (LocalIgniteCluster cluster = LocalIgniteCluster.start(CLUSTER_SIZE);
IgniteClient client = Ignition.startClient(new ClientConfiguration()
.setAddresses(cluster.clientAddresses().toArray(new
String[CLUSTER_SIZE])))
) {
ClientCache cache = client.createCache(new
ClientCacheConfiguration().setName("cache")
.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL));
CyclicBarrier barrier = new CyclicBarrier(2);
GridTestUtils.runAsync(() -> {
try {
// Another thread starts transaction here.
barrier.await(1, TimeUnit.SECONDS);
for (int i = 0; i < CLUSTER_SIZE; i++)
dropAllThinClientConnections(Ignition.allGrids().get(i));
ClientTransaction tx = client.transactions().txStart();
barrier.await(1, TimeUnit.SECONDS);
// Another thread puts to cache here.
barrier.await(1, TimeUnit.SECONDS);
tx.commit();
barrier.await(1, TimeUnit.SECONDS);
}
catch (Exception e) {
log.error("Unexpected error", e);
}
});
ClientTransaction tx = client.transactions().txStart();
barrier.await(1, TimeUnit.SECONDS);
// Another thread drops connections and create new transaction here,
which started on another node with the
// same transaction id as we started in this thread.
barrier.await(1, TimeUnit.SECONDS);
try {
cache.put(0, 0);
fail("Exception expected");
}
catch (ClientException expected) {
// No-op.
}
tx.close();
barrier.await(1, TimeUnit.SECONDS);
// Another thread commit transaction here.
barrier.await(1, TimeUnit.SECONDS);
assertFalse(cache.containsKey(0));
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12284) When service class not found on any server nodes, service not deployed without any error
[
https://issues.apache.org/jira/browse/IGNITE-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952038#comment-16952038
]
Denis Mekhanikov edited comment on IGNITE-12284 at 10/15/19 3:42 PM:
-
The error should be propagated to the deployer's site after IGNITE-3392
Current master shouldn't have this issue unless
{{IGNITE_EVENT_DRIVEN_SERVICE_PROCESSOR_ENABLED}} property is set to false.
was (Author: dmekhanikov):
The error should be propagated to the deployer's site after IGNITE-3392
Current master shouldn't have this issue.
> When service class not found on any server nodes, service not deployed
> without any error
>
>
> Key: IGNITE-12284
> URL: https://issues.apache.org/jira/browse/IGNITE-12284
> Project: Ignite
> Issue Type: Bug
> Components: managed services
>Reporter: Ilya Kasnacheev
>Priority: Major
>
> When service class presents on client node but not on server node, the
> following is printed on server:
> {code}
> [17:57:43,398][SEVERE][srvc-deploy-#44][GridServiceProcessor] Failed to
> initialize service (service will not be deployed): FService
> class org.apache.ignite.IgniteCheckedException:
> com.gridgain.datetest.MyService
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10174)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.copyAndInject(GridServiceProcessor.java:1440)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.redeploy(GridServiceProcessor.java:1361)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.processAssignment(GridServiceProcessor.java:1988)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.onSystemCacheUpdated(GridServiceProcessor.java:1615)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.access$300(GridServiceProcessor.java:126)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor$ServiceEntriesListener$1.run0(GridServiceProcessor.java:1597)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor$DepRunnable.run(GridServiceProcessor.java:2064)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
> com.gridgain.datetest.MyService
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:707)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1758)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1717)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10168)
> ... 10 more
> Caused by: java.lang.ClassNotFoundException: com.gridgain.datetest.MyService
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8775)
> at
> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:349)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:698)
> ... 16 more
> {code}
> but on client, ServiceDeploymentException is not thrown. Instead, deploy
> returns normally.
> Code to reproduce:
> {code}
> public class ServiceStarterMain {
> public static void main(String[] args) {
> Ignition.setClientMode(true);
> Ignite ignite = Ignition.start();
> ServiceConfiguration serviceConfiguration = new
> ServiceConfiguration();
> serviceConfiguration.setName("FService");
> serviceConfiguration.setMaxPerNodeCount(4);
> serviceConfiguration.setTotalCount(10);
> serviceConfiguration.setService(new
[jira] [Commented] (IGNITE-12284) When service class not found on any server nodes, service not deployed without any error
[
https://issues.apache.org/jira/browse/IGNITE-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952038#comment-16952038
]
Denis Mekhanikov commented on IGNITE-12284:
---
The error should be propagated to the deployer's site after IGNITE-3392
Current master shouldn't have this issue.
> When service class not found on any server nodes, service not deployed
> without any error
>
>
> Key: IGNITE-12284
> URL: https://issues.apache.org/jira/browse/IGNITE-12284
> Project: Ignite
> Issue Type: Bug
> Components: managed services
>Reporter: Ilya Kasnacheev
>Priority: Major
>
> When service class presents on client node but not on server node, the
> following is printed on server:
> {code}
> [17:57:43,398][SEVERE][srvc-deploy-#44][GridServiceProcessor] Failed to
> initialize service (service will not be deployed): FService
> class org.apache.ignite.IgniteCheckedException:
> com.gridgain.datetest.MyService
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10174)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.copyAndInject(GridServiceProcessor.java:1440)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.redeploy(GridServiceProcessor.java:1361)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.processAssignment(GridServiceProcessor.java:1988)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.onSystemCacheUpdated(GridServiceProcessor.java:1615)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.access$300(GridServiceProcessor.java:126)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor$ServiceEntriesListener$1.run0(GridServiceProcessor.java:1597)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor$DepRunnable.run(GridServiceProcessor.java:2064)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
> com.gridgain.datetest.MyService
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:707)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1758)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1717)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10168)
> ... 10 more
> Caused by: java.lang.ClassNotFoundException: com.gridgain.datetest.MyService
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8775)
> at
> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:349)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:698)
> ... 16 more
> {code}
> but on client, ServiceDeploymentException is not thrown. Instead, deploy
> returns normally.
> Code to reproduce:
> {code}
> public class ServiceStarterMain {
> public static void main(String[] args) {
> Ignition.setClientMode(true);
> Ignite ignite = Ignition.start();
> ServiceConfiguration serviceConfiguration = new
> ServiceConfiguration();
> serviceConfiguration.setName("FService");
> serviceConfiguration.setMaxPerNodeCount(4);
> serviceConfiguration.setTotalCount(10);
> serviceConfiguration.setService(new MyService());
> ignite.services().deploy(serviceConfiguration); // Exception
> expected, but does not happen
> }
> }
> {code}
> against a blank Ignite node started from bin distro.
> This affects Java and .Net.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952025#comment-16952025 ] Igor Sapego commented on IGNITE-6804: - [~ilyak] C++ changes looks good to me. > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 10m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11704) Write tombstones during rebalance to get rid of deferred delete buffer
[ https://issues.apache.org/jira/browse/IGNITE-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952020#comment-16952020 ] Alexei Scherbakov commented on IGNITE-11704: [~jokser] 4. Typo in the comment. My understanding is the code will be called when datastreamer initiates first update for an entry, is it true ? 6. * Looks like it's not necessary to preload 256k keys for historical rebalance, you need only one update in each partition. * Test looks similar but my idea is to delay each batch and remove all containing keys in the batch, then release batch. Such scenario should bring all partition keys to tombstones and looks interesting. In other aspects looks good. > Write tombstones during rebalance to get rid of deferred delete buffer > -- > > Key: IGNITE-11704 > URL: https://issues.apache.org/jira/browse/IGNITE-11704 > Project: Ignite > Issue Type: Improvement >Reporter: Alexey Goncharuk >Assignee: Pavel Kovalenko >Priority: Major > Labels: rebalance > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > Currently Ignite relies on deferred delete buffer in order to handle > write-remove conflicts during rebalance. Given the limit size of the buffer, > this approach is fundamentally flawed, especially in case when persistence is > enabled. > I suggest to extend the logic of data storage to be able to store key > tombstones - to keep version for deleted entries. The tombstones will be > stored when rebalance is in progress and should be cleaned up when rebalance > is completed. > Later this approach may be used to implement fast partition rebalance based > on merkle trees (in this case, tombstones should be written on an incomplete > baseline). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12189) Implement correct limit for TextQuery
[ https://issues.apache.org/jira/browse/IGNITE-12189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952012#comment-16952012 ] Yuriy Shuliha commented on IGNITE-12189: - [~amashenkov] IGNITE-12291 ticket was created and linekd > Implement correct limit for TextQuery > - > > Key: IGNITE-12189 > URL: https://issues.apache.org/jira/browse/IGNITE-12189 > Project: Ignite > Issue Type: Improvement > Components: general >Reporter: Yuriy Shuliha >Assignee: Yuriy Shuliha >Priority: Major > Fix For: 2.8 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > PROBLEM > For now each server-node returns all response records to the client-node and > it may contain ~thousands, ~hundred thousands records. > Event if we need only first 10-100. Again, all the results are added to > queue in _*GridCacheQueryFutureAdapter*_ in arbitrary order by pages. > There are no any means to deliver deterministic result. > SOLUTION > Implement _*limit*_ as parameter for _*TextQuery*_ and > _*GridCacheQueryRequest*_ > It should be passed as limit parameter in Lucene's > _*IndexSearcher.search()*_ in _*GridLuceneIndex*_. > For distributed queries _*limit*_ will also trim response queue when merging > results. > Type: long > Special value: : 0 -> No limit (Integer.MAX_VALUE); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12189) Implement correct limit for TextQuery
[ https://issues.apache.org/jira/browse/IGNITE-12189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952012#comment-16952012 ] Yuriy Shuliha edited comment on IGNITE-12189 at 10/15/19 3:17 PM: --- [~amashenkov] IGNITE-12291 ticket was created and linked was (Author: yuriy_shuliha): [~amashenkov] IGNITE-12291 ticket was created and linekd > Implement correct limit for TextQuery > - > > Key: IGNITE-12189 > URL: https://issues.apache.org/jira/browse/IGNITE-12189 > Project: Ignite > Issue Type: Improvement > Components: general >Reporter: Yuriy Shuliha >Assignee: Yuriy Shuliha >Priority: Major > Fix For: 2.8 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > PROBLEM > For now each server-node returns all response records to the client-node and > it may contain ~thousands, ~hundred thousands records. > Event if we need only first 10-100. Again, all the results are added to > queue in _*GridCacheQueryFutureAdapter*_ in arbitrary order by pages. > There are no any means to deliver deterministic result. > SOLUTION > Implement _*limit*_ as parameter for _*TextQuery*_ and > _*GridCacheQueryRequest*_ > It should be passed as limit parameter in Lucene's > _*IndexSearcher.search()*_ in _*GridLuceneIndex*_. > For distributed queries _*limit*_ will also trim response queue when merging > results. > Type: long > Special value: : 0 -> No limit (Integer.MAX_VALUE); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12291) Create controllable paged query requests / responses for TextQuery similar to current SQL result processing
Yuriy Shuliha created IGNITE-12291: --- Summary: Create controllable paged query requests / responses for TextQuery similar to current SQL result processing Key: IGNITE-12291 URL: https://issues.apache.org/jira/browse/IGNITE-12291 Project: Ignite Issue Type: Improvement Reporter: Yuriy Shuliha Assignee: Yuriy Shuliha For now query initiator node sends 1 _GridCacheQueryRequest_ and can get multiple _GridCacheQueryResponse_-s per remote. TextQuery processing finishes when all remotes send _GridCacheQueryResponse_ with 'finished' flag. _TextQuery_ processing should be reworked like it was done for SQL queries. Ignite has _GridQueryNextPageRequest \ Response_ classes for SQL result processing. Similar processing should be implemented for _TextQueries_: *GridTextQueryNextPageRequest* *GridTextQueryNextPageResponse* Proper _TextQuery_ response processing should be implemented in _GridCacheQueryFutureAdapter._ This will allow us to add correct sorting and apply limit correctly on reduce. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (IGNITE-10771) Print troubleshooting hint when exchange latch got stucked
[
https://issues.apache.org/jira/browse/IGNITE-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Kovalenko resolved IGNITE-10771.
--
Resolution: Fixed
> Print troubleshooting hint when exchange latch got stucked
> --
>
> Key: IGNITE-10771
> URL: https://issues.apache.org/jira/browse/IGNITE-10771
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.5
>Reporter: Pavel Kovalenko
>Assignee: Pavel Kovalenko
>Priority: Minor
> Labels: usability
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Sometimes users face with a problem when exchange latch can't be completed:
> {noformat}
> 2018-12-12 07:07:57:563 [exchange-worker-#42] WARN
> o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture:488 - Unable to await
> partitions release latch within timeout: ClientLatch
> [coordinator=ZookeeperClusterNode [id=6b9fc6e4-5b6a-4a98-be4d-6bc1aa5c014c,
> addrs=[172.17.0.1, 10.0.230.117, 0:0:0:0:0:0:0:1%lo, 127.0.0.1], order=3,
> loc=false, client=false], ackSent=true, super=CompletableLatch [id=exchange,
> topVer=AffinityTopologyVersion [topVer=45, minorTopVer=1]]]
> {noformat}
> It may indicate that some node in a cluster can' t finish partitions release
> (finish all ongoing operations at the previous topology version) or it can be
> silent network problem.
> We should print to log a hint how to troubleshoot it to reduce the number of
> questions about such problem.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-10771) Print troubleshooting hint when exchange latch got stucked
[
https://issues.apache.org/jira/browse/IGNITE-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952000#comment-16952000
]
Pavel Kovalenko commented on IGNITE-10771:
--
Merged to master.
> Print troubleshooting hint when exchange latch got stucked
> --
>
> Key: IGNITE-10771
> URL: https://issues.apache.org/jira/browse/IGNITE-10771
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.5
>Reporter: Pavel Kovalenko
>Assignee: Pavel Kovalenko
>Priority: Minor
> Labels: usability
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Sometimes users face with a problem when exchange latch can't be completed:
> {noformat}
> 2018-12-12 07:07:57:563 [exchange-worker-#42] WARN
> o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture:488 - Unable to await
> partitions release latch within timeout: ClientLatch
> [coordinator=ZookeeperClusterNode [id=6b9fc6e4-5b6a-4a98-be4d-6bc1aa5c014c,
> addrs=[172.17.0.1, 10.0.230.117, 0:0:0:0:0:0:0:1%lo, 127.0.0.1], order=3,
> loc=false, client=false], ackSent=true, super=CompletableLatch [id=exchange,
> topVer=AffinityTopologyVersion [topVer=45, minorTopVer=1]]]
> {noformat}
> It may indicate that some node in a cluster can' t finish partitions release
> (finish all ongoing operations at the previous topology version) or it can be
> silent network problem.
> We should print to log a hint how to troubleshoot it to reduce the number of
> questions about such problem.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-10771) Print troubleshooting hint when exchange latch got stucked
[
https://issues.apache.org/jira/browse/IGNITE-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Kovalenko updated IGNITE-10771:
-
Ignite Flags: (was: Docs Required)
> Print troubleshooting hint when exchange latch got stucked
> --
>
> Key: IGNITE-10771
> URL: https://issues.apache.org/jira/browse/IGNITE-10771
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.5
>Reporter: Pavel Kovalenko
>Assignee: Pavel Kovalenko
>Priority: Minor
> Labels: usability
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Sometimes users face with a problem when exchange latch can't be completed:
> {noformat}
> 2018-12-12 07:07:57:563 [exchange-worker-#42] WARN
> o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture:488 - Unable to await
> partitions release latch within timeout: ClientLatch
> [coordinator=ZookeeperClusterNode [id=6b9fc6e4-5b6a-4a98-be4d-6bc1aa5c014c,
> addrs=[172.17.0.1, 10.0.230.117, 0:0:0:0:0:0:0:1%lo, 127.0.0.1], order=3,
> loc=false, client=false], ackSent=true, super=CompletableLatch [id=exchange,
> topVer=AffinityTopologyVersion [topVer=45, minorTopVer=1]]]
> {noformat}
> It may indicate that some node in a cluster can' t finish partitions release
> (finish all ongoing operations at the previous topology version) or it can be
> silent network problem.
> We should print to log a hint how to troubleshoot it to reduce the number of
> questions about such problem.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-11852) Assertion errors when changing PME coordinator to locally joining node
[
https://issues.apache.org/jira/browse/IGNITE-11852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951994#comment-16951994
]
Ignite TC Bot commented on IGNITE-11852:
{panel:title=Branch: [pull/6539/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4696234buildTypeId=IgniteTests24Java8_RunAll]
> Assertion errors when changing PME coordinator to locally joining node
> --
>
> Key: IGNITE-11852
> URL: https://issues.apache.org/jira/browse/IGNITE-11852
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.5, 2.7
>Reporter: Pavel Kovalenko
>Assignee: Pavel Kovalenko
>Priority: Critical
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When PME coordinator changed to locally joining node several assertion errors
> may occur:
> 1. When some other joining nodes finished PME:
> {noformat}
> [13:49:58] (err) Failed to notify listener:
> o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$8$1$1...@27296181java.lang.AssertionError:
> AffinityTopologyVersion [topVer=2, minorTopVer=0]
> at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager$11.applyx(CacheAffinitySharedManager.java:1546)
> at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager$11.applyx(CacheAffinitySharedManager.java:1535)
> at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.lambda$forAllRegisteredCacheGroups$e0a6939d$1(CacheAffinitySharedManager.java:1281)
> at
> org.apache.ignite.internal.util.IgniteUtils.doInParallel(IgniteUtils.java:10929)
> at
> org.apache.ignite.internal.util.IgniteUtils.doInParallel(IgniteUtils.java:10831)
> at
> org.apache.ignite.internal.util.IgniteUtils.doInParallel(IgniteUtils.java:10811)
> at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.forAllRegisteredCacheGroups(CacheAffinitySharedManager.java:1280)
> at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onLocalJoin(CacheAffinitySharedManager.java:1535)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processFullMessage(GridDhtPartitionsExchangeFuture.java:4189)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onBecomeCoordinator(GridDhtPartitionsExchangeFuture.java:4731)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$3400(GridDhtPartitionsExchangeFuture.java:145)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$8$1$1.apply(GridDhtPartitionsExchangeFuture.java:4622)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$8$1$1.apply(GridDhtPartitionsExchangeFuture.java:4611)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:398)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:510)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:489)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:466)
> at
> org.apache.ignite.internal.util.future.GridCompoundFuture.checkComplete(GridCompoundFuture.java:281)
> at
> org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:143)
> at
> org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:44)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:398)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:510)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:489)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:455)
> at
>
[jira] [Commented] (IGNITE-11704) Write tombstones during rebalance to get rid of deferred delete buffer
[ https://issues.apache.org/jira/browse/IGNITE-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951983#comment-16951983 ] Pavel Kovalenko commented on IGNITE-11704: -- [~ascherbakov] I've fixed several of your concerns others are commented: 1. Tombstone object storing has optimized: Any object (key or value) has a header (object len + object type). The object type can be regular, binary or byte array. In the previous version, the tombstone will be regular cache object with marshalled "null" value. In current version, I introduced a special type of object - Tombstone that doesn't store any value, only header. All checking for the tombstone has optimized. 2. I think it's fine. Every clear tombstone task periodically check is partition become in not OWNING state. In this case, a clear tombstones operation is stopped. Yes, there can be a window of time where both clear tombstones and eviction can happen, but it shouldn't be long. 3. DropCacheContextDuringEvictionTest is reworked due to reuse test PartitionsEvictManagerAbstractTest for checking tomstones failure. cacheGroupMetricsRegistryName is added as a utility method as part of cache group tombstone metrics. GridCommandHandlerIndexingTest - merge artifact, should be ignored. 4. I've added a comment when this condition is true. 5. This test already exists (org.apache.ignite.internal.processors.cache.distributed.CacheRemoveWithTombstonesTest#testRemoveAndRebalanceRaceTx) 6. I've reworked code and now clearAll and clearTombstones have a common codebase. Could you please review it again? > Write tombstones during rebalance to get rid of deferred delete buffer > -- > > Key: IGNITE-11704 > URL: https://issues.apache.org/jira/browse/IGNITE-11704 > Project: Ignite > Issue Type: Improvement >Reporter: Alexey Goncharuk >Assignee: Pavel Kovalenko >Priority: Major > Labels: rebalance > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > Currently Ignite relies on deferred delete buffer in order to handle > write-remove conflicts during rebalance. Given the limit size of the buffer, > this approach is fundamentally flawed, especially in case when persistence is > enabled. > I suggest to extend the logic of data storage to be able to store key > tombstones - to keep version for deleted entries. The tombstones will be > stored when rebalance is in progress and should be cleaned up when rebalance > is completed. > Later this approach may be used to implement fast partition rebalance based > on merkle trees (in this case, tombstones should be written on an incomplete > baseline). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951967#comment-16951967 ] Ilya Kasnacheev commented on IGNITE-6804: - [~ptupitsyn] There's practically no way to not cause deadlock with HashMap so it makes sense to write "will". It's like with gets() function whose documentation starts with "+Never use this function.+" > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 10m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951963#comment-16951963 ] Ilya Kasnacheev commented on IGNITE-6804: - [~dmagda] please review proposed documentation changes and developer warnings. > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 10m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951962#comment-16951962 ] Ilya Kasnacheev commented on IGNITE-6804: - Failure not related to my change. > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 10m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[
https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Kasnacheev updated IGNITE-6804:
Comment: was deleted
(was: {panel:title=Branch: [pull/6976/head] Base: [master] : Possible Blockers
(113)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Platform C++ (Linux)*{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696766]]
{color:#d04437}Cache 6{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696795]]
{color:#d04437}SPI{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696757]]
{color:#d04437}ZooKeeper (Discovery) 3{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696819]]
{color:#d04437}ZooKeeper (Discovery) 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696764]]
{color:#d04437}Scala (Examples){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696762]]
{color:#d04437}Platform .NET (Long Running){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696815]]
{color:#d04437}Examples{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696738]]
{color:#d04437}PDS (Compatibility){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696804]]
{color:#d04437}Scala (Visor Console){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696763]]
{color:#d04437}RDD{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696751]]
{color:#d04437}PDS (Indexing){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696805]]
{color:#d04437}Cache (Expiry Policy){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696780]]
{color:#d04437}ZooKeeper (Discovery) 2{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696765]]
{color:#d04437}Streamers{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696755]]
{color:#d04437}Cache 2{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696791]]
{color:#d04437}Start Nodes{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696756]]
{color:#d04437}Platform .NET{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696811]]
{color:#d04437}Platform .NET (Core Linux){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696812]]
{color:#d04437}MVCC Queries{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696775]]
{color:#d04437}Continuous Query 3{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696800]]
{color:#d04437}Cache 5{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696794]]
{color:#d04437}PDS 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696807]]
{color:#d04437}Queries 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696817]]
{color:#d04437}ZooKeeper (Discovery) 4{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696820]]
{color:#d04437}MVCC PDS 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696834]]
{color:#d04437}Cache 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696790]]
{color:#d04437}Thin Client: Java{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696748]]
{color:#d04437}Compute (Grid){color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696735]]
{color:#d04437}Basic 3{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696770]]
{color:#d04437}Data Structures{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696802]]
{color:#d04437}MVCC PDS 2{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696835]]
{color:#d04437}Cache 7{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696796]]
{color:#d04437}Continuous Query 4{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696801]]
{color:#d04437}Basic 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696769]]
{color:#d04437}MVCC Cache 1{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696825]]
{color:#d04437}Cache 8{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696797]]
{color:#d04437}Cache 3{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696792]]
{color:#d04437}PDS 3{color} [[tests 0
CANCELLED|https://ci.ignite.apache.org/viewLog.html?buildId=4696809]]
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[
https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951960#comment-16951960
]
Ignite TC Bot commented on IGNITE-6804:
---
{panel:title=Branch: [pull/6976/head] Base: [master] : Possible Blockers
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Platform .NET (Inspections)*{color} [[tests 0 Failure on metric
|https://ci.ignite.apache.org/viewLog.html?buildId=4698855]]
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4697381buildTypeId=IgniteTests24Java8_RunAll]
> Print a warning if HashMap is passed into bulk update operations
>
>
> Key: IGNITE-6804
> URL: https://issues.apache.org/jira/browse/IGNITE-6804
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Reporter: Denis A. Magda
>Assignee: Ilya Kasnacheev
>Priority: Critical
> Labels: usability
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ignite newcomers tend to stumble on deadlocks simply because the keys are
> passed in an unordered HashMap. Propose to do the following:
> * update bulk operations Java docs.
> * print out a warning if not SortedMap (e.g. HashMap,
> Weak/Identity/Concurrent/Linked HashMap etc) is passed into
> a bulk method (instead of SortedMap) and contains more than 1 element.
> However, we should make sure that we only print that warning once and not
> every time the API is called.
> * do not produce warning for explicit optimistic transactions
> More details are here:
> http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-8617) Node Discovery Using AWS Application ELB
[ https://issues.apache.org/jira/browse/IGNITE-8617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951899#comment-16951899 ] Stanislav Lukyanov commented on IGNITE-8617: [~dmagda] [~uday] [~mmuzaf] LGTM. Maxim, could you please help to merge this? > Node Discovery Using AWS Application ELB > > > Key: IGNITE-8617 > URL: https://issues.apache.org/jira/browse/IGNITE-8617 > Project: Ignite > Issue Type: New Feature > Components: aws, documentation >Reporter: Uday Kale >Assignee: Uday Kale >Priority: Major > Fix For: 2.8 > > > Support for Node discovery using AWS Application ELB. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12252) Unchecked exceptions during rebalancing should be handled
[
https://issues.apache.org/jira/browse/IGNITE-12252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951880#comment-16951880
]
Anton Vinogradov commented on IGNITE-12252:
---
[~zzzadruga],
Some other pools seems to be even more critical that rebalance pool.
[~alex_pl],
could you provide some tips as an author of oomeHnd?
> Unchecked exceptions during rebalancing should be handled
> -
>
> Key: IGNITE-12252
> URL: https://issues.apache.org/jira/browse/IGNITE-12252
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Nikolai Kulagin
>Priority: Critical
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Rebalancing should handle unchecked exceptions by failure handler. In current
> implementation unchecked exceptions just ignored. They were handled by IO
> worker before IGNITE-3195.
> Reproducer:
> {code:java}
> @Test
> public void testRebalanceUncheckedError() throws Exception {
> IgniteEx ignite0 = startGrid(new
> IgniteConfiguration().setIgniteInstanceName("ignite0"));
> IgniteCache cache =
> ignite0.getOrCreateCache(DEFAULT_CACHE_NAME);
> IgniteDataStreamer streamer =
> ignite0.dataStreamer(DEFAULT_CACHE_NAME);
> for (int i = 0; i < 100_000; i++)
> streamer.addData(i, i);
> streamer.flush();
> IgniteEx ignite1 = startGrid(new
> IgniteConfiguration().setIgniteInstanceName("ignite1")
>
> .setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_OBJECT_LOADED));
> ignite1.events().localListen(e -> {
> throw new Error();
> }, EventType.EVT_CACHE_REBALANCE_OBJECT_LOADED);
> awaitPartitionMapExchange();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12252) Unchecked exceptions during rebalancing should be handled
[
https://issues.apache.org/jira/browse/IGNITE-12252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951844#comment-16951844
]
Nikolai Kulagin commented on IGNITE-12252:
--
[~avinogradov],
I believe that other thread pools are not critical, it is worth handling
exceptions the only if it is OOM. Rebalance is a critical operation, and all
exceptions must be handled by failure handler.
> Unchecked exceptions during rebalancing should be handled
> -
>
> Key: IGNITE-12252
> URL: https://issues.apache.org/jira/browse/IGNITE-12252
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Nikolai Kulagin
>Priority: Critical
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Rebalancing should handle unchecked exceptions by failure handler. In current
> implementation unchecked exceptions just ignored. They were handled by IO
> worker before IGNITE-3195.
> Reproducer:
> {code:java}
> @Test
> public void testRebalanceUncheckedError() throws Exception {
> IgniteEx ignite0 = startGrid(new
> IgniteConfiguration().setIgniteInstanceName("ignite0"));
> IgniteCache cache =
> ignite0.getOrCreateCache(DEFAULT_CACHE_NAME);
> IgniteDataStreamer streamer =
> ignite0.dataStreamer(DEFAULT_CACHE_NAME);
> for (int i = 0; i < 100_000; i++)
> streamer.addData(i, i);
> streamer.flush();
> IgniteEx ignite1 = startGrid(new
> IgniteConfiguration().setIgniteInstanceName("ignite1")
>
> .setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_OBJECT_LOADED));
> ignite1.events().localListen(e -> {
> throw new Error();
> }, EventType.EVT_CACHE_REBALANCE_OBJECT_LOADED);
> awaitPartitionMapExchange();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-10771) Print troubleshooting hint when exchange latch got stucked
[
https://issues.apache.org/jira/browse/IGNITE-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951775#comment-16951775
]
Ignite TC Bot commented on IGNITE-10771:
{panel:title=Branch: [pull/6968/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4688174buildTypeId=IgniteTests24Java8_RunAll]
> Print troubleshooting hint when exchange latch got stucked
> --
>
> Key: IGNITE-10771
> URL: https://issues.apache.org/jira/browse/IGNITE-10771
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Affects Versions: 2.5
>Reporter: Pavel Kovalenko
>Assignee: Pavel Kovalenko
>Priority: Minor
> Labels: usability
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Sometimes users face with a problem when exchange latch can't be completed:
> {noformat}
> 2018-12-12 07:07:57:563 [exchange-worker-#42] WARN
> o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture:488 - Unable to await
> partitions release latch within timeout: ClientLatch
> [coordinator=ZookeeperClusterNode [id=6b9fc6e4-5b6a-4a98-be4d-6bc1aa5c014c,
> addrs=[172.17.0.1, 10.0.230.117, 0:0:0:0:0:0:0:1%lo, 127.0.0.1], order=3,
> loc=false, client=false], ackSent=true, super=CompletableLatch [id=exchange,
> topVer=AffinityTopologyVersion [topVer=45, minorTopVer=1]]]
> {noformat}
> It may indicate that some node in a cluster can' t finish partitions release
> (finish all ongoing operations at the previous topology version) or it can be
> silent network problem.
> We should print to log a hint how to troubleshoot it to reduce the number of
> questions about such problem.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951734#comment-16951734 ] Pavel Tupitsyn commented on IGNITE-6804: [~ilyak] # I've fixed collection class references in the docs (generic specification is mandatory there), pushed the branch (hope it's ok for you). # "will lead to deadlock" is not technically correct, I would change it to "may lead to deadlock". # Yes, we can do it in C# the same way as in Java. But I don't see how the warning is "throttled" - looks like it will be displayed on every operation, am I missing something? > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 10m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11351) Basic "Starting Ignite inside an OSGi container" doesn't work
[ https://issues.apache.org/jira/browse/IGNITE-11351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951702#comment-16951702 ] Sergey Rymsha commented on IGNITE-11351: classloader issue is probably related to IGNITE-11662 > Basic "Starting Ignite inside an OSGi container" doesn't work > - > > Key: IGNITE-11351 > URL: https://issues.apache.org/jira/browse/IGNITE-11351 > Project: Ignite > Issue Type: Bug > Components: osgi >Affects Versions: 2.7 > Environment: Apache Felix, JDK 1.8 >Reporter: Xander Uiterlinden >Priority: Major > Attachments: ignite-osgi-patch.diff > > > When running the example from > [https://apacheignite.readme.io/v1.5/docs/osgi-starting-inside-a-container] > you're getting the following error. > Failed to start Ignite via OSGi Activator [errMsg=Failed to find empty > constructor for class: > o.a.i.i.processors.platform.websession.PlatformDotNetSessionLockResult]class > Looking at the codebase this class does indeed not have an empty constructor. > When adding that, you'll be running into the next issue which is a > classloader issue. Ignite needs to be started with the correct context > classloader for it to start successfully. > Please find the attached patch that resolves both problems. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12285) Remove boilerplate code in test PluginProvider implementations.
[ https://issues.apache.org/jira/browse/IGNITE-12285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PetrovMikhail updated IGNITE-12285: --- Ignite Flags: (was: Docs Required,Release Notes Required) > Remove boilerplate code in test PluginProvider implementations. > --- > > Key: IGNITE-12285 > URL: https://issues.apache.org/jira/browse/IGNITE-12285 > Project: Ignite > Issue Type: Improvement >Reporter: PetrovMikhail >Assignee: PetrovMikhail >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > It's needed to remove boilerplate code in test PluginProvider implementations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12252) Unchecked exceptions during rebalancing should be handled
[
https://issues.apache.org/jira/browse/IGNITE-12252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951642#comment-16951642
]
Anton Vinogradov commented on IGNITE-12252:
---
[~zzzadruga],
Is there any reason to keep oomeHnd (to check only OutOfMemoryError) at other
threadpools?
> Unchecked exceptions during rebalancing should be handled
> -
>
> Key: IGNITE-12252
> URL: https://issues.apache.org/jira/browse/IGNITE-12252
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Nikolai Kulagin
>Priority: Critical
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Rebalancing should handle unchecked exceptions by failure handler. In current
> implementation unchecked exceptions just ignored. They were handled by IO
> worker before IGNITE-3195.
> Reproducer:
> {code:java}
> @Test
> public void testRebalanceUncheckedError() throws Exception {
> IgniteEx ignite0 = startGrid(new
> IgniteConfiguration().setIgniteInstanceName("ignite0"));
> IgniteCache cache =
> ignite0.getOrCreateCache(DEFAULT_CACHE_NAME);
> IgniteDataStreamer streamer =
> ignite0.dataStreamer(DEFAULT_CACHE_NAME);
> for (int i = 0; i < 100_000; i++)
> streamer.addData(i, i);
> streamer.flush();
> IgniteEx ignite1 = startGrid(new
> IgniteConfiguration().setIgniteInstanceName("ignite1")
>
> .setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_OBJECT_LOADED));
> ignite1.events().localListen(e -> {
> throw new Error();
> }, EventType.EVT_CACHE_REBALANCE_OBJECT_LOADED);
> awaitPartitionMapExchange();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
