[jira] [Created] (IGNITE-14613) Mark CacheConfiguration.rebalanceDelay as deprecated.
Ilya Kasnacheev created IGNITE-14613: Summary: Mark CacheConfiguration.rebalanceDelay as deprecated. Key: IGNITE-14613 URL: https://issues.apache.org/jira/browse/IGNITE-14613 Project: Ignite Issue Type: Improvement Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Now that we have baseline topology and baseline auto-adjust, rebalance delay is rarely needed and does not see enough use/testing. See for example the mailing list thread where it became apparent it will disable historical rebalancing. Let's mark it as deprecated, target its removal in 3.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14490) cache.invoke() triggers failure handler and freezes if entry processor is not urideployed
Ilya Kasnacheev created IGNITE-14490: Summary: cache.invoke() triggers failure handler and freezes if entry processor is not urideployed Key: IGNITE-14490 URL: https://issues.apache.org/jira/browse/IGNITE-14490 Project: Ignite Issue Type: Bug Affects Versions: 2.10 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev If URI deployment is specified Caused by: java.lang.ClassNotFoundException: [Ljava.lang.StackTraceElement;" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14481) Missing dependencies for JClouds discovery (ignite-cloud)
Ilya Kasnacheev created IGNITE-14481: Summary: Missing dependencies for JClouds discovery (ignite-cloud) Key: IGNITE-14481 URL: https://issues.apache.org/jira/browse/IGNITE-14481 Project: Ignite Issue Type: Bug Components: integrations Affects Versions: 2.10 Reporter: Ilya Kasnacheev We seem to skip some essential dependencies for jclouds-based discovery, since trying to start a node results in the following error: {code} Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.ignite.spi.discovery.tcp.ipfinder.cloud.TcpDiscoveryCloudIpFinder#1877ab81' defined in URL [file:/home/ikasnacheev/Downloads/apache-ignite-2.11.0-SNAPSHOT-bin/config/cloud-config.xml]: Initialization of bean failed; nested exception is java.lang.NoClassDefFoundError: Lorg/jclouds/compute/ComputeService; at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:562) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481) at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:299) ... 28 more Caused by: java.lang.NoClassDefFoundError: Lorg/jclouds/compute/ComputeService; at java.lang.Class.getDeclaredFields0(Native Method) at java.lang.Class.privateGetDeclaredFields(Class.java:2583) at java.lang.Class.getDeclaredFields(Class.java:1916) at org.springframework.util.ReflectionUtils.getDeclaredFields(ReflectionUtils.java:713) at org.springframework.util.ReflectionUtils.findField(ReflectionUtils.java:97) at org.springframework.util.ReflectionUtils.findField(ReflectionUtils.java:80) at org.springframework.core.convert.Property.getField(Property.java:225) at org.springframework.core.convert.Property.resolveAnnotations(Property.java:202) at org.springframework.core.convert.Property.getAnnotations(Property.java:123) at org.springframework.core.convert.TypeDescriptor.(TypeDescriptor.java:115) at org.springframework.beans.BeanWrapperImpl.convertForProperty(BeanWrapperImpl.java:214) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.convertForProperty(AbstractAutowireCapableBeanFactory.java:1568) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1527) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1269) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:551) ... 30 more Caused by: java.lang.ClassNotFoundException: org.jclouds.compute.ComputeService at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 45 more {code} The following default config is used: {code} us-central1-a asia-east1-a {code} By the way, there is extra closing slash in initial tag in org.apache.ignite.spi.discovery.tcp.ipfinder.cloud.TcpDiscoveryCloudIpFinder javadoc (line 112). Makes sense to fix. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14466) Changing cache configuration once cache is created confuses PME on node join
Ilya Kasnacheev created IGNITE-14466: Summary: Changing cache configuration once cache is created confuses PME on node join Key: IGNITE-14466 URL: https://issues.apache.org/jira/browse/IGNITE-14466 Project: Ignite Issue Type: Bug Affects Versions: 2.7.5 Reporter: Ilya Kasnacheev The following code triggers the issue: {code:java} inputCache = ignite.cache(inputCacheName); CacheConfiguration configuration = inputCache.getConfiguration(CacheConfiguration.class); outputCache = ignite.getOrCreateCache(configuration.setName("cache_" + ctx.name())); {code} It is possible for user code to accidentally reuse the same cache configuration instance when creating caches (see linked example). This causes the following hard to debug NPE: {code:java} Mar 30, 2021 11:53:25 AM org.apache.ignite.logger.java.JavaLogger error SEVERE: Exception in discovery notyfier worker thread. java.lang.NullPointerException at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.addClientNode(GridDiscoveryManager.java:445) at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.processCacheChangeRequests(ClusterCachesInfo.java:596) at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onCacheChangeRequested(ClusterCachesInfo.java:430) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onCustomEvent(GridCacheProcessor.java:3827) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.onDiscovery0(GridDiscoveryManager.java:697) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.lambda$onDiscovery$0(GridDiscoveryManager.java:604) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body0(GridDiscoveryManager.java:2667) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body(GridDiscoveryManager.java:2705) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) {code} We should, maybe, clone a cache configuration before putting it to our internal data structures? Or better, serialize-deserialize as all of the remaining nodes in the cluster do. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14445) "Remote node does not observe current" after failure by not receiving metrics from client
Ilya Kasnacheev created IGNITE-14445: Summary: "Remote node does not observe current" after failure by not receiving metrics from client Key: IGNITE-14445 URL: https://issues.apache.org/jira/browse/IGNITE-14445 Project: Ignite Issue Type: Bug Affects Versions: 2.9.1 Reporter: Ilya Kasnacheev A server node might fail a client node due to pauses in the network connection: [15:07:16,330][WARNING][tcp-disco-msg-worker-[11cf0c06 10.212.120.71:57500 crd]-#2%hh_DynamicGrid_v2%][TcpDiscoverySpi] Failing client node due to not receiving metrics updates from client node within 'IgniteConfiguration.clientFailureDetectionTimeout' (consider increasing configuration property) [timeout=12, node=TcpDiscoveryNode [id=9dbcfb86-a60e-4382-904f-57bffbe18c5c,consistentId=73B5811B-9644-48FD-A533-B4609FDAD591, addrs=ArrayList [10.212.120.190], sockAddrs=HashSet [VWNV02AX07080.HH.com/10.212.120.190:0], discPort=0, order=488, intOrder=248, lastExchangeTime=1612397142960, loc=false, ver=2.8.1#20200521-sha1:86422096, isClient=true]] Then, the client node will never understand that it is dropped by cluster and will be endlessly trying to connect. I'm not sure what does discovery do on the client node: {code} [15:07:42,689][SEVERE][Thread-219][TcpCommunicationSpi] Failed to send message to remote node [node=TcpDiscoveryNode [id=83fd7c70-839d-46ca-969f-bbb9661d6ab2, consistentId=127.1.1.1:57500, addrs=ArrayList [127.1.1.1], sockAddrs=HashSet [test.com/127.1.1.1:57500], discPort=57500, order=1, intOrder=1, lastExchangeTime=1612397256785, loc=false, ver=2.8.1#20200521-sha1:86422096, isClient=false], msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, topicOrd=8, ordered=false, timeout=0, skipOnTimeout=false, msg=GridNearAtomicFullUpdateRequest [keys=ArrayList [UserKeyCacheObjectImpl [part=292, val=TestModel:TEST|bbf4da4d-c3d7-4b46-98b6-0de70c30f668, hasValBytes=true]], conflictTtls=null, conflictExpireTimes=null, expiryPlc=org.apache.ignite.internal.processors.platform.cache.expiry.PlatformExpiryPolicy@3fb1b76e, initSize=1, filter=null, parent=GridNearAtomicAbstractUpdateRequest [res=null, flags=keepBinary class org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: Remote node does not observe current node in topology : 83fd7c70-839d-46ca-969f-bbb9661d6ab2 at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioSession(TcpCommunicationSpi.java:3622) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3458) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createCommunicationClient(TcpCommunicationSpi.java:3198) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:3078) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2918) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2877) at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:2035) at org.apache.ignite.internal.managers.communication.GridIoManager.sendToGridTopic(GridIoManager.java:2132) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:1257) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:1296) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:312) {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14386) Failure handler triggered on doneFut.get() in Checkpointer.syncUpdatedStores@db-checkpoint-thread
Ilya Kasnacheev created IGNITE-14386: Summary: Failure handler triggered on doneFut.get() in Checkpointer.syncUpdatedStores@db-checkpoint-thread Key: IGNITE-14386 URL: https://issues.apache.org/jira/browse/IGNITE-14386 Project: Ignite Issue Type: Bug Components: persistence Affects Versions: 2.10 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev {code} Thread [name="binary-metadata-writer-#71", id=97, state=RUNNABLE, blockCnt=0, waitCnt=8612] at java.base@11.0.6/sun.nio.ch.FileDispatcherImpl.force0(Native Method) at java.base@11.0.6/sun.nio.ch.FileDispatcherImpl.force(Unknown Source) at java.base@11.0.6/sun.nio.ch.SimpleAsynchronousFileChannelImpl.force(Unknown Source) at app//o.a.i.i.processors.cache.persistence.file.AsyncFileIO.force(AsyncFileIO.java:216) at app//o.a.i.i.processors.cache.persistence.file.AsyncFileIO.force(AsyncFileIO.java:211) at app//o.a.i.i.processors.cache.binary.BinaryMetadataFileStore.writeMetadata(BinaryMetadataFileStore.java:143) at app//o.a.i.i.processors.cache.binary.BinaryMetadataFileStore$WriteOperationTask.execute(BinaryMetadataFileStore.java:653) at app//o.a.i.i.processors.cache.binary.BinaryMetadataFileStore$BinaryMetadataAsyncWriter.body0(BinaryMetadataFileStore.java:463) at app//o.a.i.i.processors.cache.binary.BinaryMetadataFileStore$BinaryMetadataAsyncWriter.body(BinaryMetadataFileStore.java:435) at app//o.a.i.i.util.worker.GridWorker.run(GridWorker.java:119) at java.base@11.0.6/java.lang.Thread.run(Unknown Source) Thread [name="db-checkpoint-thread-#116", id=148, state=WAITING, blockCnt=3, waitCnt=13147] at java.base@11.0.6/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.6/java.util.concurrent.locks.LockSupport.park(Unknown Source) at app//o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:186) at app//o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:149) at app//o.a.i.i.processors.cache.persistence.checkpoint.Checkpointer.syncUpdatedStores(Checkpointer.java:631) at app//o.a.i.i.processors.cache.persistence.checkpoint.Checkpointer.writePages(Checkpointer.java:550) at app//o.a.i.i.processors.cache.persistence.checkpoint.Checkpointer.doCheckpoint(Checkpointer.java:428) at app//o.a.i.i.processors.cache.persistence.checkpoint.Checkpointer.body(Checkpointer.java:264) at app//o.a.i.i.util.worker.GridWorker.run(GridWorker.java:119) at java.base@11.0.6/java.lang.Thread.run(Unknown Source) Thread [name="checkpoint-runner-IO-#131", id=163, state=RUNNABLE, blockCnt=35, waitCnt=5157152] at java.base@11.0.6/sun.nio.ch.FileDispatcherImpl.force0(Native Method) at java.base@11.0.6/sun.nio.ch.FileDispatcherImpl.force(Unknown Source) at java.base@11.0.6/sun.nio.ch.SimpleAsynchronousFileChannelImpl.force(Unknown Source) at app//o.a.i.i.processors.cache.persistence.file.AsyncFileIO.force(AsyncFileIO.java:216) at app//o.a.i.i.processors.cache.persistence.file.AsyncFileIO.force(AsyncFileIO.java:211) at app//o.a.i.i.processors.cache.persistence.file.FilePageStore.sync(FilePageStore.java:715) at app//o.a.i.i.processors.cache.persistence.checkpoint.Checkpointer.lambda$syncUpdatedStores$0(Checkpointer.java:608) at app//o.a.i.i.processors.cache.persistence.checkpoint.Checkpointer$$Lambda$760/0x7f877929a500.run(Unknown Source) at java.base@11.0.6/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base@11.0.6/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base@11.0.6/java.lang.Thread.run(Unknown Source) Thread [name="client-connector-#1849", id=3130, state=WAITING, blockCnt=11, waitCnt=77047] at java.base@11.0.6/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.6/java.util.concurrent.locks.LockSupport.park(Unknown Source) at app//o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:186) at app//o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:149) at app//o.a.i.i.processors.cache.binary.BinaryMetadataFileStore$BinaryMetadataAsyncWriter.waitForWriteCompletion(BinaryMetadataFileStore.java:586) at app//o.a.i.i.processors.cache.binary.BinaryMetadataFileStore.waitForWriteCompletion(BinaryMetadataFileStore.java:294) at app//o.a.i.i.processors.cache.binary.CacheObjectBinaryProcessorImpl.metadata0(CacheObjectBinaryProcessorImpl.java:733) at app//o.a.i.i.processors.cache.binary.CacheObjectBinaryProcessorImpl.metadata(CacheObjectBinaryProcessorImpl.java:646) at
[jira] [Created] (IGNITE-14379) Fix vulnerability of commons-codec <1.13
Ilya Kasnacheev created IGNITE-14379: Summary: Fix vulnerability of commons-codec <1.13 Key: IGNITE-14379 URL: https://issues.apache.org/jira/browse/IGNITE-14379 Project: Ignite Issue Type: Bug Components: build Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Fix For: 2.11 https://www.whitesourcesoftware.com/vulnerability-database/WS-2019-0379 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14308) IgnitePeerToPeerClassLoadingException: Could not use deployment to prepare deployable, because local node id does not correspond with class loader id
Ilya Kasnacheev created IGNITE-14308: Summary: IgnitePeerToPeerClassLoadingException: Could not use deployment to prepare deployable, because local node id does not correspond with class loader id Key: IGNITE-14308 URL: https://issues.apache.org/jira/browse/IGNITE-14308 Project: Ignite Issue Type: Bug Components: binary Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Fix For: 2.11 After IGNITE-12760, the following exception is seen: {code} IgnitePeerToPeerClassLoadingException at org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager.checkDeploymentIsCorrect(GridCacheDeploymentManager.java:717) at org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager.prepare(GridCacheDeploymentManager.java:693) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onSend(GridCacheIoManager.java:1213) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:1245) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture.proceedPrepare(GridNearOptimisticTxPrepareFuture.java:570) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture.prepareSingle(GridNearOptimisticTxPrepareFuture.java:381) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture.prepare0(GridNearOptimisticTxPrepareFuture.java:324) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFutureAdapter.prepareOnTopology(GridNearOptimisticTxPrepareFutureAdapter.java:204) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFutureAdapter.prepare(GridNearOptimisticTxPrepareFutureAdapter.java:128) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.prepareNearTxLocal(GridNearTxLocal.java:3965) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commitNearTxLocalAsync(GridNearTxLocal.java:4013) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.optimisticPutFuture(GridNearTxLocal.java:2958) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllAsync0(GridNearTxLocal.java:1033) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.invokeAsync(GridNearTxLocal.java:540) at org.apache.ignite.internal.processors.cache.GridCacheAdapter$27.op(GridCacheAdapter.java:2389) at org.apache.ignite.internal.processors.cache.GridCacheAdapter$27.op(GridCacheAdapter.java:2385) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:3830) ... 29 more {code} Turns out, sometimes local deployment belongs to other node id (previous client?). Changing behavior for local deployments was not anticipated. Let's change this error to warning for local deployments. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14285) sqllline: `Bad history file syntax!'
Ilya Kasnacheev created IGNITE-14285: Summary: sqllline: `Bad history file syntax!' Key: IGNITE-14285 URL: https://issues.apache.org/jira/browse/IGNITE-14285 Project: Ignite Issue Type: Bug Affects Versions: 2.10 Reporter: Ilya Kasnacheev When starting the updated sqlline in 2.10, I can see the following error message: {code} ~/Downloads/2.10/apache-ignite-slim-2.10.0-bin% bin/sqlline.sh мар 05, 2021 12:30:46 PM org.jline.utils.Log logr WARNING: Failed to load history java.lang.IllegalArgumentException: Bad history file syntax! The history file `/home/ikasnacheev/.sqlline/history` may be an older history: please remove it or use a different history file. at org.jline.reader.impl.history.DefaultHistory.addHistoryLine(DefaultHistory.java:185) at org.jline.reader.impl.history.DefaultHistory.addHistoryLine(DefaultHistory.java:169) at org.jline.reader.impl.history.DefaultHistory.lambda$load$0(DefaultHistory.java:86) at java.util.Iterator.forEachRemaining(Iterator.java:116) at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647) at org.jline.reader.impl.history.DefaultHistory.load(DefaultHistory.java:86) at org.jline.reader.impl.history.DefaultHistory.attach(DefaultHistory.java:69) at sqlline.SqlLine.getConsoleReader(SqlLine.java:618) at sqlline.SqlLine.begin(SqlLine.java:511) at sqlline.SqlLine.start(SqlLine.java:267) at sqlline.SqlLine.main(SqlLine.java:206) {code} Moreover, if I remove the file, and run some command, then the old sqlline will use the history incorrectly: {code} sqlline version 1.3.0 sqlline> 1614936870022:select * from person; {code} This is going to be a problem for all Ignite developers who use more than one version of Ignite on the same box, e.g. the entirely development community. I recommend changing the default history file location to e.g. ~/.sqlline/apache-ignite.history. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14284) sqlline: `Property "url" is required'
Ilya Kasnacheev created IGNITE-14284: Summary: sqlline: `Property "url" is required' Key: IGNITE-14284 URL: https://issues.apache.org/jira/browse/IGNITE-14284 Project: Ignite Issue Type: Bug Affects Versions: 2.10 Reporter: Ilya Kasnacheev Currently, when you run sqlline.sh, the following output is shown: {code} ~/Downloads/2.10/apache-ignite-slim-2.10.0-bin% bin/sqlline.sh Property "url" is required sqlline version 1.9.0 sqlline> {code} We did not have this confusing warning when using sqlline 1.3.0. From this on, it will work as expected, with !connect jdbc:ignite:thin://localhost, etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14161) Continuous Query ignores first written entry if Transformer returns user type
Ilya Kasnacheev created IGNITE-14161: Summary: Continuous Query ignores first written entry if Transformer returns user type Key: IGNITE-14161 URL: https://issues.apache.org/jira/browse/IGNITE-14161 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.10 Reporter: Ilya Kasnacheev Yup. Please see the discussion and the reproducer class. First put will come as CREATED then UPDATED {code} CacheContinuousQueryEvent [evtType=CREATED, key=CacheAccountKey{affinityAccountKey=AffinityAccountKey{accountId=8}}, newVal=Account{accountKey=DomainAccountKey{accountId=8}, name='Person#8'}, oldVal=null, partCntr=1] CacheContinuousQueryEvent [evtType=UPDATED, key=CacheAccountKey{affinityAccountKey=AffinityAccountKey{accountId=8}}, newVal=Account{accountKey=DomainAccountKey{accountId=8}, name='Person#8'}, oldVal=Account{accountKey=DomainAccountKey{accountId=8}, name='Person#8'}, partCntr=2] {code} and the event will be lost. Subsequent have just 1 event and are OK: {code} Transform CacheContinuousQueryEvent [evtType=CREATED, key=CacheAccountKey{affinityAccountKey=AffinityAccountKey{accountId=16}}, newVal=Account{accountKey=DomainAccountKey{accountId=16}, name='Person#16'}, oldVal=null, partCntr=1] Transform CacheContinuousQueryEvent [evtType=CREATED, key=CacheAccountKey{affinityAccountKey=AffinityAccountKey{accountId=18}}, newVal=Account{accountKey=DomainAccountKey{accountId=18}, name='Person#18'}, oldVal=null, partCntr=1] {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14039) Add warnings about WAL enable/disable requiring topology stability
Ilya Kasnacheev created IGNITE-14039: Summary: Add warnings about WAL enable/disable requiring topology stability Key: IGNITE-14039 URL: https://issues.apache.org/jira/browse/IGNITE-14039 Project: Ignite Issue Type: Bug Components: documentation Affects Versions: 2.9.1 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Fix For: 2.10 Due to IGNITE-13976 WAL enable/disable is not suitable for any kind of changing topology and will cause cache to go into inconsistent state. We need to add warnings to javadoc, runtime and documentation, and provide tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13976) WAL disable/enable with node restarts results in mismatching state, data loss
Ilya Kasnacheev created IGNITE-13976: Summary: WAL disable/enable with node restarts results in mismatching state, data loss Key: IGNITE-13976 URL: https://issues.apache.org/jira/browse/IGNITE-13976 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.9.1 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev If you try to enable/disable WAL on unstable topology, you will get to state when WAL status is undefined, nodes might have different wall status and the only way to fix it is to restart the cluster, which will lead to data loss because ignite removes data if WAL is disabled on restart. See the attached reproducer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13961) Hierarchical table join causes "Failed to generate REDUCE query. Data table found"
Ilya Kasnacheev created IGNITE-13961: Summary: Hierarchical table join causes "Failed to generate REDUCE query. Data table found" Key: IGNITE-13961 URL: https://issues.apache.org/jira/browse/IGNITE-13961 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.9.1, 2.8.1 Reporter: Ilya Kasnacheev {code} create table tasks (id int primary key, parent_id int); insert into tasks (id, parent_id) values (1, 1); insert into tasks (id, parent_id) values (2, 1); insert into tasks (id, parent_id) values (5, 3); insert into tasks (id, parent_id) values (7, null); SELECT * FROM (SELECT DISTINCT id, parent_id FROM tasks) AS orphan WHERE orphan.parent_id NOT IN (SELECT id FROM tasks) or orphan.parent_id is null; {code} Expected: (5, 3), (7, null) Actual: {code} Error: Failed to generate REDUCE query. Data table found: PUBLIC.TASKS SELECT ORPHAN__Z1.ID, ORPHAN__Z1.PARENT_ID FROM (SELECT DISTINCT __C0_0 ID, __C0_1 PARENT_ID FROM PUBLIC.__T0) ORPHAN__Z1 WHERE (ORPHAN__Z1.PARENT_ID IS NULL) OR (NOT (ORPHAN__Z1.PARENT_ID IN( SELECT __Z2.ID FROM PUBLIC.TASKS __Z2 ))) (state=5,code=1) {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13878) Allocated RAM is always 0 for non-persistent regions of persistent cluster
Ilya Kasnacheev created IGNITE-13878: Summary: Allocated RAM is always 0 for non-persistent regions of persistent cluster Key: IGNITE-13878 URL: https://issues.apache.org/jira/browse/IGNITE-13878 Project: Ignite Issue Type: Bug Components: persistence Affects Versions: 2.9 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager#wrapMetricsMemoryProvider The logic in overridden nextRegion() method for the DirectMemoryProvider in the case above does not work for non-persistent data regions, as an example, region with persistenceEnabled=false, initSize = maxSize = 2G has only a single chunk that writes into memMetrics.updateCheckpointBufferSize(chunkSize) instead of memMetrics.updateOffHeapSize(chunkSize). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13851) Document persistence forward compatibility policy
Ilya Kasnacheev created IGNITE-13851: Summary: Document persistence forward compatibility policy Key: IGNITE-13851 URL: https://issues.apache.org/jira/browse/IGNITE-13851 Project: Ignite Issue Type: Improvement Components: documentation Reporter: Ilya Kasnacheev Fix For: 2.10 We have a policy that you can upgrade from 2.x to 2.y, y > x, while keeping your persistence files. Let's document that fact. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13824) Suspected: "Connection reset by peer" of Communication socket triggers FH
Ilya Kasnacheev created IGNITE-13824: Summary: Suspected: "Connection reset by peer" of Communication socket triggers FH Key: IGNITE-13824 URL: https://issues.apache.org/jira/browse/IGNITE-13824 Project: Ignite Issue Type: Bug Components: networking Affects Versions: 2.9 Reporter: Ilya Kasnacheev I would expect a network error in tcp-comm-worker to never trigger a Failure Hander, yet it happens: {code} [12/3/20 16:08:26:410 GMT] 00bd IgniteKernal W Possible too long JVM pause: 2418 milliseconds. [12/3/20 16:08:27:465 GMT] 00c5 TcpCommunicat W Client disconnected abruptly due to network connection loss or because the connection was left open on application shutdown. [cls=class o.a.i.i.util.nio.GridNioException, msg=Connection reset by peer] [12/3/20 16:08:27:411 GMT] 00c5 TcpCommunicat E Failed to process selector key [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=0, bytesRcvd=48849402273, bytesSent=15994664546, bytesRcvd0=54446, bytesSent0=102, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-0, igniteInstanceName=null, finished=false, heartbeatTs=1607011706410, hashCode=433635054, interrupted=false, runner=grid-nio-worker-tcp-comm-0-#51]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=9025120, resendCnt=0, rcvCnt=9025150, sentCnt=9025152, reserved=true, lastAck=9025120, nodeLeft=false, node=TcpDiscoveryNode [id=b3ca311e-077f-42a5-884a-807b539730b6, consistentId=10.60.46.12:48500, addrs=ArrayList [10.60.46.12], sockAddrs=HashSet [hex-wgc-p-web02/10.60.46.12:48500], discPort=48500, order=1, intOrder=1, lastExchangeTime=1607006097079, loc=false, ver=2.9.0#20201015-sha1:70742da8, isClient=false], connected=false, connectCnt=1, queueLimit=4096, reserveCnt=1, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=9025120, resendCnt=0, rcvCnt=9025150, sentCnt=9025152, reserved=true, lastAck=9025120, nodeLeft=false, node=TcpDiscoveryNode [id=b3ca311e-077f-42a5-884a-807b539730b6, consistentId=10.60.46.12:48500, addrs=ArrayList [10.60.46.12], sockAddrs=HashSet [hex-wgc-p-web02/10.60.46.12:48500], discPort=48500, order=1, intOrder=1, lastExchangeTime=1607006097079, loc=false, ver=2.9.0#20201015-sha1:70742da8, isClient=false], connected=false, connectCnt=1, queueLimit=4096, reserveCnt=1, pairedConnections=false], closeSocket=true, outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1, super=GridNioSessionImpl [locAddr=/10.223.132.3:52550, rmtAddr=/10.60.46.12:48100, createTime=1607006097572, closeTime=0, bytesSent=15994657850, bytesRcvd=48849402273, bytesSent0=102, bytesRcvd0=54446, sndSchedTime=1607006097572, lastSndTime=1607011706410, lastRcvTime=1607011706410, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=o.a.i.i.util.nio.GridDirectParser@93200255, directMode=true], GridConnectionBytesVerifyFilter], accepted=false, markedForClose=false]]] java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:51) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:235) at sun.nio.ch.IOUtil.read(IOUtil.java:204) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:394) at org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1330) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2472) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2239) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1880) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:822) [12/3/20 16:08:44:437 GMT] 00c4 SystemOut O [16:08:44] Possible failure suppressed accordingly to a configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=tcp-comm-worker, igniteInstanceName=null, finished=false, heartbeatTs=1607011706420]]] [12/3/20 16:08:44:436 GMT] 00c4 W java.util.logging.LogManager$RootLogger log Possible failure suppressed accordingly to a configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
[jira] [Created] (IGNITE-13770) Ignite.dataRegionMetrics throws NPE with empty persisted region
Ilya Kasnacheev created IGNITE-13770: Summary: Ignite.dataRegionMetrics throws NPE with empty persisted region Key: IGNITE-13770 URL: https://issues.apache.org/jira/browse/IGNITE-13770 Project: Ignite Issue Type: Bug Affects Versions: 2.9 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Fix For: 2.9.1 {code} java.lang.NullPointerException at org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointBufferPagesCount(PageMemoryImpl.java:1823) at org.apache.ignite.internal.processors.cache.persistence.DataRegionMetricsImpl.getUsedCheckpointBufferPages(DataRegionMetricsImpl.java:287) at org.apache.ignite.internal.processors.cache.persistence.DataRegionMetricsSnapshot.(DataRegionMetricsSnapshot.java:108) at org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.memoryMetrics(IgniteCacheDatabaseSharedManager.java:815) at org.apache.ignite.internal.IgniteKernal.dataRegionMetrics(IgniteKernal.java:3773) {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13730) Unify fallback properties for SSL settings as per Java standard
Ilya Kasnacheev created IGNITE-13730: Summary: Unify fallback properties for SSL settings as per Java standard Key: IGNITE-13730 URL: https://issues.apache.org/jira/browse/IGNITE-13730 Project: Ignite Issue Type: Improvement Components: networking Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Right now we use e.g. "ssl.key.algorithm" which is not standard. Standard option for this setting appear to be "javax.net.ssl.trustStoreProvider" and "javax.net.ssl.keyStoreProvider" as per https://www.ibm.com/support/knowledgecenter/SSEQTP_9.0.5/com.ibm.websphere.base.doc/ae/tjj_thirdparty_ssl.html We should make sure that "javax.net.ssl.keyStoreProvider" is the default fallback property for any SSL-using code: discovery, communication, thin clients, etc. Our own properties should be kept for compatibility reasons. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13711) Exception is discarded in GridCacheProcessor
Ilya Kasnacheev created IGNITE-13711: Summary: Exception is discarded in GridCacheProcessor Key: IGNITE-13711 URL: https://issues.apache.org/jira/browse/IGNITE-13711 Project: Ignite Issue Type: Improvement Components: cache Reporter: Ilya Kasnacheev {code} locJoinCtx.initCaches() .forEach(cacheDesc -> { try { ctx.query().initQueryStructuresForNotStartedCache(cacheDesc); } catch (Exception e) { log.error("Can't initialize query structures for not started cache [cacheName=" + cacheDesc.cacheName() + "]"); } }); {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13706) Thin Client Put_All uses LinkedHashMap, causes warning and may deadlock
Ilya Kasnacheev created IGNITE-13706: Summary: Thin Client Put_All uses LinkedHashMap, causes warning and may deadlock Key: IGNITE-13706 URL: https://issues.apache.org/jira/browse/IGNITE-13706 Project: Ignite Issue Type: Bug Components: thin client Affects Versions: 2.9 Reporter: Ilya Kasnacheev Thin client protocol handler for Put_All (1004) operation will create a linked hash map and then invoke putAll on this map. This leads to couple of issues: * Thin client may easily deadlock the cluster by issuing parallel transactional putAlls with different order of keys. * "Unordered map with putAll" warning is printed to log, causing user confusion, even if client actually uses SortedMap with putAll(). What we really need is BinaryObject-aware SortedMap / making sure that keys/KeyCacheObjects are always comparable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13665) Useless failure trace "IgnitionEx$IgniteNamedInstance$2.apply"
Ilya Kasnacheev created IGNITE-13665: Summary: Useless failure trace "IgnitionEx$IgniteNamedInstance$2.apply" Key: IGNITE-13665 URL: https://issues.apache.org/jira/browse/IGNITE-13665 Project: Ignite Issue Type: Bug Components: general Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Fix For: 2.9.1 When failure handler catches an issue in striped pool, maybe in some other cases, thread dump is as folows: {code} [2020-10-10T10:10:10,100][WARN ][grid-timeout-worker-#22%cluster%][] Possible failure suppressed accordingly to a configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=sys-stripe-0, igniteInstanceName=EPE_CLUSTER_PERF, finished=false, heartbeatTs=1601234567890]]] org.apache.ignite.IgniteException: GridWorker [name=sys-stripe-0, igniteInstanceName=instance, finished=false, heartbeatTs=1601234567890] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1859) [ignite-core.jar] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1854) [ignite-core.jar] at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) [ignite-core.jar] at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:296) [ignite-core.jar] at org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:265) [ignite-core.jar] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119) [ignite-core.jar] at java.lang.Thread.run(Thread.java:834) [?:?] {code} when the actual stack trace from the relevant thread is hidden somewhere deep below. And may even be suppressed. This is a usability nightmare. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13498) Thin client throws "unexpected response id" when timeout is too small
Ilya Kasnacheev created IGNITE-13498: Summary: Thin client throws "unexpected response id" when timeout is too small Key: IGNITE-13498 URL: https://issues.apache.org/jira/browse/IGNITE-13498 Project: Ignite Issue Type: Bug Components: thin client Affects Versions: 2.8.1 Reporter: Ilya Kasnacheev Reportedly, if you set Java thin client timeout to 50 (ms), it will start throwing the following exceptions: {code} Exception: Unexpected response ID [3630521808404506928], [5259] was expected at org.apache.ignite.internal.client.thin.TcpClientChannel.receive(TcpClientChannel.java:160) at org.apache.ignite.internal.client.thin.ReliableChannel.service(ReliableChannel.java:126) at org.apache.ignite.internal.client.thin.TcpClientCache.getAll(TcpClientCache.java:165) {code} It is expected that it will throw timeout exceptions instead. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13482) Client node cannot ALTER TABLE if it was created on a different node
Ilya Kasnacheev created IGNITE-13482: Summary: Client node cannot ALTER TABLE if it was created on a different node Key: IGNITE-13482 URL: https://issues.apache.org/jira/browse/IGNITE-13482 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.8.1 Reporter: Ilya Kasnacheev Please see the attached test as well as mailing list thread. You need to CREATE table on the same client or do a SELECT before you can ALTER TABLE. Otherwise it will fail: {code} Exception in thread "main" javax.cache.CacheException at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:817) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:750) at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:424) at net.aline.cloudedh.base.database.extra.ClientNode.main(ClientNode.java:28) Caused by: class org.apache.ignite.internal.processors.query.IgniteSQLException: null at org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommandH2(CommandProcessor.java:888) at org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommand(CommandProcessor.java:418) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeCommand(IgniteH2Indexing.java:1048) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1130) at org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2406) at org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2402) at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36) at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2919) at org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$1(GridQueryProcessor.java:2422) at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2460) at org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2396) at org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2323) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:802) ... 3 more Caused by: java.lang.NullPointerException at org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommandH2(CommandProcessor.java:834) ... 15 more {code} This happens both with SqlFieldsQuery and JDBC thin driver connecting to this client! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13475) NPE on IgniteTxHandler.finishDhtLocal
Ilya Kasnacheev created IGNITE-13475: Summary: NPE on IgniteTxHandler.finishDhtLocal Key: IGNITE-13475 URL: https://issues.apache.org/jira/browse/IGNITE-13475 Project: Ignite Issue Type: Bug Components: cache Reporter: Ilya Kasnacheev Fix For: 2.8.1 {code} [05:57:16,193][SEVERE][sys-stripe-15-#16][] Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=java.lang.NullPointerException]] java.lang.NullPointerException at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:1064) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:953) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:909) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$200(IgniteTxHandler.java:123) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:217) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:215) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1142) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:591) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109) at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1847) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1472) at org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229) at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1367) at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:565) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:745) {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13474) Client node consistentId uniqueness is not checked
Ilya Kasnacheev created IGNITE-13474: Summary: Client node consistentId uniqueness is not checked Key: IGNITE-13474 URL: https://issues.apache.org/jira/browse/IGNITE-13474 Project: Ignite Issue Type: Bug Components: general Affects Versions: 2.8.1 Reporter: Ilya Kasnacheev Please see attached server and client nodes, as well as SO discussion. consistentId uniqueness is not checked on client node join, leading to multiple client nodes with same consistentId as on server node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13413) PHP using ODBC corrupts decimal numbers
Ilya Kasnacheev created IGNITE-13413: Summary: PHP using ODBC corrupts decimal numbers Key: IGNITE-13413 URL: https://issues.apache.org/jira/browse/IGNITE-13413 Project: Ignite Issue Type: Bug Components: odbc Affects Versions: 2.8.1 Reporter: Ilya Kasnacheev Assignee: Igor Sapego Attachments: update.php Please see mailing list thread. Basically, when we update 1.85 into DECIMAL column (maybe some other too), we're getting -0.57 in it instead: {code} sqlline> !connect jdbc:ignite:thin://localhost 0: jdbc:ignite:thin://localhost> create table i_price (id int primary key, price decimal); 0: jdbc:ignite:thin://localhost> insert into i_price (id, price) values (1, 1.00); Then: ~/Downloads/apache-ignite-2.8.1-bin% isql -vvv Apache\ Ignite SQL> SELECT * FROM i_price; ++-+ | ID | PRICE | ++-+ | 1 | 1 | ++-+ SQLRowCount returns 0 1 rows fetched SQL> % ~/Downloads/apache-ignite-2.8.1-bin% php update.php ## Should put 1.85 as PRICE value for ID=1 ~/Downloads/apache-ignite-2.8.1-bin% bin/sqlline.sh sqlline> !connect jdbc:ignite:thin://localhost 0: jdbc:ignite:thin://localhost> select * from i_price; +++ | ID | PRICE | +++ | 1 | -0.57 | +++ 1 row selected (0,101 seconds) {code} I will attach the php file as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13352) ScanQueryIterator needs closing or resources will leak
Ilya Kasnacheev created IGNITE-13352: Summary: ScanQueryIterator needs closing or resources will leak Key: IGNITE-13352 URL: https://issues.apache.org/jira/browse/IGNITE-13352 Project: Ignite Issue Type: Improvement Components: cache Affects Versions: 2.8.1 Reporter: Ilya Kasnacheev Let's document it in javadoc. Let's override IgniteCache.iterator() and declare it to return GridCloseableIterator which should be moved (copied?) from internal package to public API. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13267) AffinityFunction & AffinityKeyMapper are not peer class loadable
Ilya Kasnacheev created IGNITE-13267: Summary: AffinityFunction & AffinityKeyMapper are not peer class loadable Key: IGNITE-13267 URL: https://issues.apache.org/jira/browse/IGNITE-13267 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.8.1 Reporter: Ilya Kasnacheev We have a test GridP2PAffinitySelfTest/GridAffinityP2PSelfTest which suggest they should be peer loadable, but they are not, and for a good reason - they are passed as a part of cache configuration via Discovery. Maybe we should investigate this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13184) ALTER USER SET PASSWORD is valid syntax but does not find user
Ilya Kasnacheev created IGNITE-13184: Summary: ALTER USER SET PASSWORD is valid syntax but does not find user Key: IGNITE-13184 URL: https://issues.apache.org/jira/browse/IGNITE-13184 Project: Ignite Issue Type: Bug Components: security, sql Affects Versions: 2.8.1 Reporter: Ilya Kasnacheev I have noticed the following: {code} 1: jdbc:ignite:thin://localhost> ALTER USER "ignite" set password 'abcdefg' ; Error: Failed to parse query. Пользователь "ignite" не найден User "ignite" not found; SQL statement: ALTER USER "ignite" set password 'abcdefg' [90032-197] (state=42000,code=1001) {code} The correct syntax is "WITH PASSWORD" but using "SET PASSWORD" does not cause parse error (like LIMIT PASSWORD or WHERE PASSWORD) would. It will report that user does not exist, which is false, and can probably throw an user off. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13167) Cache Store fails to close connection: writeBehind + TRANSACTIONAL + readThrough
Ilya Kasnacheev created IGNITE-13167: Summary: Cache Store fails to close connection: writeBehind + TRANSACTIONAL + readThrough Key: IGNITE-13167 URL: https://issues.apache.org/jira/browse/IGNITE-13167 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.8.1 Reporter: Ilya Kasnacheev Please see referenced mailing list thread. If there is an ongoing transactional (cache is transactional, put uses an implicit transaction, tx != null) then org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter#loadFromStore will not call sessionEnd() - see sessionEnd0() method. If write behind is also enabled, then cache store is not invoked during the operation and connection will never be closed. So, a combination of TRANSACTIONAL cache, implicit transaction, loadFromCache and write behind will leak a connection every time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13163) Data storage metrics are null if persistence not configured
Ilya Kasnacheev created IGNITE-13163: Summary: Data storage metrics are null if persistence not configured Key: IGNITE-13163 URL: https://issues.apache.org/jira/browse/IGNITE-13163 Project: Ignite Issue Type: Bug Components: general Reporter: Ilya Kasnacheev ignite.dataStorageMetrics() returns null if persistence in cluster is off. This is unexpected, not mentioned in javadocs, should be fixed. See attached test. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13162) Slim docker image for Apache Ignite
Ilya Kasnacheev created IGNITE-13162: Summary: Slim docker image for Apache Ignite Key: IGNITE-13162 URL: https://issues.apache.org/jira/browse/IGNITE-13162 Project: Ignite Issue Type: New Feature Components: build Reporter: Ilya Kasnacheev We need to introduce slim docker image in line with slim binary downloadable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13161) Document & make available slim binary release
Ilya Kasnacheev created IGNITE-13161: Summary: Document & make available slim binary release Key: IGNITE-13161 URL: https://issues.apache.org/jira/browse/IGNITE-13161 Project: Ignite Issue Type: New Feature Components: documentation Affects Versions: 2.9 Reporter: Ilya Kasnacheev We can do that once 2.9 is out. Downloads page should be updated as well as web site and some docs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13146) OpenCensus example does not work from release build
Ilya Kasnacheev created IGNITE-13146: Summary: OpenCensus example does not work from release build Key: IGNITE-13146 URL: https://issues.apache.org/jira/browse/IGNITE-13146 Project: Ignite Issue Type: Bug Components: build Affects Versions: 2.9 Reporter: Ilya Kasnacheev Assignee: Nikolay Izhikov Fix For: 2.9 This happens in a binary distribution of 2.9: {code} /Downloads/apache-ignite-slim-2.9.0-SNAPSHOT-bin/examples% mvn clean install [INFO] [INFO] -< org.apache.ignite:ignite-examples >-- [INFO] Building ignite-examples 2.9.0-SNAPSHOT [INFO] [ jar ]- ... [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ ignite-examples --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 255 source files to /home/gridgain/Downloads/apache-ignite-slim-2.9.0-SNAPSHOT-bin/examples/target/classes ... [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /home/gridgain/Downloads/apache-ignite-slim-2.9.0-SNAPSHOT-bin/examples/src/main/java/org/apache/ignite/examples/opencensus/OpenCensusMetricsExporterExample.java:[20,47] package io.opencensus.exporter.stats.prometheus does not exist [ERROR] /home/gridgain/Downloads/apache-ignite-slim-2.9.0-SNAPSHOT-bin/examples/src/main/java/org/apache/ignite/examples/opencensus/OpenCensusMetricsExporterExample.java:[21,37] package io.prometheus.client.exporter does not exist [ERROR] /home/gridgain/Downloads/apache-ignite-slim-2.9.0-SNAPSHOT-bin/examples/src/main/java/org/apache/ignite/examples/opencensus/OpenCensusMetricsExporterExample.java:[34,47] package org.apache.ignite.spi.metric.opencensus does not exist [ERROR] /home/gridgain/Downloads/apache-ignite-slim-2.9.0-SNAPSHOT-bin/examples/src/main/java/org/apache/ignite/examples/opencensus/OpenCensusMetricsExporterExample.java:[54,9] cannot find symbol symbol: variable PrometheusStatsCollector location: class org.apache.ignite.examples.opencensus.OpenCensusMetricsExporterExample [ERROR] /home/gridgain/Downloads/apache-ignite-slim-2.9.0-SNAPSHOT-bin/examples/src/main/java/org/apache/ignite/examples/opencensus/OpenCensusMetricsExporterExample.java:[57,9] cannot find symbol symbol: class HTTPServer location: class org.apache.ignite.examples.opencensus.OpenCensusMetricsExporterExample [ERROR] /home/gridgain/Downloads/apache-ignite-slim-2.9.0-SNAPSHOT-bin/examples/src/main/java/org/apache/ignite/examples/opencensus/OpenCensusMetricsExporterExample.java:[57,30] cannot find symbol symbol: class HTTPServer location: class org.apache.ignite.examples.opencensus.OpenCensusMetricsExporterExample [ERROR] /home/gridgain/Downloads/apache-ignite-slim-2.9.0-SNAPSHOT-bin/examples/src/main/java/org/apache/ignite/examples/opencensus/OpenCensusMetricsExporterExample.java:[62,9] cannot find symbol symbol: class OpenCensusMetricExporterSpi location: class org.apache.ignite.examples.opencensus.OpenCensusMetricsExporterExample [ERROR] /home/gridgain/Downloads/apache-ignite-slim-2.9.0-SNAPSHOT-bin/examples/src/main/java/org/apache/ignite/examples/opencensus/OpenCensusMetricsExporterExample.java:[62,71] cannot find symbol symbol: class OpenCensusMetricExporterSpi location: class org.apache.ignite.examples.opencensus.OpenCensusMetricsExporterExample [INFO] 8 errors [INFO] - [INFO] [INFO] BUILD FAILURE [INFO] {code} This is because pom-standalone.xml and pom-standalone-lgpl.xml files needs to be updated in accordance with examples/pom.xml changes. This is currently a blocker for 2.9. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13103) Investigate marshalling errors when changing fields of lambda's capturingClass
Ilya Kasnacheev created IGNITE-13103: Summary: Investigate marshalling errors when changing fields of lambda's capturingClass Key: IGNITE-13103 URL: https://issues.apache.org/jira/browse/IGNITE-13103 Project: Ignite Issue Type: Bug Components: binary, compute Affects Versions: 2.8.1 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Trying to execute static lambda whose outer type's fields changed leads to the following error: {code} Exception in thread "main" class org.apache.ignite.IgniteException: Remote job threw user exception (override or implement ComputeTask.result(..) method if you would like to have automatic failover for this exception): Failed to serialize object [typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4] at org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:102) at org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1062) at org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1055) at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7037) at org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:1055) at org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:862) at org.apache.ignite.internal.processors.task.GridTaskWorker.processDelayedResponses(GridTaskWorker.java:711) at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:542) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:829) at org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:497) at org.apache.ignite.internal.processors.closure.GridClosureProcessor.runAsync(GridClosureProcessor.java:244) at org.apache.ignite.internal.processors.closure.GridClosureProcessor.runAsync(GridClosureProcessor.java:216) at org.apache.ignite.internal.IgniteComputeImpl.runAsync0(IgniteComputeImpl.java:702) at org.apache.ignite.internal.IgniteComputeImpl.run(IgniteComputeImpl.java:678) at com.gridgain.deployer.c.ComputeCallerStarter.start(ComputeCallerStarter.java:19) at com.gridgain.deployer.c.ComputeCallerStarter.main(ComputeCallerStarter.java:13) Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to serialize object [typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4] at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:853) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152) at org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:251) at org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:84) at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:57) at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10386) at org.apache.ignite.internal.processors.task.GridTaskWorker.sendRequest(GridTaskWorker.java:1397) at org.apache.ignite.internal.processors.task.GridTaskWorker.processMappedJobs(GridTaskWorker.java:664) at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:536) ... 9 more Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to serialize object [typeName=java.lang.invoke.SerializedLambda] at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:853) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:227) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152) at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:524) at org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObject(BinaryWriterExImpl.java:1503) at
[jira] [Created] (IGNITE-13088) Improve exception when cache not exists after client restart
Ilya Kasnacheev created IGNITE-13088: Summary: Improve exception when cache not exists after client restart Key: IGNITE-13088 URL: https://issues.apache.org/jira/browse/IGNITE-13088 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.8.1 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev After IGNITE-2766 it's possible to get the following error after client reconnects to fresh cluster, if the cache which cache proxy was pointing at no longer exists: {code} java.lang.NullPointerException: null at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.checkProxyIsValid(GatewayProtectedCacheProxy.java:1563) ~[ignite-core-8.7.10.jar:8.7.10] at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.onEnter(GatewayProtectedCacheProxy.java:1580) ~[ignite-core-8.7.10.jar:8.7.10] at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:634) ~[ignite-core-8.7.10.jar:8.7.10] {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12984) Distributed join incorrectly processed when batched:unicast on primary key is used
Ilya Kasnacheev created IGNITE-12984: Summary: Distributed join incorrectly processed when batched:unicast on primary key is used Key: IGNITE-12984 URL: https://issues.apache.org/jira/browse/IGNITE-12984 Project: Ignite Issue Type: Bug Affects Versions: 2.8 Reporter: Ilya Kasnacheev Assignee: Taras Ledkov Please see attached SQL script and userlist discussion. Summary : CASE-1 Results: Correct and as expected {code} SELECT __Z0.ID AS __C0_0, __Z0.NAME AS __C0_1, __Z1.BLOOD_GROUP AS __C0_2, __Z2.UNIVERSAL_DONOR AS __C0_3 FROM PUBLIC.PERSON__Z0 /* PUBLIC.PERSON_NAME_ASC_IDX_proxy */ LEFT OUTER JOIN PUBLIC.MEDICAL_INFO __Z1 /* batched:broadcast PUBLIC.MEDICAL_INFO_NAME_ASC_IDX: NAME = __Z0.NAME */ ON __Z0.NAME = __Z1.NAME LEFT OUTER JOIN PUBLIC.BLOOD_GROUP_INFO_PJ __Z2 /* batched:broadcast PUBLIC.BLOOD_GROUP_INFO_PJ_BLOOD_GROUP_ASC_IDX: BLOOD_GROUP = __Z1.BLOOD_GROUP */ ON __Z1.BLOOD_GROUP = __Z2.BLOOD_GROUP {code} {code} Summary : CASE-2 Results: In-correct SELECT __Z0.ID AS __C0_0, __Z0.NAME AS __C0_1, __Z1.BLOOD_GROUP AS __C0_2, __Z2.UNIVERSAL_DONOR AS __C0_3 FROM PUBLIC.PERSON __Z0 /* PUBLIC.PERSON_ID_ASC_IDX_proxy */ LEFT OUTER JOIN PUBLIC.MEDICAL_INFO __Z1 /* batched:broadcast PUBLIC.MEDICAL_INFO_NAME_ASC_IDX: NAME = __Z0.NAME */ ON __Z0.NAME = __Z1.NAME LEFT OUTER JOIN PUBLIC.BLOOD_GROUP_INFO_P __Z2 /* batched:unicast PUBLIC._key_PK_proxy: BLOOD_GROUP = __Z1.BLOOD_GROUP */ ON __Z1.BLOOD_GROUP = __Z2.BLOOD_GROUP {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12934) Change test start|stop log format for correct TC and build.log visibility.
Ilya Kasnacheev created IGNITE-12934: Summary: Change test start|stop log format for correct TC and build.log visibility. Key: IGNITE-12934 URL: https://issues.apache.org/jira/browse/IGNITE-12934 Project: Ignite Issue Type: Improvement Components: build Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev For correct TC and builds log visibility need to repeat log format, from : ">>> Stopping test: " as "##teamcity[testFinished name='" additional info: https://www.jetbrains.com/help/teamcity/build-script-interaction-with-teamcity.html#BuildScriptInteractionwithTeamCity-ReportingTests Also, make “Starting test”, “Stopping test” messages visible in Quiet mode! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12840) Leaking H2 internals when querying from pyignite
Ilya Kasnacheev created IGNITE-12840: Summary: Leaking H2 internals when querying from pyignite Key: IGNITE-12840 URL: https://issues.apache.org/jira/browse/IGNITE-12840 Project: Ignite Issue Type: Bug Components: clients, python Affects Versions: 2.8 Reporter: Ilya Kasnacheev Fix For: 2.8.1 I am sharing a slightly modified reproducer from userlist. To run: Start a node (with JVM_OPTS=-Xmx512m to see it most clearly). Run a .sql file with sqlline (!run) python3 test_insert_in_cache.py in parallel terminal: python3 read_cache_insert_update_table.py Then you should observe slowing down and long GC pauses on server node. jmap will collect ever-increasing heap dumps. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12837) Make sure Ignite leaves nothing in static fields after node is stopped, write test
Ilya Kasnacheev created IGNITE-12837: Summary: Make sure Ignite leaves nothing in static fields after node is stopped, write test Key: IGNITE-12837 URL: https://issues.apache.org/jira/browse/IGNITE-12837 Project: Ignite Issue Type: Improvement Components: general Reporter: Ilya Kasnacheev {code} There are at least two way link to IgniteKernal leaks to GC root and makes it unavailable for GC. 1. The first one: this - value: org.apache.ignite.internal.IgniteKernal #1 <\- grid - class: org.apache.ignite.internal.GridKernalContextImpl, value: org.apache.ignite.internal.IgniteKernal #1 <\- ctx - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing, value: org.apache.ignite.internal.GridKernalContextImpl #2 <\- this$0 - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing #2 <\- serializer - class: org.h2.util.JdbcUtils, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10 #1 <\- [5395] - class: java.lang.Object[], value: org.h2.util.JdbcUtils class JdbcUtils <\- elementData - class: java.util.Vector, value: java.lang.Object[] #37309 <\- classes - class: sun.misc.Launcher$AppClassLoader, value: java.util.Vector #31 <\- contextClassLoader (thread object) - class: java.lang.Thread, value: sun.misc.Launcher$AppClassLoader #1 org.h2.util.JdbcUtils has static field JavaObjectSerializer serializer, which see IgniteKernal via IgniteH2Indexing. It make closed and stopped IgniteKernal non collectable by GC. If some Ignites run in same JVM, JdbcUtils will always use only one, and it can cause some races. 2. The second way: this - value: org.apache.ignite.internal.IgniteKernal #2 <\- grid - class: org.apache.ignite.internal.GridKernalContextImpl, value: org.apache.ignite.internal.IgniteKernal #2 <\- ctx - class: org.apache.ignite.internal.processors.cache.GridCacheContext, value: org.apache.ignite.internal.GridKernalContextImpl #1 <\- cctx - class: org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry, value: org.apache.ignite.internal.processors.cache.GridCacheContext #24 <\- parent - class: org.apache.ignite.internal.processors.cache.GridCacheMvccCandidate, value: org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry #4 <\- [0] - class: java.lang.Object[], value: org.apache.ignite.internal.processors.cache.GridCacheMvccCandidate #1 <\- elements - class: java.util.ArrayDeque, value: java.lang.Object[] #43259 <\- value - class: java.lang.ThreadLocal$ThreadLocalMap$Entry, value: java.util.ArrayDeque #816 <\- [119] - class: java.lang.ThreadLocal$ThreadLocalMap$Entry[], value: java.lang.ThreadLocal$ThreadLocalMap$Entry #51 <\- table - class: java.lang.ThreadLocal$ThreadLocalMap, value: java.lang.ThreadLocal$ThreadLocalMap$Entry[] #21 <\- threadLocals (thread object) - class: java.lang.Thread, value: java.lang.ThreadLocal$ThreadLocalMap #2 {code} as reported on mailing list. I suggest we fix that (make sure Ignite node leaves nothing in static fields/GC roots after it is stopped). I also suggest we introduce a check, probably by using jmap and searching dump for IgniteKernal, etc. It would also be nice to check that if Ignite is loaded to non-root class loader, started and stopped, it would allow this class-loader to unload Ignite classes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12764) Regression: Thin JDBC streaming fails/BatchUpdateException if function is used
Ilya Kasnacheev created IGNITE-12764: Summary: Regression: Thin JDBC streaming fails/BatchUpdateException if function is used Key: IGNITE-12764 URL: https://issues.apache.org/jira/browse/IGNITE-12764 Project: Ignite Issue Type: Bug Reporter: Ilya Kasnacheev insert INTO city1(id,name,name1) VALUES(?,?,RANDOM_UUID()) happily worked in 2.7.6 but will fail on 2.8.0 with: Exception in thread "main" java.sql.BatchUpdateException: class org.apache.ignite.IgniteCheckedException: null at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection$StreamState.readResponses(JdbcThinConnection.java:1364) at java.base/java.lang.Thread.run(Thread.java:834) Suspected reason is that function RANDOM_UUID() is not processed correctly here. Column type does not matter between UUID and VARCHAR. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12752) Pass thin clients SSL certificates to security
Ilya Kasnacheev created IGNITE-12752: Summary: Pass thin clients SSL certificates to security Key: IGNITE-12752 URL: https://issues.apache.org/jira/browse/IGNITE-12752 Project: Ignite Issue Type: Bug Components: security Reporter: Ilya Kasnacheev Add SSL certificates to AuthenticationContext, so, authenticators can make additional checks based on SSL certificates. Please note that this is *Phase I* which only concerns all flavors of thin clients, but not server/client nodes! It includes JDBC, old and new flavor of Ignite thin client, control.sh and REST. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12751) callAsync(jobs, rdc) performance degrades quadratically as jobs.size() grows
Ilya Kasnacheev created IGNITE-12751: Summary: callAsync(jobs, rdc) performance degrades quadratically as jobs.size() grows Key: IGNITE-12751 URL: https://issues.apache.org/jira/browse/IGNITE-12751 Project: Ignite Issue Type: Bug Components: compute Affects Versions: 2.7.6 Reporter: Ilya Kasnacheev Attachments: word-count-reproducer.zip Please consider attached reproducer and linked report. {{compute.callAsync(jobs, reducer); Result [res=33, tookMs=81, jobs=5] //warm up Result [res=99, tookMs=21, jobs=15] Result [res=330, tookMs=22, jobs=50] Result [res=990, tookMs=57, jobs=150] Result [res=3300, tookMs=146, jobs=500] Result [res=9900, tookMs=231, jobs=1500] Result [res=33000, tookMs=840, jobs=5000] Result [res=99000, tookMs=6965, jobs=15000] Result [res=33, tookMs=118394, jobs=5]}} As soon jobs.size() grows past 5000, performance begins to degrade quadratically. I don't expect that it will be completely linear, but I would assume that it should stay linear-ish until size() hits at least 10, given that we see clusters of 100 nodes and it's not unthinkable to expect 1000 jobs to be run on each node. 5000 jobs (which still give OK performance) / 100 nodes is just 50 jobs per node, which becomes limiting factor. Linked question also mentions OOM event, which may be caused of intermediate storage of (N^2) data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12746) Regression in GridCacheColocatedDebugTest: putAll of sorted keys causes deadlock
Ilya Kasnacheev created IGNITE-12746: Summary: Regression in GridCacheColocatedDebugTest: putAll of sorted keys causes deadlock Key: IGNITE-12746 URL: https://issues.apache.org/jira/browse/IGNITE-12746 Project: Ignite Issue Type: Bug Components: cache Reporter: Ilya Kasnacheev Fix For: 2.8 After this commit: 7d4bb49264b IGNITE-12329 Invalid handling of remote entries causes partition desync and transaction hanging in COMMITTING state. the following tests: org.apache.ignite.internal.processors.cache.distributed.dht.GridCacheColocatedDebugTest#testPutsMultithreadedColocated org.apache.ignite.internal.processors.cache.distributed.dht.GridCacheColocatedDebugTest#testPutsMultithreadedMixed started to be flaky because their ordered putAll operations started deadlocking. This is a regression compared to 2.7 and should be fixed, since it may affect production clusters. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12663) Let's limit all thread pools' sizes to 3 in tests
Ilya Kasnacheev created IGNITE-12663: Summary: Let's limit all thread pools' sizes to 3 in tests Key: IGNITE-12663 URL: https://issues.apache.org/jira/browse/IGNITE-12663 Project: Ignite Issue Type: Test Reporter: Ilya Kasnacheev The rationale is, it is very painful shoving through thread dumps of test with N CPU threads in every pool. Why 3? It should allow all kinds of race conditions to still happen. Of course, we should allow specific tests to override this value. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12661) Fix GridIoManagerSelfTest
Ilya Kasnacheev created IGNITE-12661: Summary: Fix GridIoManagerSelfTest Key: IGNITE-12661 URL: https://issues.apache.org/jira/browse/IGNITE-12661 Project: Ignite Issue Type: Test Components: cache Reporter: Ilya Kasnacheev It is an old test which tries to create some Processors and stich them together, which just does no longer work. We need to rewrite this test, I think. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12600) Baseline auto-adjust makes it impossible to assemble mixed persistence clusters
Ilya Kasnacheev created IGNITE-12600: Summary: Baseline auto-adjust makes it impossible to assemble mixed persistence clusters Key: IGNITE-12600 URL: https://issues.apache.org/jira/browse/IGNITE-12600 Project: Ignite Issue Type: Bug Components: persistence Reporter: Ilya Kasnacheev If you try to start a cluster of two nodes, where A only has default non-persistent region, and B also has a persistent non-default region, it will fail with: {code} Caused by: class org.apache.ignite.spi.IgniteSpiException: Joining persistence node to in-memory cluster couldn't be allowed due to baseline auto-adjust is enabled and timeout equal to 0 {code} However, there seem to be no configuration settings to actually disable this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12558) Add UTC timestamp to local node metrics
Ilya Kasnacheev created IGNITE-12558: Summary: Add UTC timestamp to local node metrics Key: IGNITE-12558 URL: https://issues.apache.org/jira/browse/IGNITE-12558 Project: Ignite Issue Type: Improvement Components: general Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Let's also add current UTC timestamp to "Metrics for local node" message. We often see logs posted where there is no date or the time zone is unknown, making it hard to correlate logs from different nodes. Time to fix that. We should also consider any other minor improvements to this block. Maybe tone down "to disable" message a bit to make up free space? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12540) Update versions of vulnerable dependencies
Ilya Kasnacheev created IGNITE-12540: Summary: Update versions of vulnerable dependencies Key: IGNITE-12540 URL: https://issues.apache.org/jira/browse/IGNITE-12540 Project: Ignite Issue Type: Improvement Components: general Reporter: Ilya Kasnacheev -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12505) Improve logging of data regions on startup
Ilya Kasnacheev created IGNITE-12505: Summary: Improve logging of data regions on startup Key: IGNITE-12505 URL: https://issues.apache.org/jira/browse/IGNITE-12505 Project: Ignite Issue Type: Improvement Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev First we have IgniteConfiguration printed (quiet=false): {quote}2019-07-24 02:33:33.918[INFO ][Thread-139][o.a.i.i.IgniteKernal%GridNodeName] IgniteConfiguration [... dfltDataRegConf=DataRegionConfiguration [name=mem_plc, maxSize=635655159808, initSize=268435456, swapPath=null, pageEvictionMode=DISABLED, evictionThreshold=0.9, emptyPagesPoolSize=100, metricsEnabled=true, metricsSubIntervalCount=5, metricsRateTimeInterval=1000, persistenceEnabled=true, checkpointPageBufSize=17179869184], storagePath=/ssd/data, checkpointFreq=3, lockWaitTime=1, checkpointThreads=4, checkpointWriteOrder=SEQUENTIAL, walHistSize=2147483647, walSegments=10, walSegmentSize=1073741824, walPath=/ssd/data/wal, walArchivePath=/sas/wal_archive, metricsEnabled=false, walMode=LOG_ONLY, walTlbSize=131072, walBuffSize=5242880, walFlushFreq=2000, walFsyncDelay=1000, walRecordIterBuffSize=67108864, alwaysWriteFullPages=false, fileIOFactory=org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory@3612c49a, metricsSubIntervalCnt=5, metricsRateTimeInterval=6, walAutoArchiveAfterInactivity=-1, writeThrottlingEnabled=false, walCompactionEnabled=true, walCompactionLevel=1], ...]{quote} Then we have all configured Data Regions printed per IGNITE-8803 (quiet=true): {quote} [11:30:36] Data Regions Configured: [11:30:36] ^-- plcWithMetrics [initSize=256,0 MiB, maxSize=6,3 GiB, persistence=false, lazyMemoryAllocation=true] [11:30:36] ^-- plcNoMetrics [initSize=256,0 MiB, maxSize=6,3 GiB, persistence=false, lazyMemoryAllocation=true]{quote} Then we print number of Data Regions that were initialized as per IGNITE-7196, but not regions themselves (quiet=false): Configured data regions initialized successfully [total=4] I propose to keep the first one (IgniteConfiguration), remove the second one (Data Regions Configured), and promote the last one to quiet mode while also outputting the regions themselves like this: {quote} [11:30:36] Data Regions Initialized Successfully: 4 [11:30:36] ^-- plcWithMetrics [initSize=256,0 MiB, maxSize=6,3 GiB, persistence=true, lazyMemoryAllocation=true] [11:30:36] ^-- plcNoMetrics [initSize=256,0 MiB, maxSize=6,3 GiB, persistence=true, lazyMemoryAllocation=true] [11:30:36] ^-- sysMemPlc [initSize=40,0 MiB, maxSize=100,0 MiB, persistence=true, lazyMemoryAllocation=false] [11:30:36] ^-- volatileMemPlc [initSize=40,0 MiB, maxSize=100,0 MiB, persistence=false, lazyMemoryAllocation=true]{quote} Also clearly mark default region and system regions as such. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12504) Auto-adjust breaks existing code, should be disabled by default
Ilya Kasnacheev created IGNITE-12504: Summary: Auto-adjust breaks existing code, should be disabled by default Key: IGNITE-12504 URL: https://issues.apache.org/jira/browse/IGNITE-12504 Project: Ignite Issue Type: Bug Components: cache, persistence Affects Versions: 2.8 Reporter: Ilya Kasnacheev Fix For: 2.8 We have automatic baseline adjustment now. However, it is 'on' by default, which means it breaks existing code. I see new exceptions when starting an existing project after bumping Ignite dependency version: {code} Caused by: org.apache.ignite.internal.processors.cluster.BaselineAdjustForbiddenException: Baseline auto-adjust is enabled, please turn-off it before try to adjust baseline manually {code} (Please see reproducer from attached UL discussion) I think we should disable auto-adjust by default, let people enable it when they see it fit. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12503) Fix SSL configuration in Jetty REST: httpCfg assignment
Ilya Kasnacheev created IGNITE-12503: Summary: Fix SSL configuration in Jetty REST: httpCfg assignment Key: IGNITE-12503 URL: https://issues.apache.org/jira/browse/IGNITE-12503 Project: Ignite Issue Type: Bug Components: rest Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev {code} {code} should become {code} {code} Or this assignment is not being made. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12502) Document ignite-spring-data_2.2 module
Ilya Kasnacheev created IGNITE-12502: Summary: Document ignite-spring-data_2.2 module Key: IGNITE-12502 URL: https://issues.apache.org/jira/browse/IGNITE-12502 Project: Ignite Issue Type: Improvement Components: documentation Reporter: Ilya Kasnacheev After IGNITE-12259 I think there are no API changes, but we should mention that we have such module and what its dependencies are. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12483) ReflectionFactory is essential thanks to PlatformDotNetSessionLockResult
Ilya Kasnacheev created IGNITE-12483: Summary: ReflectionFactory is essential thanks to PlatformDotNetSessionLockResult Key: IGNITE-12483 URL: https://issues.apache.org/jira/browse/IGNITE-12483 Project: Ignite Issue Type: Bug Components: binary Reporter: Ilya Kasnacheev We currently treat ReflectionFactory as a nice-to-have thing, so we silently ignore failures of its reflection: {code} try { Class refFactoryCls = Class.forName("sun.reflect.ReflectionFactory"); refFac = refFactoryCls.getMethod("getReflectionFactory").invoke(null); ctorFac = refFac.getClass().getMethod("newConstructorForSerialization", Class.class, Constructor.class); } catch (NoSuchMethodException | ClassNotFoundException | IllegalAccessException | InvocationTargetException ignored) { // No-op. } {code} However, it is now essential thanks to the class PlatformDotNetSessionLockResult, which is always registered during note start-up and which does not have empty constructor. So not having access to ReflectionFactory (JBoss will hide it, for example) will lead to the following cryptic exception (courtesy stack overflow): {code} 2019-12-19 09:11:39,355 SEVERE [org.apache.ignite.internal.IgniteKernal] (ServerService Thread Pool -- 81) Got exception while starting (will rollback startup routine).: class org.apache.ignite.binary.BinaryObjectException: Failed to find empty constructor for class: org.apache.ignite.internal.processors.platform.websession.PlatformDotNetSessionLockResult at deployment.StreamsApp.ear//org.apache.ignite.internal.binary.BinaryClassDescriptor.constructor(BinaryClassDescriptor.java:981) at deployment.StreamsApp.ear//org.apache.ignite.internal.binary.BinaryClassDescriptor.(BinaryClassDescriptor.java:267) at deployment.StreamsApp.ear//org.apache.ignite.internal.binary.BinaryContext.registerPredefinedType(BinaryContext.java:1063) at deployment.StreamsApp.ear//org.apache.ignite.internal.binary.BinaryContext.registerPredefinedType(BinaryContext.java:1048) at deployment.StreamsApp.ear//org.apache.ignite.internal.binary.BinaryContext.(BinaryContext.java:350) at deployment.StreamsApp.ear//org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.start(CacheObjectBinaryProcessorImpl.java:208) at deployment.StreamsApp.ear//org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1700) at deployment.StreamsApp.ear//org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1013) at deployment.StreamsApp.ear//org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) at org.jboss.as.ee@18.0.1.Final//org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:88) {code} My suggestions are the following: - Introduce a warning when ReflectionFactory not found instead of ignoring exception. - Add empty constructor to PlatformDotNetSessionLockResult and make sure no other classes need reflection during start-up. - (optionally) instead, introduce an error when ReflectionFactory not found. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12433) Possible to create table with duplicate definition of column
Ilya Kasnacheev created IGNITE-12433: Summary: Possible to create table with duplicate definition of column Key: IGNITE-12433 URL: https://issues.apache.org/jira/browse/IGNITE-12433 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.8 Reporter: Ilya Kasnacheev {code} sqlline version 1.3.0 sqlline> !connect jdbc:ignite:thin://localhost Enter username for jdbc:ignite:thin://localhost: Enter password for jdbc:ignite:thin://localhost: 0: jdbc:ignite:thin://localhost> CREATE TABLE Person(ID INTEGER PRIMARY KEY, NAME VARCHAR(100), NAME . . . . . . . . . . . . . . . .> VARCHAR(100), AGE INTEGER (64)); No rows affected (0,229 seconds) 0: jdbc:ignite:thin://localhost> select * from person; ++++ | ID | NAME | AGE | ++++ ++++ No rows selected (0,073 seconds) {code} This is on master branch. "NAME VARCHAR(100)" twice. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12428) Cache configuration cacheLoader error after cfgs serialization changes
Ilya Kasnacheev created IGNITE-12428: Summary: Cache configuration cacheLoader error after cfgs serialization changes Key: IGNITE-12428 URL: https://issues.apache.org/jira/browse/IGNITE-12428 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.8 Reporter: Ilya Kasnacheev The attached test will work on 2.7 but will fail on master {code} [19:37:44,606][SEVERE][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Failed to initialize cache(s) (will try to rollback) [exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], discoEvt=DiscoveryCustomEvent [customMsg=DynamicCacheChangeBatch [id=18f068bee61-a388ef9b-b1b7-435f-897f-2397f3af9cff, reqs=ArrayList [DynamicCacheChangeRequest [cacheName=foo-0, hasCfg=true, nodeId=895176bf-6687-4a05-a7b3-3a1bfa81b728, clientStartOnly=false, stop=false, destroy=false, disabledAfterStartfalse]], exchangeActions=ExchangeActions [startCaches=[foo-0], stopCaches=null, startGrps=[foo-0], stopGrps=[], resetParts=null, stateChangeRequest=null], startCaches=false], affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=895176bf-6687-4a05-a7b3-3a1bfa81b728, consistentId=0:0:0:0:0:0:0:1%lo,127.0.0.1,172.17.0.1,192.168.1.2:47500, addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 127.0.0.1, 172.17.0.1, 192.168.1.2], sockAddrs=HashSet [/172.17.0.1:47500, /0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, /192.168.1.2:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1575909462941, loc=true, ver=2.7.0#20191209-sha1:, isClient=false], topVer=1, nodeId8=895176bf, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1575909464589]], nodeId=895176bf, evt=DISCOVERY_CUSTOM_EVT], caches=[o.a.i.i.processors.cache.ExchangeActions$CacheGroupActionData@76180c2b]] class org.apache.ignite.IgniteCheckedException: Cannot enable read-through (loader or store is not provided) for cache: foo-0 at org.apache.ignite.internal.processors.cache.GridCacheProcessor.validate(GridCacheProcessor.java:609) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCacheContext(GridCacheProcessor.java:1616) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheContext(GridCacheProcessor.java:2398) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:2333) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.lambda$prepareStartCaches$55a0e703$1(GridCacheProcessor.java:2208) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.lambda$prepareStartCachesIfPossible$6(GridCacheProcessor.java:2178) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareStartCaches(GridCacheProcessor.java:2205) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareStartCachesIfPossible(GridCacheProcessor.java:2176) at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.processCacheStartRequests(CacheAffinitySharedManager.java:953) at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:839) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1270) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:793) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:3031) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2880) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Failed to complete exchange process. at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1318) at org.apache.ignite.internal.IgniteKernal.createCache(IgniteKernal.java:2980) at org.apache.ignite.CacheStoreInitializationTest.createCache(CacheStoreInitializationTest.java:107) at org.apache.ignite.CacheStoreInitializationTest.deadlock_shuffledRandomOrder(CacheStoreInitializationTest.java:127) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at
[jira] [Created] (IGNITE-12425) Deadlock on CacheStore.loadAll
Ilya Kasnacheev created IGNITE-12425: Summary: Deadlock on CacheStore.loadAll Key: IGNITE-12425 URL: https://issues.apache.org/jira/browse/IGNITE-12425 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.7.6 Reporter: Ilya Kasnacheev Attachments: ignite-dataload-deadlock.zip Yes, we do have a deadlock in CacheStore.loadAll, as demonstrated by this reproducer. The obvious embarassing fix is to replace HashMap with TreeMap, which will of course not work if keys are not compabable {code} diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java index 3156d6d662d..3947800a908 100644 --- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java +++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java @@ -34,6 +34,7 @@ import java.util.List; import java.util.Map; import java.util.NoSuchElementException; import java.util.Set; +import java.util.TreeMap; import java.util.UUID; import java.util.concurrent.Callable; import java.util.concurrent.ExecutorService; @@ -2054,7 +2055,7 @@ public abstract class GridCacheAdapter implements IgniteInternalCache(); +misses = new TreeMap<>(); misses.put(key, res); {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12403) Throttle page difference output in PageMemoryTracker
Ilya Kasnacheev created IGNITE-12403: Summary: Throttle page difference output in PageMemoryTracker Key: IGNITE-12403 URL: https://issues.apache.org/jira/browse/IGNITE-12403 Project: Ignite Issue Type: Bug Components: cache Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Currently, PageMemoryTracker's dumpBytes will print a line for every byte, possibly gigabytes of them. Let's limit it up to 64k different bytes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12401) Some Text Queries return repeated results
Ilya Kasnacheev created IGNITE-12401: Summary: Some Text Queries return repeated results Key: IGNITE-12401 URL: https://issues.apache.org/jira/browse/IGNITE-12401 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.8 Reporter: Ilya Kasnacheev It came to my attention while checking for Range Queries support that we don't actually check that found query results are the correct ones. We were checking that we got some results, but not whether they were expected. And voila, it turns out that Range Queries examples, as well as some other test cases, will readily fail when run with such checks! A query will return same value repeatedly, e.g. range query will return the "1" record twice, and limited text query will return "14" record twice. It didn't really occur on non-range queries before the introduction of limits. I think we should not ship broken limit queries. Maybe also fix range queries, if it's hard let's @Ignore them for now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12365) Concurrent removeAll() on the same cache leads to deadlock
Ilya Kasnacheev created IGNITE-12365: Summary: Concurrent removeAll() on the same cache leads to deadlock Key: IGNITE-12365 URL: https://issues.apache.org/jira/browse/IGNITE-12365 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.8 Reporter: Ilya Kasnacheev That's because removeAll is implemented as invokeAll with HashSet parameter. invokeAll + HashSet = I will attach simple reproducer (please note that multiple invocations of test is needed, typically 5-10) as well as thread dumps. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12296) Implement Log Throttle in .Net and print warning for unsafe PutAll ops.
Ilya Kasnacheev created IGNITE-12296: Summary: Implement Log Throttle in .Net and print warning for unsafe PutAll ops. Key: IGNITE-12296 URL: https://issues.apache.org/jira/browse/IGNITE-12296 Project: Ignite Issue Type: Improvement Components: platforms Reporter: Ilya Kasnacheev Assignee: Pavel Tupitsyn Right now it's not possible to do one-time developer warnings in .Net as there is no Log Throttle. Please implement one. Please also issue warning if random-order collection is passed to PutAll, InvokeAll, RemoveAll, as it is prime source for deadlocks and developer frustration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12284) When service class not found on any server nodes, service not deployed without any error
Ilya Kasnacheev created IGNITE-12284: Summary: When service class not found on any server nodes, service not deployed without any error Key: IGNITE-12284 URL: https://issues.apache.org/jira/browse/IGNITE-12284 Project: Ignite Issue Type: Bug Components: managed services Reporter: Ilya Kasnacheev When service class presents on client node but not on server node, the following is printed on server: {code} [17:57:43,398][SEVERE][srvc-deploy-#44][GridServiceProcessor] Failed to initialize service (service will not be deployed): FService class org.apache.ignite.IgniteCheckedException: com.gridgain.datetest.MyService at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10174) at org.apache.ignite.internal.processors.service.GridServiceProcessor.copyAndInject(GridServiceProcessor.java:1440) at org.apache.ignite.internal.processors.service.GridServiceProcessor.redeploy(GridServiceProcessor.java:1361) at org.apache.ignite.internal.processors.service.GridServiceProcessor.processAssignment(GridServiceProcessor.java:1988) at org.apache.ignite.internal.processors.service.GridServiceProcessor.onSystemCacheUpdated(GridServiceProcessor.java:1615) at org.apache.ignite.internal.processors.service.GridServiceProcessor.access$300(GridServiceProcessor.java:126) at org.apache.ignite.internal.processors.service.GridServiceProcessor$ServiceEntriesListener$1.run0(GridServiceProcessor.java:1597) at org.apache.ignite.internal.processors.service.GridServiceProcessor$DepRunnable.run(GridServiceProcessor.java:2064) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: com.gridgain.datetest.MyService at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:707) at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1758) at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1717) at org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313) at org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102) at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82) at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10168) ... 10 more Caused by: java.lang.ClassNotFoundException: com.gridgain.datetest.MyService at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8775) at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:349) at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:698) ... 16 more {code} but on client, ServiceDeploymentException is not thrown. Instead, deploy returns normally. Code to reproduce: {code} public class ServiceStarterMain { public static void main(String[] args) { Ignition.setClientMode(true); Ignite ignite = Ignition.start(); ServiceConfiguration serviceConfiguration = new ServiceConfiguration(); serviceConfiguration.setName("FService"); serviceConfiguration.setMaxPerNodeCount(4); serviceConfiguration.setTotalCount(10); serviceConfiguration.setService(new MyService()); ignite.services().deploy(serviceConfiguration); // Exception expected, but does not happen } } {code} against a blank Ignite node started from bin distro. This affects Java and .Net. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12275) Make byte order in Persistence pages configurable
Ilya Kasnacheev created IGNITE-12275: Summary: Make byte order in Persistence pages configurable Key: IGNITE-12275 URL: https://issues.apache.org/jira/browse/IGNITE-12275 Project: Ignite Issue Type: Improvement Components: persistence Reporter: Ilya Kasnacheev Right now we force it to be in LITTLE_ENDIAN, but we don't read it in fixed order, which leads to failures. After IGNITE-12271 it will be changed to nativeOrder(). We should make it configurable so that persistence files may be moved between nodes of different archs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12271) Persistence can't read pages from disk on Big Endian architectures
Ilya Kasnacheev created IGNITE-12271: Summary: Persistence can't read pages from disk on Big Endian architectures Key: IGNITE-12271 URL: https://issues.apache.org/jira/browse/IGNITE-12271 Project: Ignite Issue Type: Improvement Components: persistence Reporter: Ilya Kasnacheev So we are trying to start master on Big Endian, and we get the following exceptions: {code} Runtime failure on row: Row@5bf1ee15[ snip, ver: GridCacheVersion [topVer=180723326, order=1569259166164, nodeOrder=1] ][ 1307496, 32211, 3, 0 ]" [5-197] at org.h2.message.DbException.getJdbcSQLException(DbException.java:357) ... 41 more Caused by: class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: Runtime failure on row: Row@5bf1ee15[ snip], ver: GridCacheVersion [topVer=180723326, order=1569259166164, nodeOrder=1] ][ 1307496, 32211, 3, 0 ] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doPut(BPlusTree.java:2320) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putx(BPlusTree.java:2267) at org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.putx(H2TreeIndex.java:323) ... 38 more Caused by: java.lang.IllegalStateException: Failed to get page IO instance (page content is corrupted) at org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions.forVersion(IOVersions.java:84) at org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions.forPage(IOVersions.java:96) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:153) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:107) at org.apache.ignite.internal.processors.query.h2.database.H2RowFactory.getRow(H2RowFactory.java:61) at org.apache.ignite.internal.processors.query.h2.database.H2Tree.createRowFromLink(H2Tree.java:221) at org.apache.ignite.internal.processors.query.h2.database.io.AbstractH2ExtrasLeafIO.getLookupRow(AbstractH2ExtrasLeafIO.java:153) at org.apache.ignite.internal.processors.query.h2.database.io.AbstractH2ExtrasLeafIO.getLookupRow(AbstractH2ExtrasLeafIO.java:35) {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12260) Fallback to {user.home}/ignite/work if {user.dir} is not writable
Ilya Kasnacheev created IGNITE-12260: Summary: Fallback to {user.home}/ignite/work if {user.dir} is not writable Key: IGNITE-12260 URL: https://issues.apache.org/jira/browse/IGNITE-12260 Project: Ignite Issue Type: Improvement Components: general Affects Versions: 2.7.6 Reporter: Ilya Kasnacheev After IGNITE-12103 we have a new program, that some software under Windows, e.g. that is installed in Program Files, tries to create ignite\work\ dir under current dir which is not writable. It was suggested to fallback to {user.home}\ignite\work dir in such cases. On each start we will try to create workdir in current dir, fail, print warning and fallback to home dir. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12256) Fix double invocation of javaMajorVersion in scripts
Ilya Kasnacheev created IGNITE-12256: Summary: Fix double invocation of javaMajorVersion in scripts Key: IGNITE-12256 URL: https://issues.apache.org/jira/browse/IGNITE-12256 Project: Ignite Issue Type: Improvement Components: general Affects Versions: 2.7.6 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Most of our shell script look as folows: {code} # # Discover path to Java executable and check it's version. # checkJava # # Discover IGNITE_HOME environment variable. # setIgniteHome # # Final JVM_OPTS for Java 9+ compatibility # javaMajorVersion "${JAVA_HOME}/bin/java" {code} It makes no sense to me since we already call javaMajorVersion in checkJava. Let's try to get rid of it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12229) Column not found when using CASE named column from result
Ilya Kasnacheev created IGNITE-12229: Summary: Column not found when using CASE named column from result Key: IGNITE-12229 URL: https://issues.apache.org/jira/browse/IGNITE-12229 Project: Ignite Issue Type: Improvement Components: sql Affects Versions: 2.7.6 Reporter: Ilya Kasnacheev Consider the following SQL: {code} create table main (key varchar primary key, val int, grouper int); create table joiner (key varchar primary key, value int, grouper int); select m.key, case when m.grouper > j.grouper then 1 else -1 end d_grouper from main m join joiner j on m.key = j.key where d_grouper = 1 order by d_grouper; {code} Expected behavior - success. Observed behavior: {code} [20:31:13,498][SEVERE][jdbc-request-handler-worker-#1828][JdbcRequestHandler] Failed to execute SQL query [reqId=0, req=JdbcQueryExecuteRequest [schemaName=PUBLIC, pageSize=1024, maxRows=0, sqlQry=select m.key, case when m.grouper > j.grouper then 1 else -1 end d_grouper from main m join joiner j on m.key = j.key where d_grouper = 1 order by d_grouper, args=[], stmtType=ANY_STATEMENT_TYPE, autoCommit=true]] class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to parse query. Столбец "D_GROUPER" не найден Column "D_GROUPER" not found; SQL statement: select m.key, case when m.grouper > j.grouper then 1 else -1 end d_grouper from main m join joiner j on m.key = j.key where d_grouper = 1 order by d_grouper [42122-197] at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.prepareStatementAndCaches(IgniteH2Indexing.java:2653) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.parseAndSplit(IgniteH2Indexing.java:2356) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:2196) at org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2128) at org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2123) at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36) at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2693) at org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2137) at org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.executeQuery(JdbcRequestHandler.java:511) at org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.doHandle(JdbcRequestHandler.java:245) at org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandlerWorker.body(JdbcRequestHandlerWorker.java:90) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Caused by: org.h2.jdbc.JdbcSQLException: Столбец "D_GROUPER" не найден Column "D_GROUPER" not found; SQL statement: select m.key, case when m.grouper > j.grouper then 1 else -1 end d_grouper from main m join joiner j on m.key = j.key where d_grouper = 1 order by d_grouper [42122-197] at org.h2.message.DbException.getJdbcSQLException(DbException.java:357) at org.h2.message.DbException.get(DbException.java:179) at org.h2.message.DbException.get(DbException.java:155) at org.h2.expression.ExpressionColumn.optimize(ExpressionColumn.java:150) at org.h2.expression.Comparison.optimize(Comparison.java:177) at org.h2.expression.ConditionAndOr.optimize(ConditionAndOr.java:130) at org.h2.command.dml.Select.prepare(Select.java:861) at org.h2.command.Parser.prepareCommand(Parser.java:283) at org.h2.engine.Session.prepareLocal(Session.java:611) at org.h2.engine.Session.prepareCommand(Session.java:549) at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1247) at org.h2.jdbc.JdbcPreparedStatement.(JdbcPreparedStatement.java:76) at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:694) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.prepare0(IgniteH2Indexing.java:539) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.prepareStatement(IgniteH2Indexing.java:509) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.prepareStatement(IgniteH2Indexing.java:476) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.prepareStatementAndCaches(IgniteH2Indexing.java:2635) ... 12 more {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12204) In binary distribution, essential dependencies for ignite-spark missing
Ilya Kasnacheev created IGNITE-12204: Summary: In binary distribution, essential dependencies for ignite-spark missing Key: IGNITE-12204 URL: https://issues.apache.org/jira/browse/IGNITE-12204 Project: Ignite Issue Type: Improvement Components: spark Affects Versions: 2.7.6 Reporter: Ilya Kasnacheev It seems that we only put direct dependencies of other JARs in our binary distribution, and not transient ones. For example, libs/optional/ignite-spark lacks the essential commons-lang3 jar, which will lead to the following error immediately: {code} Exception in thread "main" java.lang.NoClassDefFoundError: org.apache.commons.lang3.SystemUtils at org.apache.spark.util.Utils$.(Utils.scala:1915) at org.apache.spark.util.Utils$.(Utils.scala) at org.apache.spark.SparkConf.loadFromSystemProperties(SparkConf.scala:75) {code} It's almost impossible to fix without resorting to Maven source build. I understand that adding Spark module to Ignite server is something not widely used, but if we ship this module at all, we should make sure that it is usable in some form. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12178) DEBUG logging may throw exception from toString of tx objects
Ilya Kasnacheev created IGNITE-12178: Summary: DEBUG logging may throw exception from toString of tx objects Key: IGNITE-12178 URL: https://issues.apache.org/jira/browse/IGNITE-12178 Project: Ignite Issue Type: Improvement Components: general Reporter: Ilya Kasnacheev See https://stackoverflow.com/questions/57727242/apache-ignite-failed-to-create-string-representation-of-binary-object {code:java} 0 I am getting below exception and not able to figure out what is wrong with the code. I have simplified my pojo classes which are to be persisted to ignite cache, but still the complexity remains. All my pojos are serializable but few of them have business logic code, dao, application context object. These objects can't be removed. Removing these things from the code will require whole code refractoring. class org.apache.ignite.IgniteException: Failed to create string representation of binary object. at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1022) at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:864) at org.apache.ignite.internal.processors.cache.distributed.near.GridNearSingleGetResponse.toString(GridNearSingleGetResponse.java:317) at java.lang.String.valueOf(String.java:2994) at java.lang.StringBuilder.append(StringBuilder.java:131) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:1162) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:1209) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$6.apply(GridDhtCacheAdapter.java:1003) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$6.apply(GridDhtCacheAdapter.java:938) at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385) at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:355) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processNearSingleGetRequest(GridDhtCacheAdapter.java:938) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$300(GridDhtAtomicCache.java:135) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$4.apply(GridDhtAtomicCache.java:257) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$4.apply(GridDhtAtomicCache.java:252) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101) at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197) at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127) at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093) at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Caused by: class org.apache.ignite.IgniteException: Failed to create string representation of binary object. at org.apache.ignite.internal.binary.BinaryObjectExImpl.toString(BinaryObjectExImpl.java:189) at org.apache.ignite.internal.binary.BinaryObjectImpl.toString(BinaryObjectImpl.java:920) at java.lang.String.valueOf(String.java:2994) at org.apache.ignite.internal.util.GridStringBuilder.a(GridStringBuilder.java:101) at org.apache.ignite.internal.util.tostring.SBLimitedLength.a(SBLimitedLength.java:88) at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:939) at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1005) ... 27 more Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to read field: currentContacts at org.apache.ignite.internal.binary.BinaryReaderExImpl.wrapFieldException(BinaryReaderExImpl.java:446) at
[jira] [Created] (IGNITE-12175) ODBC driver does not support WVARCHAR (typeId=-9) which breaks pyodbc
Ilya Kasnacheev created IGNITE-12175: Summary: ODBC driver does not support WVARCHAR (typeId=-9) which breaks pyodbc Key: IGNITE-12175 URL: https://issues.apache.org/jira/browse/IGNITE-12175 Project: Ignite Issue Type: Improvement Components: odbc Affects Versions: 2.7.5 Reporter: Ilya Kasnacheev Subj: pyodbc.Error: ('HYC00', '[HYC00] Data type is not supported. [typeId=-9] (0) (SQLBindParameter)') https://stackoverflow.com/questions/57971252/python-string-to-varchar-mapping-using-odbc-with-apache-ignite-not-supported/57973839#57973839 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IGNITE-12134) Document default Ignite work dir location
Ilya Kasnacheev created IGNITE-12134: Summary: Document default Ignite work dir location Key: IGNITE-12134 URL: https://issues.apache.org/jira/browse/IGNITE-12134 Project: Ignite Issue Type: Improvement Components: documentation Reporter: Ilya Kasnacheev Fix For: 2.7.6 What should be documented: Contents and significance of work dir - db, marshaller, wal. Default for cases when there is IGNITE_HOME (such as running from binary distribution) Default for cases when there's no IGNITE_HOME (running Ignite as library) Changing default logs/ that sometimes comes alongside work/ -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IGNITE-12103) Change the default ignite work directory once again to avoid writing to ~/work
Ilya Kasnacheev created IGNITE-12103: Summary: Change the default ignite work directory once again to avoid writing to ~/work Key: IGNITE-12103 URL: https://issues.apache.org/jira/browse/IGNITE-12103 Project: Ignite Issue Type: Bug Components: general Affects Versions: 2.7.5 Reporter: Ilya Kasnacheev Fix For: 2.7.6 While testing 2.7.6-RC1 it came up that we no longer write data to /tmp/ignite/work, which is good, but now we would instead write to /home/username/work, which is bad. ~/work is a generic directory not linked to Ignite in any obvious way, and users will be puzzled by its appearance with possibilities of data loss or, even worse, their own documents loss if they happen to have something in Work dir and it gets clobbered/removed by accident. I suggest changing this default once more, to use either /home/username/ignite/{work,logs,etc} or ./ignite/{work,logs,etc} by leveraging user.dir property pointing to current working dir. Please note that user.dir has its own problems since it is supposed to not be changeable after JVM is up, but some code still tries to change it. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IGNITE-12075) Wrong table alias when SUM used inside CASE WHEN
Ilya Kasnacheev created IGNITE-12075: Summary: Wrong table alias when SUM used inside CASE WHEN Key: IGNITE-12075 URL: https://issues.apache.org/jira/browse/IGNITE-12075 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.7.5 Reporter: Ilya Kasnacheev https://stackoverflow.com/questions/57472293/ignite-failed-to-run-reduce-query-locally Consider the following queries: {code} create table user (id int primary key, name varchar); SELECT CASE WHEN id = 2016 THEN SUM(id) END FROM user GROUP BY id; {code} Will cause splitter to try executing wrong SQL: ```Caused by: org.h2.jdbc.JdbcSQLException: Столбец "__Z0.ID" не найден Column "__Z0.ID" not found; SQL statement: SELECT CASE WHEN (__Z0.ID = 2016) THEN SUM(__C0_0) END __C0_0 FROM PUBLIC.__T0 [42122-197]``` -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (IGNITE-12033) .Net callbacks from striped pool due to async/await may hang cluster
Ilya Kasnacheev created IGNITE-12033: Summary: .Net callbacks from striped pool due to async/await may hang cluster Key: IGNITE-12033 URL: https://issues.apache.org/jira/browse/IGNITE-12033 Project: Ignite Issue Type: Bug Components: cache, platforms Affects Versions: 2.7.5 Reporter: Ilya Kasnacheev http://apache-ignite-users.70518.x6.nabble.com/Replace-or-Put-after-PutAsync-causes-Ignite-to-hang-td27871.html#a28051 There's a reproducer project. Long story short, .Net can invoke cache operations with future callbacks, which will be invoked from striped pool. If such callbacks are to use cache operations, those will be possibly sheduled to the same stripe and cause a deadlock. The code is very simple: {code} Console.WriteLine("PutAsync"); await cache.PutAsync(1, "Test"); Console.WriteLine("Replace"); cache.Replace(1, "Testing"); // Hangs here Console.WriteLine("Wait"); await Task.Delay(Timeout.Infinite); {code} async/await should absolutely not allow any client code to be run from stripes. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (IGNITE-11956) Cache topology changes during test run on stable topology
Ilya Kasnacheev created IGNITE-11956: Summary: Cache topology changes during test run on stable topology Key: IGNITE-11956 URL: https://issues.apache.org/jira/browse/IGNITE-11956 Project: Ignite Issue Type: Bug Reporter: Ilya Kasnacheev I am fixing a test GridCachePartitionedTxMultiNodeSelfTest which starts 4 nodes with cache defined in node configuration, and proceeds running queries. Query fail with exception: "Server topology is changed during query execution inside a transaction. It's recommended to rollback and retry transaction." The following log message is generated: [2019-06-04 19:47:02,410][INFO ][exchange-worker-#712%near.GridCachePartitionedTxMultiNodeSelfTest0%][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], crd=true, evt=DISCOVERY_CUSTOM_EVT, evtNode=374107c7-953a-42b2-8595-9bc0, customEvt=CacheAffinityChangeMessage [id=e1d41732b61-5761dee7-7e89-498e-94b2-334d4b743b3c, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], exchId=null, partsMsg=null, exchangeNeeded=true], allowMerge=false] Note that test is currently broken so you just need to fix that one issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11915) Document all ways to download Ignite in YARN
Ilya Kasnacheev created IGNITE-11915: Summary: Document all ways to download Ignite in YARN Key: IGNITE-11915 URL: https://issues.apache.org/jira/browse/IGNITE-11915 Project: Ignite Issue Type: Bug Components: documentation Reporter: Ilya Kasnacheev Fix For: 2.8 That is, automatic mirror download, download by URL, local file, HDFS path, HDFS releases dir. For other properties it would be nice to mention whether they accept local or HDFS paths because it's not obvious. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11877) Transactional deadlock with Near cache enabled
Ilya Kasnacheev created IGNITE-11877: Summary: Transactional deadlock with Near cache enabled Key: IGNITE-11877 URL: https://issues.apache.org/jira/browse/IGNITE-11877 Project: Ignite Issue Type: Bug Components: cache Reporter: Ilya Kasnacheev As witnessed by unrunning test NearCachePutAllMultinodeTest, since 2.0 there is a deadlock when transaction operations are performed in parallel with near cache enabled. Without near cache it will pass. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11860) Temporarily peg SSL version to TLSv1.2 to fix Java 11/12
Ilya Kasnacheev created IGNITE-11860: Summary: Temporarily peg SSL version to TLSv1.2 to fix Java 11/12 Key: IGNITE-11860 URL: https://issues.apache.org/jira/browse/IGNITE-11860 Project: Ignite Issue Type: Improvement Components: security Affects Versions: 2.7 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Fix For: 2.7.5 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11825) Test GridCommandHandlerTest#testCacheIdleVerifyNodeFilter fails with "Duplicate row in index"
Ilya Kasnacheev created IGNITE-11825: Summary: Test GridCommandHandlerTest#testCacheIdleVerifyNodeFilter fails with "Duplicate row in index" Key: IGNITE-11825 URL: https://issues.apache.org/jira/browse/IGNITE-11825 Project: Ignite Issue Type: Bug Components: persistence Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev A freshly contributed test fails around half of runs with exceptions like: {code} [2019-04-30 14:15:14,355][ERROR][data-streamer-stripe-0-#20402%gridCommandHandlerTest0%][IgniteTestResources] Failed to set initial value for cache entry: DataStreamerEntry [key=UserKeyCach eObjectImpl [part=25, val=25, hasValBytes=true], val=UserCacheObjectImpl [val=25, hasValBytes=true]] class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: Runtime failure on search row: SearchRow [key=KeyCacheObjectImpl [part=25, val=25, hasValBytes=tru e], hash=25, cacheId=0] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1817) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1619) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1602) at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:2160) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:433) at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:4282) at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.initialValue(GridCacheMapEntry.java:3430) at org.apache.ignite.internal.processors.cache.GridCacheEntryEx.initialValue(GridCacheEntryEx.java:772) at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:2280) at org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:140) at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6845) at org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:967) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:550) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.IllegalStateException: Duplicate row in index. at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Insert.run0(BPlusTree.java:437) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Insert.run0(BPlusTree.java:423) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5643) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5629) at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:359) at org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:285) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$11400(BPlusTree.java:92) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Put.tryInsert(BPlusTree.java:3622) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Put.access$7100(BPlusTree.java:3302) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.onNotFound(BPlusTree.java:3860) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$5800(BPlusTree.java:3652) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1902) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1784) {code} which will lead to partition divergence. Maybe node filter is not implemented correctly, but otherwise it would look like a Durable Memory bug. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11770) Document PARTITON_SUPPLIED and PARTITION_MISSED events
Ilya Kasnacheev created IGNITE-11770: Summary: Document PARTITON_SUPPLIED and PARTITION_MISSED events Key: IGNITE-11770 URL: https://issues.apache.org/jira/browse/IGNITE-11770 Project: Ignite Issue Type: New Feature Components: documentation Reporter: Ilya Kasnacheev Assignee: Artem Budnikov Fix For: 2.8 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11767) GridDhtPartitionsFullMessage retains huge maps on heap ion exchange history
Ilya Kasnacheev created IGNITE-11767: Summary: GridDhtPartitionsFullMessage retains huge maps on heap ion exchange history Key: IGNITE-11767 URL: https://issues.apache.org/jira/browse/IGNITE-11767 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.7 Reporter: Ilya Kasnacheev ExchangeHistory keeps a FinishState for every topology version. FinishState contains msg, which contains at least two huge maps: partCntrs2 and partsSizesBytes. We should probably strip msg, removing those two data structures before putting msg in exchFuts linked list to be stowed away. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11757) Missed partitions during rebalancing when new blank node joins
Ilya Kasnacheev created IGNITE-11757: Summary: Missed partitions during rebalancing when new blank node joins Key: IGNITE-11757 URL: https://issues.apache.org/jira/browse/IGNITE-11757 Project: Ignite Issue Type: Bug Components: cache Reporter: Ilya Kasnacheev Assignee: Ivan Rakov Please take a look at newly added test GridCachePartitionedSupplyEventsSelfTest.testSupplyEvents There's logging of missed partitions during rebalancing, and as you can see partitions are missed even when a new node joins stable topology, with no nodes leaving. Expected behavior is that in this case no partitions will be missed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11745) Rebalancing overwhelmingly prefers some supplier nodes to others
Ilya Kasnacheev created IGNITE-11745: Summary: Rebalancing overwhelmingly prefers some supplier nodes to others Key: IGNITE-11745 URL: https://issues.apache.org/jira/browse/IGNITE-11745 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.7 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev When cache has backups, and you add third node to cluster, Ignite will only rebalance data from single node. When you add n-th node, Ignite will not rebalance from some nodes and it will pull 10x as much data from some nodes than from others. This is because we filter static nodes list by partition availability and then pick the first one. Overwhelmingly it is the first nodes in list and nodes towards the end of list will never get to supply partitions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11730) Discovery Compression check fails when nodes reconnect to cluster
Ilya Kasnacheev created IGNITE-11730: Summary: Discovery Compression check fails when nodes reconnect to cluster Key: IGNITE-11730 URL: https://issues.apache.org/jira/browse/IGNITE-11730 Project: Ignite Issue Type: Bug Components: general Affects Versions: 2.7 Reporter: Ilya Kasnacheev There is a check that Discovery Data Bag compression is supported by all nodes. Apparently this check does not work when nodes reconnect after server restart. When there is at least one client node that does not support this feature, clients will still send zipped data that server would not understand, leaving to following server error: {code} [15:46:47,101][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to unmarshal discovery data for component: 0 class org.apache.ignite.IgniteCheckedException: Failed to deserialize object with given class loader: sun.misc.Launcher$AppClassLoader@18b4aac2 at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:147) at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94) at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161) at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82) at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9922) at org.apache.ignite.spi.discovery.tcp.internal.DiscoveryDataPacket.unmarshalData(DiscoveryDataPacket.java:290) at org.apache.ignite.spi.discovery.tcp.internal.DiscoveryDataPacket.unmarshalJoiningNodeData(DiscoveryDataPacket.java:169) at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.onExchange(TcpDiscoverySpi.java:2076) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processNodeAddedMessage(ServerImpl.java:4620) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:4307) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2962) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2729) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7496) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2833) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7427) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Caused by: java.io.EOFException at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2638) at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3113) at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:853) at java.io.ObjectInputStream.(ObjectInputStream.java:349) at org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.(JdkMarshallerObjectInputStream.java:43) at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:137) ... 16 more {code} and client nodes will fail with following error: {code} [15:46:47,175][SEVERE][tcp-client-disco-msg-worker-#4][] Critical system error detected. Will be handled accordingly to configured handler [hnd=class o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.IgniteException: Node with BaselineTopology cannot join mixed cluster running in compatibility mode]] class org.apache.ignite.IgniteException: Node with BaselineTopology cannot join mixed cluster running in compatibility mode at org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.onGridDataReceived(GridClusterStateProcessor.java:727) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.onExchange(GridDiscoveryManager.java:899) at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.onExchange(TcpDiscoverySpi.java:2083) at org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.processNodeAddFinishedMessage(ClientImpl.java:2149) at org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.processDiscoveryMessage(ClientImpl.java:2028) at org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.body(ClientImpl.java:1884) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at org.apache.ignite.spi.discovery.tcp.ClientImpl$1.body(ClientImpl.java:302) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) {code} They
[jira] [Created] (IGNITE-11672) JdbcThinConnectionSelfTest.testInvalidEndpoint failed after thin client retry
Ilya Kasnacheev created IGNITE-11672: Summary: JdbcThinConnectionSelfTest.testInvalidEndpoint failed after thin client retry Key: IGNITE-11672 URL: https://issues.apache.org/jira/browse/IGNITE-11672 Project: Ignite Issue Type: Bug Components: clients Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev Broken after IGNITE-11599 Looks like error message/exception has changed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11629) Cassandra dependencies missing from deliverable
Ilya Kasnacheev created IGNITE-11629: Summary: Cassandra dependencies missing from deliverable Key: IGNITE-11629 URL: https://issues.apache.org/jira/browse/IGNITE-11629 Project: Ignite Issue Type: Bug Components: cassandra Affects Versions: 2.7 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev After IGNITE-9046 we lack an explicit netty-resolver dependency for ignite-cassandra-store module. This means that tests still run, project can be made working by fixing dependencies, but apache-ignite-bin deliverable's libs/optional/ignite-cassandra-store does not contain all required depencencies since we only put explicit ones there. Need to add this dependency explicitly, check that it works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11558) Developer warning when HashMap is passed to putAll()
Ilya Kasnacheev created IGNITE-11558: Summary: Developer warning when HashMap is passed to putAll() Key: IGNITE-11558 URL: https://issues.apache.org/jira/browse/IGNITE-11558 Project: Ignite Issue Type: Improvement Components: cache Affects Versions: 2.7 Reporter: Ilya Kasnacheev Currently when HashMap is passed to putAll it's very easy to cause deadlock since the order of keys is not stable. This is a pity because users will use HashMap by default and not expect any trouble. We should issue a warning when user passes HashMap (but not LinkedHashMap) to putAll(). On .Net we should probably check for Dictionary. Warning similar to the one issues when index cannot be efficiently inlined. Another approach is to turn keys into binary form and then sort them, if map is not a SortedMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11547) ClassCastException when creating LOCAL cache on client in Data Region which is considered persistent
Ilya Kasnacheev created IGNITE-11547: Summary: ClassCastException when creating LOCAL cache on client in Data Region which is considered persistent Key: IGNITE-11547 URL: https://issues.apache.org/jira/browse/IGNITE-11547 Project: Ignite Issue Type: Bug Components: cache, persistence Affects Versions: 2.7 Reporter: Ilya Kasnacheev If default region in cluster is persistent, then creating a LOCAL cache on client will fail with: org.apache.ignite.internal.pagemem.impl.PageMemoryNoStoreImpl cannot be cast to org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryEx This is unless there is a second non-persistent data region, and cache is created in this non-persistent region. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11424) Cannot build C++ from tarball of master branch
Ilya Kasnacheev created IGNITE-11424: Summary: Cannot build C++ from tarball of master branch Key: IGNITE-11424 URL: https://issues.apache.org/jira/browse/IGNITE-11424 Project: Ignite Issue Type: Bug Components: platforms Reporter: Ilya Kasnacheev Assignee: Igor Sapego In a tarball made by mvn initialize -Prelease on fresh master branch: {code} % libtoolize && aclocal && autoheader && automake --add-missing && autoreconf libtoolize: putting auxiliary files in '.'. libtoolize: linking file './ltmain.sh' libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. libtoolize: linking file 'm4/libtool.m4' libtoolize: linking file 'm4/ltoptions.m4' libtoolize: linking file 'm4/ltsugar.m4' libtoolize: linking file 'm4/ltversion.m4' libtoolize: linking file 'm4/lt~obsolete.m4' configure.ac:36: installing './ar-lib' configure.ac:35: installing './compile' configure.ac:24: installing './config.guess' configure.ac:24: installing './config.sub' configure.ac:28: installing './install-sh' configure.ac:28: installing './missing' configure.ac:88: error: required file 'network/include/Makefile.in' not found configure.ac:88: error: required file 'network/Makefile.in' not found Makefile.am:27: error: required directory ./network does not exist Makefile.am:22: error: required directory ./network does not exist Makefile.am:51: error: required directory ./network does not exist binary/Makefile.am: installing './depcomp' {code} Indeed it seems that network/ does not exist -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11395) GridAffinityAssignment should use list of node UUIDs instead of ClusterNodes
Ilya Kasnacheev created IGNITE-11395: Summary: GridAffinityAssignment should use list of node UUIDs instead of ClusterNodes Key: IGNITE-11395 URL: https://issues.apache.org/jira/browse/IGNITE-11395 Project: Ignite Issue Type: Bug Components: binary, compute Affects Versions: 2.7 Reporter: Ilya Kasnacheev GridAffinityAssignment and GridAffinityAssignmentV2 currently carry private List> assignment; Those are fat objects since they contain node metrics. Nevertheless they get serialized and sent away when affinityCall() is performed. We should make GridAffinityAssignmentV3 which will instead have private List> assignment; -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11299) During SSL Handshake GridNio
Ilya Kasnacheev created IGNITE-11299: Summary: During SSL Handshake GridNio Key: IGNITE-11299 URL: https://issues.apache.org/jira/browse/IGNITE-11299 Project: Ignite Issue Type: Bug Reporter: Ilya Kasnacheev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11298) TcpCommunicationSpi does not support TLSv1.3
Ilya Kasnacheev created IGNITE-11298: Summary: TcpCommunicationSpi does not support TLSv1.3 Key: IGNITE-11298 URL: https://issues.apache.org/jira/browse/IGNITE-11298 Project: Ignite Issue Type: Bug Components: general Affects Versions: 2.7 Reporter: Ilya Kasnacheev When started on Java 11 we cannot form a secure cluster - Discovery will happily use the default TLSv1.2 but Communication will fail with its custom SSLEngine-using code. Need to fix that. Until that, nodes may be salvaged by setProtocol("TLSv1.2") on SslContextFactory, or by system property -Djdk.tls.client.protocols="TLSv1.2" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11248) H2 Debug Console reports NPE when launched
Ilya Kasnacheev created IGNITE-11248: Summary: H2 Debug Console reports NPE when launched Key: IGNITE-11248 URL: https://issues.apache.org/jira/browse/IGNITE-11248 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.7 Reporter: Ilya Kasnacheev Reliably happens on invocation of IGNITE_H2_DEBUG_CONSOLE=true bin/ignite.sh {code} Внутренняя ошибка: "java.lang.NullPointerException" General error: "java.lang.NullPointerException"; SQL statement: SELECT TABLE_CAT, TABLE_SCHEM, TABLE_NAME, TABLE_TYPE, REMARKS, TYPE_CAT, TYPE_SCHEM, TYPE_NAME, SELF_REFERENCING_COL_NAME, REF_GENERATION, SQL FROM (SELECT SYNONYM_CATALOG TABLE_CAT, SYNONYM_SCHEMA TABLE_SCHEM, SYNONYM_NAME as TABLE_NAME, TYPE_NAME AS TABLE_TYPE, REMARKS, TYPE_NAME TYPE_CAT, TYPE_NAME TYPE_SCHEM, TYPE_NAME AS TYPE_NAME, TYPE_NAME SELF_REFERENCING_COL_NAME, TYPE_NAME REF_GENERATION, NULL AS SQL FROM INFORMATION_SCHEMA.SYNONYMS WHERE SYNONYM_CATALOG LIKE ? ESCAPE ? AND SYNONYM_SCHEMA LIKE ? ESCAPE ? AND SYNONYM_NAME LIKE ? ESCAPE ? AND (true) UNION SELECT TABLE_CATALOG TABLE_CAT, TABLE_SCHEMA TABLE_SCHEM, TABLE_NAME, TABLE_TYPE, REMARKS, TYPE_NAME TYPE_CAT, TYPE_NAME TYPE_SCHEM, TYPE_NAME, TYPE_NAME SELF_REFERENCING_COL_NAME, TYPE_NAME REF_GENERATION, SQL FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_CATALOG LIKE ? ESCAPE ? AND TABLE_SCHEMA LIKE ? ESCAPE ? AND TABLE_NAME LIKE ? ESCAPE ? AND (TABLE_TYPE IN(?, ?, ?, ?, ?, ?, ?)) ) ORDER BY TABLE_TYPE, TABLE_SCHEM, TABLE_NAME [5-197] HY000/5 {code} in browser window that is opened. Reportedly, used to work just fine 2.6 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11235) Sending Continuous Query with absent class on client knocks out server node
Ilya Kasnacheev created IGNITE-11235: Summary: Sending Continuous Query with absent class on client knocks out server node Key: IGNITE-11235 URL: https://issues.apache.org/jira/browse/IGNITE-11235 Project: Ignite Issue Type: Bug Affects Versions: 2.7 Reporter: Ilya Kasnacheev *In 2.7*, when you create a Continuous Query on client with class which is not present in server classpath, server node will be terminated immediately. {code} [2019-02-05 18:20:22,508][ERROR][tcp-disco-msg-worker-#2][root] Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteException: Failed to unmarshal discovery custom message: TcpDiscoveryCustomEventMessage [msg=null, super=TcpDiscoveryAbstractMessage [sndNodeId=18424a1f-a22a-4401-83f7-2040a30e28f0, id=992de3eb861-18424a1f-a22a-4401-83f7-2040a30e28f0, verifierNodeId=61ff15f0-6337-4e00-ab29-7173a0fb0e1f, topVer=2, pendingIdx=0, failedNodes=null, isClient=false class org.apache.ignite.IgniteException: Failed to unmarshal discovery custom message: TcpDiscoveryCustomEventMessage [msg=null, super=TcpDiscoveryAbstractMessage [sndNodeId=18424a1f-a22a-4401-83f7-2040a30e28f0, id=992de3eb861-18424a1f-a22a-4401-83f7-2040a30e28f0, verifierNodeId=61ff15f0-6337-4e00-ab29-7173a0fb0e1f, topVer=2, pendingIdx=0, failedNodes=null, isClient=false]] {code} This means that one rogue client (e.g. with mismatched code base version) can knock whole cluster offline. I think we should disconnect client from discovery instead when it sends incorrect discovery message. *In 2.6*, if you tried to create a Continuous Query on client with class which is not present in server classpath, you will get the following erorr on server: {code} [15:32:57,844][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to unmarshal discovery custom message. class org.apache.ignite.IgniteCheckedException: Failed to find class with given class loader for unmarshalling (make sure same versions of all classes are available on all nodes or enable peer-class-loading) [clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, cls=com.pany.CQTest$1] at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:144) at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94) at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161) at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82) at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9962) at org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:81) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:5476) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5305) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2765) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Caused by: java.lang.ClassNotFoundException: com.pany.CQTest$1 at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8608) at org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) at
[jira] [Created] (IGNITE-11177) IGNITE.NODE_METRICS view fails with "Cannot parse "TIME" constant" > 24h
Ilya Kasnacheev created IGNITE-11177: Summary: IGNITE.NODE_METRICS view fails with "Cannot parse "TIME" constant" > 24h Key: IGNITE-11177 URL: https://issues.apache.org/jira/browse/IGNITE-11177 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.7 Reporter: Ilya Kasnacheev This is because we are using TIME type for several additive measurements: {quote}SqlSystemViewNodeMetrics.class: valueTimeFromMillis(metrics.getTotalJobsExecutionTime()), valueTimeFromMillis(metrics.getTotalBusyTime()), valueTimeFromMillis(metrics.getTotalIdleTime()),{quote} which will be hundreds of hours on long-running cluster, but {{TIME}} type is limited to 24 hours and will fail to be converted otherwise, as in: {quote}0: jdbc:ignite:thin://localhost> SELECT CAST('40:52:26.548' AS TIME); Error: Failed to parse query. Невозможно преобразование строки "40:52:26.548" в тип "TIME" Cannot parse "TIME" constant "40:52:26.548"; SQL statement: SELECT CAST('40:52:26.548' AS TIME) [22007-197] (state=42000,code=1001){quote} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11162) SQL: Print a warning if SQL index inline size is overly large
Ilya Kasnacheev created IGNITE-11162: Summary: SQL: Print a warning if SQL index inline size is overly large Key: IGNITE-11162 URL: https://issues.apache.org/jira/browse/IGNITE-11162 Project: Ignite Issue Type: New Feature Components: sql Affects Versions: 2.7 Reporter: Ilya Kasnacheev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11151) Document IGNITE_OVERRIDE_CONSISTENT_ID system property
Ilya Kasnacheev created IGNITE-11151: Summary: Document IGNITE_OVERRIDE_CONSISTENT_ID system property Key: IGNITE-11151 URL: https://issues.apache.org/jira/browse/IGNITE-11151 Project: Ignite Issue Type: New Feature Components: documentation Affects Versions: 2.8 Reporter: Ilya Kasnacheev IGNITE_OVERRIDE_CONSISTENT_ID system property / env var was added. If it is supplied, then node will use specified consistentId instead of default one or even one specified in IgniteConfiguration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11127) GridDhtInvalidPartitionException not handled by GridCacheTtlManager
Ilya Kasnacheev created IGNITE-11127: Summary: GridDhtInvalidPartitionException not handled by GridCacheTtlManager Key: IGNITE-11127 URL: https://issues.apache.org/jira/browse/IGNITE-11127 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.4 Reporter: Ilya Kasnacheev Leading to either message processing problems: {code} [2019-01-27 16:57:45,474][ERROR][sys-stripe-2-#3][GridCacheIoManager] Failed to process message [senderId=4839b5a2-a295-44cf-8a44-f0cb932b689e, messageType=class o.a.i.i.processors.cache.distributed.dht.atomic.GridNearAtomicFullUpdateRequest] class org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtInvalidPartitionException [part=381, msg=Adding entry to partition that is concurrently evicted [grp=, part=381, shouldBeMoving=, belongs=false, topVer=AffinityTopologyVersion [topVer=818, minorTopVer=0], curTopVer=AffinityTopologyVersion [topVer=818, minorTopVer=0]]] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.localPartition0(GridDhtPartitionTopologyImpl.java:917) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.localPartition(GridDhtPartitionTopologyImpl.java:794) at org.apache.ignite.internal.processors.cache.distributed.dht.GridCachePartitionedConcurrentMap.localPartition(GridCachePartitionedConcurrentMap.java:69) at org.apache.ignite.internal.processors.cache.distributed.dht.GridCachePartitionedConcurrentMap.putEntryIfObsoleteOrAbsent(GridCachePartitionedConcurrentMap.java:88) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.entryEx(GridCacheAdapter.java:952) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.entryEx(GridDhtCacheAdapter.java:525) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.entryEx(GridCacheAdapter.java:943) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.expire(IgniteCacheOffheapManagerImpl.java:1047) at org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:197) at org.apache.ignite.internal.processors.cache.GridCacheUtils.unwindEvicts(GridCacheUtils.java:835) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessageProcessed(GridCacheIoManager.java:1093) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1066) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:378) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:304) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:99) at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:293) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183) at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126) at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:505) at java.lang.Thread.run(Thread.java:748) {code} or unhandled unspecified exceptions in user code (possibly violating JCache): {code} [2019-01-27 10:23:35,451][ERROR][pub-#840058][ComputeJobProcess] class org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtInvalidPartitionException [part=485, msg=Adding entry to partition that is concurrently evicted [grp=, part=485, shouldBeMoving=, belongs=false, topVer=AffinityTopologyVersion [topVer=815, minorTopVer=0], curTopVer=AffinityTopologyVersion [topVer=815, minorTopVer=0]]] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.localPartition0(GridDhtPartitionTopologyImpl.java:917) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.localPartition(GridDhtPartitionTopologyImpl.java:794) at org.apache.ignite.internal.processors.cache.distributed.dht.GridCachePartitionedConcurrentMap.localPartition(GridCachePartitionedConcurrentMap.java:69) at
[jira] [Created] (IGNITE-11104) Remove Java 7 leftovers from IgniteSystemProperties
Ilya Kasnacheev created IGNITE-11104: Summary: Remove Java 7 leftovers from IgniteSystemProperties Key: IGNITE-11104 URL: https://issues.apache.org/jira/browse/IGNITE-11104 Project: Ignite Issue Type: Improvement Components: general Affects Versions: 2.7 Reporter: Ilya Kasnacheev Assignee: Ilya Kasnacheev IGNITE_FORCE_START_JAVA7 can no longer do anything useful, for example. Should remove it right away since it's noop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)