Re: Can we avoid the PME when restarts a node in cluster.

2.7.5 may not include the feature~ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Can we avoid the PME when restarts a node in cluster.

Thank for your replay! Is there a scheduled release date of 2.8.0? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Can we avoid the PME when restarts a node in cluster.

Hi Igniters - Since the PME may hang the cluster for a while when restarting, so can we avoid it? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Memory leak.Ignite runs slower and slower after a period of time.

Emmm, Does it mean the cursor still not closed in Ignite-2.6.0? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Memory leak.Ignite runs slower and slower after a period of time.

Slava - Thank you again for your reply, I will close the cursor explicitly. BTW, I can not find the class you provided(Ignite-2.6.0), do you spell it correctly? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Memory leak.Ignite runs slower and slower after a period of time.

One more question. According to the org.apache.ignite.cache.query.QueryCursor.getAll API, I know that the resources will be closed automatically since all results are fetched. I would like to know will resources be closed after I iterate over the Cursor？ The reason why I ask this question is

Re: Memory leak.Ignite runs slower and slower after a period of time.

One more question. According to the org.apache.ignite.cache.query.QueryCursor.getAll API, I know that the resources will be closed automatically since all results are fetched. I would like to know will resources be closed after I iterate over the Cursor？ The reason why I ask this question is

Re: Memory leak.Ignite runs slower and slower after a period of time.

Slava - Thank for your reply, I will confirm it. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Question about SQL query when persistence enabled

According to the document, we know that there is only a subset of data in memory while the full data set in the disk when persistence enabled. I have some questions with regards to this point: Let's say I have a SQL like: select * from city where provinceId = '1'; 1. If some data is in memory

Re: Can I update specific field of a binaryobject

Besides the question above, I have another question. If I use a composed key like this: public class DpKey implements Serializable { //key=devId + "_" + dpId private String key; @AffinityKeyMapped private String devId; //getter setter } Now I need to add records like this,

Re: Ignite traffic isn't even in production environment

Ilya - Thank for your reply, you have helped me many times, thank you again. Q: What's the difference between chart 1 and chart 2? A: Please ignore chart 2? Another question, is there a convenient way to increase the partition number without losing the data? -- Sent from:

Re: Ignite put/query takes more than 500ms occasionally

The QPS is about 6700 SQL query operations and 2800 write operations. The QPS is not very high, I think ignite can perform better. Do I have an inappropriate configuration or miss some configurations? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Is there a mechanism that allows the user to evict cache entries that relate to an affinityKey.

Ilya - Thank for your advice, it is very helpful! Other questions: 1. If I delete data points with the same devId concurrently, is it possible occurred a deadlock? 2. Does cache entry have a timestamp? 3. Can I delete datapoints relate to a devId in PageAbstractEvictionTracker#evictDataPage, is

Re: Backup make DataStreamer performance decreased a lot.

Ilya - Thank you for your kind help. Do you mind sharing your server configuration? I re-run with your configuration, it cost more than 60 minutes to load 4000 records. And I increased data region size and checkout frequency, they improve a bit, but still too slow. According to my test,

Re: Backup make DataStreamer performance decreased a lot.

I have tried to increase QueryIndex.setInlineSize and CacheConfiguration.setSqlIndexMaxInlineSize to 128 256 and 512, but the performace became worse. Do I miss some configuration? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Backup make DataStreamer performance decreased a lot.

I have tried to increase QueryIndex.setInlineSize and CacheConfiguration.setSqlIndexMaxInlineSize to 128 256 and 512, but the performace became worse. Do I miss some configuration? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Is there a mechanism that allows the user to evict cache entries that relate to an affinityKey.

Ilya - First of all, thank for your reply. For your suggest, I have some questions: 1.Do you mean I should disable eviction policies? 2.How to scan partition by ScanQuery, I did not find an example in ignite. 3.If we have more 40 million records, does this manner has good performance? -- Sent

Re: Backup make DataStreamer performance decreased a lot.

Thank for your reply! 1. No, I did not use FULL_SYNC, because it will wait for write or commit to complete on all participating remote nodes (primary and backup), so it may lead to a drop of write performance, am I right? But I will try it. 2. Yes, please refer to the attachment, I dumped thread

Re: Performance degradation in case of high volumes

Thank for your reply, the project was created, you can refer to : https://github.com/RedBlackTreei/streamer.git Related post: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Data-Streamer-Hung-after-a-period-td21161.html Looking forward to your reply! -- Sent from:

Re: Performance degradation in case of high volumes

Thank for your reply, the project was created, you can refer to : https://github.com/RedBlackTreei/streamer.git Related post: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Data-Streamer-Hung-after-a-period-td21161.html Looking forward to your reply! -- Sent from:

Re: Performance degradation in case of high volumes

Ilya - I have tried to increase QueryIndex.setInlineSize and CacheConfiguration.setSqlIndexMaxInlineSize to 128 256 and 512, but the performace became worse. Do I miss some configuraiton? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Performance degradation in case of high volumes

Ilya - I use ignite 2.6.0, does not have IGNITE_MAX_INDEX_PAYLOAD_SIZE system property. But our index field has a fixed length:25 characters, so where can I find the algorithm to calculate the 'index inline size'. Looking forward to your reply. -- Sent from:

Re: Performance degradation in case of high volumes

Ilya - I use ignite 2.6.0, does not have IGNITE_MAX_INDEX_PAYLOAD_SIZE system property. But our index field has a fixed length:25 characters, so where can I find the algorithm to calculate the 'index inline size'. Looking forward to your reply. -- Sent from:

Re: 回复： Ignite Data Streamer Hung after a period

I have tried to load data without indexes, but it does not have any help! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: 回复： Ignite Data Streamer Hung after a period

I have tried to load data without indexes, but it does not have any help! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: 回复： Ignite Data Streamer Hung after a period

After I dig in the issue, I found that the streamer threads are waiting for building the index. This looks normal in a database based system, the more data, the slower insertion. But ignite is a wide used system, I think other people may encounter this problem, and have ways to improve the

Re: Ignite Data Streamer Hung after a period

Dmitry - I also encountered this problem. I used both persistence and indexing, when I loaded 20 million records, the loading speed became much slower than before, but the CPU of the ignite server is low.

Re: The cluster with persistence enable is stuck for 60 seconds when restarting a node. Is this normal?

Thank Sergey! >From the email that he sent to me(I do not know why the email doesn't display in this post), I know that the default checkpointFrequency(18) was too long, so it impacts the partition map exchange. After I reduced the checkpointFrequency, the block time decrease to 10 seconds.

Re: The cluster with persistence enable is stuck for 60 seconds when restarting a node. Is this normal?

Sergey - Thank for your reply. Yes, I can find them in the logs, I attached the logs of three nodes, the node that was restarted is node3. BTW, I use the default Rebalance Mode(ASYNC), so the rebalancing may not impact the whole cluster. Am I getting it wrong? ignite-003.log

Re: The cluster with persistence enable is stuck for 60 seconds when restarting a node. Is this normal?

Can someone give me some suggestions? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: The cluster with persistence enable is stuck for 60 seconds when restarting a node. Is this normal?

Sergey - Thank for your reply. Yes, I defined consistentId for every node. Is it relates to partition map exchange? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: The cluster with persistence enable is stuck for 60 seconds when restarting a node. Is this normal?

Sergey - Thank for your reply. Yes, I defined consistentId for every node. Does it relates to partition map exchange? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

The cluster with persistence enable is stuck for 60 seconds when restarting a node. Is this normal?

I have a cluster(2.7.0) with persistence enabled, the cluster has three server nodes(8c16g) and every node has 8 Gb data(1 primary and 1 backup). When I restart one of them, the whole cluster will stop about 60 seconds and do not process any request from the client. >From the logs, I saw that

Re: Failed to wait for initial partition map exchange

Ilya - Thank for your reply. Is there any configuration can help us reduce the WAL time in 2.6.0. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Failed to wait for initial partition map exchange

I also have this problem with ignite 2.6.0. Spend more than 350 seconds to restart a ignite node. Here is the system usage during partition map exchange: The CPU is at a low level and memory is enough. Here is the logs, we can find that applying WAL changes takes a long time, about

Re: Failed to wait for initial partition map exchange

I also have this problem with ignite 2.6.0. Spend more than 350 seconds to restart a ignite node. Here is the system usage during partition map exchange: The CPU is at a low level and memory is enough. Here is the logs, we can

Re: WAL size control

I have more than three hundred WAL segments in our folder -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: WAL size control

Hi Igniters - I have the problem too, but I only have one node in the server. I want to know how to delete the unused WAL segments safety. Looking forward to your reply. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

How to dump thread stacks in Ignite docker container

Hi Igniters - I tried to dump the thread stacks, but I don't know how to dump the thread stacks from a docker container since it only contains a simplified JRE, does not have JSTACK tools, and I also googled a lot of information but found that there is no suitable method. -- Sent from:

Re: How to add new nodes to a running cluster?

Do you add this node to the topology? https://apacheignite.readme.io/docs/baseline-topology -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite does not perform as expected

Sorry for my poor English, I mean only use devId as my index, and I have found the solution in ignite documents. I can force ignite use devId as index according to *USE INDEX* -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite does not perform as expected

Thank for your reply. How can I use devId as my index when only one index can be used for one table -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite does not perform as expected

Thank for your reply. How can I use devId as my index when only one index can be used for one table -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite does not perform as expected

when I run a SQL query with the two indexes, but ignite does not perform as expected, it only uses dpId does not use devId, I explain the SQL: SELECT __Z0.ID AS __C0_0, __Z0.GMTCREATE AS __C0_1, __Z0.GMTMODIFIED AS __C0_2, __Z0.DEVID AS __C0_3, __Z0.DPID AS __C0_4,

Re: If I want to know how long it takes for data reading from the server side, where should I add logs?

Andrei - Thank for your reply, sorry for missing a sentence in this post, it should be: there are almost 0.013% read requests take more than 20ms, and 0.0336% more than 200ms. When the traffic is very large, the number of requests over 200ms will be very large. So I want to know where the time

If I want to know how long it takes for data reading from the server side, where should I add logs?

I found there are almost 0.013% read requests take more than 20ms when I test the performance of ignite, so I want to know where the time takes, client or server? I do not know where to add logs clearly, can anyone tell me? Is

Re: Is there a configuration list that describes what configurations can be changed when restart

Ilya - Thank for your reply~ After testing, I found that some configurations can be modified, such as the initialSize, maxSize of DataRegionConfiguration, because I found it was changed(according to the start logs ignite printed) when I starting the cluster, is it a bug? -- Sent from:

Is there a configuration list that describes what configurations can be changed when restart

Hi Igniters - Is there a configuration list that describes what configurations can be changed and became effective when restart? In addition, is there any difference that configured in java code and spring XML file. Will appreciate to those who can give advice, looking forward to your reply!

RE: Question about add new nodes to ignite cluster.

Thank for your replies! I agree with "the node doesn’t serve any requests." But the documents write that: Asynchronous rebalancing mode. Distributed caches will start immediately and will load all necessary data from other available grid nodes in the background. under Rebalance Modes

Re: I encountered a problem when restarting ignite

Akurbanov - Thank for your reply! I have tried to dump the thread stacks, but I don't know how to dump the thread stacks from a docker container since it only contains a simplified JRE, does not have JSTACK tools, and I also googled a lot of information and found that there is no suitable

Re: I encountered a problem when restarting ignite

Sorry for forget the attachments. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

I encountered a problem when restarting ignite

I encountered a problem when restarting ignite, a node seems to be unable to process the request anymore, because its CacheStore did not process the update operations of database. Here is the log it printed: 2018-12-12 07:06:27:556 [exchange-worker-#42] WARN

Re: I encountered a problem when restarting ignite

Sorry for forgetting the attachments. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

I encountered a problem when restarting ignite

I encountered a problem when restarting ignite, a node seems to be unable to process the request anymore, because its CacheStore did not process the update operations of database. Here is the log it printed: 2018-12-12 07:06:27:556 [exchange-worker-#42] WARN

RE: Ignite cache.getAll takes a long time

Stan - Thank for your reply! Yes, the getAll and putAll(async) executed in parallel(amount of operations executed at the same time). But I think it may be caused by the write-behind, when I disabled the write-behind the timeout disappeared, and I enabled the write-behind the timeout appeared.

Re: write behind performance impacting main thread. Write behind buffer is never full

Hi lmark58 - I met a problem similar with you have met before. The performance of _cache.get()_ gets worse when enabled the write-behind, so I want to know will the performance got better when disabling the WriteBehindCoalescing? -- Sent from:

Re: Question about add new nodes to ignite cluster.

Another question: How the client APIs get or put data to the rebalancing cluster(Async Mode) when adding a new node to the cluster, from the old nodes or the new node? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite cache.getAll takes a long time

I use the default thread pool settings, can it improve the performance after I increase the system thread pool? Another question, should I add the configuration in server nodes? Looking forward to your reply -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite cache.getAll takes a long time

Hi all I have a cluster with 3 server nodes which have 4 CPU cores and 8Gb memory. When applications call cache.getAll(set), it often takes a long time almost 500ms(about 0.6%), even the set size is less than 3. And I do not think it was caused by the network, because the network latency

Ignite cache.getAll takes a long time

Hi all I have a cluster with 3 server nodes which have 4 CPU cores and 8Gb memory. When applications call cache.getAll(set), it often takes a long time almost 500ms(about 0.6%), even the set size is less than 3. And I do not think it was caused by the network, because the network latency

Re: Question about add new nodes to ignite cluster.

I understand, thank you very much. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Question about add new nodes to ignite cluster.

I added 3 nodes to existing cluster, but not add them to topology, like below: Cluster state: active Current topology version: 93 Baseline nodes: ConsistentID=0ded99c1-b19c-4ced-ba3a-06abe233c6c8, STATE=ONLINE ConsistentID=2c252722-a3c4-4f55-8d26-d216329fddbb, STATE=ONLINE

Question about add new nodes to ignite cluster.

I added 3 nodes to existing cluster, but not add them to topology, like below: Cluster state: active Current topology version: 93 Baseline nodes: ConsistentID=0ded99c1-b19c-4ced-ba3a-06abe233c6c8, STATE=ONLINE ConsistentID=2c252722-a3c4-4f55-8d26-d216329fddbb, STATE=ONLINE

Question about add new nodes to ignite cluster.

I added 3 nodes to existing cluster, but not add them to topology, like below: Cluster state: active Current topology version: 93 Baseline nodes: ConsistentID=0ded99c1-b19c-4ced-ba3a-06abe233c6c8, STATE=ONLINE ConsistentID=2c252722-a3c4-4f55-8d26-d216329fddbb, STATE=ONLINE

Re: Does it affect the put, get, query operation when add or remove a node?

Great, thank you very much! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Does it affect the put, get, query operation when add or remove a node?

Hi - When I add or remove a server node, does the performance of the cluster will be affected? become slower or even stop? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Does it affect the put, get, query operation when add or remove a node?

Hi - When I add or remove a server node, does the performance of the cluster will be affected? become slower or even stop? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: What is the username of ignite web console

wt & Denis Thank for your reply, you are right, I forgot I have registered before. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

What is the username of ignite web console

I do not know how to log in the web console as admin, and I can not find any information about admin in the ignite docs. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

An error occured when recreate cache

The following is the stack: 2018-10-25 03:47:02:992 [exchange-worker-#42] ERROR o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture:498 - Failed to reinitialize local partitions (preloading will be stopped): GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1],

Re: UPDATE query with JOIN

Anyone who can tell me how to do an update operation with *join table*? I have tried the SQL below: 1.update A set A.a1=1 join table(devId varchar=?) B on A.devId=B.devId *SQL ERROR* 2.update A, table(devId varchar=?) B set A.a2=1 where A.devId=B.devId *SQL ERROR* 3.update A set A.a1=1 where

Re: Update all data in cache spend a long time

Thank you, I will try queryParallelism. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Update all data in cache spend a long time

Ilya - Thank for your reply and point out the problem may exist in my code, I will try to split the operation into smaller ones. And I still have two problems: 1. Can I split the operation with LIMIT? 2. Where is time spent in the operation? -- Sent from:

Update all data in cache spend a long time

Hi - I have a cache with more than 1 million records, when I update the whole records in it, it spends a long time(almost 6 minutes). Here is the SQL: "update " + IgniteTableKey.T_DEVICE_ONLINE_STATUS.getCode()+ " set isOnline=0, mqttTime=" + System.currentTimeMillis() / 1000 + " where 1=1";

Re: UPDATE query with JOIN

Thank for your reply! I sure I have indexed the devId, here is the output of EXPLAIN: [SELECT __Z0.DEVID AS __C0_0, __Z0.ISONLINE AS __C0_1, __Z0.MQTTTIME AS __C0_2 FROM "device_online_status".T_DEVICE_ONLINE_STATUS __Z0 /*

Re: UPDATE query with JOIN

Hi Alexander - I have tried the SQL you suggested, but the performance got worse, I do not know why? 1. "update t_device_module set isOnline=1, mqttTime=" + System.currentTimeMillis() / 1000 + " where devId in ('0001', '0002', ..., '2048')"; The SQL may take 600ms for 2048 records. 2."update

Re: UPDATE query with JOIN

Hi Alexander - I have tried the SQL you suggested, but the performance got worse, I do not know why? 1. "update t_device_module set isOnline=1, mqttTime=" + System.currentTimeMillis() / 1000 + " where devId in ('0001', '0002', ..., '2048')"; The SQL may take 600ms for 2048 records. 2."update

Re: The system cache size was slowly increased

Evgenii - Thank for your reply! "system memory cache" means the result printed by 'free -m' command. After some test, I solved the problem, it caused by my Linux setting, not Ignite. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: The system cache size was slowly increased

Prem - Thank for your reply. You explained why Ignite container grew from 2.5G to more than 4G because there is 2G heap memory and more than 2G off-heap memory, I think what you said is very correct and solve one of my main problems. But another question, why system memory cache also grows

Re: The system cache size was slowly increased

I analyzed the log and found that there are many *checkpoint timeout*. The following are the segments of the checkpoint: [db-checkpoint-thread-#46] INFO o.a.i.i.p.c.p.GridCacheDatabaseSharedManager:478 - Checkpoint started [checkpointId=7fdb8ddd-76d7-4ddf-9fd1-cddd8bb093be,

Re: The system cache size was slowly increased

Prem - Thank for your reply. You explained why Ignite container grew from 2.5G to more than 4G because there is 2G heap memory and more than 2G off-heap memory, I think what you said is very correct and solve one of my main problems. But another question, why system memory cache also grows slow

Re: The system cache size was slowly increased

Who can give me some advice? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: The system cache size was slowly increased

Who can give me some advice? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Connection reset by peer

I also encountered this problem before when I am doing performance testing. The reason why I get the exception is the CPU of the client is used 100%, so the connection was closed occasionally. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: The system cache size was slowly increased

The second question: I run the ignite nodes in the docker container with the following command: sudo -u docker docker run -v /mnt/logs/apps/ignite:/mnt/logs/apps/ignite -v /opt/ignite/ext-libs:/opt/ignite/ext-libs -v /opt/ignite/config:/opt/ignite/config -v

The system cache size was slowly increased

Hi all - We use Ignite in our production environment, But I found that the system cache was increased slowly and never reclaim. when the free system memory less than 200M, the node seemed did not work anymore and our system cannot get any response from the server nodes. The image below is our