Multithread transactions in a C++ Thin Client

Hello all, We're developing a multithread application using one C++ Thin Client to connect to a cluster with a single Server Node. The C++ Thin Client version is "master" from January 21. We have implemented a "lock-and-update" system based on the "GetAndPut" function and

Re: Multithread transactions in a C++ Thin Client

Ok... thanks Igor! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Re[2]: Performance of Ignite as key-value datastore. C++ Thin Client.

Hello! Ok... thanks Ilya! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Performance of Ignite as key-value datastore. C++ Thin Client.

Hi Zhenya, Thanks for your quick response 1. If it is not set, the following message appears on node's startup: [13:22:33] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender

Re: Re[2]: Performance of Ignite as key-value datastore. C++ Thin Client.

2. I've found this: [16:02:23,836][WARNING][client-connector-#139][GridDhtColocatedCache] Unordered map java.util.LinkedHashMap is used for putAll operation on cache vds. This can lead to a distribut ed deadlock. Switch to a sorted map like TreeMap instead. ... but I'm using a c++ std::map.

Re: putAll() performance big difference, 2.7.5 vs 2.9.1

Hi, Maybe it has to do with this: Performance of Ignite as key-value datastore. C++ Thin Client -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Long transaction suspended

Hi all, I'm trying to commit a very large transaction (8M keys and ~4GB of data). After a while, I can see this diagnostics message in node log: [08:56:31,721][WARNING][sys-#989][diagnostic] >>> Transaction [startTime=08:55:22.095, curTime=08:56:31.712, ... *state=SUSPENDED* ... Does anyone

Re: Re[2]: Performance of Ignite as key-value datastore. C++ Thin Client.

Hello, I understand these deadlocks when there are more than one PuAll at the same time, but in this case there is only one, and it’s always sorted. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: [2.7.6] Unsupported protocol version: -76 with thin client.

Hello, Thanks for your answer. The server is already in version 2.9.1 and the c++ thin client is from master branch to get transactions support. José -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Performance of Ignite as key-value datastore. C++ Thin Client.

Hi again, As a test, I just disabled persistence in both nodes. The already mentioned transaction of 1.2 million keys and 600MB in size takes 298sec. Remember that for one single node and persistence enabled it takes 70sec, so just adding a second node makes the test more than 4 times slower.

Re: Re[2]: Performance of Ignite as key-value datastore. C++ Thin Client.

First off, thanks to you and Zhenya for your support I'm afraid that's not easy, as key's data are unrelated. As we are in an evaluation phase, we would prefer to wait for the fix and perform single node benchmarking in the meanwhile... Instead of grouping keys, we might split the big

Re: Re[4]: Performance of Ignite as key-value datastore. C++ Thin Client.

Thanks for updating! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite 2.10. Performance tests in Azure

Hi, I have the same feeling, but I think that shouldn't be the case. Small number of big batches should decrease the total latency time while would favor the total throughput. And, as Ilya said: "In a distributed system, throughput will scale with cluster growth, but latency will be steady or

Re: Ignite 2.10. Performance tests in Azure

It's strange, as that's not what is stated in the documentation: https://ignite.apache.org/docs/latest/key-value-api/basic-cache-operations#atomic-operations otherwise, why would we need specific

Cluster endpoints [2.10 c++ thin client]

Hi all, After taking a look to the c++ thin client source code (cache_client_impl.cpp) it seems partition awareness is only taken into account for single requests (Get/Put), but neither for batch (GetAll/PutAll) nor transactions. But in our tests, adding endpoints to Ignite Client configuration

Re: Ignite 2.10. Performance tests in Azure

Hello, and thanks for answering so quick Because, as you say, I should get a bigger throughput when increasing the number of nodes. Of course I can't get the best of Ignite with this configuration, but I would expect something similar to what I get writing: time decreasing while nodes increase

Re: Long transaction suspended

Hi again, For an smaller succeeding transaction 1.2M keys and 600MB in size, I noticed it changed its state something similar as follows: SUSPENDED -> ACTIVE -> COMMITTING ... and it takes around 3 min to finish. For another test with 4M keys and 2GB it is still in SUSPENDED state after 30

Re: Long transaction suspended

Another test with 2M keys and 1GB also remains in SUSPENDED state after 11 minutes... I don't understand where the difference between this one and the successful 1.2M keys and 600MB could be. Any idea is welcomed -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Long transaction suspended

Hi, First off, thanks for your help. In the test, I'm using a single server node cluster with the official 2.9.1 version. Client is a C++ Thin Client with transactions support (commit 685c1b70ca from master branch). The test is very simple: struct Blob { int8_t

Re: Long transaction suspended

Hi, Because of the kind of product we have to develop, we currently have a set of scenarios with this kind of transactions and we're evaluating several datastores as RocksDB and, sadly, timings there are quite better than the ones I've got in Ignite... :( Data streamer is not available in C++

Re: Long transaction suspended

I wouldn't mind, but I'm afraid I'm not allowed to... at least, I couldn't find the option on that page :) -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Long transaction suspended

Hello Ilya, Yes, but it has nothing to do with suspending an active transaction... the problem is that transaction never reaches ACTIVE state because it takes a long time creating the tcp message. Please, take a look to my previous post. -- Sent from:

Re: Long transaction suspended

Hello! FYI: https://issues.apache.org/jira/browse/IGNITE-14165 -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Long transaction suspended

Hello! That's exactly the reason why we would prefer to choose Ignite over RocksDB. Otherwise, we will have to implement scalability by ourselves and, believe me, that's not something we would like to do. We also know they're not directly comparable. We would agree to pay the price for

Re: Re[2]: Long transaction suspended

Hello! I'm sorry hearing that. Would you think it could be fixed to reach these 2GB? Currently it's only 1GB in the C++ Thin Client Regards -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Re[2]: Long transaction suspended

Great!... I'm really looking forward it :) -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Long transaction suspended

Hi, thanks for pointing it out This confirms our tests... moving from a single-node cluster to a two-nodes one dropped the read timings to less than the half! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Long transaction suspended

Hello! Answering to myself: Should I get a better performance in a multi-node cluster? Of course! We have switched to a 4-node cluster and reading times dropped drastically. We are really looking forward the PutAll bug to be fixed to perform writing tests. -- Sent from:

Re: Re[4]: Performance of Ignite as key-value datastore. C++ Thin Client.

Ok... thanks! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Performance of Ignite as key-value datastore. C++ Thin Client.

Hi everyone, For our project, we have the next requirements: - One single cache - Ability to lock a list of cache entries. - Large transactions. A typical one is to commit 1.2 million keys (a single PutAll call) with a total size of around 600MB. - Persistence In our proof of concept, we've

Re: [2.10 branch]cpp thin client transaction :Transaction with id 1 not found.

Hello, I've already tested it and it works like a charm... Thanks! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

[2.9.1] Out of memory in data region

Hello, I've got this simple configuration file:

Re: [2.10 branch]cpp thin client transaction :Transaction with id 1 not found.

Hello! In fact, it's very simple: int main() { IgniteClientConfiguration cfg; cfg.SetEndPoints("10.250.0.10, 10.250.0.4"); try { IgniteClient client = IgniteClient::Start(cfg); CacheClient cache = client.GetOrCreateCache("vds"); ClientTransactions

Re: [2.9.1] Out of memory in data region

Hello Thanks for your help. I know that fixes the problem, but my question was about why I’m getting that error when persistence is on. As far as I know, if persistence is on, off-heap memory holds a subset of data when they don’t fit in memory, and no OOM error should be thrown... unless there

Re: [2.9.1] Out of memory in data region

Hello! Ok but, according to the error: [15:01:23,559][SEVERE][client-connector-#134][] JVM will be halted immediately due to the failure: [failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.mem.IgniteOutOfMemoryException: *Out of memory in data region* [name=default, initSize=1.0

Re: [2.10 branch]cpp thin client transaction :Transaction with id 1 not found.

Hello, I'm facing the same problem. Just let you know my configuration in case it helps you: - Two hosts in a local network, each having a server 2.9.1 node. Different static IP addresses - A single c++ thin client. - If I add the two addresses in the SetEndPoints function, I get the

Re: [2.10 branch]cpp thin client transaction :Transaction with id 1 not found.

Hello No, both belong to the same cluster. Waiting for the PutAll fix, we want to benchmark multi-node reading performance. For that we: - create a single server node cluster - create a single cache with 1.8M keys and ~1GB in it - create a single c++ thin node with a single endpoint (the host

Re: [2.9.1] Out of memory in data region

Hello, That's what I was thinking of... thanks Ilya -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite 2.10. Performance tests in Azure

Hello all, For our project we need a distributed database with transactional support, and Ignite is one of the options we are testing. Scalability is one of our must have, so we created an Ignite Kubernetes cluster in Azure to test it, but we found that the results were not what we expected. To