Re: Force backup on different physical machine

2020-07-07 Thread Pavel Tupitsyn
Normally there are two ways to achieve this: excludeNeighbors and AffinityBackupFilter [1] However, excludeNeighbors won't work across pods on the same k8s node, since it relies on MAC addresses. So your best bet is to use ClusterNodeAttributeAffinityBackupFilter: * set ClusterNodeAttributeAffini

Re: Performance Issue with Enum Serialization in Ignite 2.8

2020-07-23 Thread Pavel Tupitsyn
I'll have a look today and get back to you. Sorry, I did not notice this thread until now. On Thu, Jul 23, 2020 at 7:27 AM zork wrote: > Can someone provide inputs on this please? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >

Re: Performance Issue with Enum Serialization in Ignite 2.8

2020-07-23 Thread Pavel Tupitsyn
12:53 PM Pavel Tupitsyn wrote: > I'll have a look today and get back to you. > Sorry, I did not notice this thread until now. > > On Thu, Jul 23, 2020 at 7:27 AM zork wrote: > >> Can someone provide inputs on this please? >> >> >> >> -- >> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >> >

Re: Performance on Windows vs Linux

2020-07-23 Thread Pavel Tupitsyn
Linux often outperforms Windows in filesystem operations. This is a big generalization, of course - every workload is different and should be benchmarked. But this is worth keeping in mind when using persistence. Also, as Andrei said, most users run Ignite on Linux in production, and Ignite docume

Re: Ignite C# client does not report correct node ID after reconnection

2020-08-18 Thread Pavel Tupitsyn
Raymond, thank you for this bug report. I've filed a ticket [1]. I'll see if that's possible to squeeze the fix into the 2.9 release, otherwise you can expect it in 2.10. Will keep you posted. [1] https://issues.apache.org/jira/browse/IGNITE-13369 On Tue, Aug 18, 2020 at 10:02 AM Raymond Wilson

Re: Failing to update metadata for type

2020-08-24 Thread Pavel Tupitsyn
> IgniteClientDisconnectedCheckedException: Failed to send custom message: client is disconnected. The client is disconnected from the cluster, so it can't update the metadata. Please check what causes the disconnect - is the cluster alive, is the network ok? Thanks, Pavel On Mon, Aug 24, 2020 a

Re: Feature request: method to test active connection in Ignite thin client

2020-09-12 Thread Pavel Tupitsyn
Hello Scott, ClientCache.getName() is a local operation, it simply returns a cached string. IgniteClient.cluster().state() is a good way to check the connectivity - it sends a lightweight request to the server. As I understood, you are asking for something like IgniteClient.ping(), right? Thank

Re: Feature request: method to test active connection in Ignite thin client

2020-09-13 Thread Pavel Tupitsyn
). > > -- Scott > > > On Sat, Sep 12, 2020 at 9:33 AM Scott Prater wrote: > >> Correct, something like a ping. But the state() method should serve the >> purpose, too. Thanks! >> >> On Sat, Sep 12, 2020 at 2:30 AM Pavel Tupitsyn >> wrote: >

Re: [DISCUSSION] Renaming Ignite's product category

2020-09-17 Thread Pavel Tupitsyn
Agree with Val, even experienced developers have a hard time understanding what "in-memory computing platform" really does. "distributed memory-first database" is right on point. On Thu, Sep 17, 2020 at 8:30 AM Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > My vote is for the "dis

Re: Ignite Thin Client - IDLE TIMEOUT

2020-09-22 Thread Pavel Tupitsyn
Hi, You can set the idle timeout on the server side with ClientConnectorConfiguration.setIdleTimeout On Tue, Sep 22, 2020 at 8:16 AM Mahesh Renduchintala < mahesh.renduchint...@aline-consulting.com> wrote: > Hi, > > is there a way one can set IdleTImeOut for a thin client? > > I did not see such

Re: apache ignite 2.9

2020-10-13 Thread Pavel Tupitsyn
Hi, The release is on vote right now [1] You can expect it next week if all goes well. [1] http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Release-Apache-Ignite-2-9-0-RC3-td49668.html On Tue, Oct 13, 2020 at 6:56 PM narges saleh wrote: > Hi All, > > Any timeline for apache ignite 2.

Re: 3.0 and messaging

2020-10-15 Thread Pavel Tupitsyn
Hi Mikael, Please check the discussion on the dev list [1] In short: - Messaging can be simulated with Services and/or Compute to some extent - Some people have the same concern as you, so it is not clear yet whether we are going to remove messaging or not [1] http://apache-ignite-developers.2

Re: Ignite Client Node Stopped With Out Of Memory

2020-10-25 Thread Pavel Tupitsyn
Hi Ravi, The exception indicates that Ignite.NET failed to allocate unmanaged memory on .NET side while trying to pass query data from Java to .NET. This indicates that your system has run out of memory. Possible reasons are: * Memory is consumed by other apps * Memory is consumed by this app * M

Re: [ANNOUNCE] Apache Ignite 2.9.0 Released

2020-10-27 Thread Pavel Tupitsyn
Congratulations, everyone, and big thanks to Alex! As always, I've prepared an overview for .NET features and improvements: https://ptupitsyn.github.io/Whats-New-In-Ignite-Net-2.9/ Pavel On Fri, Oct 23, 2020 at 10:35 PM Denis Magda wrote: > We made it! Congrats, community! Thanks, Alex for pus

Re: Ignite Client Node Stopped With Out Of Memory

2020-10-28 Thread Pavel Tupitsyn
Looks like the app is running in 32 bit mode, which can't use more than 2GB of memory. JVM and memory regions pre-allocate all of it, leaving nothing for .NET to use. Please check the `Platform` column in the Task Manager - does it say `32 bit`? If yes, then try disabling `Prefer 32 bit` in the pr

Re: Ignite Client Node Stopped With Out Of Memory

2020-10-28 Thread Pavel Tupitsyn
ding from the logs? > > Is there any way to replicate it? > > Thanks, > > > On Wed, 28 Oct 2020 at 15:47, Pavel Tupitsyn wrote: > >> Looks like the app is running in 32 bit mode, which can't use more than >> 2GB of memory. >> JVM and memory regions pre-all

Re: Ignite Client Node Stopped With Out Of Memory

2020-10-28 Thread Pavel Tupitsyn
> > On Wed, 28 Oct, 2020, 7:53 pm Ravi Makwana, > wrote: > >> >> >> >> >> On Wed, 28 Oct, 2020, 5:55 pm Pavel Tupitsyn, >> wrote: >> >>> I found a bug in Ignite [1] which probably causes the issue on your side. >>> >>&

Re: Server node started via PuTTY, C# client not able to connect to cluster

2020-11-09 Thread Pavel Tupitsyn
What kind of error do you get? 10800 is the default thin client port, are you sure it is open on the server? On Mon, Nov 9, 2020 at 8:35 PM ABDumalagan wrote: > I am starting a cluster through another computer via PuTTY, and I was > wondering how to connect a local node started programmatically

.NET 5 and Ignite

2020-11-12 Thread Pavel Tupitsyn
Igniters, Here is a short note on recently released .NET 5: https://ptupitsyn.github.io/Ignite-on-NET-5/

Re: Issue with ignite thin client - ReliableChannel

2020-11-16 Thread Pavel Tupitsyn
Which version are you using? A similar issue was fixed for 2.8.1 and 2.9: https://issues.apache.org/jira/browse/IGNITE-12743 On Tue, Nov 17, 2020 at 1:53 AM Hemambara wrote: > Ignite thin client create and use ReliableChannel which starts below async > thread, but while closing the channel it is

Re: C# SQL API on key-value entries in cache error

2020-11-18 Thread Pavel Tupitsyn
Hello, The correct query would be "SELECT AssociateID FROM Associate", because Ignite infers table name from the class name, and column names from property names. You can override table name with QueryEntity.TableName property, and column names with QuerySqlFieldAttribute.Name property. On Wed,

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-11-21 Thread Pavel Tupitsyn
Hello, *> (1)* Regarding the Load method, I was under the impression that calling Get(key) would inherently call the Load(key) method from my OracleStore.cs It would if you set CacheConfiguration.ReadThrough = true. Similarly, enable WriteThrough if you want Ignite to call OracleStore.Write on ca

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-11-22 Thread Pavel Tupitsyn
Looks like we mixed up two different delegates: 1. ICache.LoadCache method takes an *optional *filter delegate in case you want to skip some of the entries from cache store. You can pass null there when filtering is not required. 2. ICacheStore.LoadCache method gives you (an implementor of the int

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-11-23 Thread Pavel Tupitsyn
Your code seems to be correct. It works for me in a simplified form: https://gist.github.com/ptupitsyn/a64c899b32b73ab55cb706cd4a09e6e9 1. Can you try the program above - does it work for you? 2. Can you confirm that the Oracle query returns a non-empty result set? On Mon, Nov 23, 2020 at 3:00 A

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-11-23 Thread Pavel Tupitsyn
1. How many Ignite nodes do you have? 2. What if you add Thread.Sleep(5000) before the last Console.WriteLine? Does the resulting number change? On Mon, Nov 23, 2020 at 6:01 PM ABDumalagan wrote: > 1. Your program worked for me! > > 2. I added something to my LoadCache(Action, params > object[]

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-11-23 Thread Pavel Tupitsyn
Ok, here is the problem, per documentation [1]: "In case of partitioned caches, keys that are not mapped to this node, either as primary or backups, will be automatically discarded by the cache." Since you have two nodes in the cluster, but call localLoadCache only on one node, part of the cache e

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-11-24 Thread Pavel Tupitsyn
DataStreamer is the best approach for loading large amounts of data into Ignite On Tue, Nov 24, 2020 at 3:50 PM ABDumalagan wrote: > I see, that makes sense - I actually have a question regarding that: > > If I wanted to store a whole table from an underlying database as > key-value pairs in cac

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-11-24 Thread Pavel Tupitsyn
NullPointerException usually indicates a bug. Please provide full exception stack trace. On Tue, Nov 24, 2020 at 4:03 PM ABDumalagan wrote: > Also, I tried changing the code from cache.LocalLoadCache(null) to > cache.LoadCache(null) and I received a NullPointerException. > > Am I supposed to pas

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-11-24 Thread Pavel Tupitsyn
1) Yes 2) On my local machine with 2 nodes I can stream 500 000 entries in *2* seconds (I tried to recreate the Associate class with some fake data). Are you sure that Oracle is not a bottleneck? Can you try a local scenario with fake data - how much time does it take? On Tue, Nov 24, 2020 at 5:46

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-11-24 Thread Pavel Tupitsyn
> how would I check to see if the underlying database is a bottleneck Check how much time it takes to read 500 000 rows from there, without loading into Ignite. Or run a profiler. On Tue, Nov 24, 2020 at 6:25 PM ABDumalagan wrote: > I'll try a local scenario and stream with fake data right now,

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-11-24 Thread Pavel Tupitsyn
The code looks ok to me. Since the query is so simple, most likely your connection to the DB is the bottleneck. On Tue, Nov 24, 2020 at 8:30 PM ABDumalagan wrote: > So I am currently running this > > code to just pull al

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-11-24 Thread Pavel Tupitsyn
Run your application in the same data center as the database, so that network costs are minimized. On Tue, Nov 24, 2020 at 8:54 PM ABDumalagan wrote: > I see - do you have any suggestions of how I can work around the bottleneck > and speed up data loading into cache? > > > > -- > Sent from: http

Re: .Net Cache TextQuery:cannot convert from 'Apache.Ignite.Core.Cache.Query.TextQuery' to 'Apache.Ignite.Core.Cache.Query.ScanQuery

2020-11-29 Thread Pavel Tupitsyn
Hi, TextQueryis not supported in Ignite thin clients [1]: - Use thick client instead (.NET thick client supports TextQuery) - Use Services or Compute as a workaround (wrap TextQuery call in a Service or Compute Task, use thin client to call the service or the task) [1] http://apache-ig

Re: CAP Theorem

2020-12-01 Thread Pavel Tupitsyn
Hi, Short answer: CP Long answer: https://medium.com/@valentin.kulichenko/apache-ignite-and-cap-theorem-17b8aacdc0e2 On Tue, Dec 1, 2020 at 10:22 PM Wolfgang Meyerle < wolfgang.meye...@googlemail.com> wrote: > Hi, > > the CAP Theorem states that a database can only achieve two of the > following

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-12-03 Thread Pavel Tupitsyn
> are there C# equivalencies to the Java interfaces BinaryObject and Affinity Yes, please see IIgnite.GetAffinity [1] and GetBinary [2]. [1] https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.IIgnite.html#Apache_Ignite_Core_IIgnite_GetAffinity_System_String_ [2] https://ig

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-12-03 Thread Pavel Tupitsyn
okups), and C# API is on par with Java (thin client is a bit faster, AFAIR). On Thu, Dec 3, 2020 at 12:11 PM Pavel Tupitsyn wrote: > > are there C# equivalencies to the Java interfaces BinaryObject and > Affinity > > Yes, please see IIgnite.GetAffinity [1] and GetB

Re: C# CacheStoreAdapter - Customizing Load, LoadCache methods

2020-12-04 Thread Pavel Tupitsyn
> Colocation by Partition, but that API is not available for C# Partition-based APIs were added in Ignite 2.9 [1] > Is it possible to speed up loading cache by using Binary Object + Colocating by Key, or Binary Object + MapReduce API? Probably yes. But have you sorted out the DB connection bottlen

Re: Ignite cache TextQuery on JsonString data Field

2020-12-14 Thread Pavel Tupitsyn
I can't see any details in the email too. However, code is visible on Nabble [1] Weird. [1] http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-TextQuery-on-JsonString-data-Field-td34743.html On Mon, Dec 14, 2020 at 5:03 PM Ilya Kasnacheev wrote: > Hello! > > Unfortunately, nothing is v

Re: .Net Cache Store throwing error:{"Unknown pair [platformId=1, typeId=-1455665693]"}

2020-12-18 Thread Pavel Tupitsyn
> KeepBinaryInStore = false, // Cache store works with deserialized data. > I don't have any model classes You ask Ignite to deserialize the data for TenantTestCacheStore, however, Ignite does not know which class to use for that. 1. Create a corresponding model class and register it in BinaryCo

Re: Is Model class OR a value object mandatory for inserting the data thru non SQL API

2020-12-28 Thread Pavel Tupitsyn
Hi, you can use BinaryObject API without defining any classes: ignite.binary().builder("ANY_STRING_HERE"); builder.setField("id", 1); builder.setField("name", "John"); On Tue, Dec 29, 2020 at 9:56 AM Naveen wrote: > HI > > We dont need a model / pojo class or value object defined for a table wh

Re: Is Model class OR a value object mandatory for inserting the data thru non SQL API

2020-12-29 Thread Pavel Tupitsyn
How do you configure the cache? QueryEntity.valueType should be the same string as you use in ignite.binary().builder() On Tue, Dec 29, 2020 at 11:08 AM Naveen wrote: > This way if we insert > > gnite.binary().builder("ANY_STRING_HERE"); > builder.setField("id", 1); > builder.setField("name", "J

Re: Requesting mapping from grid failed issue with Ignite 2.9.0 and C# model with ICloneable interface but same was working with Ignite 2.8.1

2021-01-05 Thread Pavel Tupitsyn
Hi Charlin, I've filed a bug [1] However, it looks like [QuerySqlField] attributes are missing on TestModel29WithICloneable properties. To make your model queryable with SQL/LINQ, make sure to mark some or all properties like this: public class TestModel29WithICloneable : ICloneable, IBinarizabl

Re: Double add double get wrong result

2021-01-11 Thread Pavel Tupitsyn
Hello, You'll get the same results in plain Java code double x = -11674.46; double y = 11674.67; System.out.println(x + y); This is caused by the internal floating point representation [1] [2] You can try DECIMAL instead of DOUBLE. [1] https://stackoverflow.com/questions/3227

Re: Looking for feedback on the Ignite 3.0.0 Alpha

2021-01-13 Thread Pavel Tupitsyn
Getting Started Guide: https://ignite.apache.org/docs/3.0.0-alpha/quick-start/getting-started-guide On Wed, Jan 13, 2021 at 1:29 PM Stephen Darlington < stephen.darling...@gridgain.com> wrote: > What is the link to the Getting Started Guide? > > On 13 Jan 2021, at 03:55, Valentin Kulichenko < > v

Re: Ever increasing startup times as data grow in persistent storage

2021-01-13 Thread Pavel Tupitsyn
Raymond, Please use ICluster.SetActive [1] instead, the API linked above is obsolete [1] https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cluster.ICluster.html?#Apache_Ignite_Core_Cluster_ICluster_SetActive_System_Boolean_ On Wed, Jan 13, 2021 at 11:54 AM Raymond Wilso

Re: Ignite service

2021-01-18 Thread Pavel Tupitsyn
Hi, Yes, you can write an Ignite service in C#. I see that C# examples are missing in the new docs, we will fix that. Meanwhile, please check the old C#/.NET docs [1], the API docs [2] and a blog post with Java -> .NET service call example, if you need that [3]. [1] https://apacheignite-net.read

Re: Event Listener on Server Node

2021-01-18 Thread Pavel Tupitsyn
Hi, You can use a combination of LocalListen and Services, Messaging or Compute to achieve something similar to RemoteListen, depending on the use case. If you explain the use case in more detail, I can suggest a specific approach. Thanks, Pavel On Mon, Jan 18, 2021 at 7:42 PM Jigna wrote: >

Re: Ignite service

2021-01-18 Thread Pavel Tupitsyn
> can I register my .net service in the configuration xml file To start an Ignite service, you have to call one of IServices.Deploy* methods, XML config files can't be used for that. > Gridgain community version 8.7.28 This is Apache Ignite mailing list, for GridGain product support please contact

Re: Event Listener on Server Node

2021-01-18 Thread Pavel Tupitsyn
Continuous Queries [1] are designed specifically for use cases like that - to handle cache updates efficiently, please use them instead of Events. You can subscribe to local and remote cache updates, and define a remote filter if necessary. [1] https://ignite.apache.org/docs/latest/key-value-api/

Re: Event Listener on Server Node

2021-01-19 Thread Pavel Tupitsyn
> we will have hundreds to thousands of caches This is not a very good idea [1], every cache imposes some overhead, and starting/stopping caches is not cheap as well. Please consider a different approach where data is grouped into a smaller number of caches. > When we insert a record in cache1,

Re: Event Listener on Server Node

2021-01-19 Thread Pavel Tupitsyn
In my experience, Continuous Query is a better approach than Events. On Tue, Jan 19, 2021 at 9:39 PM Jigna wrote: > Hi, > > We use many caches because we have different schemas. > > Basically, cache2 is a consolidation from different caches. This > consolidation only takes some attributes from c

Re: Event Listener on Server Node

2021-01-19 Thread Pavel Tupitsyn
Continuous Query supports both local and remote cache events, in .NET too. Try it - create two simple programs and run them both: // Program 1: var ignite = Ignition.Start(); var cache = ignite.GetOrCreateCache("cache-name"); for (var i = 0; i < int.MaxValue; i

Re: Event Listener on Server Node

2021-01-20 Thread Pavel Tupitsyn
You have to start a continuous query for every cache separately. On Thu, Jan 21, 2021 at 1:47 AM Jigna wrote: > Hi, > > Thank you for sharing the solution. It really helped me to understand the > continuous queries for remote event listening. I have another question for > continuous queries. > >

Re: Column not found using _val and sqlfieldsquery

2021-01-21 Thread Pavel Tupitsyn
Hi, I guess it should be like this: SELECT t._val from (select _val, EntityId FROM materials ) as t where t.EntityId='123' On Fri, Jan 22, 2021 at 8:10 AM siva wrote: > Hi, > I have .Net Ignite client and server app Ignite v2.9.1. > > I am using SqlFieldsQuery using thin client to query ignite

Re: Column not found using _val and sqlfieldsquery

2021-01-21 Thread Pavel Tupitsyn
> is there any specific reason why does not working if i use like this t._val.EntityId I think the reason here is that SQL (the standardized language) does not support this kind of syntax. Think of it as a flat table, _key and _val are just additional columns. On Fri, Jan 22, 2021 at 9:04 AM siv

Re: Re[2]: Performance of Ignite as key-value datastore. C++ Thin Client.

2021-01-27 Thread Pavel Tupitsyn
> how a deadlock is possible if keys are no repeated Deadlock is possible when two putAll operations are executed at the same time on the same keys, but key order is different in the provided map. > LinkedHashMap is used for putAll operation > but I'm using a c++ std::map Unfortunately, this warni

Re: Performance of Ignite as key-value datastore. C++ Thin Client.

2021-01-28 Thread Pavel Tupitsyn
I am able to reproduce the problem - it occurs with any client, thick or thin. With one node the transaction completes in a reasonable time, but with two nodes it is orders of magnitude slower. We'll investigate and get back to you. On Thu, Jan 28, 2021 at 9:48 AM jjimeno wrote: > Hi again, >

Re: [2.7.6] Unsupported protocol version: -76 with thin client.

2021-01-28 Thread Pavel Tupitsyn
Ilya, Normally you can use any combination of thin client and server versions, the highest common protocol version is negotiated automatically during the handshake. There are some exceptions to this - not all thin clients support very old protocols, but with recent versions it should work. On Th

Re: Re[2]: Performance of Ignite as key-value datastore. C++ Thin Client.

2021-01-28 Thread Pavel Tupitsyn
Josemari, can you please describe the use case in more detail? Maybe it is possible to reduce the number of cache entries by combining the data from multiple keys into one? On Fri, Jan 29, 2021 at 7:01 AM Zhenya Stanilovsky wrote: > > I also confirm this issue, i will append additional info into

Re: Performance issues with java thin client

2021-01-31 Thread Pavel Tupitsyn
Hi Naveen, > seeing longer times for making connections Longer compared to previous Ignite versions? On Mon, Feb 1, 2021 at 9:23 AM Naveen wrote: > Hi > > We are using 2.8.1 and using Java thin client, seeing longer times for > making connections, there by seeing the higher latency. > How do we

Re: Performance issues with java thin client

2021-02-01 Thread Pavel Tupitsyn
What is the size of the cluster? On Mon, Feb 1, 2021 at 10:18 AM Naveen wrote: > Not really used with any earlier versions, most of the time have used thick > clients only. > to begin with, it takes around 100 to 200 ms, slowly it goes to 1 sec and > keep increasing. > > > > > > -- > Sent from:

Re: putAll() performance big difference, 2.7.5 vs 2.9.1

2021-02-04 Thread Pavel Tupitsyn
Hi, 1. Can you please attach the CacheConfiguration as well? 2. Have you tried DataStreamer https://ignite.apache.org/docs/latest/data-streaming? On Thu, Feb 4, 2021 at 12:27 PM jjimeno wrote: > Hi, > > Maybe it has to do with this: > > Performance of Ignite as key-value datastore. C++ Thin Cli

Re: putAll() performance big difference, 2.7.5 vs 2.9.1

2021-02-05 Thread Pavel Tupitsyn
There is a known performance drop of around 7% in 2.9 vs 2.8: [1] However, it is nowhere near the difference you are getting. Please make sure that no other apps affect the comparison. Check our performance tuning documentation [2]. [1] https://issues.apache.org/jira/browse/IGNITE-13337 [2] https

Re: System.Collections.Generic.KeyNotFoundException: The given key was not present in the cache during load test with ignite 2.9.1

2021-02-15 Thread Pavel Tupitsyn
Hi Charlin, Most likely you should use a single *TryGet *[1] call instead of ContainsKey + Get, it achieves the same logic more efficiently and without the risk of race conditions. Looks like the key gets removed in between the ContainsKey and Get calls by another thread / process / node. [1] ht

Re: SQL Query to Value

2021-02-17 Thread Pavel Tupitsyn
Copying my reply from StackOveflow [1] here. Use `_key` and `_val` keywords to retrieve the entire key and/or value objects. IgniteCache cache = ignite.cache("Person"); SqlFieldsQuery sql = new SqlFieldsQuery( "select _val from Person where age > ?", 28); try (QueryCursor> cursor = cach

Re: very fast loading of very big table

2021-02-19 Thread Pavel Tupitsyn
> 295 million rows > 3 min 35 sec Agree with Ilya, DataStreamer should do this much faster, have you tried it? > 3.7Gb I would not call this "big" by any means today, when even the cheapest laptops have 8GB of RAM. On Fri, Feb 19, 2021 at 1:33 PM Ilya Kasnacheev wrote: > Hello! > > Is there

Re: very fast loading of very big table

2021-02-19 Thread Pavel Tupitsyn
> First of all, I tried to select the whole table as once Hmm, it looks like MSSQL data retrieval may be the bottleneck here, not Ignite. Can you run a test where some dummy data of the same size as real data is generated and inserted into Ignite, so that we test Ignite perf only, excluding MSSQL

Re: Near Cache versus Continuous Query

2021-02-19 Thread Pavel Tupitsyn
Hi, > it created a near cache on each server node There are two Near Cache modes: * Server Node Near Cache - configured once before cache start for all server nodes * Client Node Near Cache - configured dynamically on per-node basis Please make sure to use the second mode, like explained in the

Re: very fast loading of very big table

2021-02-25 Thread Pavel Tupitsyn
time for 1 node and 3.5 min time for 4 nodes. Looks like a >> linear dependency (the table and the RDBMS server were the same). >> -- >> Vladimir >> >> пт, 19 февр. 2021 г. в 19:47, Pavel Tupitsyn : >> >>> > First of all, I tried to select the whole t

Re: How do I keep the Cache alive in memory between web calls of WCF on Windows/IIS

2021-02-25 Thread Pavel Tupitsyn
Hi, Looks like the Ignite instance gets disposed or stopped. Make sure that you call Ignition.Start() only once and keep the instance alive. For a lazy initialization, use the following pattern: var ignite = Ignition.TryGetIgnite() ?? Ignition.Start() You can use IIgnite.Stopped event to debug t

Re: very fast loading of very big table

2021-02-25 Thread Pavel Tupitsyn
ps://gist.github.com/ptupitsyn/4f54230636178865fc93c97e4d419f15 > [2] https://github.com/vtchernyi/FastDataLoad > > чт, 25 февр. 2021 г. в 11:01, Pavel Tupitsyn : > >> Vladimir, >> >> Thanks for getting back to us. A full example that clarifies the >> situation will

Re: IA 2.8.1 C# Client ICacheAffinity question

2021-03-04 Thread Pavel Tupitsyn
> identify all backup partitions on a given clusternode which are backups of primary partitions on another cluster node > I am running on Node A and want to know which partitions from Node B does Node A have a backup for If I understood this correctly, the following should work: IClusterNode a, b

Re: Loading data to cache using a data streamer

2021-03-09 Thread Pavel Tupitsyn
Josh, Do you have a reproducer for this issue? Can you please attach logs from all nodes? On Sat, Mar 6, 2021 at 12:59 AM Josh Katz < josh.katz.contrac...@dodgeandcox.com> wrote: > Client node gets disconnected while trying to load data to cache using a > data streamer… > > > > How can we overco

Re: Using AsCacheQueryable in LINQ to access the cache

2021-03-10 Thread Pavel Tupitsyn
Have you configured the QueryEntities [1]? [1] https://ignite.apache.org/docs/latest/SQL/indexes#configuring-indexes-using-query-entities On Wed, Mar 10, 2021 at 2:48 AM Josh Katz < josh.katz.contrac...@dodgeandcox.com> wrote: > > > We are trying to query the cache using Ignite.LINQ NUGET pack

Re: Async operations in IA C# client appear dangeroud

2021-03-10 Thread Pavel Tupitsyn
Raymond, You are right, there is no efficient AND easy way to solve this. Basically, user code has to append ContinueWith to every async Ignite API call manually: await cache.GetAsync(1).ContinueWith( t => t.Result, CancellationToken.None,

Re: Using AsCacheQueryable in LINQ to access the cache

2021-03-10 Thread Pavel Tupitsyn
configured as the following: > > CacheConfiguration config = new > CacheConfiguration(cacheName, > > new QueryEntity(typeof(HoldingsCacheKey), > typeof(List))) > > Is it setup correctly for LINQ to work? > > > > Thanks, > &g

Re: Using AsCacheQueryable in LINQ to access the cache

2021-03-10 Thread Pavel Tupitsyn
; Dodge & Cox | 415-262-7520 > > > > > > > > *From:* Pavel Tupitsyn > *Sent:* Wednesday, March 10, 2021 9:15 AM > *To:* user > *Subject:* Re: Using AsCacheQueryable in LINQ to access the cache > > > > *This is an EXTERNAL EMAIL. Stop and think before

Re: Async operations in IA C# client appear dangeroud

2021-03-12 Thread Pavel Tupitsyn
g so far. I suspect this will have only a very minor >> performance impact. >> >> Thanks, >> Raymond. >> >> >> On Wed, Mar 10, 2021 at 10:33 PM Pavel Tupitsyn >> wrote: >> >>> Raymond, >>> >>> You are r

Re: Java Thin Client: Continuous Queries

2021-03-25 Thread Pavel Tupitsyn
Yes, Java thin client will get this functionality eventually. Most likely in the next release (2.11), which should happen some time later this year. On Thu, Mar 25, 2021 at 12:07 PM ssansoy wrote: > Hi, I saw this earlier, which seems to imply continuous queries will be > supported in the .Net c

Re: Java Thin Client: Continuous Queries

2021-03-25 Thread Pavel Tupitsyn
Ticket created: https://issues.apache.org/jira/browse/IGNITE-14402 On Thu, Mar 25, 2021 at 12:16 PM ssansoy wrote: > Thats great thanks for confirming, is there a Jira or something we can keep > an eye on to get up to date with progress etc? > > > > -- > Sent from: http://apache-ignite-users.705

Re: Unit test best practices

2021-04-01 Thread Pavel Tupitsyn
1. Yes, you can start any number of nodes in the same process for unit testing - that is what we do in the Ignite tests themselves 2. You can isolate clusters by using StaticIpFinder and CommunicationSpi with different ports new IgniteConfiguration { DiscoverySpi = new TcpDiscoverySpi {

Re: Unit test best practices

2021-04-01 Thread Pavel Tupitsyn
ignite/blob/master/modules/platforms/dotnet/Apache.Ignite.Core.Tests/TestUtils.cs#L673 On Thu, Apr 1, 2021 at 5:20 PM Pavel Tupitsyn wrote: > 1. Yes, you can start any number of nodes in the same process for unit > testing > - that is what we do in the Ignite tests themselves > > 2.

Re: Persistent Atomic types

2021-04-12 Thread Pavel Tupitsyn
> Do I need to put a simple ICache.Put() inside a transaction? No. Individual entry operations are always atomic. Note that ICache has methods like PutIfAbsent(key, val), Replace(key, old, new), which are useful for conditional atomic updates without transactions and locks. - Transactions are the

Re: Java Thin Client: Continuous Queries

2021-04-13 Thread Pavel Tupitsyn
Yes, it is in master and will be included in the next release (2.11). There is no release date yet, but I think it should happen in Q3 or Q4. On Tue, Apr 13, 2021 at 6:56 PM ssansoy wrote: > Hi, I saw the changes for this were in the master branch? Is there a rough > release date for this by any

Re: Read through Write Through Example

2021-04-14 Thread Pavel Tupitsyn
Have a look at "ADO.NET As Ignite.NET Cache Store" blog post [1] It links to an example [2]. The code is a bit old and uses SQL Server Compact, but still demonstrates the point. It also uses KeepBinaryInStore, which is optional. Let me know if you need more details. [1] https://ptupitsyn.github.

Re: Continuous query with Thin client

2021-04-19 Thread Pavel Tupitsyn
There was a mistake in the release notes, I've just fixed it. Continuous queries were added to .NET thin client for 2.10, and Java thin client gets them in 2.11. On Mon, Apr 19, 2021 at 12:00 PM Stephen Darlington < stephen.darling...@gridgain.com> wrote: > I suspect it’s the backend changes req

Re: MapReduce - How to efficiently scan through subset of the caches?

2021-04-23 Thread Pavel Tupitsyn
1. Use a separate cache as an index. E.g. for every tenant store a list of IDs for quick retrieval, then use Compute.affinityRun or Cache.invokeAll to process the subset of data 2. Use SQL with index, but enable it only for the tenantId field. Get entry IDs for a given tenant with SQL, t

Re: MapReduce - How to efficiently scan through subset of the caches?

2021-04-23 Thread Pavel Tupitsyn
> For Compute.affinityRun, I am not sure how to work with it for my scenario affinityRun and affinityCall have partition-based overloads (taking int partId). Partition-based compute is the reliable way to process all data in the cluster, even in the face of topology changes/rebalance (as opposed t

Re: C# DataStreamer Best Practices

2021-04-23 Thread Pavel Tupitsyn
Hi William, > should we use a single instance within the server process Yes, use single instance per process in production. Multiple instances per process are useful for integration tests. > any benefits in trying to share/reuse DataStreamer instance DataStreamer is thread-safe and, in most cases

Re: Designing Affinity Key for more locality

2021-04-26 Thread Pavel Tupitsyn
Hi William, Can you describe the use case and domain model in more detail? 1. AffinityKey is used to colocate some data with other data. What do you achieve with user-id being the affinity key? 2. If you'd like to put all users for a given tenant/group to the same node for efficiency, the

Re: IgniteCheckedException: Requesting mapping from grid failed for [platformId=0, typeId=-1220482121]

2021-05-04 Thread Pavel Tupitsyn
Hi William, This scenario is supported, but requires a bit of extra care: 1. Class names should match on Java and C# sides, including namespace/package Alternatively, enable simple name mapper on both sides so that namespace/package part is ignored 2. Classes should be registered in Ignite.

Re: Partition Awareness in thin clients

2021-05-10 Thread Pavel Tupitsyn
> 1. Are there any drawbacks? Increased resource usage, since multiple connections are established. Some use cases look like "connect, get a small piece of data, disconnect", and in those cases partition awareness may be suboptimal. > 2. If not, why is this not enabled by default? Mostly due to

Re: System.InvalidOperationException: 'No coercion operator is defined between types 'Apache.Ignite.Core.Impl.Binary.BinaryObjectBuilder' and 'System.DateTime'

2021-05-20 Thread Pavel Tupitsyn
Josh, can you please provide the code? Ideally, a simple reproducer I could run? On Fri, May 21, 2021 at 1:47 AM Josh Katz < josh.katz.contrac...@dodgeandcox.com> wrote: > Using .NET UnitTest to connect to the cluster and persistence enabled. > > Put works without errors. When calling Get we get

Re: IN Operator not working with sub query on different cache

2021-05-26 Thread Pavel Tupitsyn
Hi, IN operator has some limitations in Ignite, it is recommended to use joins instead [1] As I understand from your second message, with JOIN you get correct results. [1] https://apacheignite.readme.io/docs/sql-performance-and-debugging#sql-performance-and-usability-considerations On Tue, May

Re: Failed to parse query. Column "...._KEY" not found;SQL statement:

2021-05-26 Thread Pavel Tupitsyn
1. What if you do "select * from CommonConstruction" - does it work? 2. Can you try listing specific columns instead of "*"? On Wed, May 26, 2021 at 6:24 AM JP wrote: > Hi, > I am with Apache Ignite V2.9.1.And using .net thin client to perform query > on cache. > And using cache query as SqlQuer

Re: Failed to parse query. Column "...._KEY" not found;SQL statement:

2021-05-26 Thread Pavel Tupitsyn
Now I see the problem - you use SqlQuery, which is deprecated (obsolete) and has some limitations. Please use SqlFieldsQuery and list specific columns. Alternatively, use Ignite LINQ provider [1] to perform strongly-typed SQL queries, for example: CommonConstruction.AsCacheQueryable() .Join(P

Re: Failed to parse query. Column "...._KEY" not found;SQL statement:

2021-05-26 Thread Pavel Tupitsyn
"_key" and "_val" are predefined column names returning full key and value objects: select _key, _val from CommonConstruction On Wed, May 26, 2021 at 12:52 PM JP wrote: > Hi, > Thanks for reply.Appriciated. > Please use SqlFieldsQuery and list specific columns. > Actually i m getting values

Thin client data structures

2021-07-16 Thread Pavel Tupitsyn
Igniters, I was asked a few times about Data Structures [1] in thin clients. While there are plans to add them eventually, it is possible to implement them on top of the existing thin client cache API. I'd like to share a short blog post which demonstrates this [2]. [1] https://ignite.apache.org

Re: Thin client data structures

2021-07-16 Thread Pavel Tupitsyn
And I'm just trying to demonstrate what is possible :) On Fri, Jul 16, 2021 at 3:20 PM Ivan Daschinsky wrote: > Hi! Pavel, it is fun stuff. But is there any real use case for this > feature? I'd rather use RabbitMQ for this :) > > пт, 16 июл. 2021 г. в 13:28, Pavel Tupitsyn :

Re: Thin client data structures

2021-07-16 Thread Pavel Tupitsyn
tore and events, you definitely can > implement queue, distributed lock and others primitives :) I'm not against > it, let it be. > > > > пт, 16 июл. 2021 г., 16:18 Pavel Tupitsyn : > > Ivan, > > > > When Ignite is already in use by the app (for caching, SQL,

  1   2   3   4   5   6   7   >