Re: IgniteOutOfMemoryException in LOCAL cache mode with persistence enabled

2019-12-11 Thread Sergey Chugunov
Hi Mitchell, I believe that research done by Anton is correct, and the root cause of the OOME is proportion of memory occupied by metapages in data region. Each cache started in data region allocates one or more metapages per initialized partition so when you run your test with only one cache

Re: How to reload a cache being queried with minimum downtime

2019-12-11 Thread Mikael
Hi! Any chance you could do it at the "client" end ? telling the client(s) to use another cache instead of the alias function, I guess that all depends on how you query the data. So create another cache, load it with the data and tell "client" to use the other cache instead and then delete

Re: PutAll Behavior Single vs Multiple Servers

2019-12-11 Thread Victor
I am aware of those nuances around distributed systems. What i am trying to understand is, with Sync mode as PRIMARY_SYNC, where the response does not wait for updates to backups, so long as primary node is updated. With this setting why should 1 node vs 3 node matter? -- Sent from:

How to reload a cache being queried with minimum downtime

2019-12-11 Thread Renato Melo
Hello, Periodically I need to reload a cache that is being massive queried by a stream application. In my current solution every reload I drop the cache, recreate it and reload data. Caches are persisted in the file system. I am looking for alternatives to minimize downtime when reloading

Re: Topology version changing very frequently

2019-12-11 Thread Denis Magda
Check the logs of all your servers nodes and client applications, you should see why the nodes stop and start. The topology changes only if a node leaves the cluster or joins it. The failureDetectionTimeout is used by default for the network communication timeouts between the nodes:

Re: Adhoc Reporting use case (Ignite+Spark?)

2019-12-11 Thread Denis Magda
Ad hoc is not a strong part of Ignite. If you have complex analytical queries than prepare to optimize them. - Denis On Wed, Dec 11, 2019 at 5:50 AM sri hari kali charan Tummala < kali.tumm...@gmail.com> wrote: > Check these examples. > > >

Re: IgniteOutOfMemoryException in LOCAL cache mode with persistence enabled

2019-12-11 Thread Denis Magda
I tend to agree with Mitchell that the cluster should not crash. If the crash is unavoidable based on the current architecture then a message should be more descriptive. Ignite persistence experts, could you please join the conversation and shed more light to the reported behavior? - Denis On

Re: Detect baseline in TopologyValidator

2019-12-11 Thread akurbanov
Hi, I would suggest going through TopologyValidator implementations in test and check out something like: https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/IgniteTopologyValidatorGridSplitCacheTest.java If you want to get the

Are data in NearCache BinaryObject?

2019-12-11 Thread Cong Guo
Hi, Are the entries stored in local NearCache on my client node in the format of deserialized java objects or BinaryObject? Will the entry in local on-heap NearCache be deserialized from BinaryObject when I call the get function? Thanks, Nap

RE: H2 version security concern

2019-12-11 Thread Scott Cote
YAY Scott Cote Senior Application Developer - Java | Electronic Transaction Consultants Corporation (ETC) 1600 N. Collins Boulevard, Suite 4000, Richardson, TX 75080 (o) 469.248.4576 | (c) 972.900.1561 [cid:image005.png@01D5B018.1D9FC010]

Re: Deadlock on concurrent calls to getAll and invokeAll on cache with read-through

2019-12-11 Thread Ilya Kasnacheev
Hello! It seems to me the issue is fixed in our master branch and the upcoming 2.8, so maybe you should just wait for the nearest available RC and use that if this scenario is important for you. I did not bisect for a fixing commit but you can do that if you like. Regards, -- Ilya Kasnacheev

Detect baseline in TopologyValidator

2019-12-11 Thread Andrey Davydov
Hello all, If there is any way to understand how many nodes from baseline topology are active in TopologyValidator.validate() method? Thanks.

Re: IgniteOutOfMemoryException in LOCAL cache mode with persistence enabled

2019-12-11 Thread Ilya Kasnacheev
Hello! putAll has to lock until all entries are written, and they can't all fit in the cache at the same time, and checkpoint can't free any memory since it can't start while putAll is running. DataStreamer does not have that limitation apparently. Regards, -- Ilya Kasnacheev ср, 11 дек.

Re: Understanding Write/Read IOPS during Ignite data load through Streamer and JDBC SQL

2019-12-11 Thread Ilya Kasnacheev
Hello! Do you have any "Indexed columns of a row cannot be fully inlined" messages in your logs? Otherwise, I'm afraid answer is no, mostly. Either increase off-heap size or live with reduced performance. Regards, -- Ilya Kasnacheev ср, 11 дек. 2019 г. в 06:57, krkumar24061...@gmail.com <

PeerClassLoader and deployment mode

2019-12-11 Thread Nicolás Azrak
I’m trying to deploy a new application in which we use Ignite as compute grid and send them broadcast messages from different client nodes. The problem is that we are storing a helper class in the workers/servers localMap while using SHARED deployment mode with the peer class loader. According to

Re: PutAll Behavior Single vs Multiple Servers

2019-12-11 Thread Stanislav Lukyanov
This is a very common pitfall with distributed systems - comparing 1 node vs 3 nodes. In short, this is not correct to compare them. When you write to one node each write does the following: 1) client sends the request to the server 2) server updates data 3) server sends the response to the

Re: Local node terminated after segmentation

2019-12-11 Thread Stanislav Lukyanov
In Ignite a node can go into "segmented" state in two cases really: 1. A node was unavailable (sleeping. hanging in full GC, etc) for a long time 2. Cluster detected a possible split-brain situation and marked the node as "segmented". Yes, split-brain protection (in GridGain implementation and in

Re: number of way segments in wal

2019-12-11 Thread akurbanov
Hi, According to mentioned ticket (https://issues.apache.org/jira/browse/IGNITE-10840), this is a critical part of reproducer. Looks like this WAL archive disabling option works for clean WAL currently. I would recommend doing following things: 1. Backup the whole work directory. 2. Remove

Re: Duplicate column name while creating table

2019-12-11 Thread Ilya Kasnacheev
Hello! I have filed an issue https://issues.apache.org/jira/browse/IGNITE-12433 Regards, -- Ilya Kasnacheev ср, 11 дек. 2019 г. в 12:43, Denis Magda : > It sounds like the implementation specificities of Ignite DML. > > SQL folks, how about throwing an exception in case of the duplicate

Re: Geometry via SQL?

2019-12-11 Thread Andrey Mashenkov
Hi, Please, refer to a Geo spatial index documentation [1] It is quite old, but should works. [1] https://apacheignite.readme.io/v1.7/docs/geospatial-queries#section-executing-geospatial-queries On Tue, Dec 10, 2019 at 10:57 PM richard.ows...@viasat.com < richard.ows...@viasat.com> wrote: > Is

Re: Duplicate column name while creating table

2019-12-11 Thread Denis Magda
It sounds like the implementation specificities of Ignite DML. SQL folks, how about throwing an exception in case of the duplicate name? - Denis On Thu, Dec 5, 2019 at 9:38 AM DS wrote: > *I am able to create the table with duplicate column name.* > > I was expecting some error saying

Re: H2 version security concern

2019-12-11 Thread Andrey Mashenkov
Hi, Mentioned CVE has no affect Ignite. Please, see discussion on dev-list. http://apache-ignite-developers.2346864.n4.nabble.com/H2-license-and-vulnerabilities-td40417.html#a40418 On Wed, Dec 11, 2019 at 2:22 AM Evgenii Zhuravlev wrote: > Hi, > > There are plans to replace H2 with Calcite.

Re: Improving Get operation performance

2019-12-11 Thread Denis Magda
Good to hear that you got to the root cause, Viktor! Do you have any suggestions for extra performance/troubleshooting tips/tricks that you had to learn hard way as long as the information was not documented? - Denis On Fri, Dec 6, 2019 at 4:11 AM Victor wrote: > Update, > > 1. So there were