Re: SQL Performance indexing performance on-heap vs off-heap

Igniters, As I understand "javadevmtl" prepared some code. May be someone experienced could run it under java profiler and may be it will give some useful information where is the slowdown? -- Alexey Kuznetsov GridGain Systems www.gridgain.com

Re: SQL Performance indexing performance on-heap vs off-heap

Sergi, Can you please clarify what reasons for performance degradation you meant? Does it worth additional investigation or this is a known issue? -Val On Wed, Sep 16, 2015 at 7:42 PM, Dmitriy Setrakyan wrote: > > > On Thu, Sep 17, 2015 at 2:00 AM, Sergi Vladykin > wrote: > >> Ok, I see. Yes,

Re: Btw anyone heard of CQ engine?

On Thu, Sep 17, 2015 at 2:58 AM, Alexey Kuznetsov wrote: > If it is not snapsohtable there could be at least one use-case: populated > once and never changed cache. > For example for some analytics. > User populate 100500 entries into cache, build index and run 100500 > queries. > > I agree with

Re: SQL Performance indexing performance on-heap vs off-heap

On Thu, Sep 17, 2015 at 2:00 AM, Sergi Vladykin wrote: > Ok, I see. Yes, I can name at least few things in off-heap > sql indexes that make things slower and less scalable > than on-heap ones, but I'm not sure when we will be > able to improve them. > I actually do not see why the performance is

Re: Btw anyone heard of CQ engine?

The docs do mention quite profusely O(1) he has a bit of info regarding performance of the various index types it supports it also support concurrency. BUT it is dn fast :) -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Btw-anyone-heard-of-CQ-engine-tp1421p1434.

Re: Btw anyone heard of CQ engine?

Alexey, patches are welcome!;) Sergi On Sep 17, 2015 2:58 AM, "Alexey Kuznetsov" wrote: > If it is not snapsohtable there could be at least one use-case: populated > once and never changed cache. > For example for some analytics. > User populate 100500 entries into cache, build index and run 100

Re: CacheStore implementation using Hibernate

Kevin, did you look at org.apache.ignite.examples.datagrid.store.hibernate.CacheHibernatePersonStore? May be it will help you to grab som ideas for your implementation. On Thu, Sep 17, 2015 at 3:25 AM, kevin wrote: > I'm trying to implement a CacheStore interface using Hibernate. > Ie. CacheSto

Re: Btw anyone heard of CQ engine?

If it is not snapsohtable there could be at least one use-case: populated once and never changed cache. For example for some analytics. User populate 100500 entries into cache, build index and run 100500 queries. On Thu, Sep 17, 2015 at 6:27 AM, Sergi Vladykin wrote: > Thanks for the pointers! >

Re: OverlappingFileLockException in nodes startup same time in Linux

Another option is to put Ignite in some common module for all the applications that require Ignite. This way Ignite classes must be loaded only once. See https://docs.jboss.org/author/display/WFLY9/Class+Loading+in+WildFly Sergi 2015-09-17 2:27 GMT+03:00 vkulichenko : > roar109 wrote > > If I un

Re: How to implement "limit 0 to 10" by Ignite SQL?

javadevmtl wrote > Val are you talking about me? I thought I was. Unless my mobile phone did > something funny. I have been posting for a while... No, this was for Mike. Sorry for the confusion :) -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-impl

Re: OverlappingFileLockException in nodes startup same time in Linux

roar109 wrote > If I understand your suggestion correctly is to make single Ignite > instance per jboss in this case to avoid this sync mechanism? This should help too. But it sounds like that in your case you can simply disable shared memory communication. To do this add the following in your con

Re: Btw anyone heard of CQ engine?

Thanks for the pointers! The main difficulty at Ignite side is that we need index data structure to be snapshotable with O(1) complexity and of course without copying of the whole thing, so I'm not sure if we will be able to use something from there as is. But nevertheless CQEngine at least deserve

Re: SQL Performance indexing performance on-heap vs off-heap

Ok, I see. Yes, I can name at least few things in off-heap sql indexes that make things slower and less scalable than on-heap ones, but I'm not sure when we will be able to improve them. Sergi 2015-09-16 21:35 GMT+03:00 javadevmtl : > Sergy, point taken, but what i'm trying to point out is the f

Re: Btw anyone heard of CQ engine?

javadevmtl wrote > Yeah it's perforamance is crazy. I hadn't experienced any "buggy" issues > when i tried. it also has off-heap otion now also. But I find the concept > of O(1) indexes very interesting... I think that O(1) is achievable only for hash indexes and therefore can be used only for equ

Re: Btw anyone heard of CQ engine?

Yeah it's perforamance is crazy. I hadn't experienced any "buggy" issues when i tried. it also has off-heap otion now also. But I find the concept of O(1) indexes very interesting... -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Btw-anyone-heard-of-CQ-engine-tp

CacheStore implementation using Hibernate

I'm trying to implement a CacheStore interface using Hibernate. Ie. CacheStore.load() and CacheStore.write() is implemented by calling Hibernate A problem I am seeing is that there are some hooks in which fields get set after an object is written to the datagrid cache (such as the "locking" number

Re: Btw anyone heard of CQ engine?

Tried to use it a couple of years ago as a local (dev-only) alternative to my data grid based framework: just plugging it instead of the Hazelcast or Ignite. Superfast, but extremely buggy. I suspect not thread-safe. Haven't followed it since then. It'd be great if data grids used something like th

Btw anyone heard of CQ engine?

It's O(1) index performance. Was wondering if it's something that can be pluged in to ignite to replace H2 indexes :) https://code.google.com/p/cqengine/#Applications -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Btw-anyone-heard-of-CQ-engine-tp1421.html Se

Re: OverlappingFileLockException in nodes startup same time in Linux

If I understand your suggestion correctly is to make single Ignite instance per jboss in this case to avoid this sync mechanism? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/OverlappingFileLockException-in-nodes-startup-same-time-in-Linux-tp1415p1420.html Sent

Re: SQL Performance indexing performance on-heap vs off-heap

Sergy, point taken, but what i'm trying to point out is the following... In on-heap mode I can easily insert and query 0-15,000,000 without any performance loss. And if anything even more then 15 million entries in the cache. The off-heap cache though instantly within a few thousand entries it's

Re: SQL Performance indexing performance on-heap vs off-heap

Regardless of whether it is off-heap or on-heap, performance of indexes *will* degrade with cache growth and this is unavoidable. For better understanding: cache is basically a hash map with O(1) operation complexity, SQL index is basically an AVL tree which has O(log(N)) operation complexity. So

Re: OverlappingFileLockException in nodes startup same time in Linux

It seems that jboss starts two instances of Ignite in the same jvm in different classloaders in parallel. This way synchronization on IpcSharedMemoryNativeLoader.class does not help much. The mutex must be changed to some JVM-wide singleton object or class instead of Ignite class. It must be easy

Re: SQL Performance indexing performance on-heap vs off-heap

Sorry Dimitry, switching to primitives definitely helped me save on heap usage, but not performance. The issue I reported with off-heap is still true, it's alot slower then on-heap and gets slower over time as the cache grows... -- View this message in context: http://apache-ignite-users.70518.

OverlappingFileLockException in nodes startup same time in Linux

I'm facing an issue with my box when I try to run 2 clients (diferent wars) in jboss eap 6.3 in Linux. Details: - 1 Jboss as server node - 1 Jboss with 2 wars, each war is a ignite as a client connecting to the server node - Ignite 1.3.0 - Jboss EAP 6.3 - Linux CentOS (both) Steps to reproduce:

Re: Is it me or primitives as properties are alot faster?

So yeah, complete puts I am! Basically I had switched my models' class name and forgot to do so for the query so it wasn't serializing back, which made it look like it was allot faster. Primitive will definitely save you heap space usage. But won't increase performance. -- View this message

Re: Is it me or primitives as properties are alot faster?

Humm maybe not, primitives might not help... More testing... i forgot to switch my model when setting the config. :( So nothing was being returned... That why it was faster. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Is-it-me-or-primitives-as-properties-are-

Re: SQL Performance indexing performance on-heap vs off-heap

On Wed, Sep 16, 2015 at 6:02 PM, javadevmtl wrote: > I think I know what it is... > > I was using object types instead of primitive types. I switched all > possible > properties of my model from Object types to primitives and off-heap seems > to > be performing as you said about 20% slower then o

Re: SQL Performance indexing performance on-heap vs off-heap

Actually might be wrong primitives might not help... -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/SQL-Performance-indexing-performance-on-heap-vs-off-heap-tp1352p1411.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: How to implement "limit 0 to 10" by Ignite SQL?

Val are you talking about me? I thought I was. Unless my mobile phone did something funny. I have been posting for a while... -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-implement-limit-0-to-10-by-Ignite-SQL-tp1392p1410.html Sent from the Apache Ignite

Re: SQL Performance indexing performance on-heap vs off-heap

I think I know what it is... I was using object types instead of primitive types. I switched all possible properties of my model from Object types to primitives and off-heap seems to be performing as you said about 20% slower then on heap, but there no performance decrease so far... Let me know i

Re: Is it me or primitives as properties are alot faster?

Ok so here is more details... Running 1.3.3 on single node single server with 32 cores I had a MyModel as... Switched to... 1- My SQL queries # of operations jumped from 18,000/sec to 54,000/sec. 2- Total cache size for 15 Million records went from 20GB on-heap usage to just about 6.5GB on-

Re: Using Jackson as JSON processor?

please use this as starting point - https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute --Yakov 2015-09-16 17:29 GMT+03:00 endian675 : > I'd be more than happy to contribute. Is there an existing example of this > kind of pattern in the Ignite code base? I am still getting to

Re: Using Jackson as JSON processor?

I'd be more than happy to contribute. Is there an existing example of this kind of pattern in the Ignite code base? I am still getting to grips with Java (not to mention Git, Ignite, et. al...) so having an example to guide me is likely to yield a better result. -- View this message in context

Re: Using Jackson as JSON processor?

It seems this is not possible now. However, I suppose that we can easily refactor GridJettyRestHandler to allow custom JSON serializers - just introduce pluggable interface and set current impl as default. Can you help us with this and contribute to Ignite? --Yakov 2015-09-16 16:03 GMT+03:00 endi

Is it me or primitives as properties are alot faster?

So I had my model with Long and Integer primitives and I switched them to long and int and I got 200% increase in performance... with SQL indexing. Still using on-heap though. Is this true? If it's the case that should be documented :P -- View this message in context: http://apache-ignite-user

Re: Using Jackson as JSON processor?

What I am actually trying to achieve is to achieve the @JsonAnySetter functionality from Jackson within Ignite, either by using Jackson directly or some other method. >From some searching it seems that it might be possible to use a custom PropertySetStrategy inside json-lib in order to get the s

Using Jackson as JSON processor?

Is it possible to use Jackson as the JSON processor rather than net.sf.json-lib? If so, how can I configure that? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Using-Jackson-as-JSON-processor-tp1402.html Sent from the Apache Ignite Users mailing list archive a