Ignite Cluster: Cache Misses happening for existing keys

2018-09-12 Thread HEWA WIDANA GAMAGE, SUBASH
Hi all, We're observing this in a 3 node server cluster(3 separate JVMs, in 3 separate VMs within same datacenter, node to node latency is 2-3 milliseconds within the network). As with following code is wrapped inside an http API, and that API is getting called by a 5000 users ramping

Re: how to configure apache ignite as cache api and as spring cache provider

2018-09-12 Thread vkulichenko
Ignition.start is supposed to start an Ignite instance, so passing spring-cache.xml file that doesn't contain any Ignite configuration doesn't make sense. The SpringCacheManager bean should be part of the Spring Application Context, it then will be used as an entry point to Ignite cluster. It

Re: Failed to wait for initial partition map exchange

2018-09-12 Thread ndipiazza3565
No. Persistence is disabled in my case. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Failed to wait for initial partition map exchange

2018-09-12 Thread eugene miretsky
Do you have persistence enabled? On Wed, Sep 12, 2018 at 6:31 PM ndipiazza3565 < nicholas.dipia...@lucidworks.com> wrote: > I'm trying to build up a list of possible causes for this issue. > > I'm only really interested in the issues that occur after successful > production deployments. Meaning

Re: Failed to wait for initial partition map exchange

2018-09-12 Thread ndipiazza3565
I'm trying to build up a list of possible causes for this issue. I'm only really interested in the issues that occur after successful production deployments. Meaning the environment has been up for some time successfully, but then later on our ignite nodes will not start and stick But as of

Re: Query 3x slower with index

2018-09-12 Thread eugene miretsky
Thanks! Tried joining with an inlined table instead of IN as per the second suggestion, and it didn't quite work. Query1: - Select COUNT(*) FROM( Select customer_id from GATABLE3 use Index( ) where category_id in (9005, 175930, 175930, 175940,175945,101450, 6453) group by customer_id

Re: How much heap to allocate

2018-09-12 Thread eugene miretsky
Thanks! For #2: wouldn't H2 need to bring the data into the heap to make the queries? Or at least some of the date to do the group_by and sum operation? On Mon, Sep 10, 2018 at 6:19 AM Vladimir Ozerov wrote: > Hi Eugene, > > Answering your questions: > 1) Grouping is performed on both mapper

how to configure apache ignite as cache api and as spring cache provider

2018-09-12 Thread mshah
I want to use ignite as cache api as well as spring cache provider. I am running following configuration in ignite-config.xml file as below http://www.springframework.org/schema/beans; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;

Re: Partition map exchange in detail

2018-09-12 Thread Вячеслав Коптилин
Hello Eugene, I hope you meant PME (partitions map exchange) instead of NPE :) > What constitutes a transaction in this context? If I am not mistaken, it is about Ignite transactions. Please take a look at this page https://apacheignite.readme.io/docs/transactions > Does it mean that if the

IGNITE-8386 question (composite pKeys)

2018-09-12 Thread eugene miretsky
Hi, A question regarding https://issues.apache.org/jira/browse/IGNITE-8386?focusedCommentId=16511394=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16511394 It states that a pkey index with a compoise pKey is "effectively useless". Could you please explain why is

Re: POJO field having wrapper type, mapped to cassandra table are getting initialized to respective default value of primitive type instead of null if column value is null.

2018-09-12 Thread Denis Magda
Igor R, Could you please review this C* contribution? -- Denis On Wed, Sep 12, 2018 at 6:33 AM Dmitriy Pavlov wrote: > Hi Igniters, > > I can see that ticket is still in patch available state. > > Denis M. > > could you please review the patch? > > Sincerely, > Dmitriy Pavlov > > вт, 26 сент.

Re: Partition map exchange in detail

2018-09-12 Thread eugene miretsky
Make sense I think the actual issue that was affecting me is https://issues.apache.org/jira/browse/IGNITE-9562. (which IEP-25 should solve). Final 2 questions: 1) If all NPE waits for all pending transactions a) What constitutes a transaction in this context? (any query, a SQL transaction, etc)

Re: Unable to connect ignite pods in Kubernetes using Ip-finder

2018-09-12 Thread Denis Magda
Hi Rishikesh, Thanks for pointing out to this. I've updated the stateless deployment doc. -- Denis On Tue, Sep 11, 2018 at 7:06 AM rishi007bansod wrote: > "serviceAccountName: ignite" should be present in Pod Deployment > specification as mentioned by Anton in post > >

Re: How to throttle/limit the cache-store read threads?

2018-09-12 Thread Denis Magda
System thread pool is used for cache and cache store operation. However, I would discourage you from limiting it to 1 thread. In general, extend the cache store implementation you use and forward all CacheStore.read() operations to your single-threaded pool. -- Denis On Sun, Sep 9, 2018 at

Re: Node keeps crashing under load

2018-09-12 Thread eugene miretsky
Good question :) yardstick does this, but not sure if it is a valid prod solution. https://github.com/apache/ignite/blob/3307a8b26ccb5f0bb7e9c387c73fd221b98ab668/modules/yardstick/src/main/java/org/apache/ignite/yardstick/jdbc/AbstractJdbcBenchmark.java We have set preferIPv4Stack=true and

Re: Partition map exchange in detail

2018-09-12 Thread Pavel Kovalenko
Eugene, In the case of Zookeeper Discovery is enabled and communication problem between some nodes, a subset of problem nodes will be automatically killed to reach cluster state where each node can communicate with each other without problems. So, you're absolutely right, dead nodes will be

Re: a node fails and restarts in a cluster

2018-09-12 Thread Pavel Kovalenko
Hi Eugene, I've reproduced your problem and filed a ticket for that: https://issues.apache.org/jira/browse/IGNITE-9562 As a temporary workaround, I can suggest you delete persistence data (cache.dat and partition files) related to that cache in starting node work directory or don't destroy

Re: Partition map exchange in detail

2018-09-12 Thread eugene miretsky
Hi Pavel, The issue we are discussing is PME failing because one node cannot communicate to another node, that's what IEP-25 is trying to solve. But in that case (where one node is either down, or there is a communication problem between two nodes) I would expect the split brain resolver to kick

Re: Partition map exchange in detail

2018-09-12 Thread Pavel Kovalenko
Hi Eugene, Sorry, but I didn't catch the meaning of your question about Zookeeper Discovery. Could you please re-phrase it? ср, 12 сент. 2018 г. в 17:54, Ilya Lantukh : > Pavel K., can you please answer about Zookeeper discovery? > > On Wed, Sep 12, 2018 at 5:49 PM, eugene miretsky < >

Re: Node keeps crashing under load

2018-09-12 Thread Ilya Kasnacheev
Hello! How would you distinguish the wrong interface (172.17.0.1) from the right one if you were Ignite? I think it's not the first time I have seen this problem but I have positively no idea how to tackle it. Maybe Docker experts could chime in? Regards, -- Ilya Kasnacheev ср, 12 сент. 2018

Re: Partition map exchange in detail

2018-09-12 Thread Ilya Lantukh
Pavel K., can you please answer about Zookeeper discovery? On Wed, Sep 12, 2018 at 5:49 PM, eugene miretsky wrote: > Thanks for the patience with my questions - just trying to understand the > system better. > > 3) I was referring to https://apacheignite.readme.io/docs/ >

Re: Partition map exchange in detail

2018-09-12 Thread eugene miretsky
Thanks for the patience with my questions - just trying to understand the system better. 3) I was referring to https://apacheignite.readme.io/docs/zookeeper-discovery#section-failures-and-split-brain-handling. How come it doesn't get the node to shut down? 4) Are there any docs/JIRAs that explain

Does ignite supported calculated columns ?

2018-09-12 Thread wengyao04
Hi, in our cache we have a column which is calculated by other columns, for example column-C is calculated by column-A / column-B When there is an update from column-A or column B, can ignite handle the calculation of column-C ? Thanks -- Sent from:

Re: Partition map exchange in detail

2018-09-12 Thread Ilya Lantukh
3) Such mechanics will be implemented in IEP-25 (linked above). 4) Partition map states include update counters, which are incremented on every cache update and play important role in new state calculation. So, technically, every cache operation can lead to partition map change, and for obvious

Re: ignite multi thread stop bug

2018-09-12 Thread ezhuravlev
Hi, it's a really bad idea to do cache operations in Discovery thread. If you want to add something to the cache, you can start a new thread and do this operation inside the new thread. Evgenii -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Data migration using persistent Queues

2018-09-12 Thread ezhuravlev
Sorry, here is the link for the BinaryObject doc: https://apacheignite.readme.io/docs/binary-marshaller -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Data migration using persistent Queues

2018-09-12 Thread ezhuravlev
Hi, Do you mean Ignite Queue? Or just your data structure that stored in simple Ignite cache? > Store data in another format let's say JSON and provide backwards > compatibility at code level. There is no need for this, Ignite internally stores everything as BinaryObject and this format supports

Re: cache doesn't exist

2018-09-12 Thread Igor Sapego
Hi, Did your thick .net client connected to the cluster? It should be visible in server node's logs. Best Regards, Igor On Wed, Sep 12, 2018 at 1:58 PM Som Som <2av10...@gmail.com> wrote: > the cache names list is empty then i call the method of non thin client > butmy TestCache is stil

Re: cache doesn't exist

2018-09-12 Thread Som Som
the cache names list is empty then i call the method of non thin client butmy TestCache is stil exists and i can see it under the thin client. i cant see the cache from NON thin client if it is created from thin client. ср, 12 сент. 2018 г., 13:29 Evgenii Zhuravlev : > Hi, > > You can always get

Re: POJO field having wrapper type, mapped to cassandra table are getting initialized to respective default value of primitive type instead of null if column value is null.

2018-09-12 Thread Dmitriy Pavlov
Hi Igniters, I can see that ticket is still in patch available state. Denis M. could you please review the patch? Sincerely, Dmitriy Pavlov вт, 26 сент. 2017 г. в 12:10, Denis Mekhanikov : > There is a page in confluence with description of the process: >

Re: cache doesn't exist

2018-09-12 Thread Evgenii Zhuravlev
Hi, You can always get the list of the all caches in cluster using Ignite.cacheNames() method. Evgenii ср, 12 сент. 2018 г. в 13:26, Som Som <2av10...@gmail.com>: > Hello > > > > I created new cache using thin .net client: > > > > public class TestEntity > > { > > public string

cache doesn't exist

2018-09-12 Thread Som Som
Hello I created new cache using thin .net client: public class TestEntity { public string ValueString { get; set; } public DateTime ValueDateTime { get; set; } } class Program { static void Main(string[] args) { var

Re: Looking for information on these methods

2018-09-12 Thread wt
hows it going with this Luqman I am still exploring an approach to single signon using kerberos without needing config files. I think i have made progress but i need to register a service principal name first. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: IgniteInterruptedException: Got interrupted while waiting for future to complete

2018-09-12 Thread Maxim.Pudov
Hello. Do you really need to add all this data in a single transaction? That's not a typical use case. Do you have any other operations running in parallel? Unfortunately, having this stack trace is not enough to help. Could you provide a full ignite log and your configuration as well? -- Sent

Re: The system cache size was slowly increased

2018-09-12 Thread Justin Ji
Evgenii - Thank for your reply! "system memory cache" means the result printed by 'free -m' command. After some test, I solved the problem, it caused by my Linux setting, not Ignite. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Partition map exchange in detail

2018-09-12 Thread eugene miretsky
2b) I had a few situations where the cluster went into a state where PME constantly failed, and could never recover. I think the root cause was that a transaction got stuck and didn't timeout/rollback. I will try to reproduce it again and get back to you 3) If a node is down, I would expect it to