Re: Ignite with Spring Cache on K8S, eviction problem

2018-02-19 Thread lukaszbyjos
I have something like in this gist. I cant share everything but in get there is just request to DB. Evict is the same and I need to have this in the separate method because sometimes I need to evict for list of ids. I know that I can't call this method directly from the same class because of how

Re: Enable Kerberos on Ignite Kafka streamer / support for new consumer config

2018-02-19 Thread jackson.dickson
Thanks Andrew. The new consumer config enable Kafka clients to connect directly to the Kafka brokers. kafka-clients 0.10.x support both style. I believe you would need another ConsumerConfig type like ZKConfig. -Jackson -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Unable to identify root cause for node failure

2018-02-19 Thread Rahul Pandey
Hi, Yes, those logs are full. Also found that root cause was O.S killing the process, the two nodes were given 48gb each heap size, on reducing it to 32gb O.S did not kill the process, but getting OutOfMemoryError while data loading. Since Ignite Persistance is turned on doesn't data should be

Node failed to join cluster with error related to : Memory configuration mismatch (fix configuration or set -DIGNITE_SKIP_CONFIGURATION_CONSISTENCY_CHECK=true system property) [rmtNodeId=47e76683-39ec

2018-02-19 Thread aMark
Hi, Before I go into error, here is our production configuration : We are using Ignite as persistent store, in Partitioned Mode having 6 data node, each running on different server. Atomicity mode is ATOMIC, and Rebalance mode is ASYNC while CacheWriteSynchronizationMode is FULL_SYNC. We got

Re: Large durable caches

2018-02-19 Thread Ivan Rakov
Hello, Seems like checkpoint-runner thread is idle. There's no active checkpoint at the moment of dump, and thus there's no work for checkpoint thread. If the problem persists, please share full thread dump. It would be better to take 2-3 consecutive thread dumps when your data streamer

Snapshot backup of Ignite native persistance data

2018-02-19 Thread arunkjn
Hi, I have an ignite cluster which has some durable caches. These caches are using amazon EBS volumes. I want to backup each EBS node daily for disaster recovery since we cannot afford losing the data and want disaster recovery capability. I was looking around to see if ignite offers a native

Re: Stale class on ignite data node

2018-02-19 Thread Alexey Kukushkin
Right now the only way to avoid re-deploying entity classes when they change is using Ignite Binary Object API to work with entities.

Ignite with mariadb

2018-02-19 Thread Sumanth Sumanth
Hi, Can Ignite be used with Mariadb? When attempted (via jdbc url: jdbc:mariadb://localhost:3306/db1?user=root=password) Ignite throws exception that it is not able to find suitable jdbc driver for the Mariadb. What is the recommended way of integrating Mariadb with Ignite? Thanks, Sumanth

Re: Ignite with mariadb

2018-02-19 Thread Alexey Kukushkin
Do you want to use mariadb as Ignite's cache store (3-rd party persistence)? You can use whatever you want if you implement your custom cache store. You might also try using out-of-the-box CacheJdbcPojoStore. It already supports DB2, Oracle, SQK Server, H2 and MySQL. It looks like mariadb should

Re: Rebalancing in Ignite (Cache loader are executing instead of just rebalancing)

2018-02-19 Thread Alexey Kukushkin
I am not sure if it helps - please note that: - loadCache() shall be called once. Call it on a client node and not on the server nodes. What loadCache() does it just broadcasts request to invoke CacheStore's loadCache() on every server node. - loadCache() is optional - you use it if

Re: Snapshot backup of Ignite native persistance data

2018-02-19 Thread Ivan Rakov
Hi! If it's possible to turn off your system for a while on a daily basis, you can stop the grid and move your Ignite Native Persistence files manually. This can be just one script run for the whole cluster if you'll use tools like Ansible. Here's step-by-step guide: 1) Turn off your Ignite

Stale class on ignite data node

2018-02-19 Thread arunkjn
Hi Guys, I have an application in which I segregate ignite nodes into data and service nodes. The data node is responsible for hosting cache and queues. The service nodes host an ignite service which uses cache defined in data nodes. These nodes are separated because during my application

Re: Rebalancing in Ignite (Cache loader are executing instead of just rebalancing)

2018-02-19 Thread Prasad Bhalerao
Hi Alex, How do you load the multiple caches in prod env? Do you always call the loadCache method from client node. What if I do not have client server mode configuration? I just have server nodes. How does one load the cache in this scenario? Is it mandatory to use client-server node

Re: Enable Kerberos on Ignite Kafka streamer / support for new consumer config

2018-02-19 Thread jackson.dickson
Hi Andrew, Yes, "bootstrap.server" option is not working for me. -Jackson -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Rebalancing in Ignite (Cache loader are executing instead of just rebalancing)

2018-02-19 Thread Alexey Kukushkin
Prasad, I am not aware about anything like what you described in Ignite (may be Community will help). To me Ignite servers work as a cluster and having a server doing something that other servers do not breaks the cluster definition . I would really

Re: Ignite with Spring Cache on K8S, eviction problem

2018-02-19 Thread lukaszbyjos
I still have a problem with evicting one cache. What's recommended setting for partitioned cache? >From service B I want to evict "ch-current" cache but service A still see old values. At the same time just after evicting this cache the other one is evicted and looks like it's working. Cache

Re: And again... Failed to get page IO instance (page content is corrupted)

2018-02-19 Thread Mikhail
Hi Sergey, The release of 2.4 should be soon, in a week or couple, however, there's no strong schedule for Apache releases. Could you please share a reproducer for the issue? Might be you can share a storage on which the issue can be reproduced? Thanks, Mike. -- Sent from:

Re: QueryEntities in Ingnite

2018-02-19 Thread Mikhail
Hi, there's two way to index objects in cache: 1. by annotations: https://apacheignite.readme.io/docs/cache-queries#section-query-configuration-by-annotations 2. by QueryEntity: https://apacheignite.readme.io/docs/cache-queries#section-query-configuration-using-queryentity But you can't mix

Re: Enable Kerberos on Ignite Kafka streamer / support for new consumer config

2018-02-19 Thread jackson.dickson
Hi Andrew, Yes, "bootstrap.server" option is not working for you? -Jackson -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Snapshot backup of Ignite native persistance data

2018-02-19 Thread arunkjn
Hi Ivan, Thanks for you reply. I have some follow up questions. 1. What is the behaviour of cluster when it is in ignite.cluster.active(false) state? Does it throw error when a cache seek operation is performed? Or does it wait for cluster to become active again? 2. How can I reliably know if

Re: Enable Kerberos on Ignite Kafka streamer / support for new consumer config

2018-02-19 Thread Andrey Mashenkov
Hi Jackson, Got it. I've created a ticket for this [1]. I'm not too close familiar with Kafka. Please, let me know if I missed smth. Seems, the fix is trivial, we've just create a KafkaStreamer directly instead of using zookeeper kafka connector. [1]

Re: Unable to identify root cause for node failure

2018-02-19 Thread Mikhail
Hi Rahul, at 12:25:03 first node detected that second one is failed: [12:45:03,251][INFO][exchange-worker-#182][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=3, minorTopVer=0], evt=NODE_FAILED,

QueryEntities in Ingnite

2018-02-19 Thread Prasad Bhalerao
Hi, As per this link ( https://stackoverflow.com/questions/36773011/what-is-the-difference-between-a-primary-key-and-a-surrogate-key), I have created the QueryEntities as follows. I have not set the fields and indexes explicitly as it is already define in my value Pojo class. private static

Re: Ignite with Spring Cache on K8S, eviction problem

2018-02-19 Thread Vinokurov Pavel
Could you show service methods with cache annotations. 2018-02-19 17:19 GMT+03:00 lukaszbyjos : > I still have a problem with evicting one cache. What's recommended setting > for partitioned cache? > From service B I want to evict "ch-current" cache but service A still

Re: Discovery on Google Cloud

2018-02-19 Thread mcherkasov
Hi, Could you please show us ignites logs? Thanks, Mike. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Only one node has a high number of SystemExecutor CompletedTask

2018-02-19 Thread dark
Thanks to your kind reply. We use Ignite to Group By Query about time series data. However, I have seen a lot of data, so when I create TimeWindow through Expiry Strategy in one cache, I see that it uses too many resources to expire entries. As a result, I have adopted a method of simply

Re: Snapshot backup of Ignite native persistance data

2018-02-19 Thread Ivan Rakov
Arun, 1. First option - you'll get something like this Method threw 'org.apache.ignite.IgniteException' exception: Can not perform the operation because the cluster is inactive. Note, that the cluster is considered inactive by default if Ignite Persistent Store is used to let all the nodes

Re: Write ahead log and early eviction of new elements

2018-02-19 Thread Dmitry Pavlov
Hi Michail, To avoid confusion between real evictions and PDS-enabled region pages rotation with disk we've desided to call second process as 'page replacement'. In future releases all messages related to page purge to disk will contain 'page replacement' instead of eviction. Hope it helps to

Re: QueryEntities in Ingnite

2018-02-19 Thread Mikhail
>Does setting index of cache configuration solves the column Groupid not found problem. yes, it does. >Cause I am not setting fields explicitly in QueryEntity. Is it required to set fields in QueryEntity when annotations are used in Pojo? you must choose only one approach to describe an object

Re: QueryEntities in Ingnite

2018-02-19 Thread Prasad Bhalerao
Does setting index of cache configuration solve the coumn Groupid not found problem. Cause I am not setting fields explicitly in QueryEntity. Is it required to set fields in QueryEntity when annotations are used in Pojo? On Feb 19, 2018 8:45 PM, "Mikhail" wrote: Hi,

[Webinar] Caching technologies evolution

2018-02-19 Thread Denis Magda
Igniters, Curious to see how the caching technologies evolved over the time? Where Ignite, Redis or Memcached fits? Join my tomorrow webinar at 11.00 PT or bookmark and get a recording later: Redis Replaced: Why Companies Now Choose Apache® Ignite™ to Improve Application Speed and Scale

Re: MultiMap

2018-02-19 Thread Michael André Pearce
Great news Amir, this is really appreciated. Is there a PR or anything already for this? Cheers Mike Sent from my iPhone > On 13 Feb 2018, at 23:38, Amir Akhmedov wrote: > > I'm planning to start working on this ticket in couple of days, hopefully > will be

Re: Large durable caches

2018-02-19 Thread lawrencefinn
Okay I am trying to reproduce. It hasn't got stuck yet, but the client got disconnected and reconnected recently. I don't think it is related to GC because I am recording GC times and it does not jump up that much. Could the system get slow on a lot of io? i see this in the ignite log:

Issues trying to force redeployment in shared mode

2018-02-19 Thread Dave Harvey
I was trying to demonstrate changing classes on a client node so that classes on servers get replaced with new code, but the symptoms make me believe that I don't understand the rules at all. The deployment mode is SHARED. I read the instructions and creates an ignite.xml with a different

Re: Large durable caches

2018-02-19 Thread Ivan Rakov
It's hard to determine the problem by these messages. I don't see anything unhealthy regarding persistence - checkpoint start is a regular event. There were cases when excessive GC load on client side seriously affected throughput/latency of data streaming. You may consider playing with the