Re: On the issue of continuous query function monitoring in thinclients

2023-10-15 Thread Pavel Tupitsyn
ry > similar to my situation. > > > > I am using gridgain-community-8.8.9 version. > > > > The server is configured in file ignite-server-config1.xml, while the thin > client and continuous query are configured in file ThinClientExample.java. > > > >

回复: On the issue of continuous query function monitoring in thinclients

2023-10-13 Thread Mengyu Jing
Hi! Mikhail, Pavel Indeed, the two pieces of information provided by Pavel may not be very similar to my situation. I am using gridgain-community-8.8.9 version. The server is configured in file ignite-server-config1.xml, while the thin client and continuous query are configured in file

Re: On the issue of continuous query function monitoring in thin clients

2023-10-13 Thread Mikhail Petrov
Pavel, in fact, I don't think that the mentioned ticket has anything to do with the described problem. Mengyu Jing, What version of the Apache Ignite are you using? Could you please attach the configuration of the Ignite server node, thin client configuration and continuous query you

Re: On the issue of continuous query function monitoring in thin clients

2023-10-12 Thread Pavel Tupitsyn
the cluster did not have > persistence enabled. > > > > I used 40 thin clients to listen for cache changes through continuous > query function, and there were also some other thin clients constantly > inserting data into the cache. When I hung up a node in the cluster using > the

On the issue of continuous query function monitoring in thin clients

2023-10-12 Thread Mengyu Jing
Hello, Igniters! I started an Ignite cluster with 12 nodes, but the cluster did not have persistence enabled. I used 40 thin clients to listen for cache changes through continuous query function, and there were also some other thin clients constantly inserting data into the cache. When I

Re: Ignite thin client continuous query listener cannot listen to all events

2023-05-25 Thread Alex Plehanov
t; By executing these two test codes in the description, it is easy to >> reproduce the problem. >> >> >> >> What should I do next? Continuously following this Jira? >> >> >> >> >> >> 发件人: Jeremy McMillan >> 发送时间: 2023年5月

Re: Ignite thin client continuous query listener cannot listen to all events

2023-05-24 Thread Pavel Tupitsyn
these two test codes in the description, it is easy to > reproduce the problem. > > > > What should I do next? Continuously following this Jira? > > > > > > *发件人: *Jeremy McMillan > *发送时间: *2023年5月24日 23:24 > *收件人: *user@ignite.apache.org > *主题: *Re: Ignite t

Re: Ignite thin client continuous query listener cannot listen to all events

2023-05-24 Thread LonesomeRain
5月24日 23:24 收件人: user@ignite.apache.org 主题: Re: Ignite thin client continuous query listener cannot listen to all events Thanks for bringing this up! https://ignite.apache.org/docs/latest/key-value-api/continuous-queries#events-delivery-guarantees This sounds like you may have found a bug

Re: Ignite thin client continuous query listener cannot listen to all events

2023-05-24 Thread Jeremy McMillan
:52 AM lonesomerain wrote: > *Hi,* > *I have a question while using ignite 2.15.0* > > *Problem scenario:* > > Start the Ignite server of one node, start one thin client and create a > continuous query listener, and then use 50 threads to add 500 data to the > cache concurr

Ignite thin client continuous query listener cannot listen to all events

2023-05-22 Thread lonesomerain
Hi, I have a question while using ignite 2.15.0 Problem scenario: Start the Ignite server of one node, start one thin client and create a continuous query listener, and then use 50 threads to add 500 data to the cache concurrently. Problem phenomenon: Through the information printed

RE: Potential continuous query memory leak?

2022-10-12 Thread Anders Melgaard
Please note that the issue only appears if backups > 0 for the cache. It also doesn't matter if the cache is transactional or atomic. On 2022/10/11 12:09:08 Anders Melgaard wrote: > Hi all, > > I have a potential issue with continuous queries and memory consumption. > I've run a test with a

Potential continuous query memory leak?

2022-10-11 Thread Anders Melgaard
Hi all, I have a potential issue with continuous queries and memory consumption. I've run a test with a single cache with 3 CQs registered on one node. Another node joins and inserts one million entries and removes the same entries afterwards. This leaves the cache empty but it seems there's ~1,4

Re: Continuous query with Thin client

2021-04-19 Thread Pavel Tupitsyn
d changes required to support continuous queries > in thin-clients. > > On 19 Apr 2021, at 07:59, sourav dihidar wrote: > > Thanks for your reply. I see in the release note of 2.10.0 under Java thin > client , it is mentioned > > >- Added thin Client Continuous Query &g

Re: Continuous query with Thin client

2021-04-19 Thread Stephen Darlington
I suspect it’s the backend changes required to support continuous queries in thin-clients. > On 19 Apr 2021, at 07:59, sourav dihidar wrote: > > Thanks for your reply. I see in the release note of 2.10.0 under Java thin > client , it is mentioned > > Added thin Clien

Re: Continuous query with Thin client

2021-04-19 Thread sourav dihidar
Thanks for your reply. I see in the release note of 2.10.0 under Java thin client , it is mentioned - Added thin Client Continuous Query Is that something else? Thanks On Fri, Apr 16, 2021, 10:11 PM Stephen Darlington < stephen.darling...@gridgain.com> wrote: > Looks like that

Re: Continuous query with Thin client

2021-04-16 Thread Stephen Darlington
Looks like that’s coming in 2.11: IGNITE-14402 <https://issues.apache.org/jira/browse/IGNITE-14402> It’s present in the .net thin client since 2.10. Regards, Stephen > On 16 Apr 2021, at 17:21, sourav dihidar wrote: > > Hi Team, > > I understood that continuous query

Continuous query with Thin client

2021-04-16 Thread sourav dihidar
Hi Team, I understood that continuous query for java thin client has been introduced in the latest ignite version(2.10.0). Do you have any documentation around this specially how to use it through thin client. Thanks

Re: Near Cache versus Continuous Query

2021-02-24 Thread Ilya Kasnacheev
e have some on-heap entries? > > 2. What is the best way to ensure the near cache is kept completely in-sync > with the server cache, would it be sufficient to use a continuous query > that > makes a cache.get() call whenever the continuous query gets a CREATED > event? > The ca

Re: Near Cache versus Continuous Query

2021-02-22 Thread bhlewka
cache, would it be sufficient to use a continuous query that makes a cache.get() call whenever the continuous query gets a CREATED event? The cache will keep a small enough amount of data that we do not have a need for an eviction policy. 3. What is the best way to keep a continuous query open

Re: Near Cache versus Continuous Query

2021-02-20 Thread Ilya Kasnacheev
ache, it > was only updated with entries it had already had stored in it. > > Is there a way I can resolve either of these two undesireable problems of > the NearCache for my situation? > > > Continuous Query Attempt: > = > This lead me to instead consider

Re: Near Cache versus Continuous Query

2021-02-19 Thread Pavel Tupitsyn
cache never received new entries from the replicated cache, it > was only updated with entries it had already had stored in it. > > Is there a way I can resolve either of these two undesireable problems of > the NearCache for my situation? > > > Continuous Query Attempt: >

Near Cache versus Continuous Query

2021-02-19 Thread tschauenberg
entries from the replicated cache, it was only updated with entries it had already had stored in it. Is there a way I can resolve either of these two undesireable problems of the NearCache for my situation? Continuous Query Attempt: = This lead me to instead consider Continuous

Near Cache versus Continuous Query

2021-02-19 Thread tschauenberg
entries from the replicated cache, it was only updated with entries it had already had stored in it. Is there a way I can resolve either of these two undesireable problems of the NearCache for my situation? Continuous Query Attempt: = This lead me to instead consider Continuous

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes and UnregisteredBinaryTypeExceptio

2021-02-02 Thread VeenaMithare
On further debugging, it looks like all the clients that had failed the count, were executing their query on the same server machinename002. When I looked at the server logs, it looks like that the records that were missing were not present on this server when the query was executed but were

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-27 Thread VeenaMithare
Hi Andrei, Did you get a chance to look at my comments . Regarding doing the continuous query as per this example : https://github.com/gridgain/gridgain/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/CacheContinuousQueryExample.java Our code is pretty much as shown

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-25 Thread VeenaMithare
HI Andrei, Some more points : 1. Also the javadocs for the querycursor says : https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/QueryCursor.html#getAll-- List getAll() Gets all query results and stores them in the collection. Use this method when you know in

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-22 Thread VeenaMithare
Hi Andrei, Also as I said it is a trimmed down version of the actual code. In the actual code, the cursor is taken into a instance variable and closed. Made modifications accordingly to the client class . Please find the same attached, InitialQueryProject-1.zip

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-21 Thread VeenaMithare
in the loop . // Create new continuous query. QueryCursor> cur = cache.query(qry); LOGGER.info( "PROJECT LISTENS: >>> Cache continuous query started successfully. Start Initial Query...{}", qry.getInitialQuery()); // Iterate throug

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-21 Thread andrei
Hi, From your source code, I can see that the QueryCursor was created inside startContinuousQuery. Then you return it from that method, but don't store it in any variable: continuousQueryHelper.startContinuousQuery(ignite,qry,testCache,subscribeParameters.callBackList,outputFieldsList); for

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-20 Thread VeenaMithare
Hi Andrei, Please find a trimmed down version of the code attached. Have a look at the ContinuousQueryHelper -> setInitialQuery to check how the initial query has been coded. regards, Veena. InitialQueryProject.zip

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-19 Thread VeenaMithare
HI Andrei, The initial query(sql query) is failing, not the continuous update. Also so far we have had no issues in the initial query.. ( for the past 1 year or so ). I can share a reproducer if that will help, regards, Veena. -- Sent from:

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-19 Thread andrei
Hi, Could you please provide your client's code where you are listening to continuous query? Probably something is wrong there. BR, Andrei 1/19/2021 4:13 PM, VeenaMithare пишет: Hi , It happened a couple of times on Friday ( 15th ) . Today I started a dummy client with a continuous query

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-19 Thread VeenaMithare
Hi , It happened a couple of times on Friday ( 15th ) . Today I started a dummy client with a continuous query against the same client multiple times against the same cluster - and I am not able to reproduce it. This cache contains 58 records , and I see all the 58 records being recorded

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-19 Thread Ilya Kasnacheev
Hello! Does this happen if you just run this SQL query during client start-up? Can you reproduce this by bringing nodes down and back up? Regards, -- Ilya Kasnacheev вт, 19 янв. 2021 г. в 15:11, VeenaMithare : > No . Its a SQL query. > > > > -- > Sent from:

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-19 Thread VeenaMithare
No . Its a SQL query. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-19 Thread Ilya Kasnacheev
Hello! What kind of initial query do you have? Is it a scan query? Regards, -- Ilya Kasnacheev пн, 18 янв. 2021 г. в 19:44, VeenaMithare : > Hi Team, > > Our env : > Servers : 3 > Cache Configuration : REPLICATED > > In one of our clients, the continuous query - Initial

2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-18 Thread VeenaMithare
Hi Team, Our env : Servers : 3 Cache Configuration : REPLICATED In one of our clients, the continuous query - Initial query did not return any result during client start whereas it was expected to return a single row. This happened for about 3-4 times over a period of 2 hours. It did return

Re: Continuous query not transactional ?

2020-10-16 Thread VeenaMithare
Hi Ilya, That is what I assume too, could someone from the developers community help confirm/comment on this ? regards, Veena. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Continuous query not transactional ?

2020-10-16 Thread Ilya Kasnacheev
e-visible-after-transaction-commit-tp33787p33861.html > > >>Continuous Query itself is not transactional and it looks like it can't > be > used for this at the moment. So, it gets notification before other entries > were committed. > > Does this mean we could get dirty re

Continuous query not transactional ?

2020-10-15 Thread VeenaMithare
Hi , This is in continuation of the below statement on this post : http://apache-ignite-users.70518.x6.nabble.com/Lag-before-records-are-visible-after-transaction-commit-tp33787p33861.html >>Continuous Query itself is not transactional and it looks like it can't be used for this at the

Re: Continuous Query

2020-10-07 Thread narges saleh
Thanks Raymond for the pointer. On Wed, Oct 7, 2020 at 4:39 PM Raymond Wilson wrote: > It's possible to configure the continuous query to return only keys in the > cache that are stored on the local node. > > On the C# client we do it like this: > > var _queryHandle = queueCac

Re: Continuous Query

2020-10-07 Thread Raymond Wilson
It's possible to configure the continuous query to return only keys in the cache that are stored on the local node. On the C# client we do it like this: var _queryHandle = queueCache.QueryContinuous (qry: new ContinuousQuery<[..KeyType..], [...ValueType...]>(listener) {Local

Re: Continuous Query

2020-10-07 Thread narges saleh
an actual Ignite APIs). > > First. You need to add a flag field to the record's class that keeps the > current processing status. Like that: > > MyRecord { > int id; > Date created; > byte status; //0 - not processed, 1 - being processed withing a > continuous query filt

Re: Continuous Query

2020-10-07 Thread Denis Magda
processing status. Like that: MyRecord { int id; Date created; byte status; //0 - not processed, 1 - being processed withing a continuous query filter, 2 - processed by the filter, all the logic successfully completed } Second. The continuous query filter (that will be executed on nodes that store

Re: Continuous Query

2020-10-07 Thread Denis Magda
> > So, a lesson for the future, would the continuous query approach still be > preferable if the calculation involves the cache with continuous query and > say a look up table? For example, if I want to see the country in the cache > employee exists in the list of the coun

Re: Continuous Query

2020-10-07 Thread narges saleh
Also, what do I need to do if I want the filter for the continuous query to execute on the cache on the local node only? Say, I have the continuous query deployed as singleton service on each node, to capture certain changes to a cache on the local node. On Wed, Oct 7, 2020 at 5:54 AM narges

Re: Continuous Query

2020-10-07 Thread narges saleh
Thank you, Denis. So, a lesson for the future, would the continuous query approach still be preferable if the calculation involves the cache with continuous query and say a look up table? For example, if I want to see the country in the cache employee exists in the list of the countries that I am

Re: Continuous Query

2020-10-06 Thread Denis Magda
Thanks Then, I would consider the continuous queries based solution as long as the records can be updated in real-time: - You can process the records on the fly and don't need to come up with any batch task. - The continuous query filter will be executed once on a node that stores

Re: Continuous Query

2020-10-05 Thread narges saleh
> On Fri, Oct 2, 2020 at 12:23 PM Denis Magda wrote: >>>> >>>>> What are your requirements? Do you need to process the records as soon >>>>> as they are put into the cluster? >>>>> >>>>> >>>>> >>>>>

Re: Continuous Query

2020-10-05 Thread Denis Magda
What are your requirements? Do you need to process the records as soon >>>> as they are put into the cluster? >>>> >>>> >>>> >>>> On Friday, October 2, 2020, narges saleh wrote: >>>> >>>>> Thank you Dennis for the

Re: Continuous Query

2020-10-05 Thread Ilya Kasnacheev
2020 2:03 AM > *To:* user@ignite.apache.org > *Subject:* Re: Continuous Query > > The latter; the server needs to perform some calculations on the data > without sending any notification to the app. > > On Fri, Oct 2, 2020 at 4:25 PM Denis Magda wrote: > > And after you detec

Re: Continuous Query

2020-10-04 Thread Priya Yadav
unsubscribe From: narges saleh Sent: Sunday, 4 October 2020 2:03 AM To: user@ignite.apache.org Subject: Re: Continuous Query The latter; the server needs to perform some calculations on the data without sending any notification to the app. On Fri, Oct 2, 2020

Re: Continuous Query

2020-10-03 Thread narges saleh
? Do you need to process the records as soon >>> as they are put into the cluster? >>> >>> >>> >>> On Friday, October 2, 2020, narges saleh wrote: >>> >>>> Thank you Dennis for the reply. >>>> From the perspective of performance/resour

Re: Continuous Query

2020-10-02 Thread Denis Magda
f performance/resource overhead and reliability, >>> which approach is preferable? Does a continuous query based approach impose >>> a lot more overhead? >>> >>> On Fri, Oct 2, 2020 at 9:52 AM Denis Magda wrote: >>> >>>> Hi Narges, >

Re: Continuous Query

2020-10-02 Thread narges saleh
, >> which approach is preferable? Does a continuous query based approach impose >> a lot more overhead? >> >> On Fri, Oct 2, 2020 at 9:52 AM Denis Magda wrote: >> >>> Hi Narges, >>> >>> Use continuous queries if you need to be notified in real-t

Re: Continuous Query

2020-10-02 Thread Denis Magda
referable? Does a continuous query based approach impose > a lot more overhead? > > On Fri, Oct 2, 2020 at 9:52 AM Denis Magda wrote: > >> Hi Narges, >> >> Use continuous queries if you need to be notified in real-time, i.e. 1) a >> record is inserted, 2) the

Re: Continuous Query

2020-10-02 Thread narges saleh
Thank you Dennis for the reply. >From the perspective of performance/resource overhead and reliability, which approach is preferable? Does a continuous query based approach impose a lot more overhead? On Fri, Oct 2, 2020 at 9:52 AM Denis Magda wrote: > Hi Narges, > > Use contin

Re: Continuous Query

2020-10-02 Thread Denis Magda
Hi Narges, Use continuous queries if you need to be notified in real-time, i.e. 1) a record is inserted, 2) the continuous filter confirms the record's time satisfies your condition, 3) the continuous queries notifies your application that does require processing. The jobs are better for a

Continuous Query

2020-10-02 Thread narges saleh
Hi All, If I want to watch for a rolling timestamp pattern in all the records that get inserted to all my caches, is it more efficient to use timer based jobs (that checks all the records in some interval) or continuous queries that locally filter on the pattern? These records can get inserted

Re: Question in Continuous Query Remote Filter

2020-08-03 Thread Evgenii Zhuravlev
I believe Java doc should be enough for that: https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/ContinuousQuery.html It says: To stop receiving updates call QueryCursor.close()

Re: Question in Continuous Query Remote Filter

2020-08-01 Thread Devakumar J
Hi, Thanks for the reply. Do we have any document reference for stopping/unsubscribing registered CQ listeners. Thanks & Regards, Devakumar J -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Question in Continuous Query Remote Filter

2020-07-31 Thread Evgenii Zhuravlev
Hi, You can just stop the CQ and then register it again if you don't want to get notifications for some period of time. Evgenii пт, 31 июл. 2020 г. в 01:51, Devakumar J : > Hi All, > > We have a setup of 2 servers and 1 client Node. Client node registers CQ on > set of caches. > > I just want

Question in Continuous Query Remote Filter

2020-07-31 Thread Devakumar J
Hi All, We have a setup of 2 servers and 1 client Node. Client node registers CQ on set of caches. I just want to temporarily pause and resume CQ notifications based on certain action at client node. I was trying to achieve this through remote filter and ignite messaging. I mean client publish

Re: Event for an update that should have been filtered is received in Local Listener of Continuous Query when a 1000 row insert is triggered

2020-07-17 Thread VeenaMithare
Hello, Looks like this issue is not related to load, but related to remote filter not getting redeployed when a server node is restarted on version 2.7.6. Related discussion at http://apache-ignite-users.70518.x6.nabble.com/Remote-Filter-Execution-td33274.html regards, Veena. -- Sent

Re: Continuous Query on a varying set of keys

2020-05-26 Thread Ilya Kasnacheev
Hello! Yes, using other continuous query to watch changes of this set looks OK. Of course there will be some yak shaving to do. Regards, -- Ilya Kasnacheev вт, 26 мая 2020 г. в 12:28, zork : > Thanks. > So I can think of two ways using which such a set could be maintained by > the

Re: Continuous Query on a varying set of keys

2020-05-26 Thread zork
and remote node can maybe listen to it using another continuous query. Is that right? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Continuous Query on a varying set of keys

2020-05-26 Thread Ilya Kasnacheev
Hello! No, updating local set will not be sent over to the remote node, but on the remote node you can probably update it from your filter: qry.setRemoteFilterFactory(new Factory>() { @Override public CacheEntryEventFilter create() { return new

Re: Continuous Query on a varying set of keys

2020-05-26 Thread zork
Hi, Sorry but I could not get it to work. The standard way of defining a remote filter as shown in sample repo is something like: qry.setRemoteFilterFactory(new Factory>() { @Override public CacheEntryEventFilter create() { return new

Re: Ordered initial query for continuous query in Ignite .Net

2020-05-19 Thread Pavel Tupitsyn
pri...@uk.bnpparibas.com> wrote: > Hi, > > I'm just upgrading to Ignite 2.8 and see that SqlQuery has been deprecated > directing us to SqlFieldsQuery, ScanQuery or Text. Previously I had a > continuous query running that took an SqlQuery which had "order by xyz" as > part o

Ordered initial query for continuous query in Ignite .Net

2020-05-19 Thread Barney Pippin
Hi, I'm just upgrading to Ignite 2.8 and see that SqlQuery has been deprecated directing us to SqlFieldsQuery, ScanQuery or Text. Previously I had a continuous query running that took an SqlQuery which had "order by xyz" as part of the initial query so I could get ordered results ba

Re: Continuous Query on a varying set of keys

2020-05-18 Thread Ilya Kasnacheev
Hello! Remote filter is code. It can execute arbitrary logic. It can adjust to what it needs to filter, change its own behavior with time. Regards, -- Ilya Kasnacheev пн, 18 мая 2020 г. в 15:40, zork : > Hi Ilya, > Thanks for your response. > I'm aware of remote filters but can these filters

Re: Continuous Query on a varying set of keys

2020-05-18 Thread zork
Hi Ilya, Thanks for your response. I'm aware of remote filters but can these filters be modified once the query is already attached? Because if not, then this would not solve my use case as the filter would always give me updates on a fixed subset of keys, however in my case this subset is varying

Re: Continuous Query on a varying set of keys

2020-05-18 Thread Ilya Kasnacheev
Hello! Continuous query has a notion of 'remote filter'. This is a piece of code which is executed near data (on server nodes) to determine if the update needs to be sent over network. https://apacheignite.readme.io/docs/continuous-queries#remote-filter If you define a proper remote filter

Continuous Query on a varying set of keys

2020-05-17 Thread zork
continuous query is attached for every key whenever it is subscribed and it is closed whenever that key is no longer of interest (or unsubscribed). The problem with this is that since there are so many continuous queries (a few thousands), the application goes out of memory. Also, it would mean all

Re: Event for an update that should have been filtered is received in Local Listener of Continuous Query when a 1000 row insert is triggered

2020-05-11 Thread akorensh
Hi, handleEntry() performs internal housekeeping chores. You are correct a notification is sent to localListeners whether an entry was filtered or not. your actual local listener will be called only if the entry has passed your filter criteria. If you place a breakpoint inside

Re: Event for an update that should have been filtered is received in Local Listener of Continuous Query when a 1000 row insert is triggered

2020-05-10 Thread VeenaMithare
HI Alex, I am not able to reproduce this issue - we have seen it happening a few times in our test env. though. I am trying to go through logs and code to understand how it can happen. It looks like all the local listeners for this cache( about 6 of them ) have received the 'untransformed'

Re: Event for an update that should have been filtered is received in Local Listener of Continuous Query when a 1000 row insert is triggered

2020-05-08 Thread akorensh
It might, I would need to see a reproducer to make a determination. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Event for an update that should have been filtered is received in Local Listener of Continuous Query when a 1000 row insert is triggered

2020-05-08 Thread VeenaMithare
I see this line being printed just before any local listener is invoked : 2020-05-06T16:28:17,909 INFO o.a.i.s.c.t.TcpCommunicationSpi [grid-nio-worker-tcp-comm-4-#255%ActivDataPublisher-ACTIVEI2-igniteclient-GREEN%]: Accepted incoming communication connection [locAddr=/x.x.x.x:yyy,

Re: Event for an update that should have been filtered is received in Local Listener of Continuous Query when a 1000 row insert is triggered

2020-05-08 Thread VeenaMithare
Hi Alex, Thank you for the reply . >> verify the continuous query definitions using the appropriate view: https://apacheignite.readme.io/docs/continuous_queries We are on 2.7.6 version. I guess this view is not available for us. Also we have been running the CQs for couple of mont

Re: Event for an update that should have been filtered is received in Local Listener of Continuous Query when a 1000 row insert is triggered

2020-05-07 Thread akorensh
Hi, It shouldn't work like you described: Continuous query clients should only be receiving the records that are pertinent to their filters/transformers. verify the continuous query definitions using the appropriate view: https://apacheignite.readme.io/docs/continuous_queries take a look

Incorrect Value Received in Local Listener of Continuous Query when a 1000 row insert is triggered

2020-05-07 Thread VeenaMithare
We have about 5 clients. Each client starts 2 different CQ to the same cache abc . (We need 2 CQ since we want different action and different lifecycle when the local listener is triggered for either query. ) Client 1 : Say CQ1 is listening to changes to a record where name is name1. (There is a

Re: Continuous query setPageSize and setTimeInterval question

2020-04-30 Thread akorensh
It is likely entries are being quickly produced and filling up the buffer giving the effect of an immediate update. You can test it via specific delays and logging. Make the server print out the counter of the object. say: if the key is an integer and the value is too then print out the key as

Re: Continuous query setPageSize and setTimeInterval question

2020-04-30 Thread crypto_ricklee
This was what I thought, however, no matter what number I set to the pageSize, e.g., 5, 20, 100, my local listener got the update immediately, 1 by 1, but not batching by the page size... -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Continuous query setPageSize and setTimeInterval question

2020-04-29 Thread akorensh
You are correct. Your local reciever will get an update every 100 entries. setPageSize sets the number of entries to batch together before sending. When the server has accumulated a number of entries larger than getPageSize it sends a message to the receiver. Like I mentioned before,

Re: Continuous query setPageSize and setTimeInterval question

2020-04-28 Thread crypto_ricklee
Thanks Alex, But I still not quite understand the expected behaviour. If I set the page size to 100 and interval to 0, should the local query be triggered for every 100 updates? Regards, Rick -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Continuous query setPageSize and setTimeInterval question

2020-04-28 Thread akorensh
Hi, setTimeInterval limits the time Ignite will wait for an internal buffer to fill up before sending. It is not a time delay setting. From the doc: https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/ContinuousQuery.html#setTimeInterval-long- When a

Continuous query setPageSize and setTimeInterval question

2020-04-28 Thread crypto_ricklee
Dear all, I tried to play around with the setPageSize and setTimeInterval on Continuous Query, but it doesn't seem to have any effect. My code is like: private void configOrderListener(Ignite ignite) { IgniteCache cache = ignite.cache(StoreHelper.getCacheName(OrderStore.class

Re: Continuous Query Questions

2020-04-01 Thread narges saleh
So, if I define the CQ as a service, and the node crashes, wouldn't ignite start a new service with CQ already registered, say if the CQ registration is in service init? I can do initial query at the as well. On Wed, Apr 1, 2020 at 7:33 PM Evgenii Zhuravlev wrote: > Well, with this use case,

Re: Continuous Query Questions

2020-04-01 Thread Evgenii Zhuravlev
Well, with this use case, if one of the nodes goes down, there is always a chance to lost notifications. I don't think that it's possible to recover lost notifications with out of the box solution, but if you will be able to track the last processed notification and store update time in entries,

Re: Continuous Query Questions

2020-04-01 Thread narges saleh
Thanks Evgenii for the recommendation and the heads up. Is there a way to recover the lost notifications or even know if a notification is lost? On Wed, Apr 1, 2020 at 12:15 PM Evgenii Zhuravlev wrote: > Hi, > > 1) I would recommend checking ContinouousQuery.setLocal: >

Re: Continuous Query Questions

2020-04-01 Thread Evgenii Zhuravlev
Hi, 1) I would recommend checking ContinouousQuery.setLocal: https://www.gridgain.com/sdk/ce/latest/javadoc/org/apache/ignite/cache/query/Query.html#setLocal-boolean-. Please check if it fits your requirements. 2) You will need to do this in a separate thread, because cache operations shouldn't

Continuous Query Questions

2020-03-31 Thread narges saleh
Hi All, I'd like to get your feedback regarding the following pattern. 1) CQ setup that listens to the changes to a cache on the local node only. 2) Upon receiving notification on a change, the listener makes additions to two other caches, one being on the local node (partitioned) and the other

Re: Continuous Query Questions

2020-03-13 Thread Evgenii Zhuravlev
I'm not sure, because the final overhead depends on the object sizes. There is a buffer for CQ, which stores 1000 entries by default, but you can decrease it using property IGNITE_CONTINUOUS_QUERY_SERVER_BUFFER_SIZE Evgenii вт, 18 февр. 2020 г. в 18:09, narges saleh : > Hi Evgeni, > > There

Re: How to reconnect continuous query?

2020-02-28 Thread Denis Magda
Hi Rick, What's the stack trace you see? Generally, creating/registring a new continuous query is a proper way of reconnection handling. The issue might be that the new query is created by a system thread that triggered your predicate - ignite.events().localListen(new IgnitePredicate() - Denis

How to reconnect continuous query?

2020-02-27 Thread crypto_ricklee
Dear all, I have a continuous query on a client node like IgniteCache cache = ignite.getOrCreateCache(StoreHelper.getCacheName(BalanceStore.class)); ContinuousQuery qry = new ContinuousQuery<>(); qry.setLocalListener(handler); qry.setRemoteFilterFactory((Factory>) ()

Re: Continuous Query Questions

2020-02-18 Thread narges saleh
Hi Evgeni, There will be several thousands notifications/day if I have it send notification only when certain patterns are visited, in about 100+ caches, which brings up another question: wouldn't having 100+ CQs be creating too much overhead? thanks. On Tue, Feb 18, 2020 at 2:17 PM Evgenii

Re: Continuous Query Questions

2020-02-18 Thread Evgenii Zhuravlev
Hi, How many notifications do you want to get? If it's just a several notifications, then you can even register separate CQ for each of the entry with its own remote filters. At the same time, if you have a requirement to send these notifications for a lot of entries, then this approach will

Re: Continuous query order on transactional cache

2020-01-16 Thread Ilya Kasnacheev
Hello! Why? I don't think that transactions, semantically have any guarantees about the order of updates inside a transaction. I'd go with A). Regards, -- Ilya Kasnacheev чт, 16 янв. 2020 г. в 17:39, Barney Pippin : > Hi, > > If I have a continuous query running and it's

Continuous query order on transactional cache

2020-01-16 Thread Barney Pippin
Hi, If I have a continuous query running and it's listening to a transactional cache, what order will I receive the notifications if say 5 updates are committed in a single transaction? Is the order: A) Undefined B) The order the cache updates are written to the cache prior to the commit C

RE: Local listener for continuous query doesn't trigger on cache keyremoval

2019-08-19 Thread Alex Shapkin
apache.org Subject: Local listener for continuous query doesn't trigger on cache keyremoval Hi, I am following the example on this page for continuous query local listener and it works great: https://apacheignite.readme.io/docs/continuous-queries#section-local-listener However, it s

  1   2   3   >