Hello! Nothing obviously wrong but it still seems to me that you're stretching it too far.
Maybe you need to rethink your cache entry granularity (put more stuff in the entry to make it self-contained) or use some kind of message queue. Regards, -- Ilya Kasnacheev вт, 20 окт. 2020 г. в 19:37, ssansoy <[email protected]>: > Following on from: > > http://apache-ignite-users.70518.x6.nabble.com/ContinuousQuery-Batch-updates-td34198.html > > The takeaway from there is that the continuous queries do not honour > transactions, so if a writer writes 100 records (e.g. > CalculationParameters) > in a transaction, the continuous query will see the updates before the > entire batch of 100 has been committed. > This is a show stopper issue for us for using ignite unfortunately so we > are > trying to think of some work arounds. > > We are considering updating the writer app so, when writing the 100 > CalculationParameters records, it: > > 1. Writes the 100 CalculationParameter records to the cluster (e.g. with > some incremented version e.g. 2) > 2. It writes a separate entry into a special "Modifications" cache, with > the > number of rows written (numRows), and the version id of those records > (versionId). > > Client apps don't subscribe to the CalculationParameter cache. Instead apps > subscribe to the Modifications cache. > > The remote filter will: > > 1. Do a cache.get or a scan query on all the records in > CalculationParameter, filtering on versionId=2. The query has to keep > repeating until all rows are visible (e.g. the numRows records are seen). > Then these records are subjected to another filter (e.g. the usual filter > criteria the the client app would have applied to CalculationParameter). If > there are any records, then the filter returns true. > > 2. The transformer does a similar process to the above, and groups all the > numRows records into a collection, and returns this collection to the > localListen in the client. The client then has access to all the records it > is interested in, in one batch. > > We would have liked to avoid waiting/retrying for all the > CalculationParameter records to appear after the Modifications update is > seen, but since this is happening on the cluster and not on the client we > can probably live with it. > > Do any ignite developers in here see any fundamental flaws with this > approach? We ultimately just want the localListen to be called with a > collection of records that were all updated at the same time - so if there > are any other things worth trying please shout. > Thanks! > Sham > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
