I think this approach works unless user does not initiate number of
concurrent cache operations greater than MSG_QUEUE_SIZE.  Where msg queue
size default is 1024, but still configurable.

Thanks!
--
Yakov Zhdanov, Director R&D
*GridGain Systems*
www.gridgain.com

2016-03-30 11:44 GMT+03:00 Vladimir Ozerov <voze...@gridgain.com>:

> Does it mean that if cache update rate is greater than filter execution
> rate, then at some point we will stop reading messages from socket? If yes,
> then it seems we still cannot execute cache operations:
> 1) Filter starts cache operation for a key. Current node is backup for this
> key.
> 2) Cache message is sent to primary node
> 3) Primary sends message back to current node.
> 4) Message is never read because of backpressure. Cache operation and
> filter never complete.
>
> Am I missing something?
>
> On Wed, Mar 30, 2016 at 11:23 AM, Yakov Zhdanov <yzhda...@apache.org>
> wrote:
>
> > Vladimir,
> >
> > Communication should stop reading from connection is there are too many
> > unprocessed messages. Sender will be blocked on putting message to queue.
> >
> > --Yakov
> >
> > 2016-03-30 11:11 GMT+03:00 Vladimir Ozerov <voze...@gridgain.com>:
> >
> > > Guys,
> > >
> > > Can you explain how backpressure control is implemented? What if event
> > > arrival speed is greater than filter processing speed?
> > >
> > > On Wed, Mar 30, 2016 at 10:37 AM, Semyon Boikov <sboi...@gridgain.com>
> > > wrote:
> > >
> > > > Andrey,
> > > >
> > > > I agree that current situation with threading in Ignite is very
> > > > inconvenient when user callbacks execute some non-trivial code. But
> > > > changing this to async dispatch is huge refactoring, even changing
> this
> > > > just for continuous queries callback is not so easy task.
> > > >
> > > > We can start with https://issues.apache.org/jira/browse/IGNITE-2004,
> > and
> > > > if
> > > > more users complains arise we can think about changing others parts
> of
> > > > system.
> > > >
> > > > For now we need decisions for these points:
> > > > - how to specify that callback should be run asynchronously (Nikolay
> > > > suggested marker interface IgniteAsyncCallback, or
> > @IgniteAsyncCallback)
> > > > - where these callbacks are executed, AFAIK Nikolay added special
> pool
> > > > which is configured in IgniteConfiguration (something like
> > > > IgniteConfiguration.asyncCallbackThreadPoolSize)
> > > >
> > > > Regards
> > > >
> > > >
> > > > On Tue, Mar 29, 2016 at 10:45 PM, Andrey Kornev <
> > > andrewkor...@hotmail.com>
> > > > wrote:
> > > >
> > > > > Vladimir, Igniters
> > > > >
> > > > > Here are my 2 cents.
> > > > >
> > > > > The current situation with threading when it comes to executing
> user
> > > > > callbacks -- the CQ filters (either local or remote), the CQ
> > listeners,
> > > > the
> > > > > event listeners, the messaging listeners, the entry processors
> (did I
> > > > miss
> > > > > anything?) -- is pretty sad. The callbacks may get executed on a
> > system
> > > > > pool's thread, public pool's, utility pool's, discovery worker
> > thread,
> > > > > application thread, to name a few. It causes a lot of grief and
> > > > suffering,
> > > > > hard-to-fix races, dead locks and other bugs.
> > > > >
> > > > > I guess it's always possible to come up with a more or less
> > reasonable
> > > > > explanation to such predicament (which usually boils down to "It is
> > so
> > > > > because this is how it's implemented"), but I, as a user, could not
> > > care
> > > > > less. I want consistency. I want all my callbacks (including Entry
> > > > > Processors!) to be executed on the public pool's threads, to be
> > > precise.
> > > > > This is not the first time I complain about this, and I really
> think
> > > it's
> > > > > time to fix this mess.
> > > > >
> > > > > For a good example of how to implement ordered async dispatch of
> > > > callbacks
> > > > > on large scale, one only needs to look at Akka (or Reactor
> > > > > https://github.com/reactor/reactor).  Coherence also managed to
> get
> > it
> > > > > right (in my opinion, that is).
> > > > >
> > > > > Regards
> > > > > Andrey
> > > > >
> > > > >
> > > >
> > >
> >
>

Reply via email to