Thanks Bela, I've moved it to the wiki: https://community.jboss.org/wiki/ClusteringMeetingLondonFeb2013
Sanne On 24 February 2013 10:26, Bela Ban <[email protected]> wrote: > > Mircea, Dan, Pedro, Sanne and I had a meeting in London this week on how > to use the new features of JGroups 3.3 in Infinispan 5.3, I've copied > the minutes from the wiki below. > > > London meeting > > Bela, Pedro, Mircea, Dan, Sanne > > > Message bundling and OOB messages > > In 3.3, all messages will be bundled, not just regular messages, but > > also OOB messages. The way this works on the sender side is: > > - A thread sending a message in the transport adds it to a queue > > - There's one thread which dequeues messages and sends them as bundles > > - It sends a messages bundle if the max size has been reached, or > > there are no more messages in the queue > > - This means single messages are sent immediately, or we fill up a > > bundle (in a few microseconds) and send it > > Impact on Infinispan: > > - Use DONT_BUNDLE instead of OOB if you don't want to bundle messages > > - However, even DONT_BUNDLE might get deprecated > > - If we have 1 sender invoking sync RPCs, we don't need to set > > DONT_BUNDLE anymore > > - If we have multiple senders invoking sync RPCs, performance should > > get better as RPCs and responses are bundled > > - Since bundling will result in message *batches* on the receiver, > > performance should increase in general > > > Message batching > > Message bundles sent by a sender are received as message batches > > (MessageBatch) by the receivers. When a batch is received, the batch > > is passed up using up(MessageBatch). > > Protocols can remove / replace / add messages in a batch and pass the > > batch further up. > > The advantage of a batch is that resources such as locks are acquired > > only once for a batch of N messages rather than N times. Example: when > > NAKACK2 receives a batch of 10 messages, it adds the 10 messages to > > the receiver table in a bulk operation, which is more efficient than > > doing this 10 times. > > Further optimizations on batching (probably 3.4): > > - Remove similar ops, e.g. UNICAST3 acks for A:15, A:25 and A:35 can > > be clubbed together into just ack(A:35) > > - Merge similar headers, e.g. multicast messages 20-30 can be orderd > > by seqno, and we simply send a range [20..30] and let the receiver > > generate the headers on the fly > > > Async Invocation API (AIA) > > JGroups only passes up messages to Infinispan, which then uses its own > > thread pool to deliver them. E.g. based on Pedro's code for TO, we > > could parallelize delivery based on the target keys of the > > transaction. E.g if we have tx1 modifying keys {A,B,C} and tx2 > > modifying keys {T,U}, then tx1 and tx2 can be run concurrently. > > If tx1 and tx2 modify overlapping key sets, then tx2 would be queued > > and executed *after* tx1, not taking up a thread from the pool, > > reducing the chances of the thread pool maxing out and also > > ensuring different threads are not going to contend on the locks > > on same keys. > > The implementation could be done in an interceptor fronting the > interceptor stack, which queues dependent TXs and > > - when ready to be executed - sends them up the interceptor stack on a > thread from > > the internal pool. > > Infinispan having its own thread pool means that JGroups threads will > > not block anymore, e.g. trying to acquire a lock for a TX. The size of > > those pools can therefore be reduced. > > The advantage of AIA is that it's up to Infinispan, not JGroups, how > > to deliver messages. JGroups delivers messages based on the order in > > which they were sent by a sender (FIFO), whereas Infinispan can make > > much more informed decisions as to how to deliver the messages. > > > Internal thread pool for JGroups > > All JGroups internal messages use the internal thread pool (message > > flag=INTERNAL). Not having to share the OOB pool with apps (such as > > Infinispan) means that internal messages can always be processed, and > > are not discarded or blocked, e.g. by a maxed out thread pool. > > The internal pool can be switched off, and - if AIA is implemented in > > Infinispan - the number of OOB and regular threads can be massively > > reduced. The internal thread pool doesn't need to be big either. > > > UNICAST3 > > Successor to UNICAST and UNICAST2, best of both worlds. Acks single > > messages quickly, so we have no first-msg-lost or last-msg-lost issues > > anymore. Doesn't generate many acks though. > > Proposed to trigger and ACK only after a certain number of messages > > rather than after any batch to avoid ACK on small batches. > > https://issues.jboss.org/browse/JGRP-1594 > > -- > Bela Ban, JGroups lead (http://www.jgroups.org) > > _______________________________________________ > infinispan-dev mailing list > [email protected] > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list [email protected] https://lists.jboss.org/mailman/listinfo/infinispan-dev
