[
https://issues.apache.org/jira/browse/CASSANDRA-15066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826823#comment-16826823
]
Alex Petrov commented on CASSANDRA-15066:
-----------------------------------------
As was discussed on the mailing list, publishing my reviewer notes that might
be helpful for anyone who would like to get an overview of the patch and have
some guidance to save time.
First of all, there are many changes that are easy to review. These are either
renames, moved, extracted classes or moved functionality:
* Some request classes were renamed, such as {{Truncation ->
TruncateRequest}}, {{EncodedHintMessage -> HintMessage}} etc.
* {{Message}} class now got {{id}} so we do not have to pass additional
{{id}} param everywhere. It also combines {{MessageIn}} and {{MessageOut}},
which is also quite a large change.
* Quite a few changes are related to adding {{TimeUnit}}, which are all
cosmetic but still important
* One of the largest changes is splitting out {{Verb}}, {{MBean}} methods and
fields and several other classes out from {{MessagingService}}
* Metrics and virtual tables (also quite large but easy to review and verify)
Now, moving to some important changes. Most of them are around net.async
package.
Two entrypoint classes that handle incoming and outgoing messages and
connections are {{OutboundConnection(s)}} and {{InboundMessageHandler(s)}}.
{{InboundMessageHandlers}} is a class that holds handler instances for
{{small}}, {{large}}, {{urgent}} and {{counter}} messages. Inbound connections
are created and initialised using {{InboundConnectionInitiator}}, which handles
netty pipeline creation in {{#initChannel}}. It is responsible for creating an
appropriate messaging pipeline (either messaging or streaming one).
{{InboundConnectionInitiator#Handler}} inner class is responsible for accepting
message, decoding it and initiating the actual connection, which is done by
receiving a handshake message ({{HandshakeProtocol.Initiate}}). After this
message is received, we check whether or not the remote node version is
compatible with current one and proceed with setting up an appropriate
pipeline. {{Handler#decode}} contains handshake logic for current and older
versions. {{HandshakeProtocol}} class contains a complete description of new
and old handshake protocols, message formats and states.
{{InboundMessageHandler}} also contains logic for handling large message. If
message size exceeds large message threshold, LargeCoprocessor comes into play.
It starts a side thread that handles feeds frame contents into
{{AsyncMessagingInputPlus}}, which calls {{#onBufConsumed}} as soon as it is
done. You can find a complete description in class javadoc.
Lastly, {{InboundMessageHandler}} handles resource limits and message dropping.
You can find acquisition logic in {{processOneContainedMessage}}, expiration in
callbacks.onArrivedExpired and {{WaitQueue#schedule}} logic.
OutboundConnections has many similarities with inbound path. It also groups
small, large and urgent messages into their own {{OutboundConnection}}
instances. OutboundConnectionInitiator creates and initialises outgoing
(messaging and streaming) connections, takes care of Netty specifics, setting
up pipelines and creating handlers. Similar to incoming path, it uses
{{HandshakeProtocol}}. {{OutboundConnections#Connect}} class handles retries
and scheduling of auxiliary tasks (for example, pruning expired messages, see
{{#scheduleMaintenanceWhileDisconnected}}).
OutboundConnection handles capacity limits (see {{#enqueue}} and
{{#acquireCapacity}}) and puts messages to {{OutboundMessageQueue}} if resource
limits allow.
Two subclasses of Delivery ({{EventLoop|LargeMessageDelivery}}) are responsible
for polling the queue (see {{#doRun}}). {{LargeMessageDelivery}} uses
{{AsyncMessagingOutputPlus}}, which buffers data, encodes it to frames and
asynchronously flushes them when buffer fills up.
Both paths have configurations, located in
{{Inbound|OutboundConnectionSettings}}.
There is already some documentation in place in javadoc and by the time the
patch is committed all important pieces will be documented. If you see / think
documentation is missing for some piece of functionality, share your thoughts.
> Improvements to Internode Messaging
> -----------------------------------
>
> Key: CASSANDRA-15066
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15066
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Internode
> Reporter: Benedict
> Assignee: Benedict
> Priority: High
> Fix For: 4.0
>
>
> CASSANDRA-8457 introduced asynchronous networking to internode messaging, but
> there have been several follow-up endeavours to improve some semantic issues.
> CASSANDRA-14503 and CASSANDRA-13630 are the latest such efforts, and were
> combined some months ago into a single overarching refactor of the original
> work, to address some of the issues that have been discovered. Given the
> criticality of this work to the project, we wanted to bring some more eyes to
> bear to ensure the release goes ahead smoothly. In doing so, we uncovered a
> number of issues with messaging, some of which long standing, that we felt
> needed to be addressed. This patch widens the scope of CASSANDRA-14503 and
> CASSANDRA-13630 in an effort to close the book on the messaging service, at
> least for the foreseeable future.
> The patch includes a number of clarifying refactors that touch outside of the
> {{net.async}} package, and a number of semantic changes to the {{net.async}}
> packages itself. We believe it clarifies the intent and behaviour of the
> code while improving system stability, which we will outline in comments
> below.
> https://github.com/belliottsmith/cassandra/tree/messaging-improvements
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
