[
https://issues.apache.org/jira/browse/IGNITE-21039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17854281#comment-17854281
]
Roman Puchkovskiy commented on IGNITE-21039:
--------------------------------------------
In IGNITE-22371, handling of big messages was fixed, for arrays of 160k
elements we now get up to 620Mb/sec.
In IGNITE-22399, handling of small messages was improved (they are collected to
batches, so a lot less channel.write() calls are made). This improved
performance with smaller messages sent on the maximum speed (like when using
array-based messages of small sizes).
With the default scenario settings, for small messages like TestByteMessage,
the benchmark is more bound to latency than to throughput. Latency became worse
due to batching, so with the default scenario settings, we now get worse
results for such messages. But, if we increase the number of in-flight messages
drastically (up to 1 million, for example), the scenario becomes
throughput-bound, and in this modified scenario for small messages batching
improves the scores.
As we prefer throughput over latency, batching is preferred even though it
(formally) worsens the results for small messages like TestByteMessage in
latency-bound scenarios.
> Network performance optimization
> --------------------------------
>
> Key: IGNITE-21039
> URL: https://issues.apache.org/jira/browse/IGNITE-21039
> Project: Ignite
> Issue Type: Improvement
> Components: networking
> Affects Versions: 3.0
> Reporter: Alexander Belyak
> Assignee: Roman Puchkovskiy
> Priority: Major
> Labels: ignite-3
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I've run several test to find out the MessagingService performance metrics
> and that is what I've found:
> {noformat}
> TestBoolaMessage 139MB/sec WARD
> TestByteaMessage 132MB/sec WARD
> TestDoubleaMessage 102MB/sec WARD
> TestFloataMessage 132MB/sec WARD
> TestDoubleaMessage 130MB/sec WARD
> TestLongaMessage 131MB/sec WARD
> TestDoubleaMessage 131MB/sec WARD
> TestStringaMessage 280MB/sec WARD
> TestBoolMessage 11MB/sec WARD WARD
> TestByteMessage 12MB/sec WARD
> TestDoubleMessage 12MB/sec WARD
> TestFloatMessage 13MB/sec WARD
> TestIntMessage 12MB/sec WARD
> TestLongMessage 11MB/sec WARD
> TestShortMessage 12MB/sec WARD
> TestStringMessage 18MB/sec WARD
> TestBool20Message 15MB/sec WARD
> TestByte20Message 12MB/sec WARD
> TestDouble20Message 32MB/sec WARD
> TestFloat20Message 22MB/sec WARD
> TestInt20Message 13MB/sec WARD
> TestLong20Message 14MB/sec WARD
> TestShort20Message 14MB/sec WARD
> TestString20Message 65MB/sec WARD
> {noformat}
> All messages were sent in the same setup: 2 server nodes, connected with a
> *10GBit* interface. *Iperf3* (iperf3 --time 30 --zerocopy --client
> 192.168.1.126 --omit 3 --interval 1 --length 16384 --window 131072 --parallel
> 2 --json --version4) shows about *850MB/sec* network throughput. But the
> *best AI3* result was only {*}280MB/sec{*}. Upper results use 3 type of
> messages:
> 1. {*}Test<Type>aMessage{*}: array of 163840 elements (primitive, except
> String) of type <Type>.
> 2. {*}Test<Type>Message{*}: single property (primitive, except String) of
> type <Type>
> 3. {*}Test<Type>20Message{*}: 20 property (primitive, except String) of type
> <Type>
> All the messages were sent in parallel from the single thread with the window
> of 100 messages (right after getting another first ack - the new message were
> sent).
> It was expected, that network utilization low for the very short messages
> (like 1 int or 20 int fields), but in comparison with the iperf3 results, the
> performance of MessagingService for 163KBytes messages was very low. It
> became significantly better only while sending huge array of strings (same
> string "{color:#067d17}Test string to check message service
> performance.{color}").
>
> I've run another butch of tests with 1KB byte[] property in the message in 1
> and 8 threads and without send window at all (each thread sends next message
> after getting the ack for the previous one):
> * *1 thread* and got *37 MBytes/sec*
> * *8 threads* and got *63 MBytes/sec* result.
> So I suppose there is pretty much contention.
> All messages were sent in the followin manner:
> {code:java}
> private void send(ClusterNode target, NetworkMessage msg) {
> messagingService.send(target, msg).handle((v, t) -> {
> if (t != null) {
> LOG.info("Error while sending huge message", t);
> }
> if (time() < timeout) {
> send(target, msg);
> }
> }{code}
>
>
> *
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
