[
https://issues.apache.org/jira/browse/ARTEMIS-2706?focusedWorklogId=421952&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-421952
]
ASF GitHub Bot logged work on ARTEMIS-2706:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 14/Apr/20 10:39
Start Date: 14/Apr/20 10:39
Worklog Time Spent: 10m
Work Description: gemmellr commented on pull request #3079: ARTEMIS-2706
Use FrameSize to decide when to flush large messages
URL: https://github.com/apache/activemq-artemis/pull/3079#discussion_r408038172
##########
File path:
artemis-protocols/artemis-amqp-protocol/src/main/java/org/apache/activemq/artemis/protocol/amqp/proton/ProtonServerSenderContext.java
##########
@@ -862,7 +862,9 @@ void deliver() {
sender.send(buf.array(), 0, size);
- connection.instantFlush();
+ if (protonSession.session.getOutgoingBytes() >=
protonSession.session.getConnection().getTransport().getOutboundFrameSizeLimit())
{
+ connection.instantFlush();
+ }
Review comment:
I dont think this makes the most sense as there isnt anything to say there
arent bytes for other messages in the session, so using that to govern
behaviour for a particular delivery isnt a reliable way to achieve the desired
behaviour.
The delivery has its own buffer, using that to govern behaviour would seem
the most appropriate, see Delivery.pending() and available() (which both do the
same thing hehe, https://issues.apache.org/jira/browse/PROTON-1409).
Just checking the brokers outbound size limit doesnt allow for the fact the
receivers frame size may be lower than it, so it may be worth calculating the
lower of the two, otherwise it may never flush here. Similarly, the return
slightly higher up may mean it returns without flushing.
The flush here will presumably also be why ARTEMIS-2707 occurred, since the
delivery isnt indicated complete (sender.advance()) or settled (which
implicitly completes) until further down just before another flush, yet would
have already had all its payload sent at this point. With the addition of an
'if outstanding is over frame size' guard here it would become less certain
whether the flush and subsquent empty transfer occurred however, depending then
on whether the threshhold was exceeded after the last send.
I'd note that by creating such a small buffer, you are forcing a similarly
small allocation on every send call as the payload is copied into the delivery.
Using a bigger buffer would seem sensible when appropriate. Perhaps the message
size can be discerned and something suitable selected based on that and the
frame sizing. Or also using NettyReadable and sendNoCopy to avoid the copy.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 421952)
Time Spent: 0.5h (was: 20m)
> outgoing AMQP messages split into an unexpectedly large number of transfer
> frames
> ---------------------------------------------------------------------------------
>
> Key: ARTEMIS-2706
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2706
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: AMQP
> Affects Versions: 2.12.0
> Reporter: Robbie Gemmell
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The broker is splitting larger messages into an unexpectedly large number of
> transfer frames when sending to consuming clients.
> For example, on a test of sending a 1MB message, with the broker having
> advertised its own max frame size of 131072 (a limit which it also uses to
> govern its max outgoing frame sizes, thus not reaching the clients advertised
> purely-coincidental 1MB max frame size), the broker is seen to send transfer
> frames including only up to 1024 bytes of message payload at a time.
> It can be seen that the message being sent by the original producer used
> frames with up to 131050 bytes of payload within the transfer frames, in
> keeping with the brokers 131072 max frame size (which includes the frame
> heading and performative itself):
> {noformat}
> [417713843:1] -> Transfer{handle=0, deliveryId=0, deliveryTag=\x00,
> messageFormat=0, settled=false, more=true, rcvSettleMode=null, state=null,
> resume=false, aborted=false, batchable=false} (131050)
> "\x00Sr\xc1)\x04\xa3\x0ex-opt-jms-destQ\x00\xa3\x12x-opt-jms-msg-typeQ\x03\x00Ss\xd0\x00\x00\x00r\x00\x00\x00\x0a\xa1/ID:947e3b66-0d17-4492-8d34-de9050aa1533:1:1:1-1@\xa1\x12testSend1MBMessage@@@\xa3\x18application/octet-stream@@\x83\x00\x00\x01qt\x0b\xe2\x86\x00Su\xb0\x00\x10\x00\x00\x98"\xeb\xfd\x84\xe0\xf8\xb0m9S\x97\xb4\x85iZ_\xa2~\x0e\x00B\x87\x9f\xa46\xe2\xfc\xd0\x9c\xf25[H\x82\x1f3x'\xcf\x0f\xab\x0fq0U^b\x1b\xd2d\x1c\xdcG\xcdLj\xf8!-\xef\xb0nP\x04y\xf7\xb8\xaa\x0d\x09\x83\xba\xd57\xef.\xb1\x1aM\xb9#\xdc\x09\xa5\xa1\xacC\xe1\x18\x14Bln\xcfSu
>
> \xc3\x1b\xd73\xad\xe1A\x81\x17\x13V\xaf\xa9\xa6G\xe1\x82Y{\xcd\x07\xfd\xdb\x11\xf2\xbe\xe6\xbd\xf4\xe9Xr\x18\x00\x12:\x88\xa1\xd0\xe5\xe3\xc1\xe5l\x9c\x8c\x99\xb6&\x01'p\xdb\xb8*\xa3\x87\xb2*on\x8f\x82\xa5j\xfc\xa8\xbd<\x06E\xe9\x888\x1d\x8e1\x17\xc3\x0cH1\x14]\x9f.ZU\xc2\x0eiG\x12T\x16\x99z\xab\xee5=r\xe6\x08\x9cZ\xde\xee\xe9\x93\x86#Y\xef$\xd8\xc9,\x04\xcf\xad\xed\xdeh<\xac\xe4\x82n\xf1k\x16\xcb3\xf0E\xf6
> \xf5\xf0 \x04=g;#\xc1\xaf\xdb\xaa=\x93Bi^\xb0N\xd2\x83\x05`"...(truncated)
> {noformat}
> The same message going back out to the consumer, only uses 1024 bytes of
> payload despite the brokers 131072 max frame size (which it applis to
> outgoing frames too) being the governing limit in the scenario:
> {noformat}
> [1933829960:1] -> Transfer{handle=1, deliveryId=0, deliveryTag=\x00,
> messageFormat=0, settled=false, more=true, rcvSettleMode=null, state=null,
> resume=false, aborted=false, batchable=false} (1024)
> "\x00Sr\xc1)\x04\xa3\x0ex-opt-jms-destQ\x00\xa3\x12x-opt-jms-msg-typeQ\x03\x00Ss\xd0\x00\x00\x00r\x00\x00\x00\x0a\xa1/ID:947e3b66-0d17-4492-8d34-de9050aa1533:1:1:1-1@\xa1\x12testSend1MBMessage@@@\xa3\x18application/octet-stream@@\x83\x00\x00\x01qt\x0b\xe2\x86\x00Su\xb0\x00\x10\x00\x00\x98"\xeb\xfd\x84\xe0\xf8\xb0m9S\x97\xb4\x85iZ_\xa2~\x0e\x00B\x87\x9f\xa46\xe2\xfc\xd0\x9c\xf25[H\x82\x1f3x'\xcf\x0f\xab\x0fq0U^b\x1b\xd2d\x1c\xdcG\xcdLj\xf8!-\xef\xb0nP\x04y\xf7\xb8\xaa\x0d\x09\x83\xba\xd57\xef.\xb1\x1aM\xb9#\xdc\x09\xa5\xa1\xacC\xe1\x18\x14Bln\xcfSu
>
> \xc3\x1b\xd73\xad\xe1A\x81\x17\x13V\xaf\xa9\xa6G\xe1\x82Y{\xcd\x07\xfd\xdb\x11\xf2\xbe\xe6\xbd\xf4\xe9Xr\x18\x00\x12:\x88\xa1\xd0\xe5\xe3\xc1\xe5l\x9c\x8c\x99\xb6&\x01'p\xdb\xb8*\xa3\x87\xb2*on\x8f\x82\xa5j\xfc\xa8\xbd<\x06E\xe9\x888\x1d\x8e1\x17\xc3\x0cH1\x14]\x9f.ZU\xc2\x0eiG\x12T\x16\x99z\xab\xee5=r\xe6\x08\x9cZ\xde\xee\xe9\x93\x86#Y\xef$\xd8\xc9,\x04\xcf\xad\xed\xdeh<\xac\xe4\x82n\xf1k\x16\xcb3\xf0E\xf6
> \xf5\xf0 \x04=g;#\xc1\xaf\xdb\xaa=\x93Bi^\xb0N\xd2\x83\x05`"...(truncated)
> {noformat}
> Whilst this is legal protocol behaviour it is fairly unexpected, and could be
> rather inefficient depending on the size of the message and a receiving
> clients behaviour reconstituting the completed message.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
