Alan Conway commented on PROTON-584:

Not necessary but need to think carefully about how/when to reduce the buffer 
size. Re-allocating a buffer for each message is expensive, so keeping a buffer 
that is "big enough for most messages" is a performance win. The buffer also 
can hold more than one frame, and reading/writing multiple frames in a single 
read is an important performance win. The current scheme works well for streams 
of messages that don't have "exceptionallly large" outlier messages - so the 
trick will be to shrink buffers that have grown because of one "exceptionally 
large" message without  penalizing the "normal sized" message stream by with 
very small reads or frequent re-allocations.

What are the stats for your message traffic? How much do "exceptional" messages 
diverge from "normal" messages in size? Would a statically configured 
min/normal-max settings work (never reduce below min, if you go over normal-max 
shrink back to normal-max ASAP)? Adaptive schemes based on past 
traffic/buffer-size/max-read might also be worth considering.

> Proton-c transport reserves large buffers for brief use
> -------------------------------------------------------
>                 Key: PROTON-584
>                 URL: https://issues.apache.org/jira/browse/PROTON-584
>             Project: Qpid Proton
>          Issue Type: Improvement
>          Components: proton-c
>    Affects Versions: 0.8
>            Reporter: Cliff Jansen
>            Assignee: Alan Conway
>              Labels: perf
>             Fix For: proton-c-0.19.0
> When processing transfer frames for incoming messages, Proton requires a 
> temporary buffer holding the whole transfer frame briefly in contiguous space 
> in the transport before moving it to the engine proper which holds the 
> content in a separate memory area.
> pn_transport_capacity grows the buffer in the non-ssl case 
> (transport->input_buf), and openssl.c largely duplicates the code (including 
> the comment about "no limit") for ssl->inbuf for ssl connections.
> Either way, a large message will trigger reserving a similarly large buffer 
> for the rest of the life of the connection.
> Is it necessary for the transport to buffer the whole message body and hang 
> on to that memory?

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

Reply via email to