[jira] [Commented] (PROTON-611) [proton-c] transport buffer increased to peer's max frame size if initial output_size is not enough
[ https://issues.apache.org/jira/browse/PROTON-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14166725#comment-14166725 ] ASF subversion and git services commented on PROTON-611: Commit 1630778 from [~gsim] in branch 'proton/branches/examples' [ https://svn.apache.org/r1630778 ] PROTON-611: Don't allocate frame_max bytes immediately, grow by factors of 2 until you get to the max - else a large frame size can blow away 4Gb of memory! > [proton-c] transport buffer increased to peer's max frame size if initial > output_size is not enough > --- > > Key: PROTON-611 > URL: https://issues.apache.org/jira/browse/PROTON-611 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.7 >Reporter: Dominic Evans >Assignee: Andrew Stitcher > Fix For: 0.8 > > Attachments: 20_fix_bad_malloc_in_transport_produce.patch > > > transport_produce attempts to allocate a negatively sized buffer > As soon as remote_max_frame is set, the code in transport_produce attempts to > increase its buffer immediately up to that size when its initial size isn't > enough. This causes a huge malloc to occur if the remote max frame size is > large and also potentially overflows MAX_INT -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-611) [proton-c] transport buffer increased to peer's max frame size if initial output_size is not enough
[ https://issues.apache.org/jira/browse/PROTON-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093435#comment-14093435 ] Andrew Stitcher commented on PROTON-611: This is issue should now be fixed (with an extended change based on the patch here). However it raises a larger issue of why there are so many different pieces of buffer code in Proton. It seems that this code might better use pn_buffer_t for its buffer management and this would avoid the entire probelm. > [proton-c] transport buffer increased to peer's max frame size if initial > output_size is not enough > --- > > Key: PROTON-611 > URL: https://issues.apache.org/jira/browse/PROTON-611 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.7 >Reporter: Dominic Evans >Assignee: Andrew Stitcher > Fix For: 0.8 > > Attachments: 20_fix_bad_malloc_in_transport_produce.patch > > > transport_produce attempts to allocate a negatively sized buffer > As soon as remote_max_frame is set, the code in transport_produce attempts to > increase its buffer immediately up to that size when its initial size isn't > enough. This causes a huge malloc to occur if the remote max frame size is > large and also potentially overflows MAX_INT -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (PROTON-611) [proton-c] transport buffer increased to peer's max frame size if initial output_size is not enough
[ https://issues.apache.org/jira/browse/PROTON-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093428#comment-14093428 ] ASF subversion and git services commented on PROTON-611: Commit 1617376 from [~astitcher] in branch 'proton/trunk' [ https://svn.apache.org/r1617376 ] PROTON-611: Don't allocate frame_max bytes immediately, grow by factors of 2 until you get to the max - else a large frame size can blow away 4Gb of memory! > [proton-c] transport buffer increased to peer's max frame size if initial > output_size is not enough > --- > > Key: PROTON-611 > URL: https://issues.apache.org/jira/browse/PROTON-611 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.7 >Reporter: Dominic Evans >Assignee: Andrew Stitcher > Attachments: 20_fix_bad_malloc_in_transport_produce.patch > > > transport_produce attempts to allocate a negatively sized buffer > As soon as remote_max_frame is set, the code in transport_produce attempts to > increase its buffer immediately up to that size when its initial size isn't > enough. This causes a huge malloc to occur if the remote max frame size is > large and also potentially overflows MAX_INT -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (PROTON-611) [proton-c] transport buffer increased to peer's max frame size if initial output_size is not enough
[ https://issues.apache.org/jira/browse/PROTON-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045839#comment-14045839 ] Dominic Evans commented on PROTON-611: -- [~astitcher] I can confirm it is a problem in the released 0.7, if you connect a proton-c client to a proton-j server that has set the OpenType idleTimeOut attribute but has not touched the maxFrameSize attribute, then proton-j will advertise a remoteMaxFrameSize of 4GB. In engine-internal.h we seem to have `#define PN_DEFAULT_MAX_FRAME_SIZE (0)` which means than transport->output_size defaults to 16 KB, so as soon as you need to send more than that, transport_produce (in its 0.7 form) would immediately attempted to malloc a new buffer for output_size that was the same as remoteMaxFrameSize, i.e., 4GB! With my patch, we simply use the behaviour that is adopted elsewhere in proton and keep doubling the output_size until we have enough space (up to a maximum of maxFrameSize). > [proton-c] transport buffer increased to peer's max frame size if initial > output_size is not enough > --- > > Key: PROTON-611 > URL: https://issues.apache.org/jira/browse/PROTON-611 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.7 >Reporter: Dominic Evans >Assignee: Andrew Stitcher > Attachments: 20_fix_bad_malloc_in_transport_produce.patch > > > transport_produce attempts to allocate a negatively sized buffer > As soon as remote_max_frame is set, the code in transport_produce attempts to > increase its buffer immediately up to that size when its initial size isn't > enough. This causes a huge malloc to occur if the remote max frame size is > large and also potentially overflows MAX_INT -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (PROTON-611) [proton-c] transport buffer increased to peer's max frame size if initial output_size is not enough
[ https://issues.apache.org/jira/browse/PROTON-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037488#comment-14037488 ] Andrew Stitcher commented on PROTON-611: Dominic, * Can we confirm that this is a problem in the released 0.7 rather than the trunk 0.8 code? * Do you have a specific test case that exhibits this bug? Thanks > [proton-c] transport buffer increased to peer's max frame size if initial > output_size is not enough > --- > > Key: PROTON-611 > URL: https://issues.apache.org/jira/browse/PROTON-611 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.7 >Reporter: Dominic Evans >Assignee: Andrew Stitcher > Attachments: 20_fix_bad_malloc_in_transport_produce.patch > > > transport_produce attempts to allocate a negatively sized buffer > As soon as remote_max_frame is set, the code in transport_produce attempts to > increase its buffer immediately up to that size when its initial size isn't > enough. This causes a huge malloc to occur if the remote max frame size is > large and also potentially overflows MAX_INT -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (PROTON-611) [proton-c] transport buffer increased to peer's max frame size if initial output_size is not enough
[ https://issues.apache.org/jira/browse/PROTON-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037136#comment-14037136 ] Dominic Evans commented on PROTON-611: -- ( similar to PROTON-516 ) > [proton-c] transport buffer increased to peer's max frame size if initial > output_size is not enough > --- > > Key: PROTON-611 > URL: https://issues.apache.org/jira/browse/PROTON-611 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.7 >Reporter: Dominic Evans > Attachments: 20_fix_bad_malloc_in_transport_produce.patch > > > transport_produce attempts to allocate a negatively sized buffer > As soon as remote_max_frame is set, the code in transport_produce attempts to > increase its buffer immediately up to that size when its initial size isn't > enough. This causes a huge malloc to occur if the remote max frame size is > large and also potentially overflows MAX_INT -- This message was sent by Atlassian JIRA (v6.2#6252)
