[jira] [Commented] (PROTON-573) proton-c: Messenger appears to have hard-coded address limits of 1024
[ https://issues.apache.org/jira/browse/PROTON-573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992836#comment-13992836 ] ASF subversion and git services commented on PROTON-573: Commit 1593307 from r...@apache.org in branch 'proton/trunk' [ https://svn.apache.org/r1593307 ] PROTON-573: removed hardcoded address limits in messenger proton-c: Messenger appears to have hard-coded address limits of 1024 - Key: PROTON-573 URL: https://issues.apache.org/jira/browse/PROTON-573 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.7 Reporter: Dominic Evans Attachments: 06_arbitrary_length_addresses_in_store.patch, 07_arbitrary_length_addresses_in_messenger.patch The AMQP 1.0 spec permits pretty much arbitrary length link names, but Messenger hard-codes some 1K limits in various places. -- This message was sent by Atlassian JIRA (v6.2#6252)
delivery.setPayload
I have been playing with the API, and there is one change that would make the API clearer IMO. Right now you have to send a delivery, and then call send(bytes) to add Bytes to the delivery what will make it copy data to the Delivery and self expand the buffer. I have played with a change that made it 5% faster than the most optimal way to expand the payload on the Delivery (using the same buffer over and over) And 15% on a brief calculation against creating the buffer every time... but there are cases where this could be a bit worse. Basically I have created an interface called Payload, and added a method setPayload on Delivery. I'm not sure yet how I would implement framing into multiple packages.. but I think it could be done.. this is just a prototyped idea: https://github.com/clebertsuconic/qpid-proton/commit/02abe61fc54911955ddcce77b792a153c5476aef in case you want to fetch the buffer from my git, it's this branch: https://github.com/clebertsuconic/qpid-proton/tree/payload In any case I liked the idea of the setPayload better than sender.send(bytes) to set the payload of a message. Ideas?
Re: codec strategies
On Thu, May 8, 2014 at 9:42 AM, Alan Conway acon...@redhat.com wrote: I vote for DispatchingDecode: it's the simplest, the fastest and is based on a well established parsing pattern with a good track record for performance (SAX). Its not so hard to ignore data in a handler. Writing a handler state machine is a bit more complex than writing a sequence of calls to a stream API, but I think you could encapsulate most of a standard state machine that given a sequence of type codes will fill a sequence of variables. Not sure about the right way to do that in Java performance-wise. Hmm. That might be worth another performance test though - if you did have such tools for making it easy to build handlers, would those tools introduce a penalty that would make the StreamingDecode look more attractive... The biggest difference from an API perspective has to do with data conversions/coersion. Say you're writing a piece of Java code that wants to operate on Java integer or a Java float and doesn't care what the underlying wire type is so long as it can be reasonable converted to that type. In a stream style API you would simple write: int i = decoder.getInt(); float f = decoder.getFloat(); The decoder implementation itself can the be smart enough to convert whatever underlying wire type there might be into the appropriate java type. The SAX API on the other hand will have a distinct callback for byte vs ubyte vs short vs ushort, etc, and it could be quite cumbersome to convert all the different possibilities into the type you actually want to operate on. Put another way, the stream style API is capable of incorporating the desired output type of the user, whereas the SAX style API is not. It might be possible to provide some kind of coercing handler that would help the SAX situation. As you say though it's probably worth checking that something like that would be usable and not introduce its own penalties. --Rafael
[jira] [Commented] (PROTON-579) Typo in proton.php use of $self should be $this
[ https://issues.apache.org/jira/browse/PROTON-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13996702#comment-13996702 ] ASF subversion and git services commented on PROTON-579: Commit 1594309 from r...@apache.org in branch 'proton/trunk' [ https://svn.apache.org/r1594309 ] PROTON-579: fixed typo in proton.php Typo in proton.php use of $self should be $this --- Key: PROTON-579 URL: https://issues.apache.org/jira/browse/PROTON-579 Project: Qpid Proton Issue Type: Bug Components: php-binding Affects Versions: 0.7 Environment: n/a Reporter: Robert Nicholson Assignee: Rafael H. Schloming Priority: Trivial On line 289 of /qpid/proton/trunk/proton-c/bindings/php/proton.php there is a reference to $self that should be $this This results in the following warning: PHP Warning: Creating default object from empty value in /home/nicholsr/bm2/worker.php/proton.php on line 289 -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: delivery.setPayload
I'm not sure this will work as an API. One problem I see with this is that it is circumventing the engine's flow control logic. If you notice there is logic inside send() to update counters on the session. Unless I've missed something your patch doesn't seem to have equivalent logic. This might just be an oversight, but I don't see how you could easily add the same logic since you don't know how many bytes the payload is until much much later on in the control flow of the engine. Can you supply some more detail as to why it got 5% faster? If it was merely avoiding the copy, then I can think of some ways we could avoid that copy without changing the API quite so drastically, e.g. just overload send() to take some sort of releasable buffer reference. FWIW, I think that a good buffer abstraction that we could use everywhere would help a lot. I suspect having distinct abstractions for payload buffers vs encodable buffers vs decodable buffers is just going to result in lots of unnecessary conversions. --Rafael On Tue, May 13, 2014 at 11:19 AM, Clebert Suconic csuco...@redhat.comwrote: I have been playing with the API, and there is one change that would make the API clearer IMO. Right now you have to send a delivery, and then call send(bytes) to add Bytes to the delivery what will make it copy data to the Delivery and self expand the buffer. I have played with a change that made it 5% faster than the most optimal way to expand the payload on the Delivery (using the same buffer over and over) And 15% on a brief calculation against creating the buffer every time... but there are cases where this could be a bit worse. Basically I have created an interface called Payload, and added a method setPayload on Delivery. I'm not sure yet how I would implement framing into multiple packages.. but I think it could be done.. this is just a prototyped idea: https://github.com/clebertsuconic/qpid-proton/commit/02abe61fc54911955ddcce77b792a153c5476aef in case you want to fetch the buffer from my git, it's this branch: https://github.com/clebertsuconic/qpid-proton/tree/payload In any case I liked the idea of the setPayload better than sender.send(bytes) to set the payload of a message. Ideas?
[jira] [Updated] (PROTON-541) Problems with multi-frame deliveries
[ https://issues.apache.org/jira/browse/PROTON-541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafael H. Schloming updated PROTON-541: --- Assignee: Andrew Stitcher Problems with multi-frame deliveries Key: PROTON-541 URL: https://issues.apache.org/jira/browse/PROTON-541 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.6 Reporter: Ted Ross Assignee: Andrew Stitcher I made a modification to messenger.c to configure the maximum frame size on transports (there is no API access to this configuration) and then ran a point-to-point test between a sender and a passive receiver using messages larger than the frame max. The receiver crashes with the following assertion after several dozen messages are transferred successfully. python: /home/ross/svn/proton/proton-c/src/message/message.c:653: pn_message_decode: Assertion `msg bytes size' failed. Another observation: When the max-frame-size is specified only by one side of the connection, there is no negotiation. Frames larger than the specified max are transferred. It is not clear from the specification (2.7.1) whether max-frame-size is intended to be symmetric and apply to transfers in both directions on the connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (PROTON-558) Make friendly protocol field logging optional for low-memory devices
[ https://issues.apache.org/jira/browse/PROTON-558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafael H. Schloming updated PROTON-558: --- Assignee: Andrew Stitcher Make friendly protocol field logging optional for low-memory devices Key: PROTON-558 URL: https://issues.apache.org/jira/browse/PROTON-558 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.6, 0.7 Reporter: Markus Horstmann Assignee: Andrew Stitcher Embedded devices tend to be memory constrained (i.e. 128KB RAM). The FIELDS structure used for protocol logging consumes significant memory. Making it optional would help reduce the memory footprint. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (PROTON-562) More gracefully handle connections to non-1.0 parties
[
https://issues.apache.org/jira/browse/PROTON-562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rafael H. Schloming updated PROTON-562:
---
Assignee: Andrew Stitcher
More gracefully handle connections to non-1.0 parties
-
Key: PROTON-562
URL: https://issues.apache.org/jira/browse/PROTON-562
Project: Qpid Proton
Issue Type: Improvement
Affects Versions: 0.7
Reporter: Justin Ross
Assignee: Andrew Stitcher
Priority: Minor
Right now, attempting a connection from messenger to a 0-10 only qpidd
results in a SASL header mismatch. That's correct but less helpful than it
might be.
{noformat}
jross@localhost bailey$ LD_LIBRARY_PATH=/tmp/tmp.mXaVrrHF8a/lib64
PYTHONPATH=/tmp/tmp.mXaVrrHF8a/lib64/proton/bindings/python/ PN_TRACE_FRM=1
python ~/test.py
[0x2418e30]: - SASL
[0x2418e30]:0 - @sasl-init(65) [mechanism=:ANONYMOUS, initial-response=b]
[0x2418e30]:ERROR[-2] SASL header mismatch: 'AMQP\x01\x01\x00\x0a'
[0x2418e30]: - EOS
[0x2418e30]:ERROR[-2] SASL header mismatch: ''
[0x2418e30]: - EOS
CONNECTION ERROR connection aborted (remote)
{noformat}
See
https://issues.apache.org/jira/browse/PROTON-561?focusedCommentId=13966905page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13966905
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: delivery.setPayload
On May 13, 2014, at 1:46 PM, Rafael Schloming r...@alum.mit.edu wrote: I'm not sure this will work as an API. One problem I see with this is that it is circumventing the engine's flow control logic. If you notice there is logic inside send() to update counters on the session. Unless I've missed something your patch doesn't seem to have equivalent logic. This might just be an oversight, but I don't see how you could easily add the same logic since you don't know how many bytes the payload is until much much later on in the control flow of the engine. as I told you this was just a prototyped idea... it's not in fact updating the window yet.. If this idea is a good idea, we could pursue the idea here. Can you supply some more detail as to why it got 5% faster? If it was merely avoiding the copy, then I can think of some ways we could avoid that copy without changing the API quite so drastically, e.g. just overload send() to take some sort of releasable buffer reference. The encoding is done directly the the FrameWriter::__outputBuffer. I've made a framework where I'm testing the send and it made it somewhat fast than copying the encoding over 1 million messages. On this case it could be a bit more if you encoded a MesasgeImpl on a new buffer every time FWIW, I think that a good buffer abstraction that we could use everywhere would help a lot. I suspect having distinct abstractions for payload buffers vs encodable buffers vs decodable buffers is just going to result in lots of unnecessary conversions. probably.. I was just trying to improve the idea of the payloads. I don't like the send API right now.. I think it would make more sense to set the payload on the delivery than send bytes through sender.
big improvement in memory usage for proton address scale-up
In my original testing for address scale-up with a Proton Messenger based client, I was measuring a memory cost of 115 KB per subscribed address in the client. Now, after Rafi's recent changes, I am seeing a better than 7x improvement, to just under 16 KB per subscribed address. The downside, of course, is that this will make it about 7x harder for me to persuade my boss to buy me a Really Big Box. ( But I'll think of something... ) Thanks for the memory!
[jira] [Updated] (PROTON-546) C++ Windows warnings: conversions and possible loss of data
[ https://issues.apache.org/jira/browse/PROTON-546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafael H. Schloming updated PROTON-546: --- Assignee: Andrew Stitcher (was: Rafael H. Schloming) C++ Windows warnings: conversions and possible loss of data --- Key: PROTON-546 URL: https://issues.apache.org/jira/browse/PROTON-546 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.6 Reporter: Chuck Rolke Assignee: Andrew Stitcher Attachments: PROTON-546-size-warnings.txt Windows has other warnings similar to PROTON-545, especially during 64-bit builds. See attached file for details. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (PROTON-579) Typo in proton.php use of $self should be $this
[ https://issues.apache.org/jira/browse/PROTON-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafael H. Schloming reassigned PROTON-579: -- Assignee: Rafael H. Schloming Typo in proton.php use of $self should be $this --- Key: PROTON-579 URL: https://issues.apache.org/jira/browse/PROTON-579 Project: Qpid Proton Issue Type: Bug Components: php-binding Affects Versions: 0.7 Environment: n/a Reporter: Robert Nicholson Assignee: Rafael H. Schloming Priority: Trivial On line 289 of /qpid/proton/trunk/proton-c/bindings/php/proton.php there is a reference to $self that should be $this This results in the following warning: PHP Warning: Creating default object from empty value in /home/nicholsr/bm2/worker.php/proton.php on line 289 -- This message was sent by Atlassian JIRA (v6.2#6252)
