Re: How about docs at top level?
I'm happy with the location although to increase consistency with other projects I have a mild preference for either docs or doc. The former seems to be the most common in other open source projects, and the latter is the name used by Qpid. Phil On 4 March 2013 19:53, Michael Goulish mgoul...@redhat.com wrote: I'm planning to start checking my docs into the proton tree soon. I was assuming I would just put them at top level, i.e. qpid-proton/documentation Anybody care to agree, object, counter-offer, praise, complain, argue, question, or muse ?
[jira] [Created] (PROTON-258) Delivery interface should expose receiver settlement mode
Keith Wall created PROTON-258: - Summary: Delivery interface should expose receiver settlement mode Key: PROTON-258 URL: https://issues.apache.org/jira/browse/PROTON-258 Project: Qpid Proton Issue Type: Improvement Components: proton-c, proton-j Reporter: Keith Wall AMQP 1.0 permits the receiver settlement mode to be overridden for an individual transfer. This overrides the receiver settlement mode that was agreed at link establishment time for the scope of a single delivery only. See: http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-transport-v1.0-os.html#type-transfer The Proton (Engine) API does not currently expose this ability to the user. The Delivery interface should be augmented so that: * as a user of Sender, I can override the receiver settlement mode for an outgoing Delivery. * as a user of Receiver, I can determine the settlement mode in force an incoming Delivery. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
proton(-j) threading model
I've been working with the proton-j engine recently and want to clarify the threading model. The Proton web site [1] says Proton is architected to be usable with any threading model as well as with non threaded applications. Turning to the implementation, I've heard that the proton-j engine is intended to be used by one thread at once. This sounds reasonable, but I want to clarify what one thread at once really means in this context. I believe we should say something like this in the proton-j documentation: Proton engine classes are not synchronized. If multiple threads access a Proton engine object concurrently, external synchronization must be used. Ditto for the Message package. Any improvements, objections etc? Out of interest, what is the threading model of Messenger? And how about proton-c? Thanks, Phil [1] http://qpid.apache.org/proton/
Publishing javadoc to site
Phil and I are currently working on PROTON-256 - Improve Engine API Javadoc. As part of this task, I'd like to start publishing the resulting javadoc to qpid.apache.org and link from http://qpid.apache.org/proton/documentation. My intention is to automate this process using Apache Buildbot http://ci.apache.org/buildbot.html. (If I understand the process correctly, we can give the builds@a.o a script that can generate our docs and publish the result into the tree). I notice we already publish python-api documentation. http://qpid.apache.org/proton/api-doc/ Has this been done manually, or has this already been automated? How? If not, is this something we'd like to see automated too? If someone can tell the steps taken to generate and publish, I'll include those steps in the script I submit to builds@. Any thoughts? cheers, Keith.
[jira] [Created] (PROTON-259) proton-j engine operations do not check if the endpoint is already closed
Philip Harvey created PROTON-259: Summary: proton-j engine operations do not check if the endpoint is already closed Key: PROTON-259 URL: https://issues.apache.org/jira/browse/PROTON-259 Project: Qpid Proton Issue Type: Bug Components: proton-j Affects Versions: 0.3 Reporter: Philip Harvey Operations such as SessionImpl#receiver(receiverName) and LinkImpl#delivery do not check the state of the endpoint. I believe they should throw an exception. Moreover, I notice that the following sequence of calls cause the transport to produce no output: --- sender.close; // deliberately use a closed sender. // The presence of this line causes the subsequent transport.output to produce zero bytes. sender.delivery(deliveryTag); transport.output(); --- -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: proton(-j) threading model
On Tue, Mar 5, 2013 at 1:52 AM, Phil Harvey p...@philharveyonline.comwrote: I've been working with the proton-j engine recently and want to clarify the threading model. The Proton web site [1] says Proton is architected to be usable with any threading model as well as with non threaded applications. Turning to the implementation, I've heard that the proton-j engine is intended to be used by one thread at once. This sounds reasonable, but I want to clarify what one thread at once really means in this context. What this really means is that a bound transport/connection pair form a single data structure and if you want to access it concurrently from multiple threads you need to protect it with your own mutex. I believe we should say something like this in the proton-j documentation: Proton engine classes are not synchronized. If multiple threads access a Proton engine object concurrently, external synchronization must be used. That sounds reasonable to me. You mind also want additional language defining a single engine instance as a bound transport/connection pair. Ditto for the Message package. Any improvements, objections etc? Out of interest, what is the threading model of Messenger? And how about proton-c? It's the same for proton-c and for Messenger. --Rafael
[jira] [Created] (PROTON-260) Messenger Documentation
michael goulish created PROTON-260: -- Summary: Messenger Documentation Key: PROTON-260 URL: https://issues.apache.org/jira/browse/PROTON-260 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.5 Reporter: michael goulish Write documentation for the Proton Messenger interface, to include: introduction API explanations theory of operation example programs programming idioms tutorials quickstarts troubleshooting -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [RESULT] [VOTE] 0.4 RC3
On 2/25/13 9:44 PM, Rafael Schloming wrote: The vote carries with 5 +1's and 0 -1's. I'll upload the artifacts shortly. --Rafael On Thu, Feb 21, 2013 at 1:54 PM, Rafael Schlomingr...@alum.mit.edu wrote: Hi everyone, I've spun an RC3 with fixes for the two immediately soluble issues that turned up with RC2. Please check it out and cast your vote. Source is here: - http://people.apache.org/~rhs/qpid-proton-0.4rc3/ Java binaries are here: - https://repository.apache.org/content/repositories/orgapacheqpid-292/ Changes since RC2: PROTON-230, PROTON-246: Copying nested data fails in some cases PROTON-245: make swig skip pn_dtag for compatibility with older versions of swig [ ] Yes, I believe we should make 0.4 RC3 into 0.4 final [ ] No, because ... Where in the repository can I see which revision/branch/tag are the release candidates? Thanks, Bozzo
[jira] [Created] (PROTON-261) pn_delivery_local_state and pn_delivery_remote_state can return a zero value (outside that defined by the pn_disposition_t enum)
Keith Wall created PROTON-261:
-
Summary: pn_delivery_local_state and pn_delivery_remote_state can
return a zero value (outside that defined by the pn_disposition_t enum)
Key: PROTON-261
URL: https://issues.apache.org/jira/browse/PROTON-261
Project: Qpid Proton
Issue Type: Bug
Components: proton-c
Affects Versions: 0.5
Reporter: Keith Wall
In proton-c, pn_delivery_local_state and pn_delivery_remote_state are defined
as returning an pn_disposition_t:
PN_EXTERN pn_disposition_t pn_delivery_local_state(pn_delivery_t *delivery);
PN_EXTERN pn_disposition_t pn_delivery_remote_state(pn_delivery_t *delivery);
with pn_disposition_t defined as:
typedef enum pn_disposition_t {
PN_RECEIVED=1,
PN_ACCEPTED=2,
PN_REJECTED=3,
PN_RELEASED=4,
PN_MODIFIED=5
} pn_disposition_t;
However, there are circumstances when the proton-c implementation returns a
value outside this enumeration - value 0 - coming for the struct initialisation
in pn_delivery(). What does this value mean and how is an application supposed
to react to this value? Are there an implicit restrictions regarding when an
application may call pn_delivery_local_state/pn_delivery_remote_state?
btw. This question arises from work on PROTON-257. Proton-jni (Swig generated
code) is failing to decode the enum when testing the state of a newly created
Delivery.
test(org.apache.qpid.proton.systemtests.ProtonEngineExampleTest) Time elapsed:
0.109 sec ERROR!
java.lang.IllegalArgumentException: No enum class
org.apache.qpid.proton.jni.pn_disposition_t with value 0
at
org.apache.qpid.proton.jni.pn_disposition_t.swigToEnum(pn_disposition_t.java:32)
at
org.apache.qpid.proton.jni.Proton.pn_delivery_local_state(Proton.java:624)
at
org.apache.qpid.proton.engine.jni.JNIDelivery.getLocalState(JNIDelivery.java:100)
at
org.apache.qpid.proton.systemtests.ProtonEngineExampleTest.test(ProtonEngineExampleTest.java:190)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PROTON-261) pn_delivery_local_state and pn_delivery_remote_state can return a zero value (outside that defined by the pn_disposition_t enum)
[
https://issues.apache.org/jira/browse/PROTON-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Keith Wall updated PROTON-261:
--
Description:
In proton-c, pn_delivery_local_state and pn_delivery_remote_state are defined
as returning an pn_disposition_t:
PN_EXTERN pn_disposition_t pn_delivery_local_state(pn_delivery_t *delivery);
PN_EXTERN pn_disposition_t pn_delivery_remote_state(pn_delivery_t *delivery);
with pn_disposition_t defined as:
typedef enum pn_disposition_t {
PN_RECEIVED=1,
PN_ACCEPTED=2,
PN_REJECTED=3,
PN_RELEASED=4,
PN_MODIFIED=5
} pn_disposition_t;
However, there are circumstances when the proton-c implementation returns a
value outside this enumeration - value 0 - coming for the struct initialisation
in pn_delivery(). What does this value mean and how is an application supposed
to react to this value? Are there an implicit restrictions regarding when an
application may call pn_delivery_local_state/pn_delivery_remote_state?
btw. This question arises from work on PROTON-257. Proton-jni (Swig generated
code) is failing to decode the enum when testing the state of a newly created
Delivery.
test(org.apache.qpid.proton.systemtests.ProtonEngineExampleTest) Time elapsed:
0.109 sec ERROR!
java.lang.IllegalArgumentException: No enum class
org.apache.qpid.proton.jni.pn_disposition_t with value 0
at
org.apache.qpid.proton.jni.pn_disposition_t.swigToEnum(pn_disposition_t.java:32)
at
org.apache.qpid.proton.jni.Proton.pn_delivery_local_state(Proton.java:624)
at
org.apache.qpid.proton.engine.jni.JNIDelivery.getLocalState(JNIDelivery.java:100)
at
org.apache.qpid.proton.systemtests.ProtonEngineExampleTest.test(ProtonEngineExampleTest.java:190)
was:
In proton-c, pn_delivery_local_state and pn_delivery_remote_state are defined
as returning an pn_disposition_t:
PN_EXTERN pn_disposition_t pn_delivery_local_state(pn_delivery_t *delivery);
PN_EXTERN pn_disposition_t pn_delivery_remote_state(pn_delivery_t *delivery);
with pn_disposition_t defined as:
typedef enum pn_disposition_t {
PN_RECEIVED=1,
PN_ACCEPTED=2,
PN_REJECTED=3,
PN_RELEASED=4,
PN_MODIFIED=5
} pn_disposition_t;
However, there are circumstances when the proton-c implementation returns a
value outside this enumeration - value 0 - coming for the struct initialisation
in pn_delivery(). What does this value mean and how is an application supposed
to react to this value? Are there an implicit restrictions regarding when an
application may call pn_delivery_local_state/pn_delivery_remote_state?
btw. This question arises from work on PROTON-257. Proton-jni (Swig generated
code) is failing to decode the enum when testing the state of a newly created
Delivery.
test(org.apache.qpid.proton.systemtests.ProtonEngineExampleTest) Time elapsed:
0.109 sec ERROR!
java.lang.IllegalArgumentException: No enum class
org.apache.qpid.proton.jni.pn_disposition_t with value 0
at
org.apache.qpid.proton.jni.pn_disposition_t.swigToEnum(pn_disposition_t.java:32)
at
org.apache.qpid.proton.jni.Proton.pn_delivery_local_state(Proton.java:624)
at
org.apache.qpid.proton.engine.jni.JNIDelivery.getLocalState(JNIDelivery.java:100)
at
org.apache.qpid.proton.systemtests.ProtonEngineExampleTest.test(ProtonEngineExampleTest.java:190)
pn_delivery_local_state and pn_delivery_remote_state can return a zero value
(outside that defined by the pn_disposition_t enum)
Key: PROTON-261
URL: https://issues.apache.org/jira/browse/PROTON-261
Project: Qpid Proton
Issue Type: Bug
Components: proton-c
Affects Versions: 0.5
Reporter: Keith Wall
In proton-c, pn_delivery_local_state and pn_delivery_remote_state are defined
as returning an pn_disposition_t:
PN_EXTERN pn_disposition_t pn_delivery_local_state(pn_delivery_t *delivery);
PN_EXTERN pn_disposition_t pn_delivery_remote_state(pn_delivery_t *delivery);
with pn_disposition_t defined as:
typedef enum pn_disposition_t {
PN_RECEIVED=1,
PN_ACCEPTED=2,
PN_REJECTED=3,
PN_RELEASED=4,
PN_MODIFIED=5
} pn_disposition_t;
However, there are circumstances when the proton-c implementation returns a
value outside this enumeration - value 0 - coming for the struct
initialisation in pn_delivery(). What does this value mean and how is an
application supposed to react to this value? Are there an implicit
restrictions regarding when an application may call
pn_delivery_local_state/pn_delivery_remote_state?
btw. This question arises from work on PROTON-257. Proton-jni (Swig
generated code) is failing to decode the enum when testing the state of a
newly created Delivery.
[jira] [Updated] (PROTON-225) Redesign Transport interface such that Transport owns the in/out buffers rather than its client
[ https://issues.apache.org/jira/browse/PROTON-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keith Wall updated PROTON-225: -- Description: This issue is intended to cover the Transport API redesign proposed on the mailing list (http://qpid.2158936.n2.nabble.com/transport-interface-changes-td7588099.html) as part of discussions around PROTON-222. The redesign is being tracked under this new because we probably want to implement it on a different timescale to the PROTON-222 bug fix. When refactoring the Java implementation, we should consider if the point when the sent/received protocol logging is done should be changed. It should also address the difference around the use of PN_EOS (-1) that proton-c uses to signal that a) the engine will accept no further input and b) produce no further output. was: This issue is intended to cover the Transport API redesign proposed on the mailing list (http://qpid.2158936.n2.nabble.com/transport-interface-changes-td7588099.html) as part of discussions around PROTON-222. The redesign is being tracked under this new because we probably want to implement it on a different timescale to the PROTON-222 bug fix. When refactoring the Java implementation, we should consider if the point when the sent/received protocol logging is done should be changed. Redesign Transport interface such that Transport owns the in/out buffers rather than its client --- Key: PROTON-225 URL: https://issues.apache.org/jira/browse/PROTON-225 Project: Qpid Proton Issue Type: Improvement Affects Versions: 0.3 Reporter: Philip Harvey Assignee: Ken Giusti Fix For: 0.5 This issue is intended to cover the Transport API redesign proposed on the mailing list (http://qpid.2158936.n2.nabble.com/transport-interface-changes-td7588099.html) as part of discussions around PROTON-222. The redesign is being tracked under this new because we probably want to implement it on a different timescale to the PROTON-222 bug fix. When refactoring the Java implementation, we should consider if the point when the sent/received protocol logging is done should be changed. It should also address the difference around the use of PN_EOS (-1) that proton-c uses to signal that a) the engine will accept no further input and b) produce no further output. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PROTON-260) Messenger Documentation
[ https://issues.apache.org/jira/browse/PROTON-260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] michael goulish updated PROTON-260: --- Description: Write documentation for the Proton Messenger interface, to include: introduction API explanations theory of operation example programs programming idioms tutorials quickstarts troubleshooting Documents should use MarkDown markup language. was: Write documentation for the Proton Messenger interface, to include: introduction API explanations theory of operation example programs programming idioms tutorials quickstarts troubleshooting Messenger Documentation --- Key: PROTON-260 URL: https://issues.apache.org/jira/browse/PROTON-260 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.5 Reporter: michael goulish Write documentation for the Proton Messenger interface, to include: introduction API explanations theory of operation example programs programming idioms tutorials quickstarts troubleshooting Documents should use MarkDown markup language. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
can't assign Jira
I created PROTON-260, but I can't seem to assign it to myself. Do I need some new permissions to do this?
Re: can't assign Jira
You should be able to assign JIRAs to yourself now -- Rob On 5 March 2013 16:50, Michael Goulish mgoul...@redhat.com wrote: I created PROTON-260, but I can't seem to assign it to myself. Do I need some new permissions to do this?
[jira] [Created] (PROTON-263) JNIDelivery.isPartial should delegate to pn_delivery_partial
Keith Wall created PROTON-263: - Summary: JNIDelivery.isPartial should delegate to pn_delivery_partial Key: PROTON-263 URL: https://issues.apache.org/jira/browse/PROTON-263 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.4 Reporter: Keith Wall Fix For: 0.5 The JNI Delivery interface currently throws a ProtonUnsupportedOperation, but should simple delegate to Proton-C pn_delivery_partial method. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (PROTON-263) JNIDelivery.isPartial should delegate to pn_delivery_partial
[ https://issues.apache.org/jira/browse/PROTON-263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keith Wall reassigned PROTON-263: - Assignee: Keith Wall JNIDelivery.isPartial should delegate to pn_delivery_partial Key: PROTON-263 URL: https://issues.apache.org/jira/browse/PROTON-263 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.4 Reporter: Keith Wall Assignee: Keith Wall Fix For: 0.5 The JNI Delivery interface currently throws a ProtonUnsupportedOperation, but should simple delegate to Proton-C pn_delivery_partial method. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (PROTON-264) Proton-J and Proton-C transports make inconsistent use of return values from methods input and output
Keith Wall created PROTON-264: - Summary: Proton-J and Proton-C transports make inconsistent use of return values from methods input and output Key: PROTON-264 URL: https://issues.apache.org/jira/browse/PROTON-264 Project: Qpid Proton Issue Type: Bug Components: proton-c, proton-j Affects Versions: 0.4 Reporter: Keith Wall Assignee: Keith Wall Priority: Minor Fix For: 0.5 This defect refers to the transport interface already in the process of being deprecated by PROTON-225. The scope of PROTON-225 has been increased (=noted in Jira) to ensure that this defect is not carried forward to the new interface and we end up with consistent behaviour. Currently, in Proton-C pn_transport_output() returns PN_EOS (=-1) to signal that the Transport will produce no further output, and pn_transport_input() returns PN_EOS to signal that the Transport will accept no further input. However, Proton-J, output always returns the number bytes output (even if zero), and input returns the number of bytes accepted (again even if zero). This difference presents a problem to tests utilising the Proton-JNI binding. As a temporary workaround, JNITransport class will changed to hide this difference from the caller. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PROTON-264) Proton-J and Proton-C transports make inconsistent use of return values within methods input and output
[ https://issues.apache.org/jira/browse/PROTON-264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keith Wall updated PROTON-264: -- Summary: Proton-J and Proton-C transports make inconsistent use of return values within methods input and output (was: Proton-J and Proton-C transports make inconsistent use of return values from methods input and output) Proton-J and Proton-C transports make inconsistent use of return values within methods input and output --- Key: PROTON-264 URL: https://issues.apache.org/jira/browse/PROTON-264 Project: Qpid Proton Issue Type: Bug Components: proton-c, proton-j Affects Versions: 0.4 Reporter: Keith Wall Assignee: Keith Wall Priority: Minor Fix For: 0.5 This defect refers to the transport interface already in the process of being deprecated by PROTON-225. The scope of PROTON-225 has been increased (=noted in Jira) to ensure that this defect is not carried forward to the new interface and we end up with consistent behaviour. Currently, in Proton-C pn_transport_output() returns PN_EOS (=-1) to signal that the Transport will produce no further output, and pn_transport_input() returns PN_EOS to signal that the Transport will accept no further input. However, Proton-J, output always returns the number bytes output (even if zero), and input returns the number of bytes accepted (again even if zero). This difference presents a problem to tests utilising the Proton-JNI binding. As a temporary workaround, JNITransport class will changed to hide this difference from the caller. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: can't assign Jira
Thanks! It worked. - Original Message - You should be able to assign JIRAs to yourself now -- Rob On 5 March 2013 16:50, Michael Goulish mgoul...@redhat.com wrote: I created PROTON-260, but I can't seem to assign it to myself. Do I need some new permissions to do this?
[jira] [Created] (PROTON-265) Change Message#decode to signal inability to decode entire buffer via exception rather than return value
Keith Wall created PROTON-265: - Summary: Change Message#decode to signal inability to decode entire buffer via exception rather than return value Key: PROTON-265 URL: https://issues.apache.org/jira/browse/PROTON-265 Project: Qpid Proton Issue Type: Improvement Components: proton-j Reporter: Keith Wall Assignee: Keith Wall Fix For: 0.5 Change Message#decode signature to throw an exception in the case where the decoder fails to decode all the bytes in the input buffer, rather than returning the number of bytes actually decoded. This will make the Java implementation more closely follow the Proton-C. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PROTON-259) proton-j engine operations do not check if the endpoint is already closed
[ https://issues.apache.org/jira/browse/PROTON-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13593726#comment-13593726 ] Rafael H. Schloming commented on PROTON-259: I wouldn't say it's a bug to create deliveries and/or receivers irrespective of endpoint state. The endpoint state captures the state of the remote/local setup handshake, but it's really kind of orthogonal to the data structures that capture the state of messages in transit. Certainly for recovery you need to create deliveries on links prior to opening them and you need to be able to fully access and examine the link data structure after the endpoint is closed. Put another way, the CLOSED state is really about controlling what you want to communicate to the remote endpoint rather than controlling how you want to use the endpoint data structures locally. proton-j engine operations do not check if the endpoint is already closed - Key: PROTON-259 URL: https://issues.apache.org/jira/browse/PROTON-259 Project: Qpid Proton Issue Type: Bug Components: proton-j Affects Versions: 0.3 Reporter: Philip Harvey Operations such as SessionImpl#receiver(receiverName) and LinkImpl#delivery do not check the state of the endpoint. I believe they should throw an exception. Moreover, I notice that the following sequence of calls cause the transport to produce no output: --- sender.close; // deliberately use a closed sender. // The presence of this line causes the subsequent transport.output to produce zero bytes. sender.delivery(deliveryTag); transport.output(); --- -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
put vs. send
quoth Rafi: The semantics of pn_messenger_put allow it to send if it can do so without blocking. So, am I understanding correctly? -- I should be able to get messages from my sender to my receiver just by calling put() -- if the receiver is ready to receive? The only transmission difference between put() and send() is that send() will actually block until they're all sent (or timeout hits). put() should get rid of all the messages that aren't blocked, and leave all that are. . . . Because what I'm seeing is -- with my receiver hanging in recv(), I put 5 messages. Sender sits there for a while. No messages arrive at receiver. Then sender calls send() -- and all 5 messages arrive at the receiver. This is true whether on the receiver side, I use pn_messenger_recv ( messenger, 100 ); pn_messenger_recv ( messenger, 5 ); pn_messenger_recv ( messenger, 1 ); or pn_messenger_recv ( messenger, -1 ); That's why it seemed two-stage to me. put() seems to gets them staged, send() seems to shove them out the door. No? Or is this a bug?
Re: How about docs at top level?
I'm +1 on docs. It would be consistent with examples, tests, and tools. --Rafael On Tue, Mar 5, 2013 at 1:01 AM, Phil Harvey p...@philharveyonline.comwrote: I'm happy with the location although to increase consistency with other projects I have a mild preference for either docs or doc. The former seems to be the most common in other open source projects, and the latter is the name used by Qpid. Phil On 4 March 2013 19:53, Michael Goulish mgoul...@redhat.com wrote: I'm planning to start checking my docs into the proton tree soon. I was assuming I would just put them at top level, i.e. qpid-proton/documentation Anybody care to agree, object, counter-offer, praise, complain, argue, question, or muse ?
Re: How about docs at top level?
+1 Rajith On Tue, Mar 5, 2013 at 1:48 PM, Rafael Schloming r...@alum.mit.edu wrote: I'm +1 on docs. It would be consistent with examples, tests, and tools. --Rafael On Tue, Mar 5, 2013 at 1:01 AM, Phil Harvey p...@philharveyonline.comwrote: I'm happy with the location although to increase consistency with other projects I have a mild preference for either docs or doc. The former seems to be the most common in other open source projects, and the latter is the name used by Qpid. Phil On 4 March 2013 19:53, Michael Goulish mgoul...@redhat.com wrote: I'm planning to start checking my docs into the proton tree soon. I was assuming I would just put them at top level, i.e. qpid-proton/documentation Anybody care to agree, object, counter-offer, praise, complain, argue, question, or muse ?
Re: put vs. send
On 03/05/2013 02:01 PM, Rafael Schloming wrote: On Tue, Mar 5, 2013 at 10:42 AM, Michael Goulish mgoul...@redhat.comwrote: So, am I understanding correctly? -- I should be able to get messages from my sender to my receiver just by calling put() -- if the receiver is ready to receive? Not necessarily, the receiver being ready just means you are unblocked on AMQP level flow control. You could also potentially block on the socket write (i.e. TCP level flow control). You need to be unblocked on both for put to succeed. Certainly there is no TCP flow control happening in Mick's scenario. What I said was put is *allowed* to send optimistically, not that it is required to. It actually did send optimistically in a previous version of the code, however I commented that line out. I would say the documented semantics of put and send should allow the implementation the flexibility to do any of the following: 1) optimistically transmit whatever it can everytime so long as it doesn't block 2) never bother transmitting anything until you force it to by calling send 3) anything in between the first two, e.g. magically transmit once you've put enough messages to reach the optimal batch size The reason for the behaviour you are observing is that we currently do option 2 in the C impl, however we've done option 1 in the past (and I think we do option 1 still in the Java impl), and we will probably do option 3 in the future. If this is the case, then Mick's original view is correct. The application must assume that messages will not ever be sent unless send is called. There is no flowing, pipelined, non-blocking producer. --Rafael
Re: put vs. send
On Tue, Mar 5, 2013 at 2:01 PM, Rafael Schloming r...@alum.mit.edu wrote: On Tue, Mar 5, 2013 at 10:42 AM, Michael Goulish mgoul...@redhat.comwrote: quoth Rafi: The semantics of pn_messenger_put allow it to send if it can do so without blocking. So, am I understanding correctly? -- I should be able to get messages from my sender to my receiver just by calling put() -- if the receiver is ready to receive? Not necessarily, the receiver being ready just means you are unblocked on AMQP level flow control. You could also potentially block on the socket write (i.e. TCP level flow control). You need to be unblocked on both for put to succeed. The only transmission difference between put() and send() is that send() will actually block until they're all sent (or timeout hits). put() should get rid of all the messages that aren't blocked, and leave all that are. . . . Because what I'm seeing is -- with my receiver hanging in recv(), I put 5 messages. Sender sits there for a while. No messages arrive at receiver. Then sender calls send() -- and all 5 messages arrive at the receiver. This is true whether on the receiver side, I use pn_messenger_recv ( messenger, 100 ); pn_messenger_recv ( messenger, 5 ); pn_messenger_recv ( messenger, 1 ); or pn_messenger_recv ( messenger, -1 ); That's why it seemed two-stage to me. put() seems to gets them staged, send() seems to shove them out the door. No? Or is this a bug? What I said was put is *allowed* to send optimistically, not that it is required to. It actually did send optimistically in a previous version of the code, however I commented that line out. I would say the documented semantics of put and send should allow the implementation the flexibility to do any of the following: 1) optimistically transmit whatever it can everytime so long as it doesn't block 2) never bother transmitting anything until you force it to by calling send 3) anything in between the first two, e.g. magically transmit once you've put enough messages to reach the optimal batch size The reason for the behaviour you are observing is that we currently do option 2 in the C impl, however we've done option 1 in the past (and I think we do option 1 still in the Java impl), and we will probably do option 3 in the future. This is a good explanation that we need to put in the docs, as Application developers certainly need to know how it behaves. If one were to use the current C impl, it certainly gives the impression that put() is meant to write messages into your internal buffer and send() will actually write it to the wire. Unfortunately some applications will depend on this behaviour, even though it's not advisable If we are to change from say #2 to #1 or even #3 we need to release note it prominently. I think the best solution is to make this behaviour configurable, and advertise the default very prominently. This way application developers will know exactly what they are getting instead of us making changes underneath. Rajith --Rafael
Re: put vs. send
On 03/05/2013 02:14 PM, Rajith Attapattu wrote: This is a good explanation that we need to put in the docs, as Application developers certainly need to know how it behaves. If one were to use the current C impl, it certainly gives the impression that put() is meant to write messages into your internal buffer and send() will actually write it to the wire. Unfortunately some applications will depend on this behaviour, even though it's not advisable If we are to change from say #2 to #1 or even #3 we need to release note it prominently. I think the best solution is to make this behaviour configurable, and advertise the default very prominently. This way application developers will know exactly what they are getting instead of us making changes underneath. Rajith Making this configurable multiplies the size of the test matrix. Can't we make this simpler? To me, this sounds like an I/O facility in which your output lines may never get sent if you don't call fflush(). This will be a surprise to most programmers, who rarely use fflush(). I think most programmers would be happier if put caused the messages to be eventually sent and send was used only for blocking until messages were flushed out. -Ted
Re: put vs. send
On Tue, Mar 5, 2013 at 2:24 PM, Ted Ross tr...@redhat.com wrote: On 03/05/2013 02:14 PM, Rajith Attapattu wrote: This is a good explanation that we need to put in the docs, as Application developers certainly need to know how it behaves. If one were to use the current C impl, it certainly gives the impression that put() is meant to write messages into your internal buffer and send() will actually write it to the wire. Unfortunately some applications will depend on this behaviour, even though it's not advisable If we are to change from say #2 to #1 or even #3 we need to release note it prominently. I think the best solution is to make this behaviour configurable, and advertise the default very prominently. This way application developers will know exactly what they are getting instead of us making changes underneath. Rajith Making this configurable multiplies the size of the test matrix. Can't we make this simpler? I do understand your concern here, but the Java impl already does both #1 and #2 and Rafi wants to do #3 in the future. The old JMS client does something similar. I agree that if we just do option #2 (as you suggest below), then the application can easily do #1 and #3 on top of that. But I'm sure they will like if the library implements those strategies for them and they have the ability to pick a strategy. To me, this sounds like an I/O facility in which your output lines may never get sent if you don't call fflush(). This will be a surprise to most programmers, who rarely use fflush(). I think most programmers would be happier if put caused the messages to be eventually sent and send was used only for blocking until messages were flushed out. -Ted
Discrepancy between Java and C is settled methods
There's a confusing difference in the meanings of the delivery is settled methods in proton-j (Delivery.isSettled) and proton-c (pn_delivery_settled): their return values represent the local and remote values respectively. proton-j has a separate remotelySettled() method, whereas proton-c appears to have no way of accessing the local state. I'd like to modify one or both apis to resolve this semantic difference. There are clearly a number of options. My favourite is to modify the proton-c function to return the local value, and add a new function to return the remote one. This is consistent with the other functions that have local and remote counterparts, eg pn_link_source and pn_link_remote_source. If this change in proton-c api semantics is too abrupt, maybe we could just deprecate the existing function and method and add new ones that are explicit about their local/remote meaning. What do people think? Thanks, Phil
Re: semantics vs. behavior
If semantics like put vs. send are not stable as documented across releases, then the API is not stable and people's applications will break when they shouldn't. In other words, even documented changes in semantics represent incompatible breaks in the API. So, you should be documenting semantics that are stable across releases. Furthermore, those semantics should be as close to what a *reasonable* developer would expect as is possible. -Ted On 03/05/2013 02:27 PM, Michael Goulish wrote: Wow, I never thought I would have a philosophy question about documentation. Should I be documenting semantics that will be stable across all[1] releases? Or observable behavior in the release you, dear developer, have in your hands right now? Or, perhaps, both ? I'll tell you, as a developer, I would certainly want to know exactly what's going to happen when I call put() and send() with this release. -- [1] for sufficiently small values of all. What I said was put is *allowed* to send optimistically, not that it is required to. It actually did send optimistically in a previous version of the code, however I commented that line out. I would say the documented semantics of put and send should allow the implementation the flexibility to do any of the following: 1) optimistically transmit whatever it can everytime so long as it doesn't block 2) never bother transmitting anything until you force it to by calling send 3) anything in between the first two, e.g. magically transmit once you've put enough messages to reach the optimal batch size The reason for the behaviour you are observing is that we currently do option 2 in the C impl, however we've done option 1 in the past (and I think we do option 1 still in the Java impl), and we will probably do option 3 in the future.
Re: semantics vs. behavior
Mick, great question! As I mentioned in the other thread we owe it to application developers to describe the behaviour. And if we change the behaviour btw releases we need to document it prominently in the release notes as is often the case applications will be written taking advantage of certain behavioural patterns. A somewhat related example is, DUPS_OK and AUTO_ACK had the same behaviour (all though wrong) and most applications that used AUTO_ACK saw a performance drop when we eventually fixed it. Some applications used the wrong ack mode, just bcos it behaved in a certain way. Chances are that applications will be written to rely on the current behaviour. So we need text to explain that all though that this is the current behaviour things may change in the future. If it does change then we need to tell them how to fall back to the previous behaviour. This is my rationale behind making the behaviour configurable. Rajith On Tue, Mar 5, 2013 at 2:27 PM, Michael Goulish mgoul...@redhat.com wrote: Wow, I never thought I would have a philosophy question about documentation. Should I be documenting semantics that will be stable across all[1] releases? Or observable behavior in the release you, dear developer, have in your hands right now? Or, perhaps, both ? I'll tell you, as a developer, I would certainly want to know exactly what's going to happen when I call put() and send() with this release. -- [1] for sufficiently small values of all. What I said was put is *allowed* to send optimistically, not that it is required to. It actually did send optimistically in a previous version of the code, however I commented that line out. I would say the documented semantics of put and send should allow the implementation the flexibility to do any of the following: 1) optimistically transmit whatever it can everytime so long as it doesn't block 2) never bother transmitting anything until you force it to by calling send 3) anything in between the first two, e.g. magically transmit once you've put enough messages to reach the optimal batch size The reason for the behaviour you are observing is that we currently do option 2 in the C impl, however we've done option 1 in the past (and I think we do option 1 still in the Java impl), and we will probably do option 3 in the future.
Re: put vs. send
On Tue, Mar 5, 2013 at 11:10 AM, Ted Ross tr...@redhat.com wrote: On 03/05/2013 02:01 PM, Rafael Schloming wrote: On Tue, Mar 5, 2013 at 10:42 AM, Michael Goulish mgoul...@redhat.com wrote: So, am I understanding correctly? -- I should be able to get messages from my sender to my receiver just by calling put() -- if the receiver is ready to receive? Not necessarily, the receiver being ready just means you are unblocked on AMQP level flow control. You could also potentially block on the socket write (i.e. TCP level flow control). You need to be unblocked on both for put to succeed. Certainly there is no TCP flow control happening in Mick's scenario. What I said was put is *allowed* to send optimistically, not that it is required to. It actually did send optimistically in a previous version of the code, however I commented that line out. I would say the documented semantics of put and send should allow the implementation the flexibility to do any of the following: 1) optimistically transmit whatever it can everytime so long as it doesn't block 2) never bother transmitting anything until you force it to by calling send 3) anything in between the first two, e.g. magically transmit once you've put enough messages to reach the optimal batch size The reason for the behaviour you are observing is that we currently do option 2 in the C impl, however we've done option 1 in the past (and I think we do option 1 still in the Java impl), and we will probably do option 3 in the future. If this is the case, then Mick's original view is correct. The application must assume that messages will not ever be sent unless send is called. There is no flowing, pipelined, non-blocking producer. It's not correct as documentation of the API semantics. It's also not correct to say that messages will never be sent unless send is called, e.g. the following code will work fine: Client: m.put(request); m.recv(); // wait for reply m.get(reply); Server: while True: m.recv(); // wait for request m.get(request) m.put(reply); As for there being no flowing, pipelined, non-blocking producer, that seems like an orthogonal issue, and depending on what you mean I wouldn't say that's necessarily true either. You can certainly set the messenger's timeout to zero and then call put followed by send to get the exact same semantics you would get if put were to optimistically send every time. --Rafael
Re: Discrepancy between Java and C is settled methods
On Tue, Mar 5, 2013 at 11:36 AM, Phil Harvey p...@philharveyonline.comwrote: There's a confusing difference in the meanings of the delivery is settled methods in proton-j (Delivery.isSettled) and proton-c (pn_delivery_settled): their return values represent the local and remote values respectively. proton-j has a separate remotelySettled() method, whereas proton-c appears to have no way of accessing the local state. I'd like to modify one or both apis to resolve this semantic difference. There are clearly a number of options. My favourite is to modify the proton-c function to return the local value, and add a new function to return the remote one. This is consistent with the other functions that have local and remote counterparts, eg pn_link_source and pn_link_remote_source. If this change in proton-c api semantics is too abrupt, maybe we could just deprecate the existing function and method and add new ones that are explicit about their local/remote meaning. What do people think? I don't think it's possible to modify the C API in that way. A locally settled delivery is actually considered freed (really it's returned to a pool), so it's not possible to actually query that state, i.e. if your modified pn_delivery_settled(pointer_to_delivery) where to ever return true then that means you just passed it a pointer to garbage memory. --Rafael
Re: put vs. send
- Original Message - On Tue, Mar 5, 2013 at 11:10 AM, Ted Ross tr...@redhat.com wrote: On 03/05/2013 02:01 PM, Rafael Schloming wrote: On Tue, Mar 5, 2013 at 10:42 AM, Michael Goulish mgoul...@redhat.com wrote: So, am I understanding correctly? -- I should be able to get messages from my sender to my receiver just by calling put() -- if the receiver is ready to receive? Not necessarily, the receiver being ready just means you are unblocked on AMQP level flow control. You could also potentially block on the socket write (i.e. TCP level flow control). You need to be unblocked on both for put to succeed. Certainly there is no TCP flow control happening in Mick's scenario. What I said was put is *allowed* to send optimistically, not that it is required to. It actually did send optimistically in a previous version of the code, however I commented that line out. I would say the documented semantics of put and send should allow the implementation the flexibility to do any of the following: 1) optimistically transmit whatever it can everytime so long as it doesn't block 2) never bother transmitting anything until you force it to by calling send 3) anything in between the first two, e.g. magically transmit once you've put enough messages to reach the optimal batch size The reason for the behaviour you are observing is that we currently do option 2 in the C impl, however we've done option 1 in the past (and I think we do option 1 still in the Java impl), and we will probably do option 3 in the future. If this is the case, then Mick's original view is correct. The application must assume that messages will not ever be sent unless send is called. There is no flowing, pipelined, non-blocking producer. It's not correct as documentation of the API semantics. It's also not correct to say that messages will never be sent unless send is called, e.g. the following code will work fine: Client: m.put(request); m.recv(); // wait for reply m.get(reply); That recv() just caused an enqueued message to be sent? I just tried it in a C program and it worked, more or less. At least the receiver got his 5 messages without send() ever having been called. Does a call to recv() also cause all unblocked messages to be sent? Is that symmetric? Does a call to send() also cause messages to be received ? Server: while True: m.recv(); // wait for request m.get(request) m.put(reply); As for there being no flowing, pipelined, non-blocking producer, that seems like an orthogonal issue, and depending on what you mean I wouldn't say that's necessarily true either. You can certainly set the messenger's timeout to zero and then call put followed by send to get the exact same semantics you would get if put were to optimistically send every time. --Rafael
Re: put vs. send
On Tue, Mar 5, 2013 at 3:20 PM, Rafael Schloming r...@alum.mit.edu wrote: On Tue, Mar 5, 2013 at 11:33 AM, Rajith Attapattu rajit...@gmail.comwrote: On Tue, Mar 5, 2013 at 2:24 PM, Ted Ross tr...@redhat.com wrote: On 03/05/2013 02:14 PM, Rajith Attapattu wrote: This is a good explanation that we need to put in the docs, as Application developers certainly need to know how it behaves. If one were to use the current C impl, it certainly gives the impression that put() is meant to write messages into your internal buffer and send() will actually write it to the wire. Unfortunately some applications will depend on this behaviour, even though it's not advisable If we are to change from say #2 to #1 or even #3 we need to release note it prominently. I think the best solution is to make this behaviour configurable, and advertise the default very prominently. This way application developers will know exactly what they are getting instead of us making changes underneath. Rajith Making this configurable multiplies the size of the test matrix. Can't we make this simpler? I do understand your concern here, but the Java impl already does both #1 and #2 and Rafi wants to do #3 in the future. The old JMS client does something similar. I agree that if we just do option #2 (as you suggest below), then the application can easily do #1 and #3 on top of that. But I'm sure they will like if the library implements those strategies for them and they have the ability to pick a strategy. I don't see why we'd make this configurable. All three options actually fit the same general semantics. Even if you're optimistically trying to transmit every single time put is called it's entirely possible for the socket to be blocked every single time you try. If this were to happen the implementation of #1 would appear to behave precisely the same as #2 behaves. In other words if you're coding correctly against the API you can't assume that put will or won't have transmitted anything regardless of which strategy is used internally. I agree with you. You make a very good point. Perhaps we should explicitly make that clear in our docs to avoid applications written against wrong assumptions. Rajith --Rafael
Re: put vs. send
- Original Message - On Tue, Mar 5, 2013 at 3:20 PM, Rafael Schloming r...@alum.mit.edu wrote: On Tue, Mar 5, 2013 at 11:33 AM, Rajith Attapattu rajit...@gmail.comwrote: On Tue, Mar 5, 2013 at 2:24 PM, Ted Ross tr...@redhat.com wrote: On 03/05/2013 02:14 PM, Rajith Attapattu wrote: This is a good explanation that we need to put in the docs, as Application developers certainly need to know how it behaves. If one were to use the current C impl, it certainly gives the impression that put() is meant to write messages into your internal buffer and send() will actually write it to the wire. Unfortunately some applications will depend on this behaviour, even though it's not advisable If we are to change from say #2 to #1 or even #3 we need to release note it prominently. I think the best solution is to make this behaviour configurable, and advertise the default very prominently. This way application developers will know exactly what they are getting instead of us making changes underneath. Rajith Making this configurable multiplies the size of the test matrix. Can't we make this simpler? I do understand your concern here, but the Java impl already does both #1 and #2 and Rafi wants to do #3 in the future. The old JMS client does something similar. I agree that if we just do option #2 (as you suggest below), then the application can easily do #1 and #3 on top of that. But I'm sure they will like if the library implements those strategies for them and they have the ability to pick a strategy. I don't see why we'd make this configurable. All three options actually fit the same general semantics. Even if you're optimistically trying to transmit every single time put is called it's entirely possible for the socket to be blocked every single time you try. If this were to happen the implementation of #1 would appear to behave precisely the same as #2 behaves. In other words if you're coding correctly against the API you can't assume that put will or won't have transmitted anything regardless of which strategy is used internally. I agree with you. You make a very good point. Perhaps we should explicitly make that clear in our docs to avoid applications written against wrong assumptions. I can certainly do that, but it seems to me that semantics should be simple, obvious, and orthogonal. What seems non-simple and non-obvious to me so far is: put() might send, or not. It doesn't send now, but it might later. recv() can cause messages to be sent. send() can cause messages to be received. I would think that 1. every verb should only mean one thing 2. there should be a simple mental model, against which every verb performs a predictable action. so for example: put( messenger, message ); // enqueue for sending send ( messenger, BLOCK ); // block till all sent. send ( messenger, DONT_BLOCK ); // send what you can. credit ( messenger, 10 ); // limit incoming queue size recv ( messenger, BLOCK ); // block till I get a message recv ( messenger, DONT_BLOCK ); // if no messages incoming, return. Rajith --Rafael
Re: put vs. send
On Tue, Mar 5, 2013 at 12:39 PM, Michael Goulish mgoul...@redhat.comwrote: - Original Message - On Tue, Mar 5, 2013 at 11:10 AM, Ted Ross tr...@redhat.com wrote: On 03/05/2013 02:01 PM, Rafael Schloming wrote: On Tue, Mar 5, 2013 at 10:42 AM, Michael Goulish mgoul...@redhat.com wrote: So, am I understanding correctly? -- I should be able to get messages from my sender to my receiver just by calling put() -- if the receiver is ready to receive? Not necessarily, the receiver being ready just means you are unblocked on AMQP level flow control. You could also potentially block on the socket write (i.e. TCP level flow control). You need to be unblocked on both for put to succeed. Certainly there is no TCP flow control happening in Mick's scenario. What I said was put is *allowed* to send optimistically, not that it is required to. It actually did send optimistically in a previous version of the code, however I commented that line out. I would say the documented semantics of put and send should allow the implementation the flexibility to do any of the following: 1) optimistically transmit whatever it can everytime so long as it doesn't block 2) never bother transmitting anything until you force it to by calling send 3) anything in between the first two, e.g. magically transmit once you've put enough messages to reach the optimal batch size The reason for the behaviour you are observing is that we currently do option 2 in the C impl, however we've done option 1 in the past (and I think we do option 1 still in the Java impl), and we will probably do option 3 in the future. If this is the case, then Mick's original view is correct. The application must assume that messages will not ever be sent unless send is called. There is no flowing, pipelined, non-blocking producer. It's not correct as documentation of the API semantics. It's also not correct to say that messages will never be sent unless send is called, e.g. the following code will work fine: Client: m.put(request); m.recv(); // wait for reply m.get(reply); That recv() just caused an enqueued message to be sent? I just tried it in a C program and it worked, more or less. At least the receiver got his 5 messages without send() ever having been called. Does a call to recv() also cause all unblocked messages to be sent? When a messenger is blocked it will do whatever outstanding work it can. This won't necessarily cause all messages to be sent, but it may cause some. Is that symmetric? Does a call to send() also cause messages to be received ? Yes it is symmetric, messages can arrive while you're blocking for send(). --Rafael
Fwd: put vs. send
Oops, meant to send this to the list.
-- Forwarded message --
From: Rafael Schloming r...@alum.mit.edu
Date: Tue, Mar 5, 2013 at 6:44 PM
Subject: Re: put vs. send
To: Bozo Dragojevic bo...@digiverse.si
On Tue, Mar 5, 2013 at 3:25 PM, Bozo Dragojevic bo...@digiverse.si wrote:
Wow, by not ever calling pn_messenger_send(), but only pn_messenger_recv()
things are, unexpectedly, better! I'll explain below what 'better' means.
But first, this begs the question, what is the purpose of
pn_messenger_send()
and where (and why) it's appropriate/required to call it.
Well one example would be if you just want to send a bunch of messages
without ever receiving any.
I think it helps if you conceptually split the API along
blocking/non-blocking lines:
non-blocking: put, subscribe, get, incoming, outgoing, etc...
blocking: send, recv
Of the non blocking portion there are things that simply access data, like
get(), incoming, outgoing, etc, and then there are things that
schedule/request asynchronous work to be done, namely put() and subscribe().
For the blocking portion, I think it helps to understand that there is
really only one fundamental operation, and that is
do_any_outstanding_work_until_condition_X_is_met. Currently the blocking
portion of the API exposes exactly two conditions, namely all messages are
sent via send() and there is a message waiting to be received via recv().
So in your example below if you're basically just a request/response server
then as you've noticed you never actually need to call send since you don't
really care about waiting for that condition, and any work you schedule in
your handlers will be performed anyways as you wait for messages to be
received.
The results are for slightly modified 0.3 release.
Most notably, I have a local change that exposes pn_driver_wakeup()
through the messenger api.
I'm curious about why you had to make this change. Can you describe the
usage/scenario that requires it?
Our API is threaded internally but all proton communication is done
via a dedicated thread that runs the following (stub) event loop:
(The same event loop is used by both, the client library, and the 'broker')
while(1) {
ret = pn_messenger_recv(m,100) // the 100 is hard to explain...
if (ret != PN_WAKED_UP) {// new error code for wakeup case
/*
* apparently there's no need to call send...
* pn_messenger_send(m);
*/
}
Command cmd = cmd_queue.get(); // cmd_queue.put() from another thread
// will call pn_driver_wakeup() and
will
// break out of pn_messenger_recv()
if (cmd)
handle(cmd); // ends up calling pn_messenger_put()
if (pn_messenger_incoming(m)) {
msg = pn_messenger_get(m);// handle just one message
// pn_messenger_recv() will not block
// until we're done
handle(msg); // can end up calling
pn_messenger_put()
}
}
So, before the change, a test client that produced messages needed to
throttle a bit, about 8ms between each 'command' that resulted in
one 'pn_messenger_put()'
If a lower delay (or no delay) was used, the client's messenger got
confused
after some fairly small number of messages sent (order of magnitude 10)
and ended up sitting in pn_driver_wait while it had unsent messages to
send.
This sounds like a bug to me. As I mentioned above, you shouldn't need to
call send, but it also shouldn't hurt.
With the one line change of commenting out the send() it can go full speed!
I know it's hard to comment on out-of-tree modified pseudo code, but
is such an event loop within the design goals of the messenger?
Yes, certainly. Your code looks to me like it is almost the same as the
server.py example.
Longer term we'll most likely be switching from messenger to engine + driver
so we can go multithreaded with the event loop.
You may find there is a fair amount you would need to reimplement if you
were to make that switch. One of the features I'd like to get into
messenger soon is factoring the messenger implementation and adding to the
API so that you can supply your own driver and write your own event loop
based on the messenger API rather than having to go all the way down to the
engine API. If what you want is more control over the I/O and threading but
you're happy with the protocol interface provided by messenger, then this
might be a better option for you than using the engine directly.
--Rafael
