On Sun, Dec 8, 2013 at 3:06 PM, Fraser Adams
<[email protected]>wrote:
> Hey all,
> I've been able to get messenger to behave fairly sensibly in a
> non-blocking way, but what I've yet to achieve is getting it to behave in a
> properly "asynchronous" event-driven way where I fire up a looping
> "notifier" after everything is initialised and the notifier calls the
> "callbacks", you know, the usual event driven pattern.
>
>
> For my send-async code I've got a "main_loop" function that performs the
> guts of the work, the key bits look like this:
>
> #define SENT_MESSAGE 0
> #define STOPPING 1
>
>
> void main_loop(void *arg) {
> printf(" *** main_loop ***\n");
>
> int err = pn_messenger_work(messenger, 0); // Sends any outstanding
> messages queued for messenger.
> int pending = pn_messenger_outgoing(messenger); // Get the number of
> pending messages in the outgoing message queue.
>
> printf("err = %d\n", err);
> printf("pending = %d\n", pending);
>
> if (state == SENT_MESSAGE && !pending) {
> printf("calling stop\n");
> pn_message_free(message); // Release message.
> pn_messenger_stop(messenger);
> state = STOPPING;
> } else if (state == STOPPING && !err) {
> printf("exiting\n");
> pn_messenger_free(messenger);
> exit(0);
> }
> }
>
>
>
> in the main method I set messenger to non-blocking, create and send the
> message and set state to SENT_MESSAGE
>
> when I have a "notifier" loop like the following:
>
> while (1) {
> main_loop(NULL);
>
> struct timeval timeout;
> timeout.tv_sec = 0;
> timeout.tv_usec = 16667;
> select(0, NULL, NULL, NULL, &timeout);
> }
>
> The approach above works fine and reaches the final state and exits, but
> It's not really what I want as it's essentially a busy-wait loop albeit
> with a 16.7 ms delay. What I *really* want is for the main notifier event
> loop to block until activity is happening.
>
>
> I tried:
>
> while (1) {
> pn_driver_wait(messenger->driver, -1); // Block indefinitely until
> there has been socket activity.
> main_loop(NULL);
> }
>
> But that doesn't even compile (error: dereferencing pointer to incomplete
> type) I guess pn_messenger_t isn't externally visible outside messenger.c
> so I can't access the driver instance?
>
> If I do:
>
> while (1) {
> pn_messenger_work(messenger, -1); // Block indefinitely until there
> has been socket activity.
> main_loop(NULL);
> }
>
>
> That "kind of" works, but it doesn't get as far as the exit state (the
> last err value is -7), so there's socket activity that I'm missing I think.
>
> In any case I *really* don't like having to do a blocking call to
> pn_messenger_work() just to do the select/poll call that I really want to
> do.
>
The -7 error code means PN_TIMEOUT, i.e. there was no work that could be
performed within the time limit you passed in, in this case 0. I'm a little
puzzled why you are calling pn_messenger_work twice. The way you have your
code structured the first and second call will happen right after each
other with the first call blocking indefinitely, and the second call
returning immediately. It seems like the second call will serve no purpose
but to return a potentially confusing error code.
> If I'm honest I don't *think* messenger is currently entirely geared up
> for asynchronous behaviour despite the ability to put it in non-blocking
> mode. What I mean is that the heart of many of the calls is
> pn_messenger_tsync, which is called in blocking and non-blocking modes and
> that's calling pn_driver_wait, potentially in a loop even in non-blocking
> mode, so even if my notifier could do a simple block on pn_driver_wait I'd
> still by calling poll multiple times - once blocking in my notifier waiting
> for activity and then non-blocking when I do.
> pn_messenger_work(messenger, 0); // Sends any outstanding messages queued
> for messenger.
>
> Even when it is working well I'm suspecting that the loop will be causing
> more calls to main_loop than really desirable as it'll trigger on all
> socket activity not just the send - perhaps that's necessary because of the
> AMQP handshaking, but it doesn't "feel" especially efficient, but I'm
> certainly far from an expert about the guts of messenger - it made my head
> explode :-)
>
You're correct that main_loop might get called a few extra times, but I
doubt it is much of a performance concern. In any performance related
scenario the bulk of what main_loop is going to be doing is
sending/receiving messages, and in this case, 99.99% of socket activity
will be related to exactly that, so even though it may occasionally get
called due to socket activity related to hand shaking or heart beating or
something like that, the odds of there not also being message related work
to process become vanishingly small the more messages you're
sending/receiving.
> I guess that most uses to date have been for traditional blocking
> scenarios, but I'm thinking that in the future a more asynchronous approach
> is likely to become important - what I mean is that as the number of
> processor cores increases I suspect that asynchronous approaches like Grand
> Central Dispatch http://en.wikipedia.org/wiki/Grand_Central_Dispatch will
> probably scale better across large numbers of cores, so it might be good to
> have an efficient asynchronous programming model for proton.
>
The asynchronous APIs have been used, but only in one or two cases, so you
are definitely exploring uncharted territory here. FWIW, I agree completely
that the asynchronous case is going to be key in the future.
>
> Am I roughly on the right track? Any ideas how to make what I'm trying to
> do a little neater?
>
I think you're close, but I'm not sure exactly what you're trying to do. I
don't see where your code is actually sending outgoing messages or
processing incoming ones. One thing I would say is that you probably only
need to call pn_messenger_work in one place, i.e. where you need to block.
I think we could definitely use a set of asynchronous examples to help get
people started here. If you can describe a bit more about what you're
trying to set up I'll see if I can take a crack at sketching out some
example code.
> At this stage I'm trying things out and trying to educate myself, so if
> there are limitations it's not necessarily a huge deal, but it might be a
> useful point for further conversation on asynchronous behaviour - anyone
> else musing over this?
>
Much of the async stuff was developed in response to some of the use cases
brought up on the proton list, so there should be at least one other user
of these APIs.
--Rafael
