On Mon, Jan 26, 2015 at 9:43 AM, Alan Conway <acon...@redhat.com> wrote:

> On Mon, 2015-01-19 at 13:57 -0500, Rafael Schloming wrote:
> > On Mon, Jan 19, 2015 at 11:52 AM, Alan Conway <acon...@redhat.com>
> wrote:
> >
> > > On Wed, 2015-01-14 at 08:28 -0500, Rafael Schloming wrote:
> > > > Hi Everyone,
> > > >
> > > > I've been doing some work on a C reactor API for proton that is
> > > > intended to fit both alongside and underneath what gordon has been
> > > > doing in pure python. I have several goals with this work.
> > > >
> > > >  - Simplify/enable a reactive style of programming in C that is
> > > > similar to what gordon has built out in pure python.
> > > >
> > > >  - Provide a C API that translates well into the python/ruby/etc
> > > > bindings, so that sophisticated handlers can be written once in C
> > > > rather than being duplicated in each scripting language.
> > > >
> > > >
> > > >  - Preserve the extensibility/flexibility that comes with being able
> > > > to define custom handlers in the bindings, so python/ruby/etc
> handlers
> > > > can intermix well with C handlers.
> > > >
> > > >
> > > >  - Provide a C API that translates well into javascript via
> > > > emscripten. In some ways this is similar to the above goals with the
> > > > other language bindings, however I mention it separately because
> there
> > > > are additional memory management constraints for javascript since it
> > > > has no finalizers.
> > > >
> > > >
> > > >
> > > > I believe I've made significant progress towards most of these goals,
> > > > although there is still plenty of work left to do. I'd like to share
> a
> > > > few examples both to illustrate where I am with this and to solicit
> > > > feedback and/or help.
> > >
> > > The goals are spot on, C API looks good at first glance, I'll try to
> > > give it more careful scrutiny. The C handler functions need a void
> > > *context so we can parameterize the callbacks. e.g.
> > >
> > > void task_dispatch(pn_handler_t *handler, pn_event_t *event, void
> > > *context) {
> > >     ...
> > >     pn_reactor_schedule(reactor, (int)context, handler);
> > > }
> > > ...
> > >       pn_handler_t *handler = pn_handler(task_dispatch, (void*)1000);
> > >
> > > I'm not sure if you intended a 1-1 relationship between handlers and
> > > callback functions. If you do then you could put the context on the
> > > handler rather than as a parameter to the callback - but that would
> > > force the 1-1 relationship in the implementation from the API.
> > >
> >
> > I've had trouble in general using just simple void pointers for contexts
> in
> > more complex scenarios. The memory management semantics quickly get
> pretty
> > cumbersome. My intention was that any state the callback needs should be
> > accessible from either the event itself or from the handler, and there
> are
> > really two ways to add your own state. One being adding "attachments"
> into
> > the object model, and two being extending the handler itself with
> > additional state.
> >
> > The attachments work like this. Most of the core python objects now
> support
> > doing something like this:
> >
> >   #define MY_STATE <any-unique-id>
> >
> >   pn_record_t *attachments = pn_connection_attachments(conn);
> >   pn_record_def(attachments, MY_STATE, PN_VOID);
> >   ...
> >   pn_record_set(attachments, MY_STATE, foo);
> >   ...
> >   void *foo = pn_record_get(attachments, MY_STATE);
> >
> > Two key things to note here is that first, when you define the
> attachment,
> > you are explicitly telling proton how to treat the pointer with respect
> to
> > memory management. You can define PN_OBJECT, or PN_WEAKREF attachments
> > also, and even define your own kind of pointer with your own finalization
> > semantics. This is how the python bindings work, the python binding
> defines
> > a PN_PYREF attachment for tracking associations between proton objects
> in C
> > and python objects.
> >
> > Second, there can be multiple independent attachments that won't conflict
> > with each other so long as they use different keys. (I've just been using
> > globally defined memory addresses to give myself guaranteed unique
> numeric
> > ids with no need to have a shared bootstrap process.)
> >
> > In the case of handlers, I've actually supplied another way to add extend
> > the handler with your own state. When you create a handler you can
> request
> > an additional chunk of memory associated with the handler that you can
> use
> > in any way you like, and when you do this you supply a finalizer along
> with
> > it. Your example above could work like this:
> >
> >   pn_handler_t *handler = pn_handler_new(task_dispatch, sizeof(int), NULL
> > /*if I were using pointers, I might want a finalizer here*/);
> >
> > You can now access that from the handler via pn_handler_mem:
> >
> >   int *my_state = pn_handler_mem(handler);
> >
> > You can check out the pn_flowcontroller handler if you want to see a full
> > working example of this.
>
> Nice! You're way ahead of me :) Agreed on the mem mgmt problems with
> void*, this seems like a nice solution. Attachments do imply an extra
> malloc, but not your second option which I like more. IMO you don't need
> attchments with the second option - you can use the second option to
> store a pointer to anything (or a struct full of pointers to manythings)
> with a proper finalizer and no void* at all. Very neat


The two options are for different scenarios, so I wouldn't necessarily
consider them alternatives. The second option is analogous to subclassing.
You are creating a new kind of object that has additional state, and you
can define whatever state you want and clean it up if necessary, but all
the state defined is under the control of a single piece of code.

Attachments on the other hand allow multiple independent pieces of code to
add state to an existing object without stepping on each others toes. Kind
of like having a map associated with the object and using string keys to
avoid naming collisions.

Say I wanted to augment the pn_flowcontroller handler to dynamically adjust
the window it uses on a per link basis using stats it gathers for that link
over time. In order to do that, my handler is going to have to track extra
state for each link, so of course it is very useful to be able to attach a
reference to the link back to whatever state that handler is tracking. Of
course it's also very useful for the application (which is really just
another handler) to be able to do the same thing, say add a pointer from an
outgoing link back to whatever source of messages it is pulling from (e.g.
a queue or a topic or something). Beyond just the memory management issues,
you can see why having a single context per link is a limitation in this
scenario. You have to globally coordinate exactly how you use that
particular context pointer. Attachments give you a much nicer way to do
this since both handlers can define their own private keys and have
independent back pointers from a link into whatever state the two handlers
are keeping on behalf of that link.

--Rafael

Reply via email to