On 20 December 2013 19:49, Rafael Schloming <r...@alum.mit.edu> wrote:

> On Fri, Dec 20, 2013 at 11:43 AM, Fraser Adams <
> fraser.ad...@blueyonder.co.uk> wrote:
> > I've been following this thread with interest and I guess that Rob's
> > comment
> >
> >
> > "
> >
> > However I think that would require us to be
> > organised differently with a recognition that the Engine and Messenger
> are
> > conceptually separate (with Messenger depending on the Engine API, and
> > having no more authority over defining/testing the engine API than any
> > other client
> > "
> >
> > Struck some resonance.
> >
> > I think perhaps the layering is OK if you're familiar with the code base
> > and perhaps it's more about "packaging" than layering, but FWIW coming
> into
> > proton quite "cold" it seems like messenger and engine are essentially
> > "combined". Now I know that's not the case now from the responses to my
> > "how does it all hang together" question of a few weeks ago, but in terms
> > of packaging that's still kind of the case.
> >
> > So what I mean by that is that qpid::messaging and proton messenger are
> as
> > far as has been discussed with me "peer" APIs - both high level that can
> > achieve similar things albeit with different semantics, whereas engine
> is a
> > lower level API.
> >
> > Why is it in that case that the proton library is built that contains
> both
> > messenger and engine? OK, so it's convenient, but as far as I'm aware
> > neither qpid::messaging nor Ted's dispatch router actually use messenger
> at
> > all, they both use engine? That would suggest to me that engine and
> > messenger probably ought to be exposed as separate libraries? (and
> > presumably there's a similar position in Java where the JMS
> implementation
> > is engine not messenger based - though I've not looked at the jar
> > packaging).
> >
> > I guess (perhaps related) it just might be better if messenger was in a
> > separate source tree, which I'd agree might be a faff, but would clarify
> > the hierarchical rather than peer relationship between messenger and
> engine.
> >
> > So I guess that my take is that Rafael is perfectly accurate when he says
> > that "I believe they are currently layered precisely the way you
> > describe" that certainly seems to be the case at the logical level, but
> at
> > the "physical" level it's an awful lot less clear of the layering -
> > certainly from the perspective of a novice trying to navigate the code
> base.
> >
> There is certainly a pragmatic element in keeping messenger and engine
> bundled the way they are. From a development perspective, having messenger
> in the same code base makes testing significantly easier.

It might make it easier for you (since you have a complete view of every
piece of the code and design) - for myself I found that it made things
*much* harder, as tests were running through multiple layers rather than
being isolated to a single component.  When a test "failed" it was entirely
unclear which layer the failure resided in or even why the outcome should
be as the test expeted.  Personally I'd say the testing is a strong
argument for separation (although it means more work upfront it makes it
easier for everyone in the long run).

> Writing a simple
> command line program to fire off a few messages in order to reproduce some
> scenario is quite trivial with messenger, but would involve lots of boiler
> plate with just the engine directly. That's something I would like to work
> on improving about the engine API, but it is very slow and difficult to
> make changes given the current setup I described in my original post.

> Given
> that, realistically I think if we were to pull messenger into a separate
> code base, there would be multiple sources of duplicate effort, not only
> having to duplicate/produce another non trivial multi lingual build system,
> but we would also need to build up a test harness that duplicates a good
> chunk of what messenger already does.

Every other project that wants to use the Proton engine will also be
writing tests for their use of the API. And they'll have different use
cases and different pattern that may not be something supported by

Sure, it's great that Messenger tests will implicitly test the Engine, but
they are not a substitute for Engine tests.

> Granted a chunk of that is
> boilerplate that should be somehow refactored into the engine API proper so
> that it is easier to use in general, and if this were to happen such a
> split would probably be more feasible, but that kind of refactoring is
> really prohibitively expensive given all the different shims and bindings
> that would be impacted.
And this points to an issue, with the focus on Messenger the Engine API is
not being enhanced... and it needs people and effort focussed on it.
Messenger should be a client of the Engine API, the Engine shouldn't be a
slave to Messenger.  Other users (whether in Qpid or not) should have equal
influence of the Engine API design, and equal ability to try to shape it.
Right now I find the Engine unattaractive to try to use within the Qpid
Java Broker... I know Gordon has also expressed thought on things that he
would like to see changed based on his work intergrating in the C++
broker.  I feel like the Engine needs to set free to evolve on its own, but
with an API that everyone (Messenger, Qpid Brokers, etc) can rely on.

> I'd also point out that from a user perspective I think there is some
> synergy in bundling a higher level tool together with the engine. It is
> very rare that you are going to want to embed the engine in something and
> not want some convenient high level way to send that thing a message. Case
> in point, Ted's dispatch router proper doesn't actually use messenger as
> you say, however he ships it with several command line scripts that do use
> messenger. Splitting them up would result in both him and his users having
> to deal with an extra dependency.
Sure, but maybe if someone else had written a different API Ted would have
wanted to use that.  Having said that, I'm not necessarily against
bundling... but I think then we need to be clearer in the loosening of the
coupling and allow for the horizontal scaling of allowing different people
to evolve the Engine and Messenger APIs.

> > I'd have to agree also with Rob about the horizontal scaling aspects of
> > lack of comments and documentation, particularly with respect to engine.
> > I'm currently trying to get a JavaScript_messenger_  working because that
> > seems tractable given the examples and documentation around messenger
> > (though as you know even there I've had a fair few complications) my
> > personal preference/familiarity is however around the JMS/qpid::messaging
> > API and I'd ultimately quite like to have a JavaScript "binding" for
> that,
> > but at this stage I wouldn't know where to begin and I'd probably end up
> > "reverse engineering" the work Gordon has done with qpid::messaging
> and/or
> > Rob and co. are doing with the new JMS implementation.
> >
> > I'd be interested to hear from Gordon and Ted about how they went about
> > building capability on top of the engine API.
> >
> > I hope I'm not coming across as over-critical - email is a bit rubbish at
> > conveying emotion :-) I guess that I just want to add my 2p from the
> > perspective of someone coming in new with no previous exposure to the
> code
> > base, which is hopefully a fairly objective position.
> >
> No worries, I'm well aware of the issues you mention and I'm as eager to
> see a solution as anyone else. I kicked off this thread asking for thoughts
> and ideas, so I certainly can't blame anyone for speaking up.
In terms of your desire to remove the JNI implementation... I agree that
it's not something that in its current form would be production
deployable.  However as an "independent" developer trying to work on the
Java implementation previously, I found having the JNI implementation
invaluable in terms of being able to compare and contrast what the C impl
was doing to the pure Java impl (I don't really see any value in the JNI
Driver or JNI Messenger... but then I'm not really interested in those
components anyway).  Using the python tests against the Java was incredibly
painful for myself and others who I work with.

I guess really it comes down to whether our aim here is to make it as
frictionless as possible for you to work on the codebase... Or whether we
should make a concerted effort to make it easier for others to work on (and
integrate with) the work we are doing.  If not removing it makes it even a
tenth as painful for you to work on as I and others found it... then
perhaps we can *all* find some motivation to reshape the process and the
mechanics. :-)

-- Rob

Reply via email to