Re: [Java] Message API design notes

[Michael Arnoldus]

Good point!

I will - with a smile - refrain from commenting further on MS and  
their insights.


Enjoy :-)

Michael

Den 23/08/2007 kl. 12.10 skrev Rupert Smith:

But hold on. We already agree that such an API should look 1:1 like  
the

protocol. Its not as if the API, even if implemented as a blind-fold
experiment, by different groups with no contact, in different  
languages, is
going to look radically different. The protocol XML itself is  
already the

language neutral spec.

BTW. I think MS has seen the light on functional languages, faster  
than

most.

On 23/08/07, Michael Arnoldus <[EMAIL PROTECTED]> wrote:


Den 23/08/2007 kl. 11.27 skrev Robert Greig:


On 23/08/07, Rupert Smith <[EMAIL PROTECTED]> wrote:


One thing that seems to have come up a few times, is the idea that
it would
be hard to have the same API across different languages. I don't
think it
would be technically hard to do this.


Yes, plus it's practically hard given other factors involved.  
Even OO

languages have different conventions - witness the rather clunky
result of transliterating the Java client into .NET.

To my mind, having idiomatic clients in each language is going to
result in a far better experience for those developers.

RG


I agree - and again agree totally!!!

As an open source thing I would expect the primary adoption of AMQP
to come from the developers doing the work. They will choose to use
AMQP if they get something they feel take care of their requirements
and is easy to start using. Some API that might be standard in some
OO specific way might look ridiculous in another language than the
one providing the original design perspective - and the resistance to
adoption will increase - or somebdy working in another language will
write their own API on top of the standardized one and everybody will
use that - in which case we end up the same place.

And BTW - I strongly oppose the OO is everything view put forward in
the previous suggestion. Even MS has started to see the light :-)

Check out http://blogs.msdn.com/charlie/archive/2007/01/26/anders-
hejlsberg-on-linq-and-functional-programming.aspx

Michael Arnoldus

Re: [Java] Message API design notes

[Rupert Smith]

But hold on. We already agree that such an API should look 1:1 like the
protocol. Its not as if the API, even if implemented as a blind-fold
experiment, by different groups with no contact, in different languages, is
going to look radically different. The protocol XML itself is already the
language neutral spec.

BTW. I think MS has seen the light on functional languages, faster than
most.

On 23/08/07, Michael Arnoldus <[EMAIL PROTECTED]> wrote:
>
> Den 23/08/2007 kl. 11.27 skrev Robert Greig:
>
> > On 23/08/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
> >
> >> One thing that seems to have come up a few times, is the idea that
> >> it would
> >> be hard to have the same API across different languages. I don't
> >> think it
> >> would be technically hard to do this.
> >
> > Yes, plus it's practically hard given other factors involved. Even OO
> > languages have different conventions - witness the rather clunky
> > result of transliterating the Java client into .NET.
> >
> > To my mind, having idiomatic clients in each language is going to
> > result in a far better experience for those developers.
> >
> > RG
>
> I agree - and again agree totally!!!
>
> As an open source thing I would expect the primary adoption of AMQP
> to come from the developers doing the work. They will choose to use
> AMQP if they get something they feel take care of their requirements
> and is easy to start using. Some API that might be standard in some
> OO specific way might look ridiculous in another language than the
> one providing the original design perspective - and the resistance to
> adoption will increase - or somebdy working in another language will
> write their own API on top of the standardized one and everybody will
> use that - in which case we end up the same place.
>
> And BTW - I strongly oppose the OO is everything view put forward in
> the previous suggestion. Even MS has started to see the light :-)
>
> Check out http://blogs.msdn.com/charlie/archive/2007/01/26/anders-
> hejlsberg-on-linq-and-functional-programming.aspx
>
> Michael Arnoldus
>
>

Re: [Java] Message API design notes

[Michael Arnoldus]

Den 23/08/2007 kl. 11.27 skrev Robert Greig:


On 23/08/07, Rupert Smith <[EMAIL PROTECTED]> wrote:

One thing that seems to have come up a few times, is the idea that  
it would
be hard to have the same API across different languages. I don't  
think it

would be technically hard to do this.


Yes, plus it's practically hard given other factors involved. Even OO
languages have different conventions - witness the rather clunky
result of transliterating the Java client into .NET.

To my mind, having idiomatic clients in each language is going to
result in a far better experience for those developers.

RG


I agree - and again agree totally!!!

As an open source thing I would expect the primary adoption of AMQP  
to come from the developers doing the work. They will choose to use  
AMQP if they get something they feel take care of their requirements  
and is easy to start using. Some API that might be standard in some  
OO specific way might look ridiculous in another language than the  
one providing the original design perspective - and the resistance to  
adoption will increase - or somebdy working in another language will  
write their own API on top of the standardized one and everybody will  
use that - in which case we end up the same place.


And BTW - I strongly oppose the OO is everything view put forward in  
the previous suggestion. Even MS has started to see the light :-)


Check out http://blogs.msdn.com/charlie/archive/2007/01/26/anders- 
hejlsberg-on-linq-and-functional-programming.aspx


Michael Arnoldus

Re: [Java] Message API design notes

[Robert Greig]

On 23/08/07, Rupert Smith <[EMAIL PROTECTED]> wrote:

> One thing that seems to have come up a few times, is the idea that it would
> be hard to have the same API across different languages. I don't think it
> would be technically hard to do this.

Yes, plus it's practically hard given other factors involved. Even OO
languages have different conventions - witness the rather clunky
result of transliterating the Java client into .NET.

To my mind, having idiomatic clients in each language is going to
result in a far better experience for those developers.

RG

Re: [Java] Message API design notes

[Robert Greig]

On 23/08/07, Gordon Sim <[EMAIL PROTECTED]> wrote:

> I think it would be premature to start trying to standardise. The java
> world is not (in my opinion) looking for a standardised messaging API;
> that role is currently fulfilled by JMS.

Yes, exactly. One other point that I would add is that my ideal
scenario when extending JMS is that if AMQP becomes prevalent, the
extensions to JMS could become part of some future JMS standard, or
JMS extension. This would be the case if other vendors needed to
expose exactly the same set of extensions.

RG

Re: [Java] Message API design notes

[Rupert Smith]

One thing that seems to have come up a few times, is the idea that it would
be hard to have the same API across different languages. I don't think it
would be technically hard to do this. If we assume OO as a base line, and
define the API as a class diagram in UML (1:1 the protocol, of course), the
mapping onto any OO language is really quite trivial.

All our current target languages for Qpid are OO, so do we really need to
care about lack of support for functional/procedural? Even if we do care
about them, there are ways that a class diagram can be mapped into non-OO
languages anyhow.

I think what is hard about this idea, is coordinating the implementation of
a common API across Qpid, given that code bases for the different languages
are in different states, implement slightly different versions of the
protocol, are being worked on by different people etc.

Would it be an idea to draw up an API convergence sheet? A spreadsheet that
documents by function, the class to which the function belongs, the name of
the function and its arguments in each of the languages. Then decide what it
is that we are converging towards, then converge?

Rupert

On 22/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
>
> On 8/22/07, Robert Greig <[EMAIL PROTECTED]> wrote:
> >
> > On 22/08/07, Robert Godfrey <[EMAIL PROTECTED]> wrote:
> >
> > > I do not particularly see the need for an API which is neither JMS
> that
> > > doesn't look exactly like the model layers of AMQP [that is the API
> need
> > not
> > > expose all the 0-10 features for connection establishment, session
> > > maintenance etc... those are functions of the client library and
> > shouldn't
> > > be exposed to the application programmer - For those familiar with
> AMQP
> > 0-10
> > > I think the API should provide hooks into all of Layer 4... with
> > > abstractions for the lower layers].
> >
> > Yes, I agree and would go further. I think it is positively
> > detrimental to Qpid to have two "competing" APIs .
>
>
> As I mentioned on the other thread [JMS extensions] there is no competing
> API's.
> The low level Qpid API and the JMS API targets different levels of users.
> If we clearly document the pros/cons there wan't be any confusion either.
>
> It is confusing to potential users - which API to choose? Remember that a
> > large
> > proportion of our target userbase is an "average" enterprise developer
> > where messaging may only be a small part of the application.
>
> It is also detrimental to the overall Qpid effort since it is more
> > code to support and document, duplicating functionality in our
> > extended JMS API. Don't forget the opportunity cost!
>
>
> The JMS implementation is built on top of the low level  API.
> Hence we maintain only one code base.
> There shouldn't be any duplication of functionality.
> As Rob describes It's L4 + abstraction of L1-L3 + a few convenience
> methods
> to help the JMS impl and direct users of the low level API.
> Nothing more.
>
> RG
> >
>

Re: [Java] Message API design notes

[Gordon Sim]

Michael Arnoldus wrote:
 From my point of view it's very cool to stop the standards at the wire 
level protocol. That means several java API's can coexist, and people 
can use several brokers. Freedom increases. 


I very much share this view. The API is in my view one of the 
distinguishing features of a client; let people innovate.


I think it would be premature to start trying to standardise. The java 
world is not (in my opinion) looking for a standardised messaging API; 
that role is currently fulfilled by JMS.


For Qpid we know we want a fully compliant JMS client. Applications 
using that API can be reasonably confident that their code is isolated 
from the details of any specific implementation.


We should be able to offer users of that client different features not 
directly exposed through the JMS API. E.g. synchronous v. asynchronous 
publication perhaps, different prefetch/flow control strategies, 
different ways of handling large messages, different ways of binding 
queues to exchanges, utilising the mandatory flag and alternate 
exchanges etc. These are all things that would be valuable to 
applications and shouldn't really require them to change the way they 
write their applications.


It is also valuable to have a usable library that works more directly 
with AMQP commands. We want this for writing the JMS client anyway, 
making it usable on its own allows others to take it and pursue 
alternative ideas as well. (For example I think it could be interesting 
to have a more tuple-space like API available that uses AMQP underneath).


It seems to me there is a lot of necessary and valuable work we can do 
within Qpid to make our software better both in the set of features it 
exposes and the manner it exposes them. That work is in my view more 
worthwhile than trying to standardise our APIs.

Re: [Java] Message API design notes

[Rajith Attapattu]

On 8/22/07, Robert Greig <[EMAIL PROTECTED]> wrote:
>
> On 22/08/07, Robert Godfrey <[EMAIL PROTECTED]> wrote:
>
> > I do not particularly see the need for an API which is neither JMS that
> > doesn't look exactly like the model layers of AMQP [that is the API need
> not
> > expose all the 0-10 features for connection establishment, session
> > maintenance etc... those are functions of the client library and
> shouldn't
> > be exposed to the application programmer - For those familiar with AMQP
> 0-10
> > I think the API should provide hooks into all of Layer 4... with
> > abstractions for the lower layers].
>
> Yes, I agree and would go further. I think it is positively
> detrimental to Qpid to have two "competing" APIs .


As I mentioned on the other thread [JMS extensions] there is no competing
API's.
The low level Qpid API and the JMS API targets different levels of users.
If we clearly document the pros/cons there wan't be any confusion either.

It is confusing to potential users - which API to choose? Remember that a
> large
> proportion of our target userbase is an "average" enterprise developer
> where messaging may only be a small part of the application.

It is also detrimental to the overall Qpid effort since it is more
> code to support and document, duplicating functionality in our
> extended JMS API. Don't forget the opportunity cost!


The JMS implementation is built on top of the low level  API.
Hence we maintain only one code base.
There shouldn't be any duplication of functionality.
As Rob describes It's L4 + abstraction of L1-L3 + a few convenience methods
to help the JMS impl and direct users of the low level API.
Nothing more.

RG
>

Re: [Java] Message API design notes

[Rajith Attapattu]

>
> I do not particularly see the need for an API which is neither JMS that
> doesn't look exactly like the model layers of AMQP [that is the API need
> not
> expose all the 0-10 features for connection establishment, session
> maintenance etc... those are functions of the client library and shouldn't
> be exposed to the application programmer - For those familiar with AMQP
> 0-10
> I think the API should provide hooks into all of Layer 4... with
> abstractions for the lower layers].


+1 I have the same view point.
Besides if you make a *very* thin client that's close to the wire protocol I
really doubt people want to go between different client libraries.
The decoding/encoding part would be the difference. Unless it is written
abysmally,  I really doubt there is much of a difference in terms of
performance.

However as Rob points out, the motivation for changing the client lib might
come from some cool vendor specific hack :)
anyways when u use that you will not be able to move again.

Rajith

Re: [Java] Message API design notes

[Robert Greig]

On 22/08/07, Robert Godfrey <[EMAIL PROTECTED]> wrote:

> I do not particularly see the need for an API which is neither JMS that
> doesn't look exactly like the model layers of AMQP [that is the API need not
> expose all the 0-10 features for connection establishment, session
> maintenance etc... those are functions of the client library and shouldn't
> be exposed to the application programmer - For those familiar with AMQP 0-10
> I think the API should provide hooks into all of Layer 4... with
> abstractions for the lower layers].

Yes, I agree and would go further. I think it is positively
detrimental to Qpid to have two "competing" APIs . It is confusing to
potential users - which API to choose? Remember that a large
proportion of our target userbase is an "average" enterprise developer
where messaging may only be a small part of the application.

It is also detrimental to the overall Qpid effort since it is more
code to support and document, duplicating functionality in our
extended JMS API. Don't forget the opportunity cost!

RG

Re: [Java] Message API design notes

[Carl Trieloff]

Robert Godfrey wrote:



On 22/08/07, *Carl Trieloff* <[EMAIL PROTECTED] 
> wrote:


Robert Godfrey wrote:
> My view...
>
> In any sane development environment people who code in Java are
going to
> want to code against JMS.

My view is...

Same as Robs for App development guys, they will most likely use JMS.

For people that are building services of infrastructure on top of us
will NOT use JMS.

So my view - we need both.



Yes - my view is that if you are building infrastructure that is the 
same level as building the JMS API - i.e. building on top of the low 
level AMQP API.  I strongly believe that this AMQP API needs to be 
public and well documented.  The point of using a different API for 
infrastructure is (one would presume) from getting the efficiencies of 
being closer to the wire protocol? 
 



yes, if I was building a proxy server for RBAC for example I would not 
want to use JMS, I would want to be closer to the wire.


Carl.

Re: [Java] Message API design notes

[Michael Arnoldus]

Den 22/08/2007 kl. 20.11 skrev Martin Ritchie:


On 22/08/07, Michael Arnoldus <[EMAIL PROTECTED]> wrote:

Den 22/08/2007 kl. 19.08 skrev Martin Ritchie:


On 22/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:


But not one that allows access to AMQP functionality. I don't see
why
the AMQP WG couldn't provide a set of Interfaces and a client lib
could then implement jmx.Session and amqp.Session. Then again I've
never had any exposure to CORBA to know what they did wrong. Is  
the

lesson simply never to provide an API? That seems a bit harsh.



Two reasons.
Defining an API is time consuming and difficult given the no of
languages we
have.
Also they differ from object oriented to procedural to functional.
So obviously there will be different views on how best it should
be done in
a given language.
I heard of client implementations in the following languages so  
far.

java, c++, python, ruby, erlang? , javascript - and most like C
and perl.

[not forgetting our own .NET :)]




The second reason is more political.
I think it is a very difficult exercise to get agreement over the
API from
different vendors.
Everybody will think their API is the best - atleast thats what
happend with
CORBA.
http://portal.acm.org/citation.cfm?
id=1142044&coll=GUIDE&dl=GUIDE&CFID=24831971&CFTOKEN=13722890&ret=1 
#F

ulltext


Isn't that why you need a standards body to lay down the
specification? Remove the vendors from the solution and allow the WG
to present the API much like Sun have done, IIRC. Vendors had
implemented JMS as it was being standardized and found it lacking so
the spec had to be updated.

AMQP only defines the wire level protocol I'm not sure that will be
enough for global domination and the breaking down of the vendor
lock-in. I could of course imagine a world where vendors simply  
supply
brokers and everyone uses a qpid or other open source client  
library.

Some risk averse institutions may still not like the idea of using
OSS. They could be forced to use a vendor client that may have a
licence that prevents them easily changing brokers.

I can understand the language issues with creating a _single_ client
API but still think you could do versions per language that are more
natural to each of the language styles. This is probably more of an
issue for languages that already have a large install base of
messaging products with an existing pervasive API. I'm thinking  
mainly

of Java/C++/.NET not sure how many messaging systems that currently
have an erlang or javascript client library. I'd also be thinking
about the openness of any of those APIs, i.e. IBMs XMS maybe widely
used in C++ but that doesn't mean I'd advocate creating a  
standardized

AMQP extension to it. I think Java and .NET are quite unique with
their messaging APIs given they are hardware agnostic their creators
have defined a messaging specification that in turn allows  
vendors to

create their own implementations.

Perhaps this is a non-issue in the real world, but if current
messaging vendors are worried about AMQP then this could be the way
the maintain their hold of clients.

just a thought.
--
Martin Ritchie


 From my point of view it's very cool to stop the standards at the
wire level protocol. That means several java API's can coexist, and
people can use several brokers. Freedom increases. If two gropus
within QPId can't agree it's actually possible to make two different
java API's - or I can use the RabbitMQ java client against the QPid
broker. This is perfect!


I would like the ability to write my java app using AMQP with Qpid
then simply swap to use the RabbitMQ java client. If I can't do this
then I am locked in to the Qpid code base. Sure I can change broker
and the AMQP Spec ensures that it will work but I'm stuck with the
Qpid client unless I re-write.

Is it just me that thinks this is an issue? In the real world does it
not matter?

Anyway time to go home for me.

--
Martin Ritchie


Why?

The client provides a binding between wire level protocol and  
language API. What do you think the RabbitMQ client will get you that  
the QPid client will not get you, if the API is exactly the same?


On the other hand, what if one API was good for one style of  
programming and another API was good for another style of programming  
- or compatible with different libraries - and I could choose based  
on my prefered API and still be fully AMQP compatible and choose  
whatever broker I would like to?


From my point of view the client is just a way to present a the wire  
protocol to a language. I could even choose to build several simple  
API's, for each kind of standard use case - and they'd still be AMQP  
compatible.


On the other hand, If the API has to go through AMQP WG, I have to  
change the API when they come with a new proposal. If the API is not  
part of the standard I can modify my old 0-8 client API to  
communicate with the super new 0-25 wire protocol and broker and not  
have to change my application to be 

Re: [Java] Message API design notes

[Robert Godfrey]

On 22/08/07, Carl Trieloff <[EMAIL PROTECTED]> wrote:
>
> Robert Godfrey wrote:
> > My view...
> >
> > In any sane development environment people who code in Java are going to
> > want to code against JMS.
>
> My view is...
>
> Same as Robs for App development guys, they will most likely use JMS.
>
> For people that are building services of infrastructure on top of us
> will NOT use JMS.
>
> So my view - we need both.



Yes - my view is that if you are building infrastructure that is the same
level as building the JMS API - i.e. building on top of the low level AMQP
API.  I strongly believe that this AMQP API needs to be public and well
documented.  The point of using a different API for infrastructure is (one
would presume) from getting the efficiencies of being closer to the wire
protocol?

-- Rob

> Carl.
>
>

Re: [Java] Message API design notes

[Carl Trieloff]

Robert Godfrey wrote:

My view...

In any sane development environment people who code in Java are going to
want to code against JMS.  


My view is...

Same as Robs for App development guys, they will most likely use JMS.

For people that are building services of infrastructure on top of us 
will NOT use JMS.


So my view - we need both.

Carl.

Re: [Java] Message API design notes

[Robert Godfrey]

My view...

In any sane development environment people who code in Java are going to
want to code against JMS.  That is THE standard for Java Messaging.  What we
want to be able to do is allow these people to progress to use features that
are in AMQP but cannot be exposed purely through a JMS implementation.  This
implies to me that either you extend the JMS interfaces or you have your
concrete implementation classes implement both the JMS interface and a
non-overlapping set of extensions.

There is a case for having a lower level API which maps as closely as
possible onto the AMQP classes and methods.  There are two reasons for
this.  This API would be what the JMS laye uses under the covers.  It will
also be the API that hardcore AMQP fanatics :-) use ...

>From and AMQP standards point of view I think that both this low level API
and the extensions to JMS should be standardised to allow people to swap
between AMQP clients (after all you may use Qpid for a while, and then
discover that Rabbit have coded a client library that is 30% faster or added
some very cool proprietary extension that you can only use via their API...
in which case you shouldn't need to rewrite all the stuff that is vanilla
AMQP).

I do not particularly see the need for an API which is neither JMS that
doesn't look exactly like the model layers of AMQP [that is the API need not
expose all the 0-10 features for connection establishment, session
maintenance etc... those are functions of the client library and shouldn't
be exposed to the application programmer - For those familiar with AMQP 0-10
I think the API should provide hooks into all of Layer 4... with
abstractions for the lower layers].

There is a genuine debate to be had about the technical design of the lower
level API and the uses of interfaces vs. classes etc...  My personal view on
this is that we let the guys finish what they are working on now, and then
review what it looks like when complete.

-- Rob

On 22/08/07, Martin Ritchie <[EMAIL PROTECTED]> wrote:
>
> On 22/08/07, Michael Arnoldus <[EMAIL PROTECTED]> wrote:
> > Den 22/08/2007 kl. 19.08 skrev Martin Ritchie:
> >
> > > On 22/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> > >>>
> > >>> But not one that allows access to AMQP functionality. I don't see
> > >>> why
> > >>> the AMQP WG couldn't provide a set of Interfaces and a client lib
> > >>> could then implement jmx.Session and amqp.Session. Then again I've
> > >>> never had any exposure to CORBA to know what they did wrong. Is the
> > >>> lesson simply never to provide an API? That seems a bit harsh.
> > >>>
> > >>
> > >> Two reasons.
> > >> Defining an API is time consuming and difficult given the no of
> > >> languages we
> > >> have.
> > >> Also they differ from object oriented to procedural to functional.
> > >> So obviously there will be different views on how best it should
> > >> be done in
> > >> a given language.
> > >> I heard of client implementations in the following languages so far.
> > >> java, c++, python, ruby, erlang? , javascript - and most like C
> > >> and perl.
> > > [not forgetting our own .NET :)]
> > >
> > >
> > >
> > >> The second reason is more political.
> > >> I think it is a very difficult exercise to get agreement over the
> > >> API from
> > >> different vendors.
> > >> Everybody will think their API is the best - atleast thats what
> > >> happend with
> > >> CORBA.
> > >> http://portal.acm.org/citation.cfm?
> > >> id=1142044&coll=GUIDE&dl=GUIDE&CFID=24831971&CFTOKEN=13722890&ret=1#F
> > >> ulltext
> > >
> > > Isn't that why you need a standards body to lay down the
> > > specification? Remove the vendors from the solution and allow the WG
> > > to present the API much like Sun have done, IIRC. Vendors had
> > > implemented JMS as it was being standardized and found it lacking so
> > > the spec had to be updated.
> > >
> > > AMQP only defines the wire level protocol I'm not sure that will be
> > > enough for global domination and the breaking down of the vendor
> > > lock-in. I could of course imagine a world where vendors simply supply
> > > brokers and everyone uses a qpid or other open source client library.
> > > Some risk averse institutions may still not like the idea of using
> > > OSS. They could be forced to use a vendor client that may have a
> > > licence that prevents them easily changing brokers.
> > >
> > > I can understand the language issues with creating a _single_ client
> > > API but still think you could do versions per language that are more
> > > natural to each of the language styles. This is probably more of an
> > > issue for languages that already have a large install base of
> > > messaging products with an existing pervasive API. I'm thinking mainly
> > > of Java/C++/.NET not sure how many messaging systems that currently
> > > have an erlang or javascript client library. I'd also be thinking
> > > about the openness of any of those APIs, i.e. IBMs XMS maybe widely
> > > used in C++ but that doesn'

Re: [Java] Message API design notes

[Martin Ritchie]

On 22/08/07, Michael Arnoldus <[EMAIL PROTECTED]> wrote:
> Den 22/08/2007 kl. 19.08 skrev Martin Ritchie:
>
> > On 22/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> >>>
> >>> But not one that allows access to AMQP functionality. I don't see
> >>> why
> >>> the AMQP WG couldn't provide a set of Interfaces and a client lib
> >>> could then implement jmx.Session and amqp.Session. Then again I've
> >>> never had any exposure to CORBA to know what they did wrong. Is the
> >>> lesson simply never to provide an API? That seems a bit harsh.
> >>>
> >>
> >> Two reasons.
> >> Defining an API is time consuming and difficult given the no of
> >> languages we
> >> have.
> >> Also they differ from object oriented to procedural to functional.
> >> So obviously there will be different views on how best it should
> >> be done in
> >> a given language.
> >> I heard of client implementations in the following languages so far.
> >> java, c++, python, ruby, erlang? , javascript - and most like C
> >> and perl.
> > [not forgetting our own .NET :)]
> >
> >
> >
> >> The second reason is more political.
> >> I think it is a very difficult exercise to get agreement over the
> >> API from
> >> different vendors.
> >> Everybody will think their API is the best - atleast thats what
> >> happend with
> >> CORBA.
> >> http://portal.acm.org/citation.cfm?
> >> id=1142044&coll=GUIDE&dl=GUIDE&CFID=24831971&CFTOKEN=13722890&ret=1#F
> >> ulltext
> >
> > Isn't that why you need a standards body to lay down the
> > specification? Remove the vendors from the solution and allow the WG
> > to present the API much like Sun have done, IIRC. Vendors had
> > implemented JMS as it was being standardized and found it lacking so
> > the spec had to be updated.
> >
> > AMQP only defines the wire level protocol I'm not sure that will be
> > enough for global domination and the breaking down of the vendor
> > lock-in. I could of course imagine a world where vendors simply supply
> > brokers and everyone uses a qpid or other open source client library.
> > Some risk averse institutions may still not like the idea of using
> > OSS. They could be forced to use a vendor client that may have a
> > licence that prevents them easily changing brokers.
> >
> > I can understand the language issues with creating a _single_ client
> > API but still think you could do versions per language that are more
> > natural to each of the language styles. This is probably more of an
> > issue for languages that already have a large install base of
> > messaging products with an existing pervasive API. I'm thinking mainly
> > of Java/C++/.NET not sure how many messaging systems that currently
> > have an erlang or javascript client library. I'd also be thinking
> > about the openness of any of those APIs, i.e. IBMs XMS maybe widely
> > used in C++ but that doesn't mean I'd advocate creating a standardized
> > AMQP extension to it. I think Java and .NET are quite unique with
> > their messaging APIs given they are hardware agnostic their creators
> > have defined a messaging specification that in turn allows vendors to
> > create their own implementations.
> >
> > Perhaps this is a non-issue in the real world, but if current
> > messaging vendors are worried about AMQP then this could be the way
> > the maintain their hold of clients.
> >
> > just a thought.
> > --
> > Martin Ritchie
>
>  From my point of view it's very cool to stop the standards at the
> wire level protocol. That means several java API's can coexist, and
> people can use several brokers. Freedom increases. If two gropus
> within QPId can't agree it's actually possible to make two different
> java API's - or I can use the RabbitMQ java client against the QPid
> broker. This is perfect!

I would like the ability to write my java app using AMQP with Qpid
then simply swap to use the RabbitMQ java client. If I can't do this
then I am locked in to the Qpid code base. Sure I can change broker
and the AMQP Spec ensures that it will work but I'm stuck with the
Qpid client unless I re-write.

Is it just me that thinks this is an issue? In the real world does it
not matter?

Anyway time to go home for me.

> Regards,
>
> Michael Arnoldus
>
>
>


-- 
Martin Ritchie

Re: [Java] Message API design notes

[Michael Arnoldus]

Den 22/08/2007 kl. 19.08 skrev Martin Ritchie:


On 22/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:


But not one that allows access to AMQP functionality. I don't see  
why

the AMQP WG couldn't provide a set of Interfaces and a client lib
could then implement jmx.Session and amqp.Session. Then again I've
never had any exposure to CORBA to know what they did wrong. Is the
lesson simply never to provide an API? That seems a bit harsh.



Two reasons.
Defining an API is time consuming and difficult given the no of  
languages we

have.
Also they differ from object oriented to procedural to functional.
So obviously there will be different views on how best it should  
be done in

a given language.
I heard of client implementations in the following languages so far.
java, c++, python, ruby, erlang? , javascript - and most like C  
and perl.

[not forgetting our own .NET :)]




The second reason is more political.
I think it is a very difficult exercise to get agreement over the  
API from

different vendors.
Everybody will think their API is the best - atleast thats what  
happend with

CORBA.
http://portal.acm.org/citation.cfm? 
id=1142044&coll=GUIDE&dl=GUIDE&CFID=24831971&CFTOKEN=13722890&ret=1#F 
ulltext


Isn't that why you need a standards body to lay down the
specification? Remove the vendors from the solution and allow the WG
to present the API much like Sun have done, IIRC. Vendors had
implemented JMS as it was being standardized and found it lacking so
the spec had to be updated.

AMQP only defines the wire level protocol I'm not sure that will be
enough for global domination and the breaking down of the vendor
lock-in. I could of course imagine a world where vendors simply supply
brokers and everyone uses a qpid or other open source client library.
Some risk averse institutions may still not like the idea of using
OSS. They could be forced to use a vendor client that may have a
licence that prevents them easily changing brokers.

I can understand the language issues with creating a _single_ client
API but still think you could do versions per language that are more
natural to each of the language styles. This is probably more of an
issue for languages that already have a large install base of
messaging products with an existing pervasive API. I'm thinking mainly
of Java/C++/.NET not sure how many messaging systems that currently
have an erlang or javascript client library. I'd also be thinking
about the openness of any of those APIs, i.e. IBMs XMS maybe widely
used in C++ but that doesn't mean I'd advocate creating a standardized
AMQP extension to it. I think Java and .NET are quite unique with
their messaging APIs given they are hardware agnostic their creators
have defined a messaging specification that in turn allows vendors to
create their own implementations.

Perhaps this is a non-issue in the real world, but if current
messaging vendors are worried about AMQP then this could be the way
the maintain their hold of clients.

just a thought.
--
Martin Ritchie


From my point of view it's very cool to stop the standards at the  
wire level protocol. That means several java API's can coexist, and  
people can use several brokers. Freedom increases. If two gropus  
within QPId can't agree it's actually possible to make two different  
java API's - or I can use the RabbitMQ java client against the QPid  
broker. This is perfect!


Regards,

Michael Arnoldus

Re: [Java] Message API design notes

[Martin Ritchie]

On 22/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> >
> > But not one that allows access to AMQP functionality. I don't see why
> > the AMQP WG couldn't provide a set of Interfaces and a client lib
> > could then implement jmx.Session and amqp.Session. Then again I've
> > never had any exposure to CORBA to know what they did wrong. Is the
> > lesson simply never to provide an API? That seems a bit harsh.
> >
>
> Two reasons.
> Defining an API is time consuming and difficult given the no of languages we
> have.
> Also they differ from object oriented to procedural to functional.
> So obviously there will be different views on how best it should be done in
> a given language.
> I heard of client implementations in the following languages so far.
> java, c++, python, ruby, erlang? , javascript - and most like C and perl.
>
> The second reason is more political.
> I think it is a very difficult exercise to get agreement over the API from
> different vendors.
> Everybody will think their API is the best - atleast thats what happend with
> CORBA.
> http://portal.acm.org/citation.cfm?id=1142044&coll=GUIDE&dl=GUIDE&CFID=24831971&CFTOKEN=13722890&ret=1#Fulltext

Interesting read. Seems like the major problem wasn't the existence of
the API but more the WG inability to weed out the poor ideas and just
throw everything in. From what I've seen the AMQP WG is much tighter
and more focused on a cohesive solution. AMQP also has a number of
implementations that will help work out the wire level protocol. Where
as CORBA didn't have any.

Not sure the article argues that having an API is a bad thing. In fact
the lack of a .NET binding was cited as part of the downfall.

I think we should be developing our qpid API, as we are doing, then
extracting the interfaces and suggesting them to the AMQP WG, it would
be a good starting point for any given languages API. The fact that we
suggested the API to the AMQP WG doesn't mean they have to accept it
but with feedback from users about its the ease of use then that would
help the WG, form an opinion and direction to take.




> Regards,
>
> Rajith
>


-- 
Martin Ritchie

Re: [Java] Message API design notes

[Martin Ritchie]

On 22/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> >
> > But not one that allows access to AMQP functionality. I don't see why
> > the AMQP WG couldn't provide a set of Interfaces and a client lib
> > could then implement jmx.Session and amqp.Session. Then again I've
> > never had any exposure to CORBA to know what they did wrong. Is the
> > lesson simply never to provide an API? That seems a bit harsh.
> >
>
> Two reasons.
> Defining an API is time consuming and difficult given the no of languages we
> have.
> Also they differ from object oriented to procedural to functional.
> So obviously there will be different views on how best it should be done in
> a given language.
> I heard of client implementations in the following languages so far.
> java, c++, python, ruby, erlang? , javascript - and most like C and perl.
[not forgetting our own .NET :)]



> The second reason is more political.
> I think it is a very difficult exercise to get agreement over the API from
> different vendors.
> Everybody will think their API is the best - atleast thats what happend with
> CORBA.
> http://portal.acm.org/citation.cfm?id=1142044&coll=GUIDE&dl=GUIDE&CFID=24831971&CFTOKEN=13722890&ret=1#Fulltext

Isn't that why you need a standards body to lay down the
specification? Remove the vendors from the solution and allow the WG
to present the API much like Sun have done, IIRC. Vendors had
implemented JMS as it was being standardized and found it lacking so
the spec had to be updated.

AMQP only defines the wire level protocol I'm not sure that will be
enough for global domination and the breaking down of the vendor
lock-in. I could of course imagine a world where vendors simply supply
brokers and everyone uses a qpid or other open source client library.
Some risk averse institutions may still not like the idea of using
OSS. They could be forced to use a vendor client that may have a
licence that prevents them easily changing brokers.

I can understand the language issues with creating a _single_ client
API but still think you could do versions per language that are more
natural to each of the language styles. This is probably more of an
issue for languages that already have a large install base of
messaging products with an existing pervasive API. I'm thinking mainly
of Java/C++/.NET not sure how many messaging systems that currently
have an erlang or javascript client library. I'd also be thinking
about the openness of any of those APIs, i.e. IBMs XMS maybe widely
used in C++ but that doesn't mean I'd advocate creating a standardized
AMQP extension to it. I think Java and .NET are quite unique with
their messaging APIs given they are hardware agnostic their creators
have defined a messaging specification that in turn allows vendors to
create their own implementations.

Perhaps this is a non-issue in the real world, but if current
messaging vendors are worried about AMQP then this could be the way
the maintain their hold of clients.

just a thought.
-- 
Martin Ritchie

Re: [Java] Message API design notes

[Rajith Attapattu]

>
> But not one that allows access to AMQP functionality. I don't see why
> the AMQP WG couldn't provide a set of Interfaces and a client lib
> could then implement jmx.Session and amqp.Session. Then again I've
> never had any exposure to CORBA to know what they did wrong. Is the
> lesson simply never to provide an API? That seems a bit harsh.
>

Two reasons.
Defining an API is time consuming and difficult given the no of languages we
have.
Also they differ from object oriented to procedural to functional.
So obviously there will be different views on how best it should be done in
a given language.
I heard of client implementations in the following languages so far.
java, c++, python, ruby, erlang? , javascript - and most like C and perl.

The second reason is more political.
I think it is a very difficult exercise to get agreement over the API from
different vendors.
Everybody will think their API is the best - atleast thats what happend with
CORBA.
http://portal.acm.org/citation.cfm?id=1142044&coll=GUIDE&dl=GUIDE&CFID=24831971&CFTOKEN=13722890&ret=1#Fulltext

Regards,

Rajith

Re: [Java] Message API design notes

[Martin Ritchie]

On 22/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> On 8/21/07, Martin Ritchie <[EMAIL PROTECTED]> wrote:
> >
> > On 21/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> > > >
> > > > This increasing usage comes with new requirements some of which native
> > > > AMQP
> > > > functionality can achieve. Should we require the client to re-write
> > > > their app using the Qpid API? I hope the answer is no, as the re-write
> > > > effort would most likely eliminate any potential saving in moving to
> > > > Qpid.
> > >
> > >
> > > This is a tough question to answer. Depending on what is needed they may
> > > need to rewrite certain portions.
> > > Rewriting doesn't necessarily mean that you loose savings.
> > > After all which application can survive feature requests forever without
> > > having to go through modifications.
> > >
> > > My argument is that you can only expose simple AMQP functionality as JMS
> > > extensions. If they want to do more then it make sense to program at the
> > > AMQP level instead of JMS. Bcos once u do that then there is no
> > advantage in
> > > sticking to JMS. You simply can't move to another JMS provider bcos your
> > > code is using AMQP extensions. You got to use the right tool for the
> > right
> > > job. Hybrid tools will only take you so far.
> >
> > But if AMQP provided an API you could move between AMQP implementations.
>
>
> As I said I doubt AMQP will ever provide language specific API.
> It shouldn't make the same mistake CORBA did.
> Besides Rupert and you keep reminding me that Java "already" has a messaging
> API :)

But not one that allows access to AMQP functionality. I don't see why
the AMQP WG couldn't provide a set of Interfaces and a client lib
could then implement jmx.Session and amqp.Session. Then again I've
never had any exposure to CORBA to know what they did wrong. Is the
lesson simply never to provide an API? That seems a bit harsh.

> > As concrete examples are good. Two recent applications that I've
> > > > assisted developing have both written code to send a ping message to
> > > > the broker to ensure the connection is up. I felt terrible telling
> > > > them that Qpid currently does that but sorry you can't hook in to the
> > > > callbacks.
> > >
> > >
> > > That is more due to poor design than anything else. Asking an
> > application to
> > > do that is not the right thing.
> > > We should provide a connection exception listener which should notify
> > the
> > > application with a proper error code that notifies when the connection
> > is
> > > lost or when the broker terminates by calling close. In the event of
> > > connection lost it should try failover and only notify when it is
> > > unsuccessful.
> > > In fact you can do that using JMS exception listener too. The only thing
> > the
> > > application needs to do is to interpret the error code properly.
> > > The client code in trunk does that.
> >
> > And on the branch but AFAICS there isn't a way to be notified of every
> > 'idle ping', which is what the applications wanted to do. I'm making
> > no comments about the reasoning behind this simply pointing out that
> > clients want access to the events on the client and when they can't
> > get it they re-implement it.
>
>
> With the restructure you can easily hook into events if you need to.
> In general an app shouldn't bother with low level pings and should only
> care/know about when the connection is lost.
> Without knowing the specifics of your use case I can't comment either -
> however I understand the need for you to be silent on the issue.

I don't know the specifics of the use case either. I agree with you
that the client shouldn't care about ping-ing and should just watch
for the connection being lost but users sometimes can't be told. They
want to write their app the way they want to and given the protocol
does what they want it seemed to make sense to be able to provide the
hooks.

> > So I am very much in favour of JMS + AMQP extensions. I
> > > > don't really see any alternative. I can see why JMS would discourage
> > > > vendor extensions but don't forget we are not adding vendor specific
> > > > extensions; We are exposing AMQP functionality.
> > >
> > >
> > > As I said this is a poor effort in exposing AMQP functionality.
> > > There are certain functionality that you cannot expose that way. For
> > example
> > > configuring the confirm mode and acquire mode for a transfer.
> > > Unless you cast it to the AMQ class and then use those methods.
> > > AMQSession ssn = (AMQSession)jmsSession;
> > > ssn.messageTransfer()
> >
> > Ideally you would cast to an AMQP interface that is moving towards a
> > standard provided by the AMQP WG.
>
>
> Qpid Session API aims to be this interface from Qpid.
> Come 0-11, people who code against this API may only see minimal changes.
>
> > If you do that, then what is the purpose of using JMS? I might as well
> > code
> > > to AMQSession class.
> >
> > If it is an interface then you 

Re: [Java] Message API design notes

[Rupert Smith]

Should just have done that the first time round!

Here it is:
http://cwiki.apache.org/confluence/display/qpid/Low-Level+API+Diagram

Rupert

On 22/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
>
> Rupert,
>
> I think the zip got filtered out.
> Howe about attaching it to the wiki and sending a link.
>
> Rajith
>
> On 8/22/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
> >
> > There is a subtle difference between the structure I have described,
> that
> > is
> > dimensions 1 + 2,  versus the comm layer + proposed client API as a
> > layered
> > system.
> >
> > As two separate dimensions, a client application would be coded against
> > both.
> > It would use dimension 1 to invoke the protocol directly as an API
> defined
> > purely in terms of interfaces (server chassis). It would use dimension 2
> > as
> > a concrete utility set for doing the common interactions that customers
> do
> > not want to have to recode themselves. This is the idea behind the
> > interfaces ProtocolBroker and ProtocolConversation that I described. For
> > example:
> >
> > // This is the utility class, it provides the same interface as the AMQP
> > protocol 1:1 API, by extending ProtocolBroker.
> > // It implements ProtocolClient, because it can be registered as a
> > delegate,
> > for the comm layer (or direct in-vm routing layer) to notify of
> > // events.
> > // It adds synchronous receive methods to wait for incoming methods, and
> > utility methods for the common interaction stuff.
> > public ProtocolConversation extends ProtocolBroker , ProtocolClient
> > {
> >// To block for incoming methods.
> >public MethodSessionAttached receiveSessionAttached();
> >... and so on.
> >
> >   // Common interactions.
> >   public ConnectionContext openConnectionSequence();
> >   public SessionContext openSessionSequence(ConnectionContext
> connection);
> >    and so on.
> > }
> >
> > In the layered approach, using the proposed client API, you are
> presenting
> > the client API as a complete abstraction with a different interface to
> > that
> > which the comm layer presents. As an abstraction it presents a complete
> > environment for client to program against. It re-presents the direct
> > protocol 1:1 API a second time, as a completely separate type, filtering
> > down to just what a client needs to program against, fair enough, but to
> > me
> > this unnecessary fragmentation or duplication of the type structure of
> the
> > program. That is, its not as if you have Session from the proposed
> client
> > API, implementing Invoker from the comm layer, even though both are
> > presenting AMQP 1:1.
> >
> > I am proposing the use of interfaces as a common abstraction that I can
> > pull
> > out of both the comm layer, and the client API, not to mention some
> future
> > broker routing layer that we will have.
> >
> > This discussion about the layering of the protocol has given me another
> > idea. If the end client only needs to see model layer + a limited amount
> > of
> > stuff from some of other layers too? This can be presented as a trimmed
> > down
> > version of the full 1:1 protocol API (you guessed it, as an interface,
> > whoopee!) for the sole purpose of writing clients apps against. So at
> the
> > moment I have ProtocolBroker, which exposes the whole of the AMQP server
> > chassis. I can also define:
> >
> > public interface ClientInvoker
> > {
> > // public void declareQueue(SessionContext session,
> MethodQueueDeclare
> > qd);
> >
> > // But I think we are preferring the exposed argument form for
> coding
> > against, fair enough, I do too.
> > public void declareQueue(SessionContext session, String queueName,
> > boolean durable, Blah blah);
> >
> > // And so on...
> > }
> >
> > // This one is equivalent to MessageListener, but expressed more
> directly
> > in
> > terms of AMQP.
> > public interface ClientDelegate
> > {
> > public void messageTransfer(SessionContext session,
> > MethodMessageTransfer mt);
> >
> > // I'm not sure now what pings are going to look like, so this is
> just
> > a
> > guess, but I think you get the idea.
> > public void pingPing(ConnectionContext connection, MethodPingPing
> pp);
> > // or is it session context?
> >
> > // And so on, for whatever else we thinks clients are going to want
> to
> > see...
> > }
> >
> > I would just make sure that I mark the ProtocolBroker and
> ProtocolClient,
> > as
> > extending both of these interfaces. Nothing else needs to be done, as
> > these
> > methods are already in the ProtocolBroker and ProtocolClient interfaces
> > because of the ClassXXX interfaces that they extend. Quick example:
> >
> > public interface ProtocolBroker extends ClassMessageBroker,
> > ClassSessionBroker, ClassExecutionBroker, ... , ClientInvoker
> > public interface ProtocolClient extends ClassMessageClient,
> > ClassSessionClient, ClassExecutionClient, ..., ClientDelegate
> >
> > Then I can add to the factory methods that return narrowed, c

Re: [Java] Message API design notes

[Rajith Attapattu]

Rupert,

I think the zip got filtered out.
Howe about attaching it to the wiki and sending a link.

Rajith

On 8/22/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
>
> There is a subtle difference between the structure I have described, that
> is
> dimensions 1 + 2,  versus the comm layer + proposed client API as a
> layered
> system.
>
> As two separate dimensions, a client application would be coded against
> both.
> It would use dimension 1 to invoke the protocol directly as an API defined
> purely in terms of interfaces (server chassis). It would use dimension 2
> as
> a concrete utility set for doing the common interactions that customers do
> not want to have to recode themselves. This is the idea behind the
> interfaces ProtocolBroker and ProtocolConversation that I described. For
> example:
>
> // This is the utility class, it provides the same interface as the AMQP
> protocol 1:1 API, by extending ProtocolBroker.
> // It implements ProtocolClient, because it can be registered as a
> delegate,
> for the comm layer (or direct in-vm routing layer) to notify of
> // events.
> // It adds synchronous receive methods to wait for incoming methods, and
> utility methods for the common interaction stuff.
> public ProtocolConversation extends ProtocolBroker , ProtocolClient
> {
>// To block for incoming methods.
>public MethodSessionAttached receiveSessionAttached();
>... and so on.
>
>   // Common interactions.
>   public ConnectionContext openConnectionSequence();
>   public SessionContext openSessionSequence(ConnectionContext connection);
>    and so on.
> }
>
> In the layered approach, using the proposed client API, you are presenting
> the client API as a complete abstraction with a different interface to
> that
> which the comm layer presents. As an abstraction it presents a complete
> environment for client to program against. It re-presents the direct
> protocol 1:1 API a second time, as a completely separate type, filtering
> down to just what a client needs to program against, fair enough, but to
> me
> this unnecessary fragmentation or duplication of the type structure of the
> program. That is, its not as if you have Session from the proposed client
> API, implementing Invoker from the comm layer, even though both are
> presenting AMQP 1:1.
>
> I am proposing the use of interfaces as a common abstraction that I can
> pull
> out of both the comm layer, and the client API, not to mention some future
> broker routing layer that we will have.
>
> This discussion about the layering of the protocol has given me another
> idea. If the end client only needs to see model layer + a limited amount
> of
> stuff from some of other layers too? This can be presented as a trimmed
> down
> version of the full 1:1 protocol API (you guessed it, as an interface,
> whoopee!) for the sole purpose of writing clients apps against. So at the
> moment I have ProtocolBroker, which exposes the whole of the AMQP server
> chassis. I can also define:
>
> public interface ClientInvoker
> {
> // public void declareQueue(SessionContext session, MethodQueueDeclare
> qd);
>
> // But I think we are preferring the exposed argument form for coding
> against, fair enough, I do too.
> public void declareQueue(SessionContext session, String queueName,
> boolean durable, Blah blah);
>
> // And so on...
> }
>
> // This one is equivalent to MessageListener, but expressed more directly
> in
> terms of AMQP.
> public interface ClientDelegate
> {
> public void messageTransfer(SessionContext session,
> MethodMessageTransfer mt);
>
> // I'm not sure now what pings are going to look like, so this is just
> a
> guess, but I think you get the idea.
> public void pingPing(ConnectionContext connection, MethodPingPing pp);
> // or is it session context?
>
> // And so on, for whatever else we thinks clients are going to want to
> see...
> }
>
> I would just make sure that I mark the ProtocolBroker and ProtocolClient,
> as
> extending both of these interfaces. Nothing else needs to be done, as
> these
> methods are already in the ProtocolBroker and ProtocolClient interfaces
> because of the ClassXXX interfaces that they extend. Quick example:
>
> public interface ProtocolBroker extends ClassMessageBroker,
> ClassSessionBroker, ClassExecutionBroker, ... , ClientInvoker
> public interface ProtocolClient extends ClassMessageClient,
> ClassSessionClient, ClassExecutionClient, ..., ClientDelegate
>
> Then I can add to the factory methods that return narrowed, client only
> interfaces (for dummies):
>
> public interface ProtocolFactory
> {
>// To get full invokers to call the protocol through.
>public ProtocolBroker getBrokerInvoker();
>public ProtocolClient getClientInvoker();
>
>// To register delegates to be informed of incoming methods-as-events.
>public void registerBrokerDelegate(ProtocolBroker delegate);
>public void registerClientDelegate(ProtocolClient delegate);
>
>// To get/reg

Re: [Java] Message API design notes

[Robert Godfrey]

Dentoing both subchannel and layer in the XML will be done...  we've started
talking about this issue now in the AMQP 0-10 group... hopefully the xml
will soon be updated with this information.

-- Rob

On 22/08/07, Alan Conway <[EMAIL PROTECTED]> wrote:
>
> On Tue, 2007-08-21 at 14:44 -0400, Carl Trieloff wrote:
> > > Is it the case that every method belongs to one of these layers?
> >
> >
> > I would expect so.
>
> Yes. The layering is described in the spec text and in various JIRAs.
>
> > > In which case, will the layer numbers be put into the 0-10 xml?
> > >
> >
> > No idea, or a least I don't know of any vote to do so. However the
> > channels in 0-10 should be enough
> > to derive this, so I don't think we need it in the XML.
>
> The "subchannel" id provides this kind of association. Noting it in the
> XML has been discussed but I'm not sure if/when it will actually happen.
> Logically it belongs in the XML as its part of the definition of a
> method.
>
>
>
>

Re: [Java] Message API design notes

[Rajith Attapattu]

On 8/21/07, Martin Ritchie <[EMAIL PROTECTED]> wrote:
>
> On 21/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> > >
> > > This increasing usage comes with new requirements some of which native
> > > AMQP
> > > functionality can achieve. Should we require the client to re-write
> > > their app using the Qpid API? I hope the answer is no, as the re-write
> > > effort would most likely eliminate any potential saving in moving to
> > > Qpid.
> >
> >
> > This is a tough question to answer. Depending on what is needed they may
> > need to rewrite certain portions.
> > Rewriting doesn't necessarily mean that you loose savings.
> > After all which application can survive feature requests forever without
> > having to go through modifications.
> >
> > My argument is that you can only expose simple AMQP functionality as JMS
> > extensions. If they want to do more then it make sense to program at the
> > AMQP level instead of JMS. Bcos once u do that then there is no
> advantage in
> > sticking to JMS. You simply can't move to another JMS provider bcos your
> > code is using AMQP extensions. You got to use the right tool for the
> right
> > job. Hybrid tools will only take you so far.
>
> But if AMQP provided an API you could move between AMQP implementations.


As I said I doubt AMQP will ever provide language specific API.
It shouldn't make the same mistake CORBA did.
Besides Rupert and you keep reminding me that Java "already" has a messaging
API :)

> As concrete examples are good. Two recent applications that I've
> > > assisted developing have both written code to send a ping message to
> > > the broker to ensure the connection is up. I felt terrible telling
> > > them that Qpid currently does that but sorry you can't hook in to the
> > > callbacks.
> >
> >
> > That is more due to poor design than anything else. Asking an
> application to
> > do that is not the right thing.
> > We should provide a connection exception listener which should notify
> the
> > application with a proper error code that notifies when the connection
> is
> > lost or when the broker terminates by calling close. In the event of
> > connection lost it should try failover and only notify when it is
> > unsuccessful.
> > In fact you can do that using JMS exception listener too. The only thing
> the
> > application needs to do is to interpret the error code properly.
> > The client code in trunk does that.
>
> And on the branch but AFAICS there isn't a way to be notified of every
> 'idle ping', which is what the applications wanted to do. I'm making
> no comments about the reasoning behind this simply pointing out that
> clients want access to the events on the client and when they can't
> get it they re-implement it.


With the restructure you can easily hook into events if you need to.
In general an app shouldn't bother with low level pings and should only
care/know about when the connection is lost.
Without knowing the specifics of your use case I can't comment either -
however I understand the need for you to be silent on the issue.

> So I am very much in favour of JMS + AMQP extensions. I
> > > don't really see any alternative. I can see why JMS would discourage
> > > vendor extensions but don't forget we are not adding vendor specific
> > > extensions; We are exposing AMQP functionality.
> >
> >
> > As I said this is a poor effort in exposing AMQP functionality.
> > There are certain functionality that you cannot expose that way. For
> example
> > configuring the confirm mode and acquire mode for a transfer.
> > Unless you cast it to the AMQ class and then use those methods.
> > AMQSession ssn = (AMQSession)jmsSession;
> > ssn.messageTransfer()
>
> Ideally you would cast to an AMQP interface that is moving towards a
> standard provided by the AMQP WG.


Qpid Session API aims to be this interface from Qpid.
Come 0-11, people who code against this API may only see minimal changes.

> If you do that, then what is the purpose of using JMS? I might as well
> code
> > to AMQSession class.
>
> If it is an interface then you may very well do that, as your
> application evolves.
>
> > Anyone implementing
> > > AMQP based applications need to be acutely aware that the protocol is
> > > still at a beta level.
> >
> >
> > Folks you want to do that will do with that explicit understanding.
> > From 0-10 onwards I doubt we will see drastic changes.
>
> That may be true but there are no guarantees offered by the WG until 1.0


Well given the way things are going right now, I feel there want be any
radical changes.
I agree that there are no garuntees though.

> When AMQP hits 1.0 and historic version
> > > compatibility is required then use of these extensions should be less
> > > risky than they are just now. This will be greatly assisted when AMQP
> > > provide an API
> >
> >
> > I doubt the AMQP spec group will ever provide an API. I don't see a need
> to
> > do so.
> > The semantic model already defines how a broker and client should
> 

Re: [Java] Message API design notes

[Alan Conway]

On Tue, 2007-08-21 at 14:44 -0400, Carl Trieloff wrote:
> > Is it the case that every method belongs to one of these layers?
> 
> 
> I would expect so.

Yes. The layering is described in the spec text and in various JIRAs.

> > In which case, will the layer numbers be put into the 0-10 xml?
> >
> 
> No idea, or a least I don't know of any vote to do so. However the 
> channels in 0-10 should be enough
> to derive this, so I don't think we need it in the XML.

The "subchannel" id provides this kind of association. Noting it in the
XML has been discussed but I'm not sure if/when it will actually happen.
Logically it belongs in the XML as its part of the definition of a
method.

 

Re: [Java] Message API design notes

[Rupert Smith]

There is a subtle difference between the structure I have described, that is
dimensions 1 + 2,  versus the comm layer + proposed client API as a layered
system.

As two separate dimensions, a client application would be coded against both.
It would use dimension 1 to invoke the protocol directly as an API defined
purely in terms of interfaces (server chassis). It would use dimension 2 as
a concrete utility set for doing the common interactions that customers do
not want to have to recode themselves. This is the idea behind the
interfaces ProtocolBroker and ProtocolConversation that I described. For
example:

// This is the utility class, it provides the same interface as the AMQP
protocol 1:1 API, by extending ProtocolBroker.
// It implements ProtocolClient, because it can be registered as a delegate,
for the comm layer (or direct in-vm routing layer) to notify of
// events.
// It adds synchronous receive methods to wait for incoming methods, and
utility methods for the common interaction stuff.
public ProtocolConversation extends ProtocolBroker , ProtocolClient
{
   // To block for incoming methods.
   public MethodSessionAttached receiveSessionAttached();
   ... and so on.

  // Common interactions.
  public ConnectionContext openConnectionSequence();
  public SessionContext openSessionSequence(ConnectionContext connection);
   and so on.
}

In the layered approach, using the proposed client API, you are presenting
the client API as a complete abstraction with a different interface to that
which the comm layer presents. As an abstraction it presents a complete
environment for client to program against. It re-presents the direct
protocol 1:1 API a second time, as a completely separate type, filtering
down to just what a client needs to program against, fair enough, but to me
this unnecessary fragmentation or duplication of the type structure of the
program. That is, its not as if you have Session from the proposed client
API, implementing Invoker from the comm layer, even though both are
presenting AMQP 1:1.

I am proposing the use of interfaces as a common abstraction that I can pull
out of both the comm layer, and the client API, not to mention some future
broker routing layer that we will have.

This discussion about the layering of the protocol has given me another
idea. If the end client only needs to see model layer + a limited amount of
stuff from some of other layers too? This can be presented as a trimmed down
version of the full 1:1 protocol API (you guessed it, as an interface,
whoopee!) for the sole purpose of writing clients apps against. So at the
moment I have ProtocolBroker, which exposes the whole of the AMQP server
chassis. I can also define:

public interface ClientInvoker
{
// public void declareQueue(SessionContext session, MethodQueueDeclare
qd);

// But I think we are preferring the exposed argument form for coding
against, fair enough, I do too.
public void declareQueue(SessionContext session, String queueName,
boolean durable, Blah blah);

// And so on...
}

// This one is equivalent to MessageListener, but expressed more directly in
terms of AMQP.
public interface ClientDelegate
{
public void messageTransfer(SessionContext session,
MethodMessageTransfer mt);

// I'm not sure now what pings are going to look like, so this is just a
guess, but I think you get the idea.
public void pingPing(ConnectionContext connection, MethodPingPing pp);
// or is it session context?

// And so on, for whatever else we thinks clients are going to want to
see...
}

I would just make sure that I mark the ProtocolBroker and ProtocolClient, as
extending both of these interfaces. Nothing else needs to be done, as these
methods are already in the ProtocolBroker and ProtocolClient interfaces
because of the ClassXXX interfaces that they extend. Quick example:

public interface ProtocolBroker extends ClassMessageBroker,
ClassSessionBroker, ClassExecutionBroker, ... , ClientInvoker
public interface ProtocolClient extends ClassMessageClient,
ClassSessionClient, ClassExecutionClient, ..., ClientDelegate

Then I can add to the factory methods that return narrowed, client only
interfaces (for dummies):

public interface ProtocolFactory
{
   // To get full invokers to call the protocol through.
   public ProtocolBroker getBrokerInvoker();
   public ProtocolClient getClientInvoker();

   // To register delegates to be informed of incoming methods-as-events.
   public void registerBrokerDelegate(ProtocolBroker delegate);
   public void registerClientDelegate(ProtocolClient delegate);

   // To get/register narrowed down invokers/delegates for client
applications.
   public ClientInvoker getClientInvoker();
   public void registerClientDelegate(ClientDelegate delegate);
}

Implementation of getClientInvoker will just be:

ProtocolBroker invoker;

public ClientInvoker getClientInvoker()
{
   return (ClientInvoker)invoker;
}

Sorry, I need to think of some better names for these thi

Re: [Java] Message API design notes

[Martin Ritchie]

On 21/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> >
> > This increasing usage comes with new requirements some of which native
> > AMQP
> > functionality can achieve. Should we require the client to re-write
> > their app using the Qpid API? I hope the answer is no, as the re-write
> > effort would most likely eliminate any potential saving in moving to
> > Qpid.
>
>
> This is a tough question to answer. Depending on what is needed they may
> need to rewrite certain portions.
> Rewriting doesn't necessarily mean that you loose savings.
> After all which application can survive feature requests forever without
> having to go through modifications.
>
> My argument is that you can only expose simple AMQP functionality as JMS
> extensions. If they want to do more then it make sense to program at the
> AMQP level instead of JMS. Bcos once u do that then there is no advantage in
> sticking to JMS. You simply can't move to another JMS provider bcos your
> code is using AMQP extensions. You got to use the right tool for the right
> job. Hybrid tools will only take you so far.

But if AMQP provided an API you could move between AMQP implementations.

> As concrete examples are good. Two recent applications that I've
> > assisted developing have both written code to send a ping message to
> > the broker to ensure the connection is up. I felt terrible telling
> > them that Qpid currently does that but sorry you can't hook in to the
> > callbacks.
>
>
> That is more due to poor design than anything else. Asking an application to
> do that is not the right thing.
> We should provide a connection exception listener which should notify the
> application with a proper error code that notifies when the connection is
> lost or when the broker terminates by calling close. In the event of
> connection lost it should try failover and only notify when it is
> unsuccessful.
> In fact you can do that using JMS exception listener too. The only thing the
> application needs to do is to interpret the error code properly.
> The client code in trunk does that.

And on the branch but AFAICS there isn't a way to be notified of every
'idle ping', which is what the applications wanted to do. I'm making
no comments about the reasoning behind this simply pointing out that
clients want access to the events on the client and when they can't
get it they re-implement it.

> So I am very much in favour of JMS + AMQP extensions. I
> > don't really see any alternative. I can see why JMS would discourage
> > vendor extensions but don't forget we are not adding vendor specific
> > extensions; We are exposing AMQP functionality.
>
>
> As I said this is a poor effort in exposing AMQP functionality.
> There are certain functionality that you cannot expose that way. For example
> configuring the confirm mode and acquire mode for a transfer.
> Unless you cast it to the AMQ class and then use those methods.
> AMQSession ssn = (AMQSession)jmsSession;
> ssn.messageTransfer()

Ideally you would cast to an AMQP interface that is moving towards a
standard provided by the AMQP WG.

> If you do that, then what is the purpose of using JMS? I might as well code
> to AMQSession class.

If it is an interface then you may very well do that, as your
application evolves.

> Anyone implementing
> > AMQP based applications need to be acutely aware that the protocol is
> > still at a beta level.
>
>
> Folks you want to do that will do with that explicit understanding.
> From 0-10 onwards I doubt we will see drastic changes.

That may be true but there are no guarantees offered by the WG until 1.0

> When AMQP hits 1.0 and historic version
> > compatibility is required then use of these extensions should be less
> > risky than they are just now. This will be greatly assisted when AMQP
> > provide an API
>
>
> I doubt the AMQP spec group will ever provide an API. I don't see a need to
> do so.
> The semantic model already defines how a broker and client should interact.
> Not sure what more you can do.

I think this would be a mistake. AMQP is positioning itself as a
vendor independent messaging protocol. Attempting to commoditise
messaging so that one solution can be swapped for another. This
requires interoperability, for which I'm sure an official protocol
test will come from the WG at some point. However, I would be
surprised if we got to the state where vendors only provided the
broker and if we can't swap clients as well as brokers then clients
will still end up with a client code lock in unless there is a
standard AMQP API which we can write our code to and compile clients
against.


> at which point it would be a Qpid-AMQP-API only
> > extension, it would be AMQP-API which we were exposing.
>
>
> The proposed Qpid API is a java version of the 1:1 mapping of the protocol
> for the client side (with a few exceptions + sugar).
> As Rupert points out the comm layer is 1:1 mapping of the protocol for both
> the client and the broker.
>
> > Your case for JMS 

Re: [Java] Message API design notes

[Carl Trieloff]

Is it the case that every method belongs to one of these layers?



I would expect so.


In which case, will the layer numbers be put into the 0-10 xml?



No idea, or a least I don't know of any vote to do so. However the 
channels in 0-10 should be enough

to derive this, so I don't think we need it in the XML.

Carl.

Re: [Java] Message API design notes

[Rajith Attapattu]

>
> This increasing usage comes with new requirements some of which native
> AMQP
> functionality can achieve. Should we require the client to re-write
> their app using the Qpid API? I hope the answer is no, as the re-write
> effort would most likely eliminate any potential saving in moving to
> Qpid.


This is a tough question to answer. Depending on what is needed they may
need to rewrite certain portions.
Rewriting doesn't necessarily mean that you loose savings.
After all which application can survive feature requests forever without
having to go through modifications.

My argument is that you can only expose simple AMQP functionality as JMS
extensions. If they want to do more then it make sense to program at the
AMQP level instead of JMS. Bcos once u do that then there is no advantage in
sticking to JMS. You simply can't move to another JMS provider bcos your
code is using AMQP extensions. You got to use the right tool for the right
job. Hybrid tools will only take you so far.

As concrete examples are good. Two recent applications that I've
> assisted developing have both written code to send a ping message to
> the broker to ensure the connection is up. I felt terrible telling
> them that Qpid currently does that but sorry you can't hook in to the
> callbacks.


That is more due to poor design than anything else. Asking an application to
do that is not the right thing.
We should provide a connection exception listener which should notify the
application with a proper error code that notifies when the connection is
lost or when the broker terminates by calling close. In the event of
connection lost it should try failover and only notify when it is
unsuccessful.
In fact you can do that using JMS exception listener too. The only thing the
application needs to do is to interpret the error code properly.
The client code in trunk does that.

So I am very much in favour of JMS + AMQP extensions. I
> don't really see any alternative. I can see why JMS would discourage
> vendor extensions but don't forget we are not adding vendor specific
> extensions; We are exposing AMQP functionality.


As I said this is a poor effort in exposing AMQP functionality.
There are certain functionality that you cannot expose that way. For example
configuring the confirm mode and acquire mode for a transfer.
Unless you cast it to the AMQ class and then use those methods.
AMQSession ssn = (AMQSession)jmsSession;
ssn.messageTransfer()

If you do that, then what is the purpose of using JMS? I might as well code
to AMQSession class.

Anyone implementing
> AMQP based applications need to be acutely aware that the protocol is
> still at a beta level.


Folks you want to do that will do with that explicit understanding.
>From 0-10 onwards I doubt we will see drastic changes.

When AMQP hits 1.0 and historic version
> compatibility is required then use of these extensions should be less
> risky than they are just now. This will be greatly assisted when AMQP
> provide an API


I doubt the AMQP spec group will ever provide an API. I don't see a need to
do so.
The semantic model already defines how a broker and client should interact.
Not sure what more you can do.

at which point it would be a Qpid-AMQP-API only
> extension, it would be AMQP-API which we were exposing.


The proposed Qpid API is a java version of the 1:1 mapping of the protocol
for the client side (with a few exceptions + sugar).
As Rupert points out the comm layer is 1:1 mapping of the protocol for both
the client and the broker.

> Your case for JMS extentions based on feedback from users ? what is your
> > basis?
>
>
> > Do you think you can provide first class representations of AMQP
> concepts
> > using JMS extentions? - I gave concreate examples where you cannot do
> so.
> >
> > Regards,
> >
> > Rajith
> >
> > On 8/21/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
> > >
> > > Thanks for private mails, I'd prefer to discuss on the list, that way
> we
> > > can
> > > all contribute and learn something. I think good ideas stand up to
> > > scrutiny
> > > and are best shared. I've nothing to hide, and I put my ideas out as
> ideas
> > > without being afraid of being wrong. By sharing my ideas I will find
> out;
> > > I
> > > will also learn something and develop better ideas.
> > >
> > > Thanks for your explanation, and yes, you are right that I do need to
> > > spend
> > > more time reading the 0-10 specs.
> > >
> > > I think my confusion over your proposed API largely stems from the
> fact
> > > that
> > > we seemed to be in agreement some months ago that the common API for
> Qpid
> > > should be 1:1 the protocol. Which I have interpreted as meaning
> literally
> > > the protocol 'classes' and 'methods' directly represented in the code.
> I
> > > thought that was a very orthogonal approach, in that it guarantees
> that
> > > the
> > > protocol commands in their entirety are there to code against,
> completely
> > > taking care of one dimension of the design.
> > >

Re: [Java] Message API design notes

[Rajith Attapattu]

Yes I believe the Qpid API sits above the API you talk about.
And I would like that to be the API we promote to users as a client.

What you are essentially talking about is a slightly different approach to
how the comm layer should expose it self.
We need that API for sanity and design purposes. End users may also use it
if they want to build their own client or broker.
So lets not confuse that with the Client API discussion.

I believe Rafi can answer you better as to why he didn't use the interface
approach or as to why he didn't split into seperate classes.
Also I am not sure spliting up server and client chasis make much sense.
Bcos a broker can be a client to anothe Broker.
Can u elaborate a bit more as to why you would need to make such a
distinction.

I think an Invoker and a Delegate makes more sense.
Invoker being the methods you can call and Delegate being the method that
somebody else can call you with out making any distinction as server or
client.


On 8/21/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
>
> Yes, but my proposal differs slightly, in that it uses interfaces, not
> concrete classes, and splits server and client chassis up. Other than
> that,
> yes, the comm layer seems to fit the bill perfectly. No surprise, given
> that
> I started with Rafeal's stub and pulled an interface only API out of it.
>
> On 21/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> >
> > On 8/21/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
> > >
> > > On 21/08/07, Carl Trieloff <[EMAIL PROTECTED]> wrote:
> > > >
> > > > Watching the debate, I think you guys are actually quite close to
> each
> > > > other.
> > >
> > >
> > > Yes, as I say, we are to some extent talking at cross purposes. I'm
> > > primarily coming from the point of view of modularizing, and allowing
> > > multiple implementations of a common set of interfaces. I think Rajith
> > is
> > > primarily looking at things from the point of view of offering a
> client
> > > API.
> > > I'm actually thinking of an API that is for both the client and the
> > > broker,
> > > to call into a shared comm layer.
> >
> >
> > Rupert,  there is code in common module that satisfies your criteria
> > a.k.acomm layer.
> > I guess we can disucss this more in detail when we all meet at the f2f.
> > The idea is to have a common API that will provide a basis for a client
> > and
> > broker implementation.
> > If a user wants to hook into this layer, then that is possible as well.
> >
> > The client API that I am talking about is a thin layer on top of this
> > common
> > API.
> > Which provides connection negotiation, failover and a few convinience
> > methods.
> > 90% of the method invocations are directly called on the comm layer.
> > If u look at the code.
> > ClientSession implements o.a.q.client.Session extends
> o.a.q.Sessionextends
> > Invoker.
> >
> > The o.a.q.Session is part of the comm layer and will be used by both
> > broker
> > and client.
> > The o.a.q.client.Session is just a mask for exposing only what is
> > appropriate and useful for a client + sugar.
> >
> > So we seem to be in agreement.
> >
> > > L1 - Transport
> > > > L2 - Session
> > > > L3 - Execution
> > > > L4 - Model
> > >
> > > This is something for me to think about, because I have not tried to
> > take
> > > this into account. My idea to expose the whole protocol, is completely
> > > flat
> > > wrt this layering. The reason I thought every method has to be
> exposed,
> > is
> > > so that the client can be plugged direct into the broker in-vm or into
> a
> > > comm layer for going over the network. Sounds like I need to expose
> > layers
> > > 2-4 in full, in order to do this.
> > >
> > > Is it the case that every method belongs to one of these layers? In
> > which
> > > case, will the layer numbers be put into the 0-10 xml?
> > >
> > > Rupert
> > >
> >
>

Re: [Java] Message API design notes

[Rupert Smith]

Yes, but my proposal differs slightly, in that it uses interfaces, not
concrete classes, and splits server and client chassis up. Other than that,
yes, the comm layer seems to fit the bill perfectly. No surprise, given that
I started with Rafeal's stub and pulled an interface only API out of it.

On 21/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
>
> On 8/21/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
> >
> > On 21/08/07, Carl Trieloff <[EMAIL PROTECTED]> wrote:
> > >
> > > Watching the debate, I think you guys are actually quite close to each
> > > other.
> >
> >
> > Yes, as I say, we are to some extent talking at cross purposes. I'm
> > primarily coming from the point of view of modularizing, and allowing
> > multiple implementations of a common set of interfaces. I think Rajith
> is
> > primarily looking at things from the point of view of offering a client
> > API.
> > I'm actually thinking of an API that is for both the client and the
> > broker,
> > to call into a shared comm layer.
>
>
> Rupert,  there is code in common module that satisfies your criteria
> a.k.acomm layer.
> I guess we can disucss this more in detail when we all meet at the f2f.
> The idea is to have a common API that will provide a basis for a client
> and
> broker implementation.
> If a user wants to hook into this layer, then that is possible as well.
>
> The client API that I am talking about is a thin layer on top of this
> common
> API.
> Which provides connection negotiation, failover and a few convinience
> methods.
> 90% of the method invocations are directly called on the comm layer.
> If u look at the code.
> ClientSession implements o.a.q.client.Session extends o.a.q.Sessionextends
> Invoker.
>
> The o.a.q.Session is part of the comm layer and will be used by both
> broker
> and client.
> The o.a.q.client.Session is just a mask for exposing only what is
> appropriate and useful for a client + sugar.
>
> So we seem to be in agreement.
>
> > L1 - Transport
> > > L2 - Session
> > > L3 - Execution
> > > L4 - Model
> >
> > This is something for me to think about, because I have not tried to
> take
> > this into account. My idea to expose the whole protocol, is completely
> > flat
> > wrt this layering. The reason I thought every method has to be exposed,
> is
> > so that the client can be plugged direct into the broker in-vm or into a
> > comm layer for going over the network. Sounds like I need to expose
> layers
> > 2-4 in full, in order to do this.
> >
> > Is it the case that every method belongs to one of these layers? In
> which
> > case, will the layer numbers be put into the 0-10 xml?
> >
> > Rupert
> >
>

Re: [Java] Message API design notes

[Rajith Attapattu]

On 8/21/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
>
> On 21/08/07, Carl Trieloff <[EMAIL PROTECTED]> wrote:
> >
> > Watching the debate, I think you guys are actually quite close to each
> > other.
>
>
> Yes, as I say, we are to some extent talking at cross purposes. I'm
> primarily coming from the point of view of modularizing, and allowing
> multiple implementations of a common set of interfaces. I think Rajith is
> primarily looking at things from the point of view of offering a client
> API.
> I'm actually thinking of an API that is for both the client and the
> broker,
> to call into a shared comm layer.


Rupert,  there is code in common module that satisfies your criteria
a.k.acomm layer.
I guess we can disucss this more in detail when we all meet at the f2f.
The idea is to have a common API that will provide a basis for a client and
broker implementation.
If a user wants to hook into this layer, then that is possible as well.

The client API that I am talking about is a thin layer on top of this common
API.
Which provides connection negotiation, failover and a few convinience
methods.
90% of the method invocations are directly called on the comm layer.
If u look at the code.
ClientSession implements o.a.q.client.Session extends o.a.q.Session extends
Invoker.

The o.a.q.Session is part of the comm layer and will be used by both broker
and client.
The o.a.q.client.Session is just a mask for exposing only what is
appropriate and useful for a client + sugar.

So we seem to be in agreement.

> L1 - Transport
> > L2 - Session
> > L3 - Execution
> > L4 - Model
>
> This is something for me to think about, because I have not tried to take
> this into account. My idea to expose the whole protocol, is completely
> flat
> wrt this layering. The reason I thought every method has to be exposed, is
> so that the client can be plugged direct into the broker in-vm or into a
> comm layer for going over the network. Sounds like I need to expose layers
> 2-4 in full, in order to do this.
>
> Is it the case that every method belongs to one of these layers? In which
> case, will the layer numbers be put into the 0-10 xml?
>
> Rupert
>

Re: [Java] Message API design notes

[Martin Ritchie]

On 21/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> Rupert,
>
> I think you need to get yourself familiar with the 0-10 protocol before we
> continue the discussion.
> Most of your comments stem from lack of understanding of the 0-10 spec.
> I am not fully aware of all the details either, but u need atleast a
> resonable knowledge to discuss.
>
> I left out message.qos bcos it's going to be dropped from 0-10.
> So is session.ping and session.pong
> Can you please list any other method?
>
> Session.ack has nothing do with with messsageAcknowledgment. It's for
> execution layer functionality.
>
> >Cover the entire protocol, both client and server chassis, as a 100%
> literal
> This is not a useful excerisie at all (for a client API). 0-10 defines 4-5
> layers (depending on whom you talk to:) ).
> L1 - Transport
> L2 - Session
> L3 - Execution
> L4 - Model
>
> Methods defined in the XML corresponds to each of these layers.
> For an API it would be usefull only to expose L4 in 100% and certain method
> in the other layers.
> Some of the methods like in the session layer are for frame replay etc -
> absolutely no point in exposing them in a client API.
>
> In summary there is little value in exposing a raw set of methods. You need
> a thorough knowledge in AMQP to work at that level.
> I doubt our user base wants to write their own client impl. Do u think
> otherwise? Do you have users asking for that?
> What they need (based on the feedback received so far) is a usable client
> that can work at AMQP level (in addition to our JMS impl)
>
> >Follow a protocol methods-as-events model, with ability to register listen
> to any incoming event.
> You could already do this with the comm layer. This would only be done by
> somebody who has a good knowledge on AMQP.
> Please don't confuse this with the client API. Our main goal is to provide a
> client not a toolkit for people to build their own client implementation.
> However we also allow the flexibility to achieve the latter option.
>
> >With no risk of running into JMS licensing issues?
> This is a red herring and a very weak argument. There is always a risk.
> The entire AMQP protocol runs the risk, not just the Qpid java impl.
>
> >Here is a summary of how I picture things at the moment:
> >Orthogonal dimension 1:
> [..]
> This is the comm layer
>
> Orthogonal dimension 2:
> [..]
> This is the Qpid API
>
> So what is the disagreement here?
>
> Horizontal extentions for JMS
> -
> Why do we need this when we have your "Orthogonal dimension 2:" ?
> Why not use JMS for what it is intended for?
> Many folks that I spoke to thought of it as an akward way of exposing AMQP.
> Most of them liked a Qpid Java client.

> Besdies JMS (or any standard API) discourage vendor specific extentions.
> IMO encouraging people to use JMS extentions defeats the very purpose of
> JMS.

I know of many users that currently use JMS and are looking to use
Qpid for messaging. Many of these users are looking to Qpid for cost
and as the cost of messaging falls their usage increases. This
increasing usage comes with new requirements some of which native AMQP
functionality can achieve. Should we require the client to re-write
their app using the Qpid API? I hope the answer is no, as the re-write
effort would most likely eliminate any potential saving in moving to
Qpid.

As concrete examples are good. Two recent applications that I've
assisted developing have both written code to send a ping message to
the broker to ensure the connection is up. I felt terrible telling
them that Qpid currently does that but sorry you can't hook in to the
callbacks. So I am very much in favour of JMS + AMQP extensions. I
don't really see any alternative. I can see why JMS would discourage
vendor extensions but don't forget we are not adding vendor specific
extensions; We are exposing AMQP functionality. Anyone implementing
AMQP based applications need to be acutely aware that the protocol is
still at a beta level. When AMQP hits 1.0 and historic version
compatibility is required then use of these extensions should be less
risky than they are just now. This will be greatly assisted when AMQP
provide an API at which point it would be a Qpid-AMQP-API only
extension, it would be AMQP-API which we were exposing.

> Your case for JMS extentions based on feedback from users ? what is your
> basis?


> Do you think you can provide first class representations of AMQP concepts
> using JMS extentions? - I gave concreate examples where you cannot do so.
>
> Regards,
>
> Rajith
>
> On 8/21/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
> >
> > Thanks for private mails, I'd prefer to discuss on the list, that way we
> > can
> > all contribute and learn something. I think good ideas stand up to
> > scrutiny
> > and are best shared. I've nothing to hide, and I put my ideas out as ideas
> > without being afraid of being wrong. By sharing my ideas I will find out;
> > I
> > will also l

Re: [Java] Message API design notes

[Rupert Smith]

On 21/08/07, Carl Trieloff <[EMAIL PROTECTED]> wrote:
>
> Watching the debate, I think you guys are actually quite close to each
> other.


Yes, as I say, we are to some extent talking at cross purposes. I'm
primarily coming from the point of view of modularizing, and allowing
multiple implementations of a common set of interfaces. I think Rajith is
primarily looking at things from the point of view of offering a client API.
I'm actually thinking of an API that is for both the client and the broker,
to call into a shared comm layer.

> L1 - Transport
> L2 - Session
> L3 - Execution
> L4 - Model

This is something for me to think about, because I have not tried to take
this into account. My idea to expose the whole protocol, is completely flat
wrt this layering. The reason I thought every method has to be exposed, is
so that the client can be plugged direct into the broker in-vm or into a
comm layer for going over the network. Sounds like I need to expose layers
2-4 in full, in order to do this.

Is it the case that every method belongs to one of these layers? In which
case, will the layer numbers be put into the 0-10 xml?

Rupert

Re: [Java] Message API design notes

[Carl Trieloff]

I shall give a few concrete examples of protocol 'methods' that I cannot see
in your API, as this is what you have asked me for. Picked at random, and
not a complete list, simply to prove the point that not everything is there:

session.ping
session.closed
message.qos
... (there are more, its simply a case of comparing your API to the XML)

  


Watching the debate, I think you guys are actually quite close to each 
other.  For example,
message.qos  -- voted out and replaced with the flow commands in message 
class

session.ping -- has been renamed to no op I believe
...

So I think the debate is somewhat connected to 0-10. From what I can 
see, by the time
0-10 is completely voted and the client is done there should be a 1:1 
mapping, with some

utility classes/functions. I might be wrong, but that is my take.

It might be useful doing a review once all the 0-10 votes are complete,I 
for one have
enjoyed following the debate. It might also be worth linking and posting 
more of the spec

on our wiki. Maybe even link to the amqp.org JIRA on spec issues.
Carl.

Re: [Java] Message API design notes

[Rajith Attapattu]

Rupert,

I think you need to get yourself familiar with the 0-10 protocol before we
continue the discussion.
Most of your comments stem from lack of understanding of the 0-10 spec.
I am not fully aware of all the details either, but u need atleast a
resonable knowledge to discuss.

I left out message.qos bcos it's going to be dropped from 0-10.
So is session.ping and session.pong
Can you please list any other method?

Session.ack has nothing do with with messsageAcknowledgment. It's for
execution layer functionality.

>Cover the entire protocol, both client and server chassis, as a 100%
literal
This is not a useful excerisie at all (for a client API). 0-10 defines 4-5
layers (depending on whom you talk to:) ).
L1 - Transport
L2 - Session
L3 - Execution
L4 - Model

Methods defined in the XML corresponds to each of these layers.
For an API it would be usefull only to expose L4 in 100% and certain method
in the other layers.
Some of the methods like in the session layer are for frame replay etc -
absolutely no point in exposing them in a client API.

In summary there is little value in exposing a raw set of methods. You need
a thorough knowledge in AMQP to work at that level.
I doubt our user base wants to write their own client impl. Do u think
otherwise? Do you have users asking for that?
What they need (based on the feedback received so far) is a usable client
that can work at AMQP level (in addition to our JMS impl)

>Follow a protocol methods-as-events model, with ability to register listen
to any incoming event.
You could already do this with the comm layer. This would only be done by
somebody who has a good knowledge on AMQP.
Please don't confuse this with the client API. Our main goal is to provide a
client not a toolkit for people to build their own client implementation.
However we also allow the flexibility to achieve the latter option.

>With no risk of running into JMS licensing issues?
This is a red herring and a very weak argument. There is always a risk.
The entire AMQP protocol runs the risk, not just the Qpid java impl.

>Here is a summary of how I picture things at the moment:
>Orthogonal dimension 1:
[..]
This is the comm layer

Orthogonal dimension 2:
[..]
This is the Qpid API

So what is the disagreement here?

Horizontal extentions for JMS
-
Why do we need this when we have your "Orthogonal dimension 2:" ?
Why not use JMS for what it is intended for?
Many folks that I spoke to thought of it as an akward way of exposing AMQP.
Most of them liked a Qpid Java client.

Besdies JMS (or any standard API) discourage vendor specific extentions.
IMO encouraging people to use JMS extentions defeats the very purpose of
JMS.

Your case for JMS extentions based on feedback from users ? what is your
basis?
Do you think you can provide first class representations of AMQP concepts
using JMS extentions? - I gave concreate examples where you cannot do so.

Regards,

Rajith

On 8/21/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
>
> Thanks for private mails, I'd prefer to discuss on the list, that way we
> can
> all contribute and learn something. I think good ideas stand up to
> scrutiny
> and are best shared. I've nothing to hide, and I put my ideas out as ideas
> without being afraid of being wrong. By sharing my ideas I will find out;
> I
> will also learn something and develop better ideas.
>
> Thanks for your explanation, and yes, you are right that I do need to
> spend
> more time reading the 0-10 specs.
>
> I think my confusion over your proposed API largely stems from the fact
> that
> we seemed to be in agreement some months ago that the common API for Qpid
> should be 1:1 the protocol. Which I have interpreted as meaning literally
> the protocol 'classes' and 'methods' directly represented in the code. I
> thought that was a very orthogonal approach, in that it guarantees that
> the
> protocol commands in their entirety are there to code against, completely
> taking care of one dimension of the design.
>
> I shall give a few concrete examples of protocol 'methods' that I cannot
> see
> in your API, as this is what you have asked me for. Picked at random, and
> not a complete list, simply to prove the point that not everything is
> there:
>
> session.ping
> session.closed
> message.qos
> ... (there are more, its simply a case of comparing your API to the XML)
>
> So can I write a client against your API that is capable of attaching a
> listener delegate to be informed of the session.closed event? Whether or
> not
> being able to do so is useful is, I admit, debatable. The JMS impl on top
> of
> this API will want to know that session.closed has been received, however.
>
> Also, there are some naming inconsistencies between your API and AMQP, for
> example you have 'messageAcknowledege' in your Session interface, whereas
> in
> the protocol XML it is session.ack. Not hugely important, its just if
> something is 1:1 I expected that the names would map 1:1 in

Re: [Java] Message API design notes

[Rupert Smith]

Thanks for private mails, I'd prefer to discuss on the list, that way we can
all contribute and learn something. I think good ideas stand up to scrutiny
and are best shared. I've nothing to hide, and I put my ideas out as ideas
without being afraid of being wrong. By sharing my ideas I will find out; I
will also learn something and develop better ideas.

Thanks for your explanation, and yes, you are right that I do need to spend
more time reading the 0-10 specs.

I think my confusion over your proposed API largely stems from the fact that
we seemed to be in agreement some months ago that the common API for Qpid
should be 1:1 the protocol. Which I have interpreted as meaning literally
the protocol 'classes' and 'methods' directly represented in the code. I
thought that was a very orthogonal approach, in that it guarantees that the
protocol commands in their entirety are there to code against, completely
taking care of one dimension of the design.

I shall give a few concrete examples of protocol 'methods' that I cannot see
in your API, as this is what you have asked me for. Picked at random, and
not a complete list, simply to prove the point that not everything is there:

session.ping
session.closed
message.qos
... (there are more, its simply a case of comparing your API to the XML)

So can I write a client against your API that is capable of attaching a
listener delegate to be informed of the session.closed event? Whether or not
being able to do so is useful is, I admit, debatable. The JMS impl on top of
this API will want to know that session.closed has been received, however.

Also, there are some naming inconsistencies between your API and AMQP, for
example you have 'messageAcknowledege' in your Session interface, whereas in
the protocol XML it is session.ack. Not hugely important, its just if
something is 1:1 I expected that the names would map 1:1 in a mechanical
fashion? I think that is what I was expecting to see, a completely
inclusive, literal, 1:1, transliteration of the entire set of protocol
'methods' into an API, both for the client and server chassis.

For example, instead of a MessageListener, I was kind of expecting there to
be a 'delegate' or listener interface for the entire set of incoming
protocol 'methods' that a client can be sent by a broker. Possibly for no
better reason than the ideas of orthogonality and completeness that I am
always on the look out for in a design.

After reading your explanation I realise that your messaging API does stuff
that JMS does not. For example, in JMS if I want to send some messages and
get confirmation that my messages have really been received by the broker, I
must use a transaction. Which in turn means that none of my messages can be
delivered until the commit. In the 0-10 protocol, I can do a sync, like you
describe, to get confirmation, but the messages can already have continued
on their way before then? Keep me right 0-10 experts. This certainly sounds
more asynchronous. It certainly sounds necessary, for client applications to
be able to code against these 0-10 concepts, that are not available in JMS,
in order to take full advantage of its capabilities.

We are to some extend talking at cross purposes, because I have identified a
set of interfaces, that covers the entire protocol, largely with a view to
creating a modular design, and with some possible implementation efficiency
benefits that I think could be derived from the method-as-class model. This
is what the existing comm layer does, I just pulled out the interfaces,
splitting up the client and server chassis, that's all. This is already
approximately what you are placing your client layer on top of.

You say you are opposed to horizontal extension of JMS in general, (hope you
don't mind me quoting from a private mail here?):

> I totally oppose horizontal extentions to JMS as it does not provide first
class representation for AMQP concepts.

So your API must duplicate some parts of JMS (as any messaging API would
have to). Just as I am constantly on the look out for completeness and
orthogonality in a design, I am also always on the look out for designs that
eliminate duplication. I would like to put out the idea, that this can be
represented as a JMS extension, simply by adding a sync() method into the
extended Session interface? So I could send a JMS message, then call sync?
Eventually, of course, weaving things into existing API's starts to become
too cludgy, so I think there are definitely trade-offs to consider in these
design choices. I think I am concerned that your replace rather than extend
strategy means that the end user must entirely throw out JMS to use advanced
AMQP features. The extend strategy is more of a gradual migration. JMS is a
very good and highly successful API, which makes me reluctant to abandon
rather than try and extend it.

One thing about the low-level API that I have proposed, is that it is
entirely different to JMS in its structure. It is based around the 

Re: [Java] Message API design notes

[Rajith Attapattu]

>
>
> I find it strange though because it is like JMS in a parallel universe:
>
> JMS  Qpid
> ConnectionConnection
> Session Session
> MessageListenerMessageListener
> ExceptionListener   ExceptionListener
> MessageMessage
> BytesMessage   ByteBufferMessage
> ?   FileMessage (why in terms of files? could be
> more abstract in terms of InputStream?)
>

Quick clarification to go along with my other email. I am very very glad
that u bought this up.
Infact would have been extreamly happy if u stated this in your very first
email.

Connection and Session are classes directly from AMQP - so no argument
there.

Message interface is a true reflection of the AMQP spec. Nothing to do with
JMS. Note the distinction.
AMQP defines different message properties,
DeliveryProperties, ApplicationProperties ..and can extend. In 0-11 we might
have MsgGroupProperties.
read(), appendData() is obviously common to any message model. So nothing to
claim that it looks like JMS.

The broker sends replies asynchronously and that includes errors too.
ExceptionListener is a generic way of notifying errors asynchronously.
It's nothing JMS specific. Do you agree?

If u notice.
MessageListener is just suger, not part of the core API. I am moving it to
the util package as u can only use it with the utility adapter class.

BytesMessage is a utiility class that belongs to the implementation not the
API. So why the fuss ?
If u are not happy with the name u can implement the message API with any
name u want.
I actually had an inline implementation for the message interface, only
reason I moved it was to allow reuse for Arnaud.

FileMessage : Another utility class - not part of the API.

>>(why in terms of files? could be more abstract in terms of InputStream?
Did u look at the code or at the least read the java doc?
I am using the FileChannel from nio to make use of the map function which
mapps regions of a file into memory.

Does this explanation help u?

Regards,

Rajith

Re: [Java] Message API design notes

[Rajith Attapattu]

Rupert,

I still think I haven't done a good job explaining. I also think that once
you get familliar with 0-10 protocol you will also understand it better.
Instead of answering your questions directly I will put forward a summary.
Please read it carefully before answering.
Also reading just the 0-10 xml will not help, you need to read the proposal
on the AMQP wiki to understand subtle points.
I highly recomend reading the new message proposal and others.

The API exposes every AMQP protocol method except for connection negotiation
and failover. So it's a very close 1:1 map of the protocol.
I have listed the methods in the API and highlighted the significance
brieftly.
You can decide for yourself after comparing with JMS and reading the
protocol specs/proposals.

Message Receiving
---
messageSubscribe  - pay attention to the acquire mode and confirm mode -
very useful
messageFlow and Stop - credit based flow control
messageRelease and messageAcquire - ownership of messages
messageFlush - asking the broker to exaust the credit and send messages
immediately if available
executionComplete for transfer method - acking the message
(messageAcknowledge)
messageReject - rejecting the message.
messageAcquired - the broker notifying u what messages were given to u.

The protocl will send the messages in the following sequence. (Segment is a
collection of frames)
messageTransfer, Header Segment, Body Segments.
Hence it naturally maps to message part listener.
messageTransfer(commandId)
headers(Struct[] ...)
data(ByteBuffer) - This will be called each time a frame arrives
end () will be called after the last frame of the last segment is received.

Everything else like the MessageListener is just sugar - not part of the
core API.

MessageSending

As I explained above sending is the same aequence,
messageTransfer , headers(Struct[] ), one or more data(ByteBuffer)  , end().
messageReject () - notifies u when a message is rejected

Everything else like messageStream or messageTransfer with message as an
argument is just suger - not part of the core API.

ExecutionLayer
-
sync() - Very very useful - it notifies when a particular method has been
executed.
In other words executionComplete has arrived.  You can think of them as
synchronization points.

You can do things like.
queueDeclare(A)
queueDeclare(B)
sync()
messageTransfer (A)
messageTransfer(B)
sync()

The Rest of the API is obvious 1:1 mapping of Exchange,Queue and Session
methods.

Rupert, what do u think?
Does it still look like JMS?
Can u point out which protocol method is not exposed?

Regards,

Rajith


On 8/20/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
>
> Ok, sorry about the delay, I got the tests fixed up. First off, why I
> think
> the proposed API (http://people.apache.org/~rajith/qpid_docs/client_api/)
> offers almost nothing that JMS does not already cover. And then a second
> email about what my opinions/ideas about the low-level API are.
>
> The proposed API, is primarily geared towards listening for messages, of
> course, combined with sending messages, the most important thing a
> messaging
> API is interested in providing. So you have a 'MessageListener', as does
> JMS.
>
> You also present a little example, of sending messages through a push
> driven
> streaming API:
>
> //Option3 - can use it with any message size, recomended for large
> messages
> public void messageTransfer(String destination, short confirmMode,
> short acquireMode);
> public void headers(Struct... headers);
> public void data(byte[] data);
> public void data(ByteBuffer buf);
> public void data(String str);
> public void endData();
>
> as if this is an important innovation. But then JMS has the StreamMessage
> class, that provides this capability. There is a difference, and that is
> that in the example, the transfer comes before the appending of data,
> followed by an explicit end to terminate. Does this mean we are doing real
> streaming over the protocol 'message class'? Again, I am not clear on
> this,
> 0-8 had separate 'message' and 'stream' classes, does 'message' in 0-10
> now
> covers streaming too? If so, then it looks to me like this would be a
> useful
> JMS API extension, that would sit side by side with the JMS
> implementation,
> rather than as a lower-level that sits beneath it. In a similar way to how
> the current 0-8 implementation provides horizontal extension of JMS for
> AMQP
> specific features.
>
> This proposed API lets me bind exchanges, about the only thing it offers
> that JMS does not already cover.
>
> I have looked at MessagePartListener, and it does not convince me that
> this
> API is a 1:1 mapping of the protocol. Looking inside the protocol XML, I
> am
> looking at every method in it, some are marked as:
>
> 
> and some as
>
> 
> and some as both of these. Now looking at the Session class, I certainly
> see
> a lot more of the 'server' chassis exposed, but n

Re: [Java] Message API design notes

[Rupert Smith]

Arnuad,

You should unset your reply-to, as I thought I was replying to the list, not
realising I was sending messages only to you. We should discuss these things
on the list.

rupert

On 20/08/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
>
>
>
> -- Forwarded message --
> From: Rupert Smith <[EMAIL PROTECTED]>
> Date: 20-Aug-2007 16:23
> Subject: Re: [Java] Message API design notes
> To: [EMAIL PROTECTED]
>
> I find it strange though because it is like JMS in a parallel universe:
>
> JMS  Qpid
> ConnectionConnection
> Session Session
> MessageListenerMessageListener
> ExceptionListener   ExceptionListener
> MessageMessage
> BytesMessage   ByteBufferMessage
> ?   FileMessage (why in terms of files? could be
> more abstract in terms of InputStream?)
>
> Only, it is not as if the Qpid API is a super-type of the JMS API that you
> propose to put on top of it. Therefore, the JMS api must extend the Qpid one
> by 'delegation' or 'wrapping' the Qpid one, and translating. In other words
> the Qpid and JMS APIs are at a similar level of abstraction, therefore the
> Qpid API does not naturally form a lower layer to implement JMS on top of,
> but sits uncomfortably as an extra layer to be translated through.
>
> Also, the fact that the Qpid API is a reasonably close copy of the JMS API
> renders it... kind of pointless. Why bother to repeat concepts like these,
> when horizontal extension means you can take what JMS already provides and 
> only
> add to it, rather than duplicating. I think it is the duplicating of the
> same level of abstraction that bothers me.
>
> Rupert
>
>
> On 20/08/07, Arnaud Simon <[EMAIL PROTECTED]> wrote:
> >
> > On Mon, 2007-08-20 at 15:54 +0100, Rupert Smith wrote:
> > > On 20/08/07, Arnaud Simon <[EMAIL PROTECTED] > wrote:
> > > So today the current API could be seen as part of the JMS
> > > implementation
> > > i.e. some useful methods for interfacing with the common
> > > layer.
> > > Back on the JMS extensions, I haven't implemented them yet.
> > > The idea is
> > > to have first the JMS tck passing then to provide support for
> > > the JMS
> > > extensions. What do you think? Does that make sense?
> > >
> > > It makes some sense, totally fine with things being done in stages.
> > >
> > > I just think that this proposed API does not actually look very
> > > useful, as a 'useful' layer to put between the JMS implementation and
> > > the comm layer. It looks to me like it would be better to be putting
> > > the JMS layer straight onto to comm layer, without this 'strange
> > > cousin' of JMS getting in the way;
> >
> > You should have a closer look at it as this API is a close mapping of
> > the protocol (at least what the JMS layer needs from the AMQP protocol).
> >
> > The only thing that it does is providing:
> > - connection negotiation
> > - synchronization points
> > - message assembly facilities
> > - clustering support
> >
> > >  as I've said before it just seems like an extra layer for no good
> > > reason (other than perhaps, creating work for people?). Be great if
> > > the JMS layer can implements a ProtocolClient delegate, and call a
> > > ProtocolBroker invoker, and be capable of being aware of every event
> > > in the protocol if it needs to. That is, expose the whole protocol
> > > through a interface only API, and put the JMS on top of it, handling
> > > and calling just what it needs to.
> >
> > As I said this is pretty much what the current API doe so it should not
> > be a big deal to move to such a solution if we decide to go for it. Note
> > that I have nothing against your proposal that I think makes sense.
> >
> > > Also, as far as a Qpid API is concerned, there was general agreement
> > > some months ago, when this was all debated before, that such an API
> > > should 'look like the protocol'.
> >
> > +1, however I may slightly disagree with you as I think that the current
> > API is very close to the AMQP protocol.
> >
> > Arnaud
> >
> >
>

Re: [Java] Message API design notes

[Arnaud Simon]

On Mon, 2007-08-20 at 14:41 +0100, Rupert Smith wrote:

> 
> Exposing the entire AMQP protocol 1:1 does seem like overkill from the
> point of view of writing applications against, since in the vast
> majority of cases it is just create connection, create session, send
> messages (in which case use JMS, or JMS + AMQP extensions). I am
> making a case for it more from the point of modularizing our
> implementation, that is, being able to plug clients into comm layer or
> routing layer or native impl and so on, and keeping open the
> possibility of alternate implementations of the same set of
> interfaces. 
> 
> Suppose you implemented the comm layer to an identical (wrt languge)
> set of interfaces in C++ and Java. Dead easy to call the C++ direct
> from the Java as a native implementation, supposing that doing so
> might be more performant. I'm a little unsure but its pretty easy to
> call C++ from .Net? 

What you are saying confirms my understanding of your proposal. That is
to say that your approach should not be compared nor confronted with the
current "API". You approach subsumes the notion of a Qpid messaging API.
IF we go for it (I have nothing against) then a potential Qpid API would
be implemented on top of it. 
Am I right? 

Arnaud 

Re: [Java] Message API design notes

[Rupert Smith]

On 20/08/07, Arnaud Simon <[EMAIL PROTECTED]> wrote:
>
> > 3. Use of the inverse symmetry of the API so that the outgoing interface
> > that the client calls, is identical to the incoming interface that the
> > broker routing layer implements.
>
> This is a requirement for the common layer and I think that it already
> does that, doesn't it?



Yes, but best way to have alternate implementations of things is through
interfaces. Comm layer does look like it is leaning towards allowing this,
but it defines stuff in terms of concrete classes. All I've really done is
pull the exact same API out into a clean set of interfaces + factories,
primarily with the intent of doing this sort of stuff.

> 4. Be language neutral, that is a suitable API for implementing in any of
> > our target languages. Will assume OO as a base-line.
>
> I am not sure that this is realistic and this should be left to AMQP to
> provide such an API not to Qpid.
>

Also, again. I'm thinking about a clean modular design for our
implementation, Qpid, not about a wonderful API for AMQP.

Re: [Java] Message API design notes

[Rupert Smith]

On 20/08/07, Arnaud Simon <[EMAIL PROTECTED]> wrote:

> > 2. Full mapping of the protocol 1:1 onto the API.
>
> This is again a debatable criteria. I personally think that some
> convenient features like providing support for connection negotiation
> would be very helpful. This does not mean that the raw AMQP methods
> don't have to be exposed though.
>

Already described. The 'conversation' interface, or an equivalent encodes
these common synchronous interactions.

Exposing the entire AMQP protocol 1:1 does seem like overkill from the point
of view of writing applications against, since in the vast majority of cases
it is just create connection, create session, send messages (in which case
use JMS, or JMS + AMQP extensions). I am making a case for it more from the
point of modularizing our implementation, that is, being able to plug
clients into comm layer or routing layer or native impl and so on, and
keeping open the possibility of alternate implementations of the same set of
interfaces.

Suppose you implemented the comm layer to an identical (wrt languge) set of
interfaces in C++ and Java. Dead easy to call the C++ direct from the Java
as a native implementation, supposing that doing so might be more
performant. I'm a little unsure but its pretty easy to call C++ from .Net?

Re: [Java] Message API design notes

[Rupert Smith]

On 20/08/07, Arnaud Simon <[EMAIL PROTECTED]> wrote:
>
> > 1. Methods as classes, rather than exposed arguments.
>
> This is debatable indeed. I am not sure that the performance should
> disfavor usability and ease of programming.
>


Ok, but there is a very easy way out of that one.

In my API I have to do:

messageTranfser(session, factory.createMethodMessageTransfer(...));

which is pretty ugly, and not designed for ease of programming.

No problem, just add a convenience method to the interface

public interface ClassMessageBroker
{
   public void messageTransfer(SessionContext session,
MethodMesssageTransfer mt);

   // for convenience, something along the lines of
   public void messageTransfer(SessionContext session, byte[] data, String
destination, ...);

And do the method creation in the convenience method. Then you get the best
of both options.

Re: [Java] Message API design notes

[Arnaud Simon]

On Mon, 2007-08-20 at 13:19 +0100, Rupert Smith wrote:
> On 20/08/07, Arnaud Simon <[EMAIL PROTECTED]> wrote:
> I like what Rupert is proposing but it may be too low level.
> Rupert you
> are saying that the current API does not expose the AMQP ping
> mechanism.
> Is that all? Can you provide an exhaustive list of the AMQP
> features
> that are not exposed?
> 
> As a typing exercise yes, but not one I am inclined to do. I shall
> instead furnish you with a procedure for creating this list. Look at
> the protocol XML, compare every method marked as  name="server" ...>, and look at the API to see if it is there or not.
> Look for every method marked as 

Re: [Java] Message API design notes

[Robert Greig]

On 20/08/07, Rupert Smith <[EMAIL PROTECTED]> wrote:

> So is this what the example, 'Option3', is describing? Is it a streamed
> messages split into chunks? And we are doing streaming through the 'message
> class' now?

I think this should be described as chunking not streaming, since that
is all it is - allowing the client to control the sending of a message
in chunks. It is not "end to end" streaming.

> I'm perfectly happy with that, but I see it as a JMS extensions, not
> replacement.

Absolutely.

RG

Re: [Java] Message API design notes

[Arnaud Simon]

On Mon, 2007-08-20 at 12:09 +0100, Rupert Smith wrote:
> Onto some ideas about low-level API, what I had imagined such a thing would
> look like. I've been hacking this around on my laptop, but without internet
> access for it (or time to rebuild my kernel with WiFi support ;-), should
> get hooked up at home again on thursday), I will just have to give an
> outline of what my ideas look like. I don't like to mail code for something
> that I'm not completely satisfied with anyway; yes, my ideas are ideas,
> their merit is debatable, and I am interested in that debate to sound out
> opinion and shape-up and strengthen the ideas to our mutual satisfaction. I
> think this interface/API is an important one, because it is what the clean
> modular design of our implementation will hinge around.

You are right we should discuss especially because nobody has ever said
that the current proposal was written in stone. 

> As I mentioned previously I had 3, now 4 criteria:
> 
> 1. Methods as classes, rather than exposed arguments.

This is debatable indeed. I am not sure that the performance should
disfavor usability and ease of programming.  

> 2. Full mapping of the protocol 1:1 onto the API.

This is again a debatable criteria. I personally think that some
convenient features like providing support for connection negotiation
would be very helpful. This does not mean that the raw AMQP methods
don't have to be exposed though. 

> 3. Use of the inverse symmetry of the API so that the outgoing interface
> that the client calls, is identical to the incoming interface that the
> broker routing layer implements.

This is a requirement for the common layer and I think that it already
does that, doesn't it?

> 4. Be language neutral, that is a suitable API for implementing in any of
> our target languages. Will assume OO as a base-line.

I am not sure that this is realistic and this should be left to AMQP to
provide such an API not to Qpid. 

Arnaud

Re: [Java] Message API design notes

[Rupert Smith]

And I do agree, there needs to be a slightly higher level client API, to
make this very low-level direct protocol API usable to code against. In Java
I believe that it is JMS + horizontal extensions, in other languages,
something similar wrt JMS licensing issues, or better if you come up with a
nicer abstraction.

On 20/08/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
>
> On 20/08/07, Arnaud Simon <[EMAIL PROTECTED]> wrote:
>
> > I like what Rupert is proposing but it may be too low level. Rupert you
> > are saying that the current API does not expose the AMQP ping mechanism.
> > Is that all? Can you provide an exhaustive list of the AMQP features
> > that are not exposed?
> >
>
> As a typing exercise yes, but not one I am inclined to do. I shall instead
> furnish you with a procedure for creating this list. Look at the protocol
> XML, compare every method marked as , and look at
> the API to see if it is there or not. Look for every method marked as
> 

Re: [Java] Message API design notes

[Rupert Smith]

On 20/08/07, Arnaud Simon <[EMAIL PROTECTED]> wrote:

> I like what Rupert is proposing but it may be too low level. Rupert you
> are saying that the current API does not expose the AMQP ping mechanism.
> Is that all? Can you provide an exhaustive list of the AMQP features
> that are not exposed?
>

As a typing exercise yes, but not one I am inclined to do. I shall instead
furnish you with a procedure for creating this list. Look at the protocol
XML, compare every method marked as , and look at
the API to see if it is there or not. Look for every method marked as

Re: [Java] Message API design notes

[Rupert Smith]

On 20/08/07, Arnaud Simon <[EMAIL PROTECTED]> wrote:
>
> On Mon, 2007-08-20 at 10:42 +0100, Rupert Smith wrote:
> > //Option3 - can use it with any message size, recomended for large
> messages
> > public void messageTransfer(String destination, short confirmMode,
> > short acquireMode);
> > public void headers(Struct... headers);
> > public void data(byte[] data);
> > public void data(ByteBuffer buf);
> > public void data(String str);
> > public void endData();
> >
> > as if this is an important innovation. But then JMS has the
> StreamMessage
> > class, that provides this capability.
>
> Sorry but I must disagree with this statement as a JMS streamMessage is
> just a convenient way of building a message that is eventually sent as
> if. What you are describing would be equivalent to a JMS way of handling
> large messages, that is to say splitting the message and sending small
> batches to the broker.
>

So is this what the example, 'Option3', is describing? Is it a streamed
messages split into chunks? And we are doing streaming through the 'message
class' now?

I'm perfectly happy with that, but I see it as a JMS extensions, not
replacement.

Re: [Java] Message API design notes

[Arnaud Simon]

On Mon, 2007-08-20 at 10:42 +0100, Rupert Smith wrote:
> //Option3 - can use it with any message size, recomended for large messages
> public void messageTransfer(String destination, short confirmMode,
> short acquireMode);
> public void headers(Struct... headers);
> public void data(byte[] data);
> public void data(ByteBuffer buf);
> public void data(String str);
> public void endData();
> 
> as if this is an important innovation. But then JMS has the StreamMessage
> class, that provides this capability.

Sorry but I must disagree with this statement as a JMS streamMessage is
just a convenient way of building a message that is eventually sent as
if. What you are describing would be equivalent to a JMS way of handling
large messages, that is to say splitting the message and sending small
batches to the broker. 

>  There is a difference, and that is
> that in the example, the transfer comes before the appending of data,
> followed by an explicit end to terminate. Does this mean we are doing real
> streaming over the protocol 'message class'? Again, I am not clear on this,
> 0-8 had separate 'message' and 'stream' classes, does 'message' in 0-10 now
> covers streaming too? If so, then it looks to me like this would be a useful
> JMS API extension, that would sit side by side with the JMS implementation,
> rather than as a lower-level that sits beneath it. In a similar way to how
> the current 0-8 implementation provides horizontal extension of JMS for AMQP
> specific features.
> This proposed API lets me bind exchanges, about the only thing it offers
> that JMS does not already cover.

I think that the real debate that is going on is whether we want to
expose the AMQP protocol as it or if we want to provide a slightly
higher level API that provides few convenient features. 

I like what Rupert is proposing but it may be too low level. Rupert you
are saying that the current API does not expose the AMQP ping mechanism.
Is that all? Can you provide an exhaustive list of the AMQP features
that are not exposed? 

Arnaud

Re: [Java] Message API design notes

[Rupert Smith]

And

public interface ProtocolBroker extends ClassMethodBroker,
ClassSessionBroker, ClassExecutionBroker, ...

should be

public interface ProtocolBroker extends ClassMessageBroker,
ClassSessionBroker, ClassExecutionBroker, ...

On 20/08/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
>
> Typo:
>
> public interface ClassSessionAttached
> {
>public void sessionAttached(SessionContext context,
> MethodSessionAttached attached);
>...
> }
>
> should be:
>
> public interface ClassSessionClient
> {
>public void sessionAttached(SessionContext context,
> MethodSessionAttached attached);
>...
> }
>
> On 20/08/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
> >
> > Here's the updated diagram.
> >
> > On 20/08/07, Rupert Smith < [EMAIL PROTECTED]> wrote:
> > >
> > > I've updated my diagram, because the first one I sent was a bit
> > > misleading, in that it erased the invoker/delegate design that the comm
> > > layer uses. I'll send that seperately as a zip, as Apache won't let me 
> > > post
> > > a gif to the list.
> > >
> >
> >
> >
> >
> >
>

Re: [Java] Message API design notes

[Rupert Smith]

Typo:

public interface ClassSessionAttached
{
   public void sessionAttached(SessionContext context, MethodSessionAttached
attached);
   ...
}

should be:

public interface ClassSessionClient
{
   public void sessionAttached(SessionContext context, MethodSessionAttached
attached);
   ...
}

On 20/08/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
>
> Here's the updated diagram.
>
> On 20/08/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
> >
> > I've updated my diagram, because the first one I sent was a bit
> > misleading, in that it erased the invoker/delegate design that the comm
> > layer uses. I'll send that seperately as a zip, as Apache won't let me post
> > a gif to the list.
> >
>
>
>
>
>

Re: [Java] Message API design notes

[Rupert Smith]

Onto some ideas about low-level API, what I had imagined such a thing would
look like. I've been hacking this around on my laptop, but without internet
access for it (or time to rebuild my kernel with WiFi support ;-), should
get hooked up at home again on thursday), I will just have to give an
outline of what my ideas look like. I don't like to mail code for something
that I'm not completely satisfied with anyway; yes, my ideas are ideas,
their merit is debatable, and I am interested in that debate to sound out
opinion and shape-up and strengthen the ideas to our mutual satisfaction. I
think this interface/API is an important one, because it is what the clean
modular design of our implementation will hinge around.

As I mentioned previously I had 3, now 4 criteria:

1. Methods as classes, rather than exposed arguments.
2. Full mapping of the protocol 1:1 onto the API.
3. Use of the inverse symmetry of the API so that the outgoing interface
that the client calls, is identical to the incoming interface that the
broker routing layer implements.
4. Be language neutral, that is a suitable API for implementing in any of
our target languages. Will assume OO as a base-line.

I have pulled out of the stub code that Rafael sent, an API purely in terms
of interfaces. No great surprises, once you see how the comm layer works. As
I've mentioned already, the only thing I have really tried to change is the
splitting each of the protocol 'classes' into the server and client chassis
parts. As it is in terms of interfaces only, there is also a factory to work
with this API. I have dropped the 'context's as type parameters from the
API, because I feel that in terms of the API the context is never abstract
enough to need parametric polymorphism; class subtype polymorphism will
suffice.

I've updated my diagram, because the first one I sent was a bit misleading,
in that it erased the invoker/delegate design that the comm layer uses. I'll
send that seperately as a zip, as Apache won't let me post a gif to the
list.

You can see that on the client side, the JMS implementation is calling the
protocol broker interface on the comm layer through its invoker. It is
providing an implementation of the client delegate interface to the comm
layer, for the comm layer to inform it of all incoming client method calls
initiated by the broker side. The JMS layer need only implement those calls
it is interested in, as the abstract base class provide no-ops for
everything by default.

The comm layer translates those incoming protocol calls into frames, these
go over the transport to another comm layer instance on the broker side.
That comm layer instance translates the incoming frames into protocol method
calls, some of which it handles internally too, as it is stateful. The
broker routing engine has provided this second comm layer instance with a
delegate, to inform it of incoming protocol methods as events. It does its
routing and calls the comm layer client interface through its invoker. Those
method calls go back to the clients in a similar manner.

This is a modular design, and it is an API that exposes in a minimal way,
what is necessary to interact with the full AMQP protocol. This is in accord
with Parnas' principles of modular design.

As it is modular and in terms of interfaces, not concrete classes, you can
plug a client straight into a routing engine. The routing engine implements
the broker side interface for incoming methods. The client code can be
written in such a way that it is unaware of whether it is to be plugged into
the comm layer or routing engine direct. Just supply it with a suitably
configured factory for the situation.

Also, for a laugh we could do an implementation of the comm/transport layer,
that implements this API, but just uses the default java RMI over IIOP. It'd
be AMQP at the method/class level but not the correct wire format. A quick
way of getting something up and running.

Rajith, you rightly point out that our customers are not going to want to
have to code the open/negotiate connection interaction or the open/attach
session interaction and so on themselves. Also, they will want default
implementations of the ping/pong, execution and transaction model, fail-over
and so on. So a pure low-level API is too low-level to be API that in the
vast majority of cases, applications are coded against (naturally, they will
use JMS).

For these common interactions, there needs to be a way to make a blocking
wait for a response. For example, open session, wait for session attached
and so on. For these cases I have defined a 'conversation' api, which
represents the common interactions that the client can instigate that
require a synchronous interaction. The conversation interface defines
'receive' methods that block for a particular incoming method. It also
provides convenience methods that encode the common interaction sequences.
This is in a very similar style to the
org.apache.qpid.util.ConversationFactory class, ex

Re: [Java] Message API design notes

[Rupert Smith]

Ok, sorry about the delay, I got the tests fixed up. First off, why I think
the proposed API (http://people.apache.org/~rajith/qpid_docs/client_api/)
offers almost nothing that JMS does not already cover. And then a second
email about what my opinions/ideas about the low-level API are.

The proposed API, is primarily geared towards listening for messages, of
course, combined with sending messages, the most important thing a messaging
API is interested in providing. So you have a 'MessageListener', as does
JMS.

You also present a little example, of sending messages through a push driven
streaming API:

//Option3 - can use it with any message size, recomended for large messages
public void messageTransfer(String destination, short confirmMode,
short acquireMode);
public void headers(Struct... headers);
public void data(byte[] data);
public void data(ByteBuffer buf);
public void data(String str);
public void endData();

as if this is an important innovation. But then JMS has the StreamMessage
class, that provides this capability. There is a difference, and that is
that in the example, the transfer comes before the appending of data,
followed by an explicit end to terminate. Does this mean we are doing real
streaming over the protocol 'message class'? Again, I am not clear on this,
0-8 had separate 'message' and 'stream' classes, does 'message' in 0-10 now
covers streaming too? If so, then it looks to me like this would be a useful
JMS API extension, that would sit side by side with the JMS implementation,
rather than as a lower-level that sits beneath it. In a similar way to how
the current 0-8 implementation provides horizontal extension of JMS for AMQP
specific features.

This proposed API lets me bind exchanges, about the only thing it offers
that JMS does not already cover.

I have looked at MessagePartListener, and it does not convince me that this
API is a 1:1 mapping of the protocol. Looking inside the protocol XML, I am
looking at every method in it, some are marked as:

 wrote:
>
> >
> >
> > As for JMS, yes things can change, but personally, if I were trying out
> a
> > new an as yet unproved technology (Qpid), I would program to JMS, just
> to
> > be
> > sure I keep my options open, so I could switch easily to a different JMS
> > provider if need be.
>
>
> The entire Qpid community is  working towards making it a proven AMQP
> implementation.
> The AMQP spec group is working very hard to be the messaging protocol of
> choice.
> I have heard from users who are keen to take the risk and I have provided
> who they are in my previous email.
> I am willing to help people who put faith in AMQP and Qpid.
> I like to think that we as a community will provide the best solution out
> there. !!
>
> I think your API copies too much from JMS and is not
> > low-level enough. The API that the comm layer exposes, gets it just
> about
> > right IMO.
> >
>
> Can you please provide concrete examples of where it copies from the JMS??
> The message listener is not a JMS concept. It's been use before and will
> be
> used after JMS
>
> Rupert you are making vague comments  and beating around the bush.
> What low level AMQP method it doesn't expose? or where it fails to be low
> level. (other than connection negotiation and failover)
> You are yet to answer this question.
>
> On 16/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> > >
> > > >
> > > > > > 1. Methods as classes, rather than exposed arguments.
> > > > >
> > > > >
> > > > > [RA] On a general note, as an API user I like to use arguments
> > rather
> > > > than
> > > > > classes.
> > > > > I think that's what a majority of users would prefer as it looks
> > more
> > > > > natural. YMMV
> > > > >
> > > >
> > > >
> > > > Yes, but two points:
> > > >
> > > > Most users will use JMS, not the low-level API. The low-level is to
> > > > implement JMS on top of, and to provide extra capabilities in some
> > > > scenarioes.
> > >
> > >
> > > [RA] Well as AMQP gets more popular people may  want to use AMQP
> > directly.
> > > Rabbit already has a no JMS java client.
> > > Synapse is interested in using AMQP without JMS
> > > We already have a customer who doesn't care about JMS but a java
> client.
> > >
> > > I am not discounting JMS. But let the users take what they want.
> > > Nothing is forever. EJB proved that beyond dobut.
> > >
> > > Can provide the exposed argument forms as convenience methods on top
> of
> > > the
> > > > method-as-class/factory style.
> > > >
> > >
> >
>

Re: [Java] Message API design notes

[Rajith Attapattu]

>
>
> As for JMS, yes things can change, but personally, if I were trying out a
> new an as yet unproved technology (Qpid), I would program to JMS, just to
> be
> sure I keep my options open, so I could switch easily to a different JMS
> provider if need be.


The entire Qpid community is  working towards making it a proven AMQP
implementation.
The AMQP spec group is working very hard to be the messaging protocol of
choice.
I have heard from users who are keen to take the risk and I have provided
who they are in my previous email.
I am willing to help people who put faith in AMQP and Qpid.
I like to think that we as a community will provide the best solution out
there. !!

I think your API copies too much from JMS and is not
> low-level enough. The API that the comm layer exposes, gets it just about
> right IMO.
>

Can you please provide concrete examples of where it copies from the JMS??
The message listener is not a JMS concept. It's been use before and will be
used after JMS

Rupert you are making vague comments  and beating around the bush.
What low level AMQP method it doesn't expose? or where it fails to be low
level. (other than connection negotiation and failover)
You are yet to answer this question.

On 16/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> >
> > >
> > > > > 1. Methods as classes, rather than exposed arguments.
> > > >
> > > >
> > > > [RA] On a general note, as an API user I like to use arguments
> rather
> > > than
> > > > classes.
> > > > I think that's what a majority of users would prefer as it looks
> more
> > > > natural. YMMV
> > > >
> > >
> > >
> > > Yes, but two points:
> > >
> > > Most users will use JMS, not the low-level API. The low-level is to
> > > implement JMS on top of, and to provide extra capabilities in some
> > > scenarioes.
> >
> >
> > [RA] Well as AMQP gets more popular people may  want to use AMQP
> directly.
> > Rabbit already has a no JMS java client.
> > Synapse is interested in using AMQP without JMS
> > We already have a customer who doesn't care about JMS but a java client.
> >
> > I am not discounting JMS. But let the users take what they want.
> > Nothing is forever. EJB proved that beyond dobut.
> >
> > Can provide the exposed argument forms as convenience methods on top of
> > the
> > > method-as-class/factory style.
> > >
> >
>

Re: [Java] Message API design notes

[Rupert Smith]

Ok, but I have some tests to run and write up, so maybe not till tomorow
afternoon or monday.

On 16/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
>
> >
> > My comments are more directed at:
> > http://cwiki.apache.org/confluence/display/qpid/Message+API+Design,
> which
> > I
> > don't feel is really the low-level protocol 1:1 API.
>
>
> As I pointed out in the previous emails. The API expose everything  except
> for the connection negotiation and failover.
> And have added a few convenience methods for message handling.
> I don't understand what is not low level about it?
> The API has the same level of granularity as the com layer except for
> connection negotiation and failover.
> Do u envision users wanting to do all that?
>
> Yes MessagePartListener exposes more, but not everything. I was hoping to
> > see every method that the broker can call on the client, not just for
> > 'message' but everything.
>
>
> Again I am confused as to what you want. The MessagePartListener exposed
> message content Frame by Frame using the data method ?
> What more do you want? What do mean by not everything? What are these
> methods?
>
> Can you please provide concreate examples? Lets start it from there.
>
> Rupert
> >
> > On 16/08/07, Arnaud Simon <[EMAIL PROTECTED]> wrote:
> > >
> > > Hi,
> > >
> > > It looks to me that the solution you are describing is very similar to
> > > the current common layer (I am not speaking about the API described @
> > > http://cwiki.apache.org/confluence/display/qpid/Message+API+Design).
> > > I would help (me at least) if you can highlight the main differences
> > > between the two approaches. What would change, What are the advantages
> > > of using your approach when compare with talking directly to the
> current
> > > common layer?
> > >
> > > Thanks
> > >
> > > Arnaud
> > >
> > > On Thu, 2007-08-16 at 15:46 +0100, Rupert Smith wrote:
> > > > Here is a picture that might help explain my idea more clearly.
> > > >
> > > > 'B' is the broker interface, callable by the client, 'C' is the
> > > > methods the client handles, callable by the broker. Comm layer
> > > > implements both, turning method calls into frames, frames into
> method
> > > > calls. An instance of the comm layer created through its factory,
> will
> > > > be specific to client or broker usage scenario.
> > > >
> > > > The broker routing layer also implements the protocol factory. As it
> > > > is a broker it only supplies the broker interface, if you try and
> gets
> > > > its client interface, it will throw an exception, which is why I put
> a
> > > > line through the 'C'.
> > > >
> > > > As you can see, the client that is talking to the comm layer, could
> > > > just as easily be talking to an in-vm broker.
> > > >
> > > >
> > >
> > >
> >
>

Re: [Java] Message API design notes

[Rajith Attapattu]

>
> My comments are more directed at:
> http://cwiki.apache.org/confluence/display/qpid/Message+API+Design, which
> I
> don't feel is really the low-level protocol 1:1 API.


As I pointed out in the previous emails. The API expose everything  except
for the connection negotiation and failover.
And have added a few convenience methods for message handling.
I don't understand what is not low level about it?
The API has the same level of granularity as the com layer except for
connection negotiation and failover.
Do u envision users wanting to do all that?

Yes MessagePartListener exposes more, but not everything. I was hoping to
> see every method that the broker can call on the client, not just for
> 'message' but everything.


Again I am confused as to what you want. The MessagePartListener exposed
message content Frame by Frame using the data method ?
What more do you want? What do mean by not everything? What are these
methods?

Can you please provide concreate examples? Lets start it from there.

Rupert
>
> On 16/08/07, Arnaud Simon <[EMAIL PROTECTED]> wrote:
> >
> > Hi,
> >
> > It looks to me that the solution you are describing is very similar to
> > the current common layer (I am not speaking about the API described @
> > http://cwiki.apache.org/confluence/display/qpid/Message+API+Design).
> > I would help (me at least) if you can highlight the main differences
> > between the two approaches. What would change, What are the advantages
> > of using your approach when compare with talking directly to the current
> > common layer?
> >
> > Thanks
> >
> > Arnaud
> >
> > On Thu, 2007-08-16 at 15:46 +0100, Rupert Smith wrote:
> > > Here is a picture that might help explain my idea more clearly.
> > >
> > > 'B' is the broker interface, callable by the client, 'C' is the
> > > methods the client handles, callable by the broker. Comm layer
> > > implements both, turning method calls into frames, frames into method
> > > calls. An instance of the comm layer created through its factory, will
> > > be specific to client or broker usage scenario.
> > >
> > > The broker routing layer also implements the protocol factory. As it
> > > is a broker it only supplies the broker interface, if you try and gets
> > > its client interface, it will throw an exception, which is why I put a
> > > line through the 'C'.
> > >
> > > As you can see, the client that is talking to the comm layer, could
> > > just as easily be talking to an in-vm broker.
> > >
> > >
> >
> >
>

Re: [Java] Message API design notes

[Rupert Smith]

I think saying that sping/hibernate/tomcat disproved EJB is nearer the mark.
I think spring/hibernate sold themselves on their intuitive API's.

As for JMS, yes things can change, but personally, if I were trying out a
new an as yet unproved technology (Qpid), I would program to JMS, just to be
sure I keep my options open, so I could switch easily to a different JMS
provider if need be. I think your API copies too much from JMS and is not
low-level enough. The API that the comm layer exposes, gets it just about
right IMO.

On 16/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
>
> >
> > > > 1. Methods as classes, rather than exposed arguments.
> > >
> > >
> > > [RA] On a general note, as an API user I like to use arguments rather
> > than
> > > classes.
> > > I think that's what a majority of users would prefer as it looks more
> > > natural. YMMV
> > >
> >
> >
> > Yes, but two points:
> >
> > Most users will use JMS, not the low-level API. The low-level is to
> > implement JMS on top of, and to provide extra capabilities in some
> > scenarioes.
>
>
> [RA] Well as AMQP gets more popular people may  want to use AMQP directly.
> Rabbit already has a no JMS java client.
> Synapse is interested in using AMQP without JMS
> We already have a customer who doesn't care about JMS but a java client.
>
> I am not discounting JMS. But let the users take what they want.
> Nothing is forever. EJB proved that beyond dobut.
>
> Can provide the exposed argument forms as convenience methods on top of
> the
> > method-as-class/factory style.
> >
>

Re: [Java] Message API design notes

[Rupert Smith]

Yes, the current common layer, the stuff Rafael has been working on, looks
very good to me. As I say, all I have really done is to split the
invoker/delegate concept up into methods that client can call vs methods
that broker can call, and to expose everything through interfaces rather
than as concrete classes. That is, I have pulled out what I consider to be
pure API. There is essentially very little difference other than that.

The advantage of using interfaces, rather than concrete classes, is that
broker routing layer, can partially implement the comm layer, for direct
client to in-vm broker. That's really the only difference.

My comments are more directed at:
http://cwiki.apache.org/confluence/display/qpid/Message+API+Design, which I
don't feel is really the low-level protocol 1:1 API.

Yes MessagePartListener exposes more, but not everything. I was hoping to
see every method that the broker can call on the client, not just for
'message' but everything.

Rupert

On 16/08/07, Arnaud Simon <[EMAIL PROTECTED]> wrote:
>
> Hi,
>
> It looks to me that the solution you are describing is very similar to
> the current common layer (I am not speaking about the API described @
> http://cwiki.apache.org/confluence/display/qpid/Message+API+Design).
> I would help (me at least) if you can highlight the main differences
> between the two approaches. What would change, What are the advantages
> of using your approach when compare with talking directly to the current
> common layer?
>
> Thanks
>
> Arnaud
>
> On Thu, 2007-08-16 at 15:46 +0100, Rupert Smith wrote:
> > Here is a picture that might help explain my idea more clearly.
> >
> > 'B' is the broker interface, callable by the client, 'C' is the
> > methods the client handles, callable by the broker. Comm layer
> > implements both, turning method calls into frames, frames into method
> > calls. An instance of the comm layer created through its factory, will
> > be specific to client or broker usage scenario.
> >
> > The broker routing layer also implements the protocol factory. As it
> > is a broker it only supplies the broker interface, if you try and gets
> > its client interface, it will throw an exception, which is why I put a
> > line through the 'C'.
> >
> > As you can see, the client that is talking to the comm layer, could
> > just as easily be talking to an in-vm broker.
> >
> >
>
>

Re: [Java] Message API design notes

[Rajith Attapattu]

>
> > > 1. Methods as classes, rather than exposed arguments.
> >
> >
> > [RA] On a general note, as an API user I like to use arguments rather
> than
> > classes.
> > I think that's what a majority of users would prefer as it looks more
> > natural. YMMV
> >
>
>
> Yes, but two points:
>
> Most users will use JMS, not the low-level API. The low-level is to
> implement JMS on top of, and to provide extra capabilities in some
> scenarioes.


[RA] Well as AMQP gets more popular people may  want to use AMQP directly.
Rabbit already has a no JMS java client.
Synapse is interested in using AMQP without JMS
We already have a customer who doesn't care about JMS but a java client.

I am not discounting JMS. But let the users take what they want.
Nothing is forever. EJB proved that beyond dobut.

Can provide the exposed argument forms as convenience methods on top of the
> method-as-class/factory style.
>

Re: [Java] Message API design notes

[Rupert Smith]

On 16/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
>
> > gets up to such tricks. There is also an out-of-date project,
> > http://www.eecs.harvard.edu/~mdw/proj/old/jaguar/, that does this sort
> of
> > thing too, no idea but maybe the Terracotta folks got some ideas from
> > there.
>
>
> [RA] Sounds interesting. However  my experience is that fancy solutions
> often result in complex code hard to comprehend and difficult to maintain.
> You heard about YAGNI? "suggests to programmers that they should not add
> functionality until it is necessary"
> We can adopt a strategy to keep the door open for all these
> customizations,
> but my experience most projects don't even end up doing these things.
>

I'm definitely not suggesting that we do this stuff now, and it certainly
sounds complex to do. But the best way to keep options open is through a
carefully chosen interface. I think choosing the right interface is vital,
because once it is chosen it is not likely to change.

Re: [Java] Message API design notes

[Rupert Smith]

On 16/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
>
> Rupert,
>
> first of all thank you for talking the time to go through and comment.
> Let me try answering the first 4 questions.
>
> On 8/16/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
> >
> > Hi, I have tried putting a case forward for three things.
> >
> > 1. Methods as classes, rather than exposed arguments.
>
>
> [RA] On a general note, as an API user I like to use arguments rather than
> classes.
> I think that's what a majority of users would prefer as it looks more
> natural. YMMV
>


Yes, but two points:

Most users will use JMS, not the low-level API. The low-level is to
implement JMS on top of, and to provide extra capabilities in some
scenarioes.

Can provide the exposed argument forms as convenience methods on top of the
method-as-class/factory style.

Re: [Java] Message API design notes

[Arnaud Simon]

Hi,

It looks to me that the solution you are describing is very similar to
the current common layer (I am not speaking about the API described @
http://cwiki.apache.org/confluence/display/qpid/Message+API+Design). 
I would help (me at least) if you can highlight the main differences
between the two approaches. What would change, What are the advantages
of using your approach when compare with talking directly to the current
common layer?

Thanks 

Arnaud

On Thu, 2007-08-16 at 15:46 +0100, Rupert Smith wrote:
> Here is a picture that might help explain my idea more clearly.
> 
> 'B' is the broker interface, callable by the client, 'C' is the
> methods the client handles, callable by the broker. Comm layer
> implements both, turning method calls into frames, frames into method
> calls. An instance of the comm layer created through its factory, will
> be specific to client or broker usage scenario. 
> 
> The broker routing layer also implements the protocol factory. As it
> is a broker it only supplies the broker interface, if you try and gets
> its client interface, it will throw an exception, which is why I put a
> line through the 'C'. 
> 
> As you can see, the client that is talking to the comm layer, could
> just as easily be talking to an in-vm broker.
> 
> 

Re: [Java] Message API design notes

[Rajith Attapattu]

Rupert,

first of all thank you for talking the time to go through and comment.
Let me try answering the first 4 questions.

On 8/16/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
>
> Hi, I have tried putting a case forward for three things.
>
> 1. Methods as classes, rather than exposed arguments.


[RA] On a general note, as an API user I like to use arguments rather than
classes.
I think that's what a majority of users would prefer as it looks more
natural. YMMV

Reason: Methods created by factories, allowing multi-version support
> (although Raphael points out that if the args change significantly then
> the
> protocol is changing too much between versions, so has gone for a
> different
> solution to the multi-version issue).


[RA] I agree with Rafi here. The point is any significant change will need
more than multi version support.
So a client or broker may need to rework stuff anyways - changing method
arguments is the least of concerns at that time.
So we can keep the multi version support simple and nice.

Allows one implementation of methods
> as byte buffers, another as java fields, for in-vm transport. Keeps open
> possibility of native implementation. Allows pass-through processing,
> where
> method is never decoded or only partially decoded. Also, it might be
> possible to do faster serialization/deserialization, using a JVM tweaked
> at
> the byte code level. I believe that Terracotta (http://terracottatech.com/
> )
> gets up to such tricks. There is also an out-of-date project,
> http://www.eecs.harvard.edu/~mdw/proj/old/jaguar/, that does this sort of
> thing too, no idea but maybe the Terracotta folks got some ideas from
> there.


[RA] Sounds interesting. However  my experience is that fancy solutions
often result in complex code hard to comprehend and difficult to maintain.
You heard about YAGNI? "suggests to programmers that they should not add
functionality until it is necessary"
We can adopt a strategy to keep the door open for all these customizations,
but my experience most projects don't even end up doing these things.

2. Full mapping of the protocol 1:1 onto the API.
>
> I agree that onMessage is the most frequently handled incoming method on
> the
> client side, but there is the rest of the protocol to consider too. The
> API
> just isn't low-level if it does not expose everything.


[RA] This question tells me that u haven't gone through the doc fully :)
MessageListener interface is not even the primary means of exposing message
receiving.

Did u see the MessagePartListener which is very natural to the way AMQP
works ??
It directly exposes (1:1 mapping) the way the protocol works when receiving
messages.
MassageListener is supported as an Adapter on top of this for users who
needs a convenient way of handling small messages.

Just to be clear, The API exposes the entire protocol (or the COM layer)
except that it provides 3 convenience methods.
Connect - which handles connection negotiation
Convenient message methods - small messages + streaming support for large
messages.
Failover.

Other than that it exposes everything else 1:1. Can you please point out
concrete examples of what is not exposed?
Lets start there and work our way through.

If for some reason if a user wants to do connection negotiation and failover
themselves then they could directly use the com layer.
But 99% of the users will prefer the thin API layer outlined in the current
code.

3. Use of the inverse symmetry of the API so that the outgoing interface
> that the client calls, is identical to the incoming interface that the
> broker routing layer implements.


[RA] This is actually the case. As Rafi explained before, the Delegate is
the inverse of the Invoker.

Comm layer can be completely bypassed, and a client can be directly welded
> onto the broker.


[RA] This can be easily done. Rafi will work on the transport layer to
abstract the underlying transport (TCP,SCTP..etc) when he comes back.
He just wanted the common use case working before anything else.

I think I should add a fourth to this list:
>
> 4. Be language neutral, that is a suitable API for implementing in any of
> our target languages.


[RA] This is a question for the AMQP group. From the discussions so far, I
gather that the AMQP goups is not ready for providing an AMQP API even if it
wants, as the protocol is not stable yet.
CORBA made that mistake and eventually suffered the consequences of it.
I don't think there is a need for a separate language neutral API from AMQP.

IMO the  semantics defined by way of classes in the sepc already provides
this language neutral API.
Implementations in multiple languages (in OO, Functional and procedural - if
a pure c client exists) proves that this suffices.

Can assume target language is OO? Not sure I'd like to invent an API that
> equally well maps onto an OO as well as a functional language ;).


[RA] There u go. Defining a language neutral API is not a value bearing
exercise.
An AMQP implementation 

Re: [Java] Message API design notes

[Rupert Smith]

Hi, I have tried putting a case forward for three things.

1. Methods as classes, rather than exposed arguments.

Reason: Methods created by factories, allowing multi-version support
(although Raphael points out that if the args change significantly then the
protocol is changing too much between versions, so has gone for a different
solution to the multi-version issue). Allows one implementation of methods
as byte buffers, another as java fields, for in-vm transport. Keeps open
possibility of native implementation. Allows pass-through processing, where
method is never decoded or only partially decoded. Also, it might be
possible to do faster serialization/deserialization, using a JVM tweaked at
the byte code level. I believe that Terracotta (http://terracottatech.com/)
gets up to such tricks. There is also an out-of-date project,
http://www.eecs.harvard.edu/~mdw/proj/old/jaguar/, that does this sort of
thing too, no idea but maybe the Terracotta folks got some ideas from there.

2. Full mapping of the protocol 1:1 onto the API.

I agree that onMessage is the most frequently handled incoming method on the
client side, but there is the rest of the protocol to consider too. The API
just isn't low-level if it does not expose everything.

3. Use of the inverse symmetry of the API so that the outgoing interface
that the client calls, is identical to the incoming interface that the
broker routing layer implements.

Comm layer can be completely bypassed, and a client can be directly welded
onto the broker.

I think I should add a fourth to this list:

4. Be language neutral, that is a suitable API for implementing in any of
our target languages.

Can assume target language is OO? Not sure I'd like to invent an API that
equally well maps onto an OO as well as a functional language ;).

To these ends, I have been playing around with the Stub that Raphael
originally posted to this list. I am slowly getting an API into shape, on my
laptop on the way to and from work on the train, but haven't really the time
to dedicate to completing it yet.

Here is a rough outline:

Everything in the API is an interface first, the comm layer implements it. I
like interfaces, because you can have multiple implementations. There is
also a toy broker that implements some of the interfaces too, allowing
direct connection of the client onto an in-vm broker.

Every 'class' in the protocol has two interfaces in the API. One for the
outgoing calls that the client may make to the broker, one for the calls the
broker may make to the client.

Every 'method' in the protocol has an interface.

All the outgoing 'classes' are pulled together into a single interface, and
all the incoming 'classes' are pulled together into a single interface that
extends them all. The reason for splitting down on per-'class' basis, is in
case someone wants to implement a delegate for just one 'class', rather than
having to implement all methods. The were probably also be no-op abstract
implementations, that can be extended to write delegates that handle just
one 'method' event out of a 'class'.

Methods are created by a method factory.

Methods can be called in the context of a connection, or a session. The
initial connection creation and session creation are performed by calling
their 'open' methods.

The comm layer implements both the incoming and outgoing 'classes', you have
to ask the factory for the flavour you want depending on the context. For
example the interface for the client to call the broker is called
'ProtocolBroker' (extends ClassSessionBroker, ClassMessageBroker, etc), and
the one for the broker to call the client is 'ProtocolClient'.(extends
ClassSessionClient, ClassMessageClient, etc).

The factory allows you to register delegates, with the comm layer. When the
comm layer receives incoming frames, it delegates the decoded method calls
onto the delegate (I think just one delegate for the moment, rather than an
interceptor pattern style chain, not sure, will probably have to do some
kind of chain eventually, xwork/webwork may provide some inspiration for
this).

The 'toy' broker I have sketched out, implements ProtocolBroker. It also
provides a protocol factory, this factory will throw an exception if you ask
it to create a ProtocolClient, but it will give you a ProtocolBroker.

So to write a client against the comm layer, ask its ProtocoFactory
implementation for a ProtocolBroker. To write a client directly against the
broker routing engine, ask its ProtocolFactory implementation for a
ProtocolBroker. This is point 3 above, about the inverse symmetry of the
interfaces.

Some methods in the protocol require a response, but all are asynchronous.
For this I am providing a pair of conversation interfaces, with wait for
methods. So for example to open and attach a session, I do:

ProtocolFactory factory = ...
ProtocolBroker invoker = ...
ConnectionContext connection = ...
ProtoclClient delegate = factory.defaultClientDelegate();

MethodOpenSession = fact