Onto some ideas about low-level API, what I had imagined such a thing would
look like. I've been hacking this around on my laptop, but without internet
access for it (or time to rebuild my kernel with WiFi support ;-), should
get hooked up at home again on thursday), I will just have to give an
outline of what my ideas look like. I don't like to mail code for something
that I'm not completely satisfied with anyway; yes, my ideas are ideas,
their merit is debatable, and I am interested in that debate to sound out
opinion and shape-up and strengthen the ideas to our mutual satisfaction. I
think this interface/API is an important one, because it is what the clean
modular design of our implementation will hinge around.
As I mentioned previously I had 3, now 4 criteria:
1. Methods as classes, rather than exposed arguments.
2. Full mapping of the protocol 1:1 onto the API.
3. Use of the inverse symmetry of the API so that the outgoing interface
that the client calls, is identical to the incoming interface that the
broker routing layer implements.
4. Be language neutral, that is a suitable API for implementing in any of
our target languages. Will assume OO as a base-line.
I have pulled out of the stub code that Rafael sent, an API purely in terms
of interfaces. No great surprises, once you see how the comm layer works. As
I've mentioned already, the only thing I have really tried to change is the
splitting each of the protocol 'classes' into the server and client chassis
parts. As it is in terms of interfaces only, there is also a factory to work
with this API. I have dropped the 'context's as type parameters from the
API, because I feel that in terms of the API the context is never abstract
enough to need parametric polymorphism; class subtype polymorphism will
suffice.
I've updated my diagram, because the first one I sent was a bit misleading,
in that it erased the invoker/delegate design that the comm layer uses. I'll
send that seperately as a zip, as Apache won't let me post a gif to the
list.
You can see that on the client side, the JMS implementation is calling the
protocol broker interface on the comm layer through its invoker. It is
providing an implementation of the client delegate interface to the comm
layer, for the comm layer to inform it of all incoming client method calls
initiated by the broker side. The JMS layer need only implement those calls
it is interested in, as the abstract base class provide no-ops for
everything by default.
The comm layer translates those incoming protocol calls into frames, these
go over the transport to another comm layer instance on the broker side.
That comm layer instance translates the incoming frames into protocol method
calls, some of which it handles internally too, as it is stateful. The
broker routing engine has provided this second comm layer instance with a
delegate, to inform it of incoming protocol methods as events. It does its
routing and calls the comm layer client interface through its invoker. Those
method calls go back to the clients in a similar manner.
This is a modular design, and it is an API that exposes in a minimal way,
what is necessary to interact with the full AMQP protocol. This is in accord
with Parnas' principles of modular design.
As it is modular and in terms of interfaces, not concrete classes, you can
plug a client straight into a routing engine. The routing engine implements
the broker side interface for incoming methods. The client code can be
written in such a way that it is unaware of whether it is to be plugged into
the comm layer or routing engine direct. Just supply it with a suitably
configured factory for the situation.
Also, for a laugh we could do an implementation of the comm/transport layer,
that implements this API, but just uses the default java RMI over IIOP. It'd
be AMQP at the method/class level but not the correct wire format. A quick
way of getting something up and running.
Rajith, you rightly point out that our customers are not going to want to
have to code the open/negotiate connection interaction or the open/attach
session interaction and so on themselves. Also, they will want default
implementations of the ping/pong, execution and transaction model, fail-over
and so on. So a pure low-level API is too low-level to be API that in the
vast majority of cases, applications are coded against (naturally, they will
use JMS).
For these common interactions, there needs to be a way to make a blocking
wait for a response. For example, open session, wait for session attached
and so on. For these cases I have defined a 'conversation' api, which
represents the common interactions that the client can instigate that
require a synchronous interaction. The conversation interface defines
'receive' methods that block for a particular incoming method. It also
provides convenience methods that encode the common interaction sequences.
This is in a very similar style to the
org.apache.qpid.util.ConversationFactory class, except with protocol
specific methods.
So here is the pure interface API outlined:
For each 'class' in the protocol there is a 'client' and 'broker' version
for the 'client' and 'server' chassis respectively. 'client' describes the
methods that a client can receive, 'broker' the methods that the broker can
receive.
For example:
public interface ClassSessionBroker
{
public void sessionOpen(ConnectionContext context, MethodSessionOpen
open);
...
}
public interface ClassSessionAttached
{
public void sessionAttached(SessionContext context, MethodSessionAttached
attached);
...
}
All 'broker' and 'client' interfaces are gathered together as ProtocolBroker
and ProtocolClient. For example:
public interface ProtocolBroker extends ClassMethodBroker,
ClassSessionBroker, ClassExecutionBroker, ...
There is a method factory, for creating methods. Every method is defined as
an interface, providing 'get' access to its fields. For example:
public interface MethodExchangeDeclare extends Method
{
public String getName();
public boolean isDirable();
...
}
'Method' is a common root interface for all methods, fully opaque; its just
a marker interface, to allow methods that handle all 'Methods' to be
written.
It seems a bit unnatural to me to put the full protocol in one interface per
side, ProtocolBroker and ProtocolClient, because the context of the methods
is connection in some cases or session in others, and these things have
different lifecycles. However, as they are given as explicit arguments, it
seems ok to do it that way.
Connections are repreneted by the fully opaque 'ConnectionContext'
interface, and sessions by 'SessionContext'. You have to go to the factory
and through the open/.. procedure to get one of these.
There is a factory for the protocol, that lets you create methods, register
delegates, acquire a handle on an invoker, and get a conversation
implementation for those common synchronous interactions:
public interface ProtocolFactory
{
public MethodFactory getMethodFactory();
// To get invokers to call thr protocol through.
public ProtocolBroker getBrokerInvoker();
public ProtocolClient getClientInvoker();
// To register delegates to be informed of incoming methods-as-events.
public void registerBrokerDelegate(ProtocolBroker delegate);
public void registerClientDelegate(ProtocolClient delegate);
// Conversation for synchronous common iteractions.
public ProtocolConversation createConversation(ProtocolBroker invoker);
}
The conversation interface looks like:
/** Implements ProtocolClient as is delegate for incoming client methods,
ProtocolBroker as is invoker for outgoing broker methods. */
public interface ProtocolConversation extends ProtocolBroker, ProtocolClient
{
// To block for incoming methods.
public MethodSessionAttached receiveSessionAttached();
... and so on.
// Common interactions.
public ConnectionContext openConnectionSequence();
public SessionContext openSessionSequence(ConnectionContext connection);
.... and so on.
}
Comm layer provides a factory implementation. Broker routing engine/layer
provides one, but one that only lets you create a broker invoker, and
register a client delegate.
A quick example of what using this low-level API might look like:
ProtocolFactory pFactory = CommLayerProtoclFactory.getInstance();
MethodFactory mFactory = pFactory.getMethodFactory();
ProtocolBroker invoker = pFactory.getBrokerInvoker();
ProtocolConversation conversation = pFactory.createConversation(invoker);
pFactory.registerClientDelegate(conversation); // Probably unnecessary as
can auto-register upon creation.
// Open a connection.
ConnectionContext connection = conversation.openConnectionSequence();
// Open a session.
SessionContext session = conversation.openSessionSequence(connection);
// Send a message.
invoker.messageTransfer(session, mFactory.createMessageTransfer(...));
and inside the conversation implementation something like:
public SessionContext openSessionSequence(ConnectionContext connection)
{
invoker.openSession(connection, mFactory.createSessionOpen(...));
SessionContext session = receiveSessionAttached();
return session;
}
It fits my criteria, has anyone else any requirements to add?
Rupert
On 20/08/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
>
> Ok, sorry about the delay, I got the tests fixed up. First off, why I
> think the proposed API (
> http://people.apache.org/~rajith/qpid_docs/client_api/<http://people.apache.org/%7Erajith/qpid_docs/client_api/>)
> offers almost nothing that JMS does not already cover. And then a second
> email about what my opinions/ideas about the low-level API are.
>
> The proposed API, is primarily geared towards listening for messages, of
> course, combined with sending messages, the most important thing a messaging
> API is interested in providing. So you have a 'MessageListener', as does
> JMS.
>
> You also present a little example, of sending messages through a push
> driven streaming API:
>
> //Option3 - can use it with any message size, recomended
> for large messages
> public void messageTransfer(String destination, short confirmMode,
> short acquireMode);
> public void headers(Struct... headers);
> public void data(byte[] data);
>
> public void data(ByteBuffer buf);
> public void data(String str);
> public void endData();
>
> as if this is an important innovation. But then JMS has the StreamMessage
> class, that provides this capability. There is a difference, and that is
> that in the example, the transfer comes before the appending of data,
> followed by an explicit end to terminate. Does this mean we are doing real
> streaming over the protocol 'message class'? Again, I am not clear on this,
> 0-8 had separate 'message' and 'stream' classes, does 'message' in 0-10 now
> covers streaming too? If so, then it looks to me like this would be a useful
> JMS API extension, that would sit side by side with the JMS implementation,
> rather than as a lower-level that sits beneath it. In a similar way to how
> the current 0-8 implementation provides horizontal extension of JMS for AMQP
> specific features.
>
> This proposed API lets me bind exchanges, about the only thing it offers
> that JMS does not already cover.
>
> I have looked at MessagePartListener, and it does not convince me that
> this API is a 1:1 mapping of the protocol. Looking inside the protocol XML,
> I am looking at every method in it, some are marked as:
>
> <chassis name="server" ...
>
> and some as
>
> <chassis name="client" ...
>
> and some as both of these. Now looking at the Session class, I certainly
> see a lot more of the 'server' chassis exposed, but not all of it. The only
> part of the protocol 'client' chassis I see exposed in this API is
> 'messageTransfer', and there is a lot more besides. Whilst I do agree that
> it is questionable whether or not we need to expose the full protocol as our
> low-level API, doing so would be undoubtedly low-level.
>
> As to the question of whether or not we should, we have had customers
> implementing their own heart beat pings, when there is a perfectly good
> ping/pong API in the protocol session 'class' that they could use. I do
> concede, that in the majority of cases, client code will not want to
> interact with the protocol so directly, being primarily interested in
> sending and receiving messages. But in that case they will use JMS.
>
> There seems to be a difference of opinion about what this so called
> low-level Qpid API is for. I see it as being a 1:1 mapping of the protocol,
> that JMS will be implemented on top of (and a similar or better client API
> in the other languages). All features of the protocol exposed for the
> convenience and symmetry of programming against. I see the low-level API as
> being mostly internal to Qpid, as an interface and abstraction that will
> help us to build the software in a cleanly modular fashion.
>
> Rajith, you keep arguing that you are inventing a new messaging API for
> Java, that people will somehow shift over to using as they defect from JMS.
> Comparing JMS to EJB's is an analogy that attempts to confer the faults of
> EJB onto JMS, but a weak argument; there was a lot wrong with EJB, hence
> people were all to keen to defect to the funky spring/hibernate model. What
> exactly is so wrong with JMS? Just as I have been asked to drop my argument
> that too much layering will slow things down, I think this JMS replacement
> argument should also be dropped.
>
> I'm for horizontal extension of JMS to provide AMQP specific features, not
> layering.
>
> Rupert
>
> On 16/08/07, Rajith Attapattu < [EMAIL PROTECTED]> wrote:
> >
> > >
> > >
> > > As for JMS, yes things can change, but personally, if I were trying
> > out a
> > > new an as yet unproved technology (Qpid), I would program to JMS, just
> > to
> > > be
> > > sure I keep my options open, so I could switch easily to a different
> > JMS
> > > provider if need be.
> >
> >
> > The entire Qpid community is working towards making it a proven AMQP
> > implementation.
> > The AMQP spec group is working very hard to be the messaging protocol of
> > choice.
> > I have heard from users who are keen to take the risk and I have
> > provided
> > who they are in my previous email.
> > I am willing to help people who put faith in AMQP and Qpid.
> > I like to think that we as a community will provide the best solution
> > out
> > there. !!!!!!
> >
> > I think your API copies too much from JMS and is not
> > > low-level enough. The API that the comm layer exposes, gets it just
> > about
> > > right IMO.
> > >
> >
> > Can you please provide concrete examples of where it copies from the
> > JMS??
> > The message listener is not a JMS concept. It's been use before and will
> > be
> > used after JMS
> >
> > Rupert you are making vague comments and beating around the bush.
> > What low level AMQP method it doesn't expose? or where it fails to be
> > low
> > level. (other than connection negotiation and failover)
> > You are yet to answer this question.
> >
> > On 16/08/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
> > > >
> > > > >
> > > > > > > 1. Methods as classes, rather than exposed arguments.
> > > > > >
> > > > > >
> > > > > > [RA] On a general note, as an API user I like to use arguments
> > > rather
> > > > > than
> > > > > > classes.
> > > > > > I think that's what a majority of users would prefer as it looks
> >
> > > more
> > > > > > natural. YMMV
> > > > > >
> > > > >
> > > > >
> > > > > Yes, but two points:
> > > > >
> > > > > Most users will use JMS, not the low-level API. The low-level is
> > to
> > > > > implement JMS on top of, and to provide extra capabilities in some
> > > > > scenarioes.
> > > >
> > > >
> > > > [RA] Well as AMQP gets more popular people may want to use AMQP
> > > directly.
> > > > Rabbit already has a no JMS java client.
> > > > Synapse is interested in using AMQP without JMS
> > > > We already have a customer who doesn't care about JMS but a java
> > client.
> > > >
> > > > I am not discounting JMS. But let the users take what they want.
> > > > Nothing is forever. EJB proved that beyond dobut.
> > > >
> > > > Can provide the exposed argument forms as convenience methods on top
> > of
> > > > the
> > > > > method-as-class/factory style.
> > > > >
> > > >
> > >
> >
>
>
