Rupert, thanks for the comments. I have marked my comments with [RA]
I also invite you to look at the code for the Qpid java client and the AMQP
low level client.
There is also a design doc that explains the AMQP low level client.
[RA] Let me start with a general comment that explains the rationale behind
the new client.
We all seems to be converging on a JMS --> Qpid Client --> AMQP Framing type
of layering for our client design.
The architecture of the AMQP protocol is evolving rapidly and our existing
client design was not able to support it with out hacks and many ugly
modifications.
As you correctly pointed out, lack of clean encapsulation of
responsibilities, classes tightly coupled and absence of a proper layering
mechanism in the old client provided major obstacles in implementing
fundamental protocol changes. Ex the execution sub layer or the new session
layer or failover.
The old client is a direct implementation of JMS interfaces. AMQP on the
other hand is evolving with some nice features that cannot be exploited
using the JMS API. Also there was no way to tap any protocol level features
for more fine grained control.
There is also some demand for a very low level protocol level client that
works at AMQP method level from other projects who want fast AMQP support
w/o bothering to use any JMS. There is also a demand for a slightly higher
level java client w/o any JMS. And then finally most end user wants JMS.
While most end users prefer JMS, most infrastructure projects wants native
support and likes a framework like client to work with.
This is NOT in any way to criticize the existing client. I rather see that
as a precursor and the new client certainly reflects the all the hard work
that has gone into the existing client.
On 6/5/07, Rupert Smith <[EMAIL PROTECTED]> wrote:
On 04/06/07, Rajith Attapattu <[EMAIL PROTECTED]> wrote:
>
> > Rather than having to remember the format of this URL scheme, I'd
> > prefer (overloaded) method arguments for the constitutent parts. E.g.
> > con.connect("localhost", 5672, "test", "guest", "guest"). Or perhaps a
> > properties object with some defined keys (i.e. similar to how JDBC
> > connections are often configured).
The existing AMQConnection class provides such a method...
> > or (probably even more controversially!):
> >
> > session.exchangeDeclare(false, false ...);
> > session.queueDeclare(false, false, ...);
> > session.queueBind(QpidConstants.DIRECT_EXCHANGE_NAME, ...);
The existing AMQSession does it this way... Length of the class is not the
problem, undocumented code, lack of clean encapsulation of responsibilies,
several classes all tightly bound together is. This can all be solved with
my refactoring axe! The session implementation is always going to be long,
just by the number of methods defined in javax.jms.Session, that by itself
does not make it bad.
[RA] The length of the class affects readability. I would rather read a
class with 300-500 lines of code than something with close to 2000 lines of
code.
The problem is compounded by adding more methods to the already long
javax.jms.Session. Ex queueCreate, queueBind.
Adding more methods to JMS is IMHO not the correct way to expose AMQP
features, neither does it do any justice for the AMQP features.
If we are trapped in the JMS mentality for long we will not evolve. It's
wise to have support for JMS. However it's prudent to be a bit more radical
and have an AMQP client. It certainly has enough merit to deserve it's own.
Gordon, see my above comment. Creating messages can be hidden behind
> factories.
Like the existing set of message factories?
[RA] No the existing message factories are too tightly coupled with JMS.
I am proposing three generic message factories.
TextMessageFactory, XMLMessageFactory, BytesMessageFactory. (We can add more
in the future .. example SOAPMessageFactory).
All JMS messages can be easily built on top of the above message factories.
We are planning some work with native support for XML messages, hence the
XMLMessageFactory.
Stream and Map messages can be built on top of the BytesMessageFactory and
frankly I have not seen any applications using these two types of messages.
That doesn't mean we shouldn't support it.
I would like to ask my question again, about what is the reason for writing
a new client from scratch? and what is the plan for integrating it into
the
existing code? Is it just an experiment with a new design that will be
used
to influence the existing client, or is the intention that the existing
client is to be completely scrapped and replaced?
[RA] The intent is to replace the existing client. Please read my general
comment about the rational behind it.
Several of us tried to re-factor the existing client with limited success.
Hence we decided to rewrite the client.
I don't want to try and defend the existing code too heavily, as already
pointed out, it is ugly in places but it seems to me that there is
re-invention going on when there is already an adequate solution for the
problem. Rather than have an API that I can call in a slightly different
way
to what is there already, I think I'd prefer some javadoc to explain what
all the existing parameters are for. What concerns me about throwing away
the existing solution is the time spend re-solving the same problems
again,
and the danger of throwing out bug fixes that have been put in as feedback
from customers required them.
[RA] Rupert its not an API with a different look and feel. Architecturally
it's way different.
re-invention should be used where necessary. It doesn't mean we are throwing
away the existing hard work.
Instead we are building better solutions based on the lessons we learned in
the previous outcomes.
The existing client is maturing. Probably it was coded for the success
pathway first, where things could go wrong the original authors just threw
an AMQException or JMSException and forgot about it. Then some of these
failure pathways have been examined more closely, in response to testing
and
bug reports and time spent thinking. There are still ways this client can
fail in unexpected ways, but it is getting there, and I feel more than
anything it is in need of completion, by closing off these existing
failures, and fully testing the expected error pathways as well as the
success ones (even better, by simply removing some of the error pathways
that cannot occurr), and by writing the javadoc. Starting from scratch
means
undergoing this maturation process again.
[RA] It's a risk we have to take. However this time we have more in depth
understanding of the AMQP protocol and the direction we are talking.
This time around we have more experience with all the problems/pit falls and
solutions that the original authors did not have.
If we were complacent with horse driven carriage and spent time just
improving it instead of thinking outside the box, where would the today's
cars , air planes be ??
Continuous innovation is the way forward. That's what people expect from
open source projects as opposed to proprietary projects.
One of the disadvantages of implementing a new client from scratch in a
completely new package is that existing code, tests for example, will not
be
refactored along at the same time. If you refactor, using say intellij,
you
will keep this code up-to-date as you go. For example, if I safe delete a
method, I will have to go through the tests and refactor to the new API to
get things to compile again, then re-run the tests keeping the test suite
up-to-date with developments.
[RA] We need to add more unit and integration tests for any code we write.
So new tests will be written for the client.
Going off on a tangent, here's a simple scheme I have used succesfully in
the past, for regression testing old bugs as code evolves - bug report
goes
in Jira Qpid-203, so you write a test called testQpid203 to expose the
bug,
so long as the test is passing the bug remains solved.
Here is a list of things that are buried somewhere in the existing code,
that a complete re-implementation might forget about, or have to re-invent
in a very similar fashion:
Hashed password implementation for SASL.
[RA] Any improvements has to be inherited while weeding out the mistakes.
Adding this would be easy, but as u said it's also easy to forget.
The ability to turn off prefetch on new connections to work with MULE (MULEs
fault and a pitty we have to have stuff like this in our code, but thats
the
real world creeping in).
ConnectionListener to listen for failovers (unfortunately we have an
existing customer using this non-JMS feature... their risk to bear for
being
non-JMS, I suppose).
[RA] The new client has better infrastructure for this kind of things. It's
written with AMQP in mind and not JMS.
It has Events and State listeners to listen to protocol level method
events/states.
Existing system properties such as amq strict flag.
[RA] One of the problems of the existing client is the indiscriminate use of
the system properties with virtually no documentation. The code is littered
with the use of system properties. One improvement I made is to unify the
configuration using commons configuration. Now all configuration has been
moved to an xml file which can be documented properly and configured easily.
However u can continue to override these properties using system properties
if u will.
w.r.t amqp strict flag, what does this do? the java implementation is
notorious for introducing non standard features. If u set this false does it
mean we are doing something non standard ? why do we want to do that?
Synchronous request/response that can be interrupted by fail-over. The way
fail-over works will change for the new protocol, because it supports it
better, but a similar solution involving interruptable tasks and some sort
of retry will inevitably be invented.
[RA] Failover will anyhow have to change and I don't see how u can implement
with the existing client with out major modifications.
This is just stuff that I know of, I haven't fully examined the existing
code, so there are probably other things too.
The danger is leaving existing customers behind on an old branch, and not
being able to bring them easily onto the new branch, for fear of
re-introducing old bugs,
[RA] Well when we change AMQP versions this is inevitable. Introducing a new
client doesn't mean we are re-introducing old bugs.
Some of these bugs will not even be relevant with the new client
architecture.
or incompatible API changes.
[RA] If customers are using JMS what is the issue?
I think u must be referring to the non JMS AMQP specific methods. That's
precisely what we are trying to address in the new client.
Frankly there was no API to start with, so there want be any incompatible
API changes. Did we have an AMQP API visible before ?? (other than a few
methods added as extensions to the JMS interfaces). I don't think u can call
that an API.
I did post about APIs
months ago, to make the point that an API needs to be decided early and
preferably set in stone from then on (things are a bit different for the
.Net because there are less people using it in the wild yet).
Unfortunately,
you need to give some carefull consideration before changing a public API
method, and the nice way to do it is to mark the old one @deprecated for a
while, before removing it.
Rupert
