Re: camel spring xml restructure

Tue, 02 Sep 2008 13:15:24 -0700 

Hi Vadim,
This has been on my radar (on and off) for almost a year now. I still don't have a good solution. 


We definitely want code completion in the ui, and we'd like the  
language to be extensible.  One of the biggest issue is that once you  
add support for a new pattern, it has to go in ProcessorType, which is  
why it grew like this.  It is quite easy now for a component to add  
support for new TypeConverters, Components, even Expression language,  
but not for DSL.  Ideally the DSL would be pluggable in a non  
intrusive way.



The biggest issue, I think is that we have to define the dsl in terms  
of not yet defined structures (actually I think there's exactly one:  
the pipeline, which in your example is called process).  That is,  
whatever the pipeline is a choice would be:


when_clause:
        WHEN pipeline;

otherwise_clause:
        OTHERWISE pipeline;

choice_pattern:
        (when_clause) +
        [otherwise_clause;

pattern:
        ...
        | choice_pattern
        ...
        /* should be extensible */

pipeline:
        (pattern)+



Well, there is some recursive definition there.  One should be able to  
define a new pattern in a separate BNF description, but still be added  
to the language, even if the price for this would be to rebuild the  
whole language again (which i don't see yet how one could avoid).



Lastly, one thing that would be nice to solve and is a bit messed up  
now, is not supporting constructs like:

from().choice().when().when() (notice the second when)

from 
().choice().when().pipeline().otherwise().pipeline().when().pipeline()  
(notice a when after otherwise)
from 
().choice 
().when 
().choice().when().pipeline().otherwise().pipeline().when().pipeline()  
(but this would be legal, as the last when would belong to the first  
choice)



Things could get pretty complicated because java is not a dynamic  
language, but I feel is should be possible.  Even in one programming  
language we could/should be able to support multiple syntaxes.  For  
instance a good subset of bpel could be supported, not just spring  
xml.  BAM does a bit of that, but it's built on top of the existing  
camel dsl, it does not add to it.


Cheers
Hadrian 




process_list :
      /* can be empty*/
      | process
      | process process_list /* works as pipiline */
      ;






On Sep 2, 2008, at 4:24 AM, Vadim Chekan wrote:


Thanks for your comments James,
It is good to know that BNF is supported in eclipse out of the box.

I'll try not to concentrate on the grammar too much and implement  
from-to syntax. And when it works, we can discuss the final syntax.


Vadim.

James Strachan wrote:

2008/8/30 Vadim Chekan <[EMAIL PROTECTED]>:

Claus,

I'm glad you see the problem. Arguably, difficulties with routing

configurations is one of the the biggest obstacles on wider Camel  
adoption.
There is nothing more frustrating than knowing exactly what you  
want in

terms of EIP but spending many hours with so minor issue as syntax.


Interesting enough, I was thinking about it yesterday too and my  
conclusion
is such that it is very difficult to achieve smooth configuration  
using xml.
It is not the tool for the job. I mean it can do it (it does it  
currently)
and we can keep improving it but will we get the easiest possible  
way of

configuring camel routes?

I'll skip large philosophical discussion but in my opinion xml  
sometimes

becomes victim of its own success and programmers try to stretch its

application beyond the reasons because "xml is so great, I would  
add it to

my salad if I could" :)


Basically we are trying to define Abstract Syntax Tree in xml  
serialized

form. People don't usually think in terms of AST.

Totally agree BTW. FWIW I'd always envisioned we'd have multiple  
DSLs;

Java, XML, Groovy, Ruby, Scala and even a real DSL - hopefully all
using the same AST underneath...
http://activemq.apache.org/camel/maven/camel-core/apidocs/org/apache/camel/model/package-summary.html
allowing folks to create routes using one language and save them in
another DSL if folks wanted to etc.

We can give a whirl an idea of Domain Specific Language (DSL). I  
have some
under my belt and it is not too difficult. The result would look  
like this

(syntax can be any, but I'm trying to be java script alike):

routes {
route {
  from 'queue:Incoming'
  process {
    bean 'org.my.company.camel' 'MyFunction'
    aggregator()
  }
  if (header('myHeader') == 'value1') {
    to 'queue:Outgoing'
  } else {
    to 'queue:Alternative'
  }
}
}

Have you looked at the Scala DSL? Its pretty close to this  
already! :)

http://activemq.apache.org/camel/scala-dsl.html
http://activemq.apache.org/camel/scala-dsl-eip.html

I thought about juel (or others evaluators).

I see a problem that those languagas will dictate syntax which may  
be not

optimal for camel route definition domain.

Another and bigger problem is that external language will not  
detect camel
invalid constructions. For example creating serializer but not  
assigning it
data format would be invalid in camel but from Juel point of view  
it would

be perfectly ok.

Yeah - we need to make sure we validate as much of the AST up front  
as

we can along with giving great error messages.

Having our own syntax parser will let us detect any syntax  
deviations and
provide user with very clear and informative error messages for  
example "

'bean' or 'aggregator' are expected inside in 'process' ".

Another benefit we get is Backus-Naur Form (BNF) file which is human

readable and is very easy to use in documentation. As opposite to  
xml schema
which can be used for learning purposes but I would not risk  
claiming it a

recommended way.

Here are fragments of BNF as I see it:
route :
      ROUTE
      | from_list process_list to_list
      ;
...
process_list :
      /* can be empty*/
      | process
      | process process_list /* works as pipiline */
      ;

process :
      /* can be empty */
      | BEAN LITERAL
      | MULTICAST '{ process_list '}'
      | PIPELINE '{' process_list '}'
      | SPLIT LITERAL
      | AGGREGATE expression
      | AGGREGATE
      | resequence
      ;

resequence :
      BATCH RESEQUENCE
      | BATCH RESEQUENCE expression
      | BATCH RESEQUENCE NUMBER NUMBER
      | BATCH RESEQUENCE NUMBER NUMBER '(' expression_list ')'
      | STREAM RESEQUENCE resequence_options
      | STREAM RESEQUENCE resequence_options '(' expression ')'
      ;

to_list :
      to
      | to to_list
      ;

to :
      LITERAL
      | IF '(' bool_expression ')' THEN to
      | IF '(' bool_expression ')' THEN to ELSE to
      ;

If you guys think it worth a try I can mock up a prototype.

I think I would build an AST and then have another process which  
would

traverse AST and invoke RouteBuilder to build the actual route.

This way we preserve backward compatibility with existing ways of  
building

routes.


This route builder would be a separate plugin in order to be as  
little

invasive as possible.


AST can be used in future to build graph representation of camel  
routes
without building the actual routes. I recall James recent comment  
that
building route for display purpose only (without running it) added  
some

complexity.

Let me know what do you think folks.

I've always fancied a real DSL as well :). As you mention - the
hardest part of any real DSL is what language is used for
predicates/expressions. e.g. do we let any language be used...
xpath( /foo/bar )
or something?
But it'd be great to try make one. Incidentally if you do fancy
experimenting with this; have a look at the xtext library in eclipse.
I created a little prototype project here...
https://svn.apache.org/repos/asf/activemq/camel/ide/camel-eclipse/
which attempts to take an Antlr grammar for the DSL and auto-create
both a parser but also an eclipse based editor for the language. It
takes an Antlr grammer here...
https://svn.apache.org/repos/asf/activemq/camel/ide/camel-eclipse/camel.routing/src/camel.xtxt
which as you can see is kinda basic :)

Having both a nice textual DSL along with an IDE to edit it would  
rock :)

Its well worth looking at the Scala DSL which looks great; maybe they
could both look and feel kinda similar? FWIW the only thing I don't

like with the scala DSL is the use of _ which is a bit odd - but  
thats

a scala thing


























					Reply via email to