We looked into antlr. It appears to be very similar to javacc, with the added feature that the java code it generates is humanly readable. That isn't why we want to switch off of javacc. Olga listed the 3 things we want out of a parser that javacc isn't giving us (lack of docs, no easy customization of error handle, decoupling of scanning and parsing). So antlr doesn't look viable.

In response to Pi's suggestion that we could use the logical plan, I hope we could use something close to it. Whatever we choose we want it to be flexible enough to represent richer language constructs (like branch and loop). I'm not sure our current logical plan can do that. At the same time, we don't need another layer of translation (we already have logical -> physical -> mapreduce). I would like to find a representation that could handle expressing the syntax and what is currently the logical plan.


On Feb 20, 2009, at 5:15 PM, pi song wrote:

Should be pretty close but we may need to cleanup the interface a bit. Then
the new parser  module can be switched in easily.
BTW, have we already got the solution for the new parser generator?


On Fri, Feb 20, 2009 at 9:03 PM, Ted Dunning <ted.dunn...@gmail.com> wrote:

Probably nearly the same effect as you suggest. Are the concepts at the
logical plan layer similar to those expressed in pig latin?  Or has a
significant transformation occurred by then?

On Fri, Feb 20, 2009 at 1:59 AM, pi song <pi.so...@gmail.com> wrote:

Sounds good but how about exposing the logical plan layer instead?
that yield the same effect?  From python for example you still can
a logical plan and give to Pig to execute.

Ted Dunning, CTO

Reply via email to