Hi!
On Apr 7, 2009, at 4:27 AM, Stewart Smith wrote:
On Mon, Apr 06, 2009 at 12:03:15PM -0700, Eric Day wrote:
Not sure if I agree here. I think a pre-parser plugin/module would
be better than sticking this in the client library.
What about post-parser? i.e. trap the parse error and then try
remapping. This way wouldn't affect the 99.99% of real queries.
I wouldn't do it in a pre-parser stage, preprocessors are evil. In
reality you almost always want to stay inside the language and do
rewriting on trees rather than strings.
Just look at how hard it can be to understand C/C++ syntax errors when
they result from complex macro expansions (the problem only is that
cpp is not operating on C syntax trees but on strings).
How about a recognition pipeline?
I can see two different approaches:
The first idea would be to chain the recognizers (= lexer + parser),
where the main SQL recognizer pair is the first one to have a shot at
an incoming query.
Should recognition fail, the entire thing get's passed to the next one
(multiple parsers could share lexers, thus avoiding expensive re-
tokenization, but they don't have to).
In case none of the recognizers can make sense of the input, the first
one would supply the error message, because it's the "main one".
In this approach both efficiency and quality of error messages are
likely to be problems, though. However, it's easier to implement than
the next one.
The second idea would be to have multiple recognizers that recognize
_disjoint_ sets of query syntax. They would form a tree, with the
easiest layout having a tree depth of 1.
Here the main problem is to efficiently pick a parser (not lexer
+parser!). This can be done very efficiently by retrieving the
possible start symbols from all the parsers and then delegating to the
one that knows about the symbol (e.g. a token "SHOW" would cause the
"SHOW" parser to run, the real SQL parser would have "SELECT",
"INSERT", ... as start symbols).
The implementation would simply be a big switch based on the first
token (or even deeper into the token stream, if that's what you want).
It can be implemented by hand as a recursive descent parser with one
token lookahead (LL(1)) pretty easily.
The downside is that you need to have exactly one lexer to yield the
token used in picking the right parser. This lexer doesn't not need to
be the one used by the invoked recognizer, but then it would have to
be generated or manually kept in sync as the superset of all
individual lexers. Doing this at runtime, esp looking at supporting
dynamically loaded recognizers, is not impossible, but might not be
the most efficient nor most trivial code ever written ;)
What's the point of all this?
Well, given this architecture, you could support SHOW commands by
having the SHOW-parser generate a tree that looks exactly like the
corresponding information_schema query. This is completely
transparent for the client and eases maintenance should the I_S
schema ever change: Doing the preprocessing on the client side would
mean that an old client would constantly generate invalid I_S queries.
Having it server side you can keep both the SHOW commands and I_S
schema compatible.
Another upside to the second approach are the improved error messages
you can generate. There's no confusion which recognizer is responsible
for what type of query.
cheers,
-k
P.S.: The concept of recognition delegation could easily extend to
other plugins, too, as long as the token that triggers this delegation
is part of the "main" lexer. Token stream implementation permitting
you could also stack lexers that then feed off of the same character
stream. Ambiguities are the biggest problem, especially when allowing
plugins to "inject" token rules into the main lexer. However, given
it's upsides for modularity and extensibility I think it's worth the
risk of backtracking a lot.
Have a look at:
- http://www.program-transformation.org/Transform/IslandGrammars
- http://www.antlr.org/wiki/display/ANTLR3/Island+Grammars+Under+Parser+Control
--
Kay Roepke
Software Engineer, MySQL Enterprise Tools
Sun Microsystems GmbH Sonnenallee 1, DE-85551 Kirchheim-Heimstetten
Geschaeftsfuehrer: Thomas Schroeder, Wolfang Engels, Dr. Roland Boemer
Vorsitz d. Aufs.rat.: Martin Haering HRB MUC 161028
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
