andygrove commented on issue #2893:
URL:
https://github.com/apache/datafusion-sqlparser-rs/issues/2893#issuecomment-6742865702
@edmondop Yes, v2 solves #2612. Dialects get lexer hooks that run before the
core lexer, and each one receives the rest of the SQL as a `&str`, so lookahead
is ordinary string handling. Here is the Jinja comment case from that issue,
plus a new `//` operator:
```rust
#[derive(Debug, Clone, PartialEq)]
struct Dbt;
impl Dialect for Dbt {
type Ext = Builtin;
// Skip Jinja's `{# ... #}` and `#` line comments.
fn lex_comment(rest: &str) -> Option<Result<usize, &'static str>> {
if rest.starts_with("{#") {
return Some(match rest.find("#}") {
Some(end) => Ok(end + 2),
None => Err("unterminated Jinja comment"),
});
}
if rest.starts_with('#') {
return Some(Ok(lexer::line_comment(rest)));
}
lexer::lex_core_comment(rest)
}
// Read `//` as one token. Returning `None` falls back to the core lexer.
fn lex_token(rest: &str) -> Option<LexResult> {
rest.starts_with("//").then_some(Ok(Lexeme {
kind: TokenKind::Op,
len: 2,
}))
}
// Give the new token a meaning: integer division, binding like `*`.
fn infix_rule<'a>(p: &Parser<'a, Self>) -> Option<InfixRule<'a, Self>> {
if p.at_op("//") {
return Some(InfixRule::binary(prec::MUL, BinaryOp::Named("//")));
}
grammar::infix_rule(p)
}
}
let sql = "SELECT a // 2 {# half of a #} FROM t # trailing note";
let statements = parse_sql::<Dbt>(sql).unwrap();
assert_eq!(statements[0].to_string(), "SELECT a // 2 FROM t");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
