g4lb opened a new pull request, #3007: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/3007
Dialects: none `Tokenizer::tokenize_word` collected the rest of an identifier into a temporary `String` with `peeking_take_while`, then copied it into the word. Now it appends the characters to the word directly. The tokens do not change. | Benchmark | Before | After | Change | |---|---:|---:|---:| | tokenize TPC-H Q1 (local) | 6.93 µs | 6.00 µs | -13.4% | | `parse_compound_chain/chain_20` | 4.43 µs | 4.20 µs | -5.2% | No other `sqlparser_bench` benchmark changed significantly. I used 10 interleaved rounds of criterion and a Mann-Whitney test (Apple M5, rustc 1.98.0). Tokens, ASTs, `Display` output and errors matched the old code on 1M SQL statements in 14 dialects. The statements came from the tests and fuzz seeds, also with random changes. AI assistance: an AI agent found and measured this change. I read the diff and the results. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
