Hi all, Over the weekend, I started from a clean slate and implemented a new version of the sqlparser-rs crate using the design that I had originally wanted but did not know how to build (I was still quite new to Rust at the time).
The new version has one trait per dialect and all dialect-specific parsing lives in that trait. There is no longer one huge "core" parser containing conditional logic for different dialects. Related dialects can also share logic. For example, the Hive and Databricks dialects both call into the Spark dialect for common pieces. The new version also has ~10x fewer memory allocations (no more copying Strings everywhere) and is ~10x faster. I am nearly finished porting all of the existing dialects over. The actual parsing code is taken from the current implementation, so that is not re-written. I have ported over all the tests as well, and am working towards 100% parity with v1, although v2 contains quite a lot of correctness fixes compared to v1. What is the appetite in the community for me pushing this to a new 2.0 branch in the Apache repo? Another option would be for me to file a huge issue explaining the design in detail, and let the community rebuild it. WDYT? I filed an issue for this discssion as well, so that is probably the best place to continue the discussion. https://github.com/apache/datafusion-sqlparser-rs/issues/2893 Thanks, Andy. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
