andygrove commented on issue #2893: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/2893#issuecomment-6747161248
> [@andygrove](https://github.com/andygrove) is the AST representation changed too, or is it just the parser? AST has changed for multiple reasons, but a key one is using slices instead of `String`. > In general I think it would be cool to see a massive rewrite of sqlparser-rs using AI, and i think parsing crates like this is uniquely amenable to it, especially as we have a significant corpus of tests and can just turn the tools loose I agree that this is an ideal use of AI. We have all the existing tests, so we know when we are done. > In my mind the key gates are: > > 1. Support all sql that is currently parsed (as proven by an integration into DataFusion and other systems) Agreed. > 3. Support limits like stack overflow avoidance, etc.> This is already improved. > > Another critical correctness issue in the current crate, before speed or allocations are concerned, is the spans. At this time the vast majority of them is plainly wrong, and will need to be addressed before refactorings for the same reason I suggest to first extend the AST & fix the fuzzer bugs. > > Yes, I agree the spans are just wrong and we should fix that (probably by including a span with each new AST node. I will look at this as well. Have not started on this part yet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
