iffyio commented on code in PR #2352:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/2352#discussion_r3317306048
##########
src/parser/mod.rs:
##########
@@ -1717,6 +1731,23 @@ impl<'a> Parser<'a> {
return prefix;
}
+ // Memoize parse_prefix failures to break 2^N speculation when both
+ // prefix arms fail at every level (e.g. `IF(current_time(...x`).
+ // The per-arm cache in `parse_prefix_inner` complements this for
+ // chains where the reserved arm fails but the unreserved fallback
+ // succeeds (e.g. `case-case-...c`).
+ let start_index = self.index;
+ if let Some(cached) = self.failed_prefix_positions.get(&start_index) {
+ return Err(cached.clone());
+ }
+ let result = self.parse_prefix_inner();
+ if let Err(ref e) = result {
+ self.failed_prefix_positions.insert(start_index, e.clone());
Review Comment:
> valid SQL costs nothing
My understanding is that e.g. on the `parse_expr_prefix_by_reserved_word`
path, each unique attempt will add an error string into the map, if so then
we're potentially looking at one entry in that map per word roughly. is that
the the case?
To be clear, its not about a particular amount of memory, main thing is that
we're not increasing memory usage of the parser significantly - if the
additional memory usage grows as a function of the sql string, already that is
problematic, then to improve it we would like to have each entry as minimal as
we can, or potentially consider other solutions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
