askalt commented on code in PR #2184:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/2184#discussion_r2735116922
##########
src/tokenizer.rs:
##########
@@ -917,6 +937,23 @@ impl<'a> Tokenizer<'a> {
self
}
+ /// Return an iterator over tokens
+ pub fn iter(&mut self) -> TokenWithLocationIter<'a, '_> {
Review Comment:
1. General name is used because I cannot imagine why an other iterator
method could be needed for tokenizer. So when I call `tokenizer.iter()` I
expect that it will iterate over tokens. So, IMO the API should be named as
simple as possible if it reflects semantic good enough. If someone need raw
Token iterator, it could `.map(|t| t.token)` an existing one.
2. Renamed to `TokenWithSpanIter`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
