Hello everyone! I'm new to Scheme, and I need some help wrapping my head around
it.
I want to write a simple tokenizer, which is just a function that takes a
string as input and outputs a list of "tokens", which to keep thing simple is a
tuple of NAME and VALUE.
So, for example:
(tokenize "foo = 1") => '((IDENTIFIER . "foo") (EQUALS . "=") (NUMBER . 1))
I didn't want to use global variables to keep things functional, so my solution
doesn't use any, but I wouldn't mind much using them. This is some code for my
current implementation, I omit some code for brevity as it's quite repetitive
for now:
(define (match-identifier input)
(regexp-match #rx"^[a-zA-Z_]+"))
(define (consume-identifier input)
(let ([match (first (match-identifier input))])
(cons `(IDENTIFIER . ,match) (tokenize (substring input (string-
length match))))))
;; some more matchers and consumers, all pretty similar
;; ...
(define (tokenize input)
(cond
((match-identifier input) (consume-identifier input))
((match-equals input) (consume-equals input))
((match-number input) (consume-number input))
(else '())))
What bothers me is that it's calling the matchers twice for each token, which
isn't very good for performance and it's not pretty :p
I want to get rid of that double-check. I figured I could use some globals in
order to do it but I'd rather not.
Does anyone have a suggestion/fix on my implementation?
Thank you for your time :)
--
You received this message because you are subscribed to the Google Groups
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
