On 11/16/2014 5:43 PM, "Ola Fosheim Grøstad"
<ola.fosheim.grostad+dl...@gmail.com>" wrote:
On Monday, 17 November 2014 at 01:39:38 UTC, Walter Bright wrote:
Notice the total lack of strlen()'s in Warp.
Why would you need that? You know where the lexeme begins and ends? If we are
talking about old architectures you have to acknowledge that storage was premium
and that the major cost was getting the strings into memory in the first place.
The preprocessor stores lots of strings. Things like identifiers, keywords,
string literals, expanded macro text, etc.
The C preprocessor I wrote in C years ago is filled with strlen(), as is about
every C string processing program ever written. Heck, how do you think strcat()
works?
(Another problem with strlen() is that the string pointed to is in a different
piece of memory, and it'll have to be loaded into the cache to scan for the 0.
Whereas with slices, the length data is in the hot cache.)
Nah, if you know that the file ends with zero then you can build an efficient
finite automata as a classifier.
deadalnix busted that myth a while back with benchmarks.
I haven't seen it,
It's in the n.g. archives somewhere in a thread about implementing lexers.
but it is difficult to avoid lexers being bandwidth limited
these days.
Besides, how do you actually implement a lexer without constructing a FA one way
or the other?
That's the wrong question. The question is does a trailing sentinel result in a
faster FA? deadalnix demonstrated that the answer is 'no'.
You know, Ola, I've been in the trenches with this problem for decades.
Sometimes I still learn something new, as I did with deadalnix's benchmark. But
the stuff you are positing is well-trodden ground. There's a damn good reason
why D uses slices and not 0 terminated strings.
