On Sun, 22 Aug 2010, Graeme Geldenhuys wrote:
Well, does it hurt having them tokenized?
Of course.
It slows the code. Marginally, I agree, but nevertheless.
If anybody other than me comes up with a new idea/way of using the
fcl-passrc parser on tokenizer - maybe they would like to rebuild
source code at some point. Without tokens telling you where EOL used
to be or TAB characters used to be (though the latter is not to
important), it does limit it's functionality.
Attached is a file processed with the profiler and fcl-passrc that
doesn't use tkLineEnding or tkTab. It's rather hard to read and fix
compiler errors introduced by the profile inserting code.
I don't see why introducing such new tokens is bad? It can only make
the tokenizer and parser more useful in the long term.
I never said it is bad; I just don't see why you need it.
You could perfectly do it differently, e.g. insert a newline after certain
tokens:
then
begin
;
end
else
var
type
const
That would perfectly be enough for your purposes, which is, to debug your
profiler.
And once that is done, it is no longer needed at all.
Don't worry, I'll apply the patch, but you have not convinced me it is a
needed patch, even for your purposes. I'm surprised you didn't think of
the above ways to do what you need.
Michael
_______________________________________________
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
