I implemented a patch which allows a bytelist to be read directly by a lexersource. This ends up with the following (plusses and minuses):
+ We do not convert the bytelist to a String in order to parse during evals + We do not save a copy of that converted String in ByteList (bytelist caching mechanism) + I discovered an issue with our caching logic which is a little scary (e.g. a bug) - Evals are the same speed or a little slower because of the abstraction of lexing One thing which sucks about the patch is that the ByteListLexerSource could not just be a bytelist and an index. It needs to defer back to a bunch of special logic in the base class. To make a long explanation short, the lexer sometimes pushes back data to the source. What it pushes back isn't guaranteed to be identical to original source (think \r\n and \n). With quite a bit more work I could maybe work around this, but the newline handling stuff sucks. *BING* *BING* *READ THIS* It would be useful if someone could test this patch on a Rails application and see if the application consumes less memory. I could see that as a good reason for continuing to improve this patch. -Tom PS - Patch against trunk -- Blog: http://www.bloglines.com/blog/ThomasEEnebo Email: [EMAIL PROTECTED] , [EMAIL PROTECTED]
bytelistlexersource.patch
Description: Binary data
---------------------------------------------------------------------
To unsubscribe from this list please visit:
http://xircles.codehaus.org/manage_email
