On Sat, 08 Jul 2006, Charles O Nutter defenestrated me: > > On 7/8/06, Thomas E Enebo <[EMAIL PROTECTED]> wrote: > > When I originally changed SourcePosition for RDT and jEdit > folks I had planned on leaving our impl at the single > SourcePosition > per line (that is all Ruby cares about for a runtime). After > changing > SourcePosition I thought of some ways to get the memory usage much > lower > for SourcePosition (I wrote about this some time ago but I believe > the idea > was): > 1. Make Node impl SourcePosition which saves one Object reference > [4 bytes] > 2. Make line number a derived thing which calls back on position > to the LexerSource newline table [2 x 4 bytes]. Line endings > would go from O(1) to O(ln n) in int accesses. > 3. Consider splitting start and end line into bit operations and > storing them into a single int [4 bytes]. This one is most > dubious to me. This obviously limits precision to less than int > (Ruby uses int in C so this potentially would make JRuby have > less > precision -- what is the largest byte offset you have seen in > Ruby code?) > > I have a patch for this third one I can toss out (it's at home). I > wouldn't be concerned about line having a problem (>65k lines??) but > I'm a little sketch about offset (since I'm pretty sure I've seen some > .rb files >65k bytes). > The Node < Position idea would be a good one, especially if we can > reduce the size of positioning info. Also, I see that name is stored > in SourcePosition for the name of the file. I assume this is a single > object and not a copy for each, yes? > You'll have to explain #2 to me a bit more. > All this will have bearing on compilation as well (how to handle stack > traces in compiled code) and on pre-parsing (pre-parsing more of the > libraries may eat up more memory up front).
Yeah the filename can be retrieved from the same source as #2 above so that removes a reference as well. When we parse a file we actually maintain a list of offsets where newlines occur. A divide and conquer search can easily give us the line for start or end. > yyparse is pretty transient in the memory department no? Once we > are > done parsing that memory gets reclaimed. > > The profiling I did referred to retained memory; I can look at the > profiling information more to see for sure. Yeah, unless it is memory to store all the production states, then I think all memory should get cleared for the parser (unless we errantly hold a reference to someting). -- + http://www.tc.umn.edu/~enebo +---- mailto:[EMAIL PROTECTED] ----+ | Thomas E Enebo, Protagonist | "Luck favors the prepared | | | mind." -Louis Pasteur | ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jruby-devel mailing list Jruby-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jruby-devel