Hey guys, good to hear from you.
Don, I am not in favor of long lived objects, at all :)
But generating a lot of short-lived objects makes garbage collector
work extra hard and ruby GC compared to java GC is ... well, weak. I
used Java for good 8 years before switching to ruby, and I had
opportunity to get into heavy optimizations... Java GC was a bit
better then Ruby's even in earlier days, and it got way better over
time.
Also, I completely agree with you that columns with time should be
loaded only when they are going to be absolutely used. Timezone
conversion problem is not unique to Sequel, I noticed it with AR
before as well, I guess that comes with territory (MRI ruby, that is).
Jeremy, yes, I have not provided any data for you, you are right. I
can not give you the code since this is a non-open-source app, but if
you are really curious, I can privately send you ruby-prof file to
have a look at (with SQL queries and short chunks of code to accompany
it). I am using postgres 8.4 ('pg' driver), tested described behavior
on both Mac and Ubuntu. Initially some of my queries were returning
the 'unused' columns that you referred to, not anymore, now I load
only required columns for each and every query used it that (test)
action, and that certainly helps. The funny thing is, I noticed the
problem when I replaced 4 AR queries with 4 queries in Sequel. All 4
of them were returning (short) naked list of ids either before or
after the change. That is when I noticed that while 'best case
scenario' speed improved (prepared statements help), but GC times
suffered badly enough to more than make up for that.
Now that I have dropped each and every unused field from every query,
GC times dropped, yes, but in more or less apple-to-apple comparison
with AR, total time used by GC during 500 requests went up from 9s to
12s when I switched. It is 7s now, but at a cost of having to
carefully craft every query to load only required parameters. Using
better (patched and tuned) GC drops this GC time down to 2-3s, but for
variety of reasons I can not put patched ruby on my servers yet (will
surely do in the future).
If you consider that 500 requests, each about 20ms, should take 10s to
complete, 9-12s of GC time overhead is very sad, app is spending as
much time collecting garbage as doing useful work (that means that in
reality it takes 20s to serve those 500 requests). Yikes!
Sorry for the very unscientific nature of my post and 'data', but I am
reporting those finding 'hot off the press' and they were done in a
very non-academic environment by doing a practical exercise of
converting existing code :)
--
You received this message because you are subscribed to the Google Groups
"sequel-talk" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sequel-talk?hl=en.
