[chromium-dev] Re: Buildbot performance issue.

2009-07-14 Thread Evan Martin
On Tue, Jul 14, 2009 at 2:18 PM, Nicolas Sylvainnsylv...@chromium.org wrote:   The underlying problem with buildbot is the database format, which is just hundred of thousand of files on the harddrive, with no seek capability, and the fact that the webserver itself is single threaded.   We

[chromium-dev] Re: Buildbot performance issue.

2009-07-14 Thread Ojan Vafai
On Tue, Jul 14, 2009 at 2:18 PM, Nicolas Sylvain nsylv...@chromium.orgwrote: Q3: What kind of auto-refresh do we need? We used to be at 60 secs for a long time, and I changed it a couple of weeks ago to 90 secs. No one complained, so I guess this is good. Should we go even higher than

[chromium-dev] Re: Buildbot performance issue.

2009-07-14 Thread Paweł Hajdan Jr .
I was just thinking... doesn't it have a reverse proxy in front of it? It could also force content cache time of 60s or even more... Something like Squid or Varnish. Oh, and today morning was probably me scraping the logs. :-( Sorry. On Tue, Jul 14, 2009 at 14:18, Nicolas

[chromium-dev] Re: Buildbot performance issue.

2009-07-14 Thread Scott Hess
If I understand right, simply serving the auto-refresh requests is substantial? At least for the main page, a reverse in accelerator mode could turn that into a constant load. [I'd offer to help, but I don't know what kind of technology we're talking about, here.] -scott On Tue, Jul 14, 2009

[chromium-dev] Re: Buildbot performance issue.

2009-07-14 Thread Nicolas Sylvain
On Tue, Jul 14, 2009 at 2:25 PM, Evan Martin e...@chromium.org wrote: On Tue, Jul 14, 2009 at 2:18 PM, Nicolas Sylvainnsylv...@chromium.org wrote: The underlying problem with buildbot is the database format, which is just hundred of thousand of files on the harddrive, with no seek

[chromium-dev] Re: Buildbot performance issue.

2009-07-14 Thread 王重傑
On Tue, Jul 14, 2009 at 2:35 PM, Nicolas Sylvain nsylv...@chromium.orgwrote: On Tue, Jul 14, 2009 at 2:25 PM, Evan Martin e...@chromium.org wrote: On Tue, Jul 14, 2009 at 2:18 PM, Nicolas Sylvainnsylv...@chromium.org wrote: The underlying problem with buildbot is the database format,

[chromium-dev] Re: Buildbot performance issue.

2009-07-14 Thread Ryosuke Niwa
On Tue, Jul 14, 2009 at 2:35 PM, Nicolas Sylvainnsylv...@chromium.org wrote: - Get a better machine. It's already running on a dedicated dual quad core nehalem server   with 24gb of RAM and 15k rpm drives. This is absurdly powerful!  It should have all the data necessary to generate

[chromium-dev] Re: Buildbot performance issue.

2009-07-14 Thread Adam Langley
On Tue, Jul 14, 2009 at 2:40 PM, Albert J. Wong (王重傑)ajw...@chromium.org wrote: That is pretty nuts.  Is it calling fsync or something crazy?  Since you said strace, I'm assmuming linux. In that case, the buffer cache should be saving you from disk accesses for most everything. Of course,

[chromium-dev] Re: Buildbot performance issue.

2009-07-14 Thread Ojan Vafai
On Tue, Jul 14, 2009 at 2:35 PM, Nicolas Sylvain nsylv...@chromium.orgwrote: On Tue, Jul 14, 2009 at 2:25 PM, Evan Martin e...@chromium.org wrote: On Tue, Jul 14, 2009 at 2:18 PM, Nicolas Sylvainnsylv...@chromium.org wrote: It seems to me a caching layer that only ever hit the backend every

[chromium-dev] Re: Buildbot performance issue.

2009-07-14 Thread Jeremy Orlow
On Tue, Jul 14, 2009 at 2:42 PM, Adam Langley a...@chromium.org wrote: On Tue, Jul 14, 2009 at 2:40 PM, Albert J. Wong (王重傑)ajw...@chromium.org wrote: That is pretty nuts. Is it calling fsync or something crazy? Since you said strace, I'm assmuming linux. In that case, the buffer cache

[chromium-dev] Re: Buildbot performance issue.

2009-07-14 Thread Mohamed Mansour
I would like to see auto refresh turned off by default. That might help the load. -- Mohamed Mansour On Tue, Jul 14, 2009 at 7:15 PM, Michael Nordman micha...@google.comwrote: Turning off auto refresh by default sounds reasonable idea right now... with an option to enable it if really