Re: ReviewBoard example instance

[Stephen Gallagher]

On 07/19/2010 03:12 PM, Christian Hammond wrote:
> Can you verify that the settings_local.py in Review Board and the
> settings.py in other Django sites are using the exact same database
> backend?
>
> Are all the sites running on the same server?
>
> The logs you provided show a lag of ~2 seconds rather than 8, though.
> If you're seeing 8 from your end, but the logs are showing 2, then
> there's a whole different issue happening, but I'm assuming it just
> varies on attempt and page.

Well, I think I may be seeing 2s of lag for each component being
displayed on the dashboard, which would account for the 8 seconds total.

I just confirmed that both Transifex and ReviewBoard are both using
postgresql-psycopg2 to talk to the database.

Transifex and ReviewBoard themselves are both running on separate
servers. Transifex has a dedicated machine at the moment, while
ReviewBoard is shared with a number of Trac instances (but the load is
low and there's plenty of free memory).


-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Christian Hammond]

There is one query in the list that happens twice and takes 0.352s on
average each time. That's the query to get the auth info from the Fedora
auth backend. I don't know why it'd be so slow, but it might also point to
an issue in talking to the database.

Christian

-- 
Christian Hammond - chip...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com


On Mon, Jul 19, 2010 at 12:12 PM, Christian Hammond wrote:

> The SQL query time isn't that slow, but something definitely is. Especially
> if it's happening on every page. The function that's really taking all that
> time is precompute_objects. It does the following:
>
> 1) Builds a list of fields to sort in order, which will be used in the SQL
> query (this is fairly simple and shouldn't be slow)
>
> 2) Builds a paginator, which is just a simple object used to increment the
> offset in the query. Also simple, part of Django, and used in lots of
> installs.
>
> 3) Fetches the list of IDs for the objects that will be shown in the
> dashboard.
>
> 4) Takes those IDs and uses them to get the actual objects in one batch.
>
> 5) Renders them (the logs show this to not be taking a lot of time).
>
> So basically, some simple logic to split a string and stuff it into a
> dictionary (which will at most have 2 items, and usually 0 or 1), perform
> the query (which the logs say are actually fairly quick, once the database
> connection is established), and then render it (which the logs say are also
> quick). So I can't imagine it being anything other than the PostgreSQL
> configuration.
>
> Can you verify that the settings_local.py in Review Board and the
> settings.py in other Django sites are using the exact same database backend?
>
> Are all the sites running on the same server?
>
> The logs you provided show a lag of ~2 seconds rather than 8, though. If
> you're seeing 8 from your end, but the logs are showing 2, then there's a
> whole different issue happening, but I'm assuming it just varies on attempt
> and page.
>
>
> Christian
>
> --
> Christian Hammond - chip...@chipx86.com
> Review Board - http://www.reviewboard.org
> VMware, Inc. - http://www.vmware.com
>
>
> On Mon, Jul 19, 2010 at 12:04 PM, Stephen Gallagher <
> step...@gallagherhome.com> wrote:
>
>> On 07/19/2010 03:00 PM, Christian Hammond wrote:
>> > Hi Stephen,
>> >
>> > Sorry, I missed the e-mail where you attached the profile log. I'm
>> > looking through it now.
>> >
>> > The render_to_response was misleading. The slowdown is actually within
>> > precompute_objects, which does some database queries. That primarily
>> > does database queries, but the SQL logs show that they're not taking
>> > very long individually. Certainly not a total of 2 seconds. The
>> > rendering doesn't appear to be the slow part either.
>> >
>> > I don't know much about PostgreSQL, but it almost sounds like there's
>> > some issue in the configuration or something. If there's a delay in
>> > talking to the database (locking? bad connection?) then I could see
>> > that causing this. The low SQL query time is interesting. That makes
>> > me wonder if it's just an issue in establishing the first connection.
>>
>> Well, the problem is that I'm seeing a lag of about 8 seconds for every
>> page load. I can't imagine the SQL query time being that slow...
>> Furthermore, there are other Django-based web applications talking to
>> this same database server without any obvious performance issues.
>> (Specifically, it's a Transifex instance, so it's low-traffic. That
>> wouldn't be wasting DB time).
>>
>> --
>> Want to help the Review Board project? Donate today at
>> http://www.reviewboard.org/donate/
>> Happy user? Let us know at http://www.reviewboard.org/users/
>> -~--~~~~--~~--~--~---
>> To unsubscribe from this group, send email to
>> reviewboard+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/reviewboard?hl=en
>>
>
>

-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Christian Hammond]

The SQL query time isn't that slow, but something definitely is. Especially
if it's happening on every page. The function that's really taking all that
time is precompute_objects. It does the following:

1) Builds a list of fields to sort in order, which will be used in the SQL
query (this is fairly simple and shouldn't be slow)

2) Builds a paginator, which is just a simple object used to increment the
offset in the query. Also simple, part of Django, and used in lots of
installs.

3) Fetches the list of IDs for the objects that will be shown in the
dashboard.

4) Takes those IDs and uses them to get the actual objects in one batch.

5) Renders them (the logs show this to not be taking a lot of time).

So basically, some simple logic to split a string and stuff it into a
dictionary (which will at most have 2 items, and usually 0 or 1), perform
the query (which the logs say are actually fairly quick, once the database
connection is established), and then render it (which the logs say are also
quick). So I can't imagine it being anything other than the PostgreSQL
configuration.

Can you verify that the settings_local.py in Review Board and the
settings.py in other Django sites are using the exact same database backend?

Are all the sites running on the same server?

The logs you provided show a lag of ~2 seconds rather than 8, though. If
you're seeing 8 from your end, but the logs are showing 2, then there's a
whole different issue happening, but I'm assuming it just varies on attempt
and page.

Christian

-- 
Christian Hammond - chip...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com


On Mon, Jul 19, 2010 at 12:04 PM, Stephen Gallagher <
step...@gallagherhome.com> wrote:

> On 07/19/2010 03:00 PM, Christian Hammond wrote:
> > Hi Stephen,
> >
> > Sorry, I missed the e-mail where you attached the profile log. I'm
> > looking through it now.
> >
> > The render_to_response was misleading. The slowdown is actually within
> > precompute_objects, which does some database queries. That primarily
> > does database queries, but the SQL logs show that they're not taking
> > very long individually. Certainly not a total of 2 seconds. The
> > rendering doesn't appear to be the slow part either.
> >
> > I don't know much about PostgreSQL, but it almost sounds like there's
> > some issue in the configuration or something. If there's a delay in
> > talking to the database (locking? bad connection?) then I could see
> > that causing this. The low SQL query time is interesting. That makes
> > me wonder if it's just an issue in establishing the first connection.
>
> Well, the problem is that I'm seeing a lag of about 8 seconds for every
> page load. I can't imagine the SQL query time being that slow...
> Furthermore, there are other Django-based web applications talking to
> this same database server without any obvious performance issues.
> (Specifically, it's a Transifex instance, so it's low-traffic. That
> wouldn't be wasting DB time).
>
> --
> Want to help the Review Board project? Donate today at
> http://www.reviewboard.org/donate/
> Happy user? Let us know at http://www.reviewboard.org/users/
> -~--~~~~--~~--~--~---
> To unsubscribe from this group, send email to
> reviewboard+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/reviewboard?hl=en
>

-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Stephen Gallagher]

On 07/19/2010 03:00 PM, Christian Hammond wrote:
> Hi Stephen,
>
> Sorry, I missed the e-mail where you attached the profile log. I'm
> looking through it now.
>
> The render_to_response was misleading. The slowdown is actually within
> precompute_objects, which does some database queries. That primarily
> does database queries, but the SQL logs show that they're not taking
> very long individually. Certainly not a total of 2 seconds. The
> rendering doesn't appear to be the slow part either.
>
> I don't know much about PostgreSQL, but it almost sounds like there's
> some issue in the configuration or something. If there's a delay in
> talking to the database (locking? bad connection?) then I could see
> that causing this. The low SQL query time is interesting. That makes
> me wonder if it's just an issue in establishing the first connection.

Well, the problem is that I'm seeing a lag of about 8 seconds for every
page load. I can't imagine the SQL query time being that slow...
Furthermore, there are other Django-based web applications talking to
this same database server without any obvious performance issues.
(Specifically, it's a Transifex instance, so it's low-traffic. That
wouldn't be wasting DB time).

-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Christian Hammond]

Hi Stephen,

Sorry, I missed the e-mail where you attached the profile log. I'm looking
through it now.

The render_to_response was misleading. The slowdown is actually within
precompute_objects, which does some database queries. That primarily does
database queries, but the SQL logs show that they're not taking very long
individually. Certainly not a total of 2 seconds. The rendering doesn't
appear to be the slow part either.

I don't know much about PostgreSQL, but it almost sounds like there's some
issue in the configuration or something. If there's a delay in talking to
the database (locking? bad connection?) then I could see that causing this.
The low SQL query time is interesting. That makes me wonder if it's just an
issue in establishing the first connection.

Christian

-- 
Christian Hammond - chip...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com


On Mon, Jul 19, 2010 at 5:08 AM, Stephen Gallagher <
step...@gallagherhome.com> wrote:

> On 07/08/2010 07:17 AM, Stephen Gallagher wrote:
> > On 07/07/2010 06:04 PM, Christian Hammond wrote:
> >
> >> Do you happen to have the raw profiling log file too? I want to see
> >> the SQL queries as well.
> >>
> >> There were some heavy optimizations made in Djblets for the datagrid
> >> rendering, but I'm not sure if all that made it into beta 2 or not.
> >>
> >> Christian
> >>
> > Sure, attached is the matching .prof file (gzipped since it was rather
> > large uncompressed)
> >
> *crickets*
>
> Christian, have you had a chance to look at this? I'm really unsure how
> to track this issue down.
>
> --
> Want to help the Review Board project? Donate today at
> http://www.reviewboard.org/donate/
> Happy user? Let us know at http://www.reviewboard.org/users/
> -~--~~~~--~~--~--~---
> To unsubscribe from this group, send email to
> reviewboard+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/reviewboard?hl=en
>

-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Stephen Gallagher]

On 07/08/2010 07:17 AM, Stephen Gallagher wrote:
> On 07/07/2010 06:04 PM, Christian Hammond wrote:
>   
>> Do you happen to have the raw profiling log file too? I want to see
>> the SQL queries as well.
>>
>> There were some heavy optimizations made in Djblets for the datagrid
>> rendering, but I'm not sure if all that made it into beta 2 or not.
>>
>> Christian
>> 
> Sure, attached is the matching .prof file (gzipped since it was rather
> large uncompressed)
>   
*crickets*

Christian, have you had a chance to look at this? I'm really unsure how
to track this issue down.

-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Christian Hammond]

Do you happen to have the raw profiling log file too? I want to see the SQL
queries as well.

There were some heavy optimizations made in Djblets for the datagrid
rendering, but I'm not sure if all that made it into beta 2 or not.

Christian

-- 
Christian Hammond - chip...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com


On Wed, Jul 7, 2010 at 2:45 PM, Stephen Gallagher  wrote:

> On 06/12/2010 03:39 AM, Christian Hammond wrote:
> > Hmm, it might be due to not having cProfile installed? Does the
> > reviewboard.log show anything?
> >
> > Christian
> >
> > --
> > Christian Hammond - chip...@chipx86.com 
> > Review Board - http://www.reviewboard.org
> > VMware, Inc. - http://www.vmware.com
> >
> >
> > On Fri, Jun 11, 2010 at 1:30 PM, Stephen Gallagher
> > mailto:step...@gallagherhome.com>> wrote:
> >
> > On 06/10/2010 03:24 PM, Christian Hammond wrote:
> >
> > Hi Stephen,
> >
> > We're running on MySQL. I don't know why you'd be seeing such
> > a huge delay with PostgreSQL though. Can you tell me which
> > pages are showing delays? (Is it all of them?)
> >
> > if it's the dashboard, there are some known slowdowns when
> > reaching large numbers of review requests. If the diff viewer,
> > make sure your memcached is caching aggressively.
> >
> > One thing you could try doing is going into Settings ->
> > Logging and turn on Profiling. Then you can visit a page with
> > slowdowns and run with ?profiling=1. This should output to a
> > reviewboard.prof file in the logging directory with some
> > details on the SQL queries and the code paths hit. If you
> > could pass that on to me, I could take a look.
> >
> >
> > Still working on this, but I thought I should mention. I get a 500
> > error on every page that I try to use ?profiling=1 on. Is this a
> > known bug on 1.5 beta 2?
> >
>
>
> Okay, it turned out that I needed to hack python-djblets to use profile
> instead of cProfile (since cProfile requires python 2.5 and RHEL5 only
> has 2.4)
>
> I got the following profile results (graphed for easier consumption):
> http://sgallagh.fedorapeople.org/rb-profile.svg
>
> If I'm reading it right, it looks like the majority of the time is being
> lost in grids:696:render_to_response
>
> * That SVG should be viewable in Firefox, but if not I can generate a
> PNG as well - it's just much larger.
>
> --
> Want to help the Review Board project? Donate today at
> http://www.reviewboard.org/donate/
> Happy user? Let us know at http://www.reviewboard.org/users/
> -~--~~~~--~~--~--~---
> To unsubscribe from this group, send email to
> reviewboard+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/reviewboard?hl=en
>

-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Stephen Gallagher]

On 06/12/2010 03:39 AM, Christian Hammond wrote:
> Hmm, it might be due to not having cProfile installed? Does the
> reviewboard.log show anything?
>
> Christian
>
> -- 
> Christian Hammond - chip...@chipx86.com 
> Review Board - http://www.reviewboard.org
> VMware, Inc. - http://www.vmware.com
>
>
> On Fri, Jun 11, 2010 at 1:30 PM, Stephen Gallagher
> mailto:step...@gallagherhome.com>> wrote:
>
> On 06/10/2010 03:24 PM, Christian Hammond wrote:
>
> Hi Stephen,
>
> We're running on MySQL. I don't know why you'd be seeing such
> a huge delay with PostgreSQL though. Can you tell me which
> pages are showing delays? (Is it all of them?)
>
> if it's the dashboard, there are some known slowdowns when
> reaching large numbers of review requests. If the diff viewer,
> make sure your memcached is caching aggressively.
>
> One thing you could try doing is going into Settings ->
> Logging and turn on Profiling. Then you can visit a page with
> slowdowns and run with ?profiling=1. This should output to a
> reviewboard.prof file in the logging directory with some
> details on the SQL queries and the code paths hit. If you
> could pass that on to me, I could take a look.
>
>
> Still working on this, but I thought I should mention. I get a 500
> error on every page that I try to use ?profiling=1 on. Is this a
> known bug on 1.5 beta 2?
>


Okay, it turned out that I needed to hack python-djblets to use profile
instead of cProfile (since cProfile requires python 2.5 and RHEL5 only
has 2.4)

I got the following profile results (graphed for easier consumption):
http://sgallagh.fedorapeople.org/rb-profile.svg

If I'm reading it right, it looks like the majority of the time is being
lost in grids:696:render_to_response

* That SVG should be viewable in Firefox, but if not I can generate a
PNG as well - it's just much larger.

-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Christian Hammond]

Hmm, it might be due to not having cProfile installed? Does the
reviewboard.log show anything?

Christian

-- 
Christian Hammond - chip...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com


On Fri, Jun 11, 2010 at 1:30 PM, Stephen Gallagher <
step...@gallagherhome.com> wrote:

> On 06/10/2010 03:24 PM, Christian Hammond wrote:
>
>> Hi Stephen,
>>
>> We're running on MySQL. I don't know why you'd be seeing such a huge delay
>> with PostgreSQL though. Can you tell me which pages are showing delays? (Is
>> it all of them?)
>>
>> if it's the dashboard, there are some known slowdowns when reaching large
>> numbers of review requests. If the diff viewer, make sure your memcached is
>> caching aggressively.
>>
>> One thing you could try doing is going into Settings -> Logging and turn
>> on Profiling. Then you can visit a page with slowdowns and run with
>> ?profiling=1. This should output to a reviewboard.prof file in the logging
>> directory with some details on the SQL queries and the code paths hit. If
>> you could pass that on to me, I could take a look.
>>
>
> Still working on this, but I thought I should mention. I get a 500 error on
> every page that I try to use ?profiling=1 on. Is this a known bug on 1.5
> beta 2?
>
>
> --
> Want to help the Review Board project? Donate today at
> http://www.reviewboard.org/donate/
> Happy user? Let us know at http://www.reviewboard.org/users/
> -~--~~~~--~~--~--~---
> To unsubscribe from this group, send email to
> reviewboard+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/reviewboard?hl=en
>

-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Stephen Gallagher]

On 06/10/2010 03:24 PM, Christian Hammond wrote:

Hi Stephen,

We're running on MySQL. I don't know why you'd be seeing such a huge 
delay with PostgreSQL though. Can you tell me which pages are showing 
delays? (Is it all of them?)


if it's the dashboard, there are some known slowdowns when reaching 
large numbers of review requests. If the diff viewer, make sure your 
memcached is caching aggressively.


One thing you could try doing is going into Settings -> Logging and 
turn on Profiling. Then you can visit a page with slowdowns and run 
with ?profiling=1. This should output to a reviewboard.prof file in 
the logging directory with some details on the SQL queries and the 
code paths hit. If you could pass that on to me, I could take a look.


Still working on this, but I thought I should mention. I get a 500 error 
on every page that I try to use ?profiling=1 on. Is this a known bug on 
1.5 beta 2?


--
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[David Trowbridge]

The reviews.reviewboard.org server is running under mysql. We've done  
a little tuning but not much. I haven't tried with pgsql at all,  
personally.


-David

On Jun 10, 2010, at 11:56 AM, Stephen Gallagher > wrote:


Well, right now 4-8s for a page refresh is just barely within the  
tolerable limits of my users. I'm trying to determine whether it  
makes more sense to migrate to MySQL (if it would be better  
performance)


I was also curious whether ReviewBoard's reference implementation  
was running on postgresql and whether there were any tuning tricks I  
should be aware of.


On 06/10/2010 02:53 PM, Jan Koprowski wrote:

It is probably between SQLite and PostgreSQL. But this is delay is
proportional to scalable. SQLite is not scalable at all, PostgreSQL  
is

most scalable free SQL database.

On Jun 9, 9:39 pm, Stephen Gallagher
wrote:

I'm curious, what database is thehttp:// 
reviews.reviewboard.orgserver
using? We deployed on PostgreSQL recently, and I've noticed a  
distinct

performance issue with it.

I can only directly compare it to SQLite, but of course that's local
access. But we went from<  1s responses on webpages to 4-8s per page
when backed by Postgresql (on a private network with very little  
traffic).


How is the performance when backed by MySQL?





--
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en


--
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Jan Koprowski]

You can check Your efficient of memcached caching. This could be
critical. Read about getting statistic from memcached:
http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt
this is the fastest element of ReviewBoard.
If You still belive this is database engine bottle neck here is good
article witch talk about Postgres tuning also:
http://jacobian.org/writing/django-performance-tips/


On Jun 10, 8:56 pm, Stephen Gallagher 
wrote:
> Well, right now 4-8s for a page refresh is just barely within the
> tolerable limits of my users. I'm trying to determine whether it makes
> more sense to migrate to MySQL (if it would be better performance)
>
> I was also curious whether ReviewBoard's reference implementation was
> running on postgresql and whether there were any tuning tricks I should
> be aware of.
>
> On 06/10/2010 02:53 PM, Jan Koprowski wrote:
>
>
>
> > It is probably between SQLite and PostgreSQL. But this is delay is
> > proportional to scalable. SQLite is not scalable at all, PostgreSQL is
> > most scalable free SQL database.
>
> > On Jun 9, 9:39 pm, Stephen Gallagher
> > wrote:
>
> >> I'm curious, what database is thehttp://reviews.reviewboard.orgserver
> >> using? We deployed on PostgreSQL recently, and I've noticed a distinct
> >> performance issue with it.
>
> >> I can only directly compare it to SQLite, but of course that's local
> >> access. But we went from<  1s responses on webpages to 4-8s per page
> >> when backed by Postgresql (on a private network with very little traffic).
>
> >> How is the performance when backed by MySQL?

-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Stephen Gallagher]

On 06/10/2010 03:24 PM, Christian Hammond wrote:

Hi Stephen,

We're running on MySQL. I don't know why you'd be seeing such a huge 
delay with PostgreSQL though. Can you tell me which pages are showing 
delays? (Is it all of them?)


Yeah, it's every page but the login page. That includes the admin UI.



if it's the dashboard, there are some known slowdowns when reaching 
large numbers of review requests. If the diff viewer, make sure your 
memcached is caching aggressively.


For the record, our test instance currently has fewer than thirty review 
requests so far, so I guarantee it's not this.




One thing you could try doing is going into Settings -> Logging and 
turn on Profiling. Then you can visit a page with slowdowns and run 
with ?profiling=1. This should output to a reviewboard.prof file in 
the logging directory with some details on the SQL queries and the 
code paths hit. If you could pass that on to me, I could take a look.




Yeah, I'm going to do some additional profiling. Something I noticed 
when doing testing on django_evolution (using the test suite you 
directed me to) was that I saw a difference of 7s vs. 45s for the time 
it took to run the suite. This is leading me to believe that the problem 
is more likely to be in python-pyscopg2 or Django itself, rather than 
ReviewBoard.


I'll let you know what I find.

--
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Christian Hammond]

Hi Stephen,

We're running on MySQL. I don't know why you'd be seeing such a huge delay
with PostgreSQL though. Can you tell me which pages are showing delays? (Is
it all of them?)

if it's the dashboard, there are some known slowdowns when reaching large
numbers of review requests. If the diff viewer, make sure your memcached is
caching aggressively.

One thing you could try doing is going into Settings -> Logging and turn on
Profiling. Then you can visit a page with slowdowns and run with
?profiling=1. This should output to a reviewboard.prof file in the logging
directory with some details on the SQL queries and the code paths hit. If
you could pass that on to me, I could take a look.

Christian

-- 
Christian Hammond - chip...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com


On Thu, Jun 10, 2010 at 11:56 AM, Stephen Gallagher <
step...@gallagherhome.com> wrote:

> Well, right now 4-8s for a page refresh is just barely within the tolerable
> limits of my users. I'm trying to determine whether it makes more sense to
> migrate to MySQL (if it would be better performance)
>
> I was also curious whether ReviewBoard's reference implementation was
> running on postgresql and whether there were any tuning tricks I should be
> aware of.
>
>
> On 06/10/2010 02:53 PM, Jan Koprowski wrote:
>
>> It is probably between SQLite and PostgreSQL. But this is delay is
>> proportional to scalable. SQLite is not scalable at all, PostgreSQL is
>> most scalable free SQL database.
>>
>> On Jun 9, 9:39 pm, Stephen Gallagher
>> wrote:
>>
>>
>>> I'm curious, what database is thehttp://reviews.reviewboard.orgserver
>>> using? We deployed on PostgreSQL recently, and I've noticed a distinct
>>> performance issue with it.
>>>
>>> I can only directly compare it to SQLite, but of course that's local
>>> access. But we went from<  1s responses on webpages to 4-8s per page
>>> when backed by Postgresql (on a private network with very little
>>> traffic).
>>>
>>> How is the performance when backed by MySQL?
>>>
>>>
>>
>>
>
> --
> Want to help the Review Board project? Donate today at
> http://www.reviewboard.org/donate/
> Happy user? Let us know at http://www.reviewboard.org/users/
> -~--~~~~--~~--~--~---
> To unsubscribe from this group, send email to
> reviewboard+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/reviewboard?hl=en
>

-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Stephen Gallagher]

Well, right now 4-8s for a page refresh is just barely within the 
tolerable limits of my users. I'm trying to determine whether it makes 
more sense to migrate to MySQL (if it would be better performance)


I was also curious whether ReviewBoard's reference implementation was 
running on postgresql and whether there were any tuning tricks I should 
be aware of.


On 06/10/2010 02:53 PM, Jan Koprowski wrote:

It is probably between SQLite and PostgreSQL. But this is delay is
proportional to scalable. SQLite is not scalable at all, PostgreSQL is
most scalable free SQL database.

On Jun 9, 9:39 pm, Stephen Gallagher
wrote:
   

I'm curious, what database is thehttp://reviews.reviewboard.orgserver
using? We deployed on PostgreSQL recently, and I've noticed a distinct
performance issue with it.

I can only directly compare it to SQLite, but of course that's local
access. But we went from<  1s responses on webpages to 4-8s per page
when backed by Postgresql (on a private network with very little traffic).

How is the performance when backed by MySQL?
 
   


--
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Re: ReviewBoard example instance

[Jan Koprowski]

It is probably between SQLite and PostgreSQL. But this is delay is
proportional to scalable. SQLite is not scalable at all, PostgreSQL is
most scalable free SQL database.

On Jun 9, 9:39 pm, Stephen Gallagher 
wrote:
> I'm curious, what database is thehttp://reviews.reviewboard.orgserver
> using? We deployed on PostgreSQL recently, and I've noticed a distinct
> performance issue with it.
>
> I can only directly compare it to SQLite, but of course that's local
> access. But we went from < 1s responses on webpages to 4-8s per page
> when backed by Postgresql (on a private network with very little traffic).
>
> How is the performance when backed by MySQL?

-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

ReviewBoard example instance

[Stephen Gallagher]

I'm curious, what database is the http://reviews.reviewboard.org server 
using? We deployed on PostgreSQL recently, and I've noticed a distinct 
performance issue with it.


I can only directly compare it to SQLite, but of course that's local 
access. But we went from < 1s responses on webpages to 4-8s per page 
when backed by Postgresql (on a private network with very little traffic).


How is the performance when backed by MySQL?

--
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~--~~~~--~~--~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en