[PATCH] Fetch all series for patch/cover viewing

2018-08-08 Thread Stewart Smith
e.g. a 10 comment patch goes from 26 queries in 17-20ms down to 20 queries in 12ms. A 67 comment cover letter goes from 14 queries in 16ms down to 8 queries in 8ms. So, effectively, a near 2x perf improvement. Previously, at several points we were asking for the latest series and then asking

[PATCH v2] 4x performance improvement for viewing patch with many comments

2018-08-08 Thread Stewart Smith
Using the example of id:20180720035941.6844-1-khand...@linux.vnet.ibm.com with my test dataset of a chunk of a variety of mailing lists, has this cover letter have 67 comments from a variety of people. Thus, it's on the larger side of things. Originally, displaying the /patch/550/ for this

[PATCH] Add index for patchwork_comment (submission_id,date)

2018-08-08 Thread Stewart Smith
This (at least theoretically) should speed up displaying comments on patches/cover letters. It's an index that will return rows in-order for the query that we always do ("give me the comments on this submission in date order"), rather than having to have the database server do a sort for us. I

[PATCH] 4x performance improvement for viewing patch with many comments

2018-08-08 Thread Stewart Smith
Using the example of id:20180720035941.6844-1-khand...@linux.vnet.ibm.com with my test dataset of a chunk of a variety of mailing lists, has this cover letter have 67 comments from a variety of people. Thus, it's on the larger side of things. Originally, displaying the /patch/550/ for this

Re: [RFC PATCH] 3x improvement in patch listing

2018-08-08 Thread Stewart Smith
Daniel Axtens writes: > Daniel Axtens writes: > >> Konstantin Ryabitsev writes: >> >>> On Wed, Aug 08, 2018 at 05:40:06PM +1000, Stewart Smith wrote: There's two main bits that are really expensive when composing the list of patches for a project: the query getting the list, and the

Re: [RFC PATCH] 3x improvement in patch listing

2018-08-08 Thread Stewart Smith
Konstantin Ryabitsev writes: > On Wed, Aug 08, 2018 at 05:40:06PM +1000, Stewart Smith wrote: >>There's two main bits that are really expensive when composing the list >>of patches for a project: the query getting the list, and the query >>finding the series for each patch. > > Stewart: > >

Re: [RFC PATCH] 3x improvement in patch listing

2018-08-08 Thread Daniel Axtens
Daniel Axtens writes: > Konstantin Ryabitsev writes: > >> On Wed, Aug 08, 2018 at 05:40:06PM +1000, Stewart Smith wrote: >>>There's two main bits that are really expensive when composing the list >>>of patches for a project: the query getting the list, and the query >>>finding the series for

Re: Database consistency issues

2018-08-08 Thread Konstantin Ryabitsev
On Thu, Aug 09, 2018 at 02:01:11AM +1000, Daniel Axtens wrote: In the case of Patchwork 1.1.3, I am more confused - the Comment model has a foreign key to Patch here. So if you try to delete Patch you should have got a referential integrity error like I did with e.g. Events. (Patchwork pre 2.0

Re: [RFC PATCH] 3x improvement in patch listing

2018-08-08 Thread Daniel Axtens
Konstantin Ryabitsev writes: > On Wed, Aug 08, 2018 at 05:40:06PM +1000, Stewart Smith wrote: >>There's two main bits that are really expensive when composing the list >>of patches for a project: the query getting the list, and the query >>finding the series for each patch. > > Stewart: > >

Re: [PATCH v2 3/3] travis: add mariadb-10.3 as test

2018-08-08 Thread Daniel Axtens
Hi Daniel, All look good to me (sparse commit messages aside) - I'll do some testing and hopefully push them this week. Thanks for your contributions to Patchwork! Regards, Daniel > Signed-off-by: Daniel Black > --- > .travis.yml | 5 + > 1 file changed, 5 insertions(+) > > diff --git

Re: Database consistency issues

2018-08-08 Thread Daniel Axtens
> > I will prepare a patch that adds the foreign keys. Further to this: Patchwork 2.1 has that Comment has a foreign key relationship to submission, not patch. Patch *inherits* from Submission. This was, in hindsight, one of the cases where an ORM and object-orientation do not sit well

Re: Database consistency issues

2018-08-08 Thread Konstantin Ryabitsev
On Thu, Aug 09, 2018 at 01:16:25AM +1000, Daniel Axtens wrote: SELECT id, submission_id FROM patchwork_comment WHERE submission_id=9896319; +--+---+ | id | submission_id | +--+---+ | 20810589 | 9896319 | +--+---+ 1 row in

Re: [RFC PATCH] 3x improvement in patch listing

2018-08-08 Thread Konstantin Ryabitsev
On Wed, Aug 08, 2018 at 05:40:06PM +1000, Stewart Smith wrote: There's two main bits that are really expensive when composing the list of patches for a project: the query getting the list, and the query finding the series for each patch. Stewart: Thanks for working on this! Do you think this

[RFC PATCH] 3x improvement in patch listing

2018-08-08 Thread Stewart Smith
There's two main bits that are really expensive when composing the list of patches for a project: the query getting the list, and the query finding the series for each patch. If we look at the query getting the list, it gets a lot of unnesseccary fields such as 'headers' and 'content', even

Re: [RFC PATCH] REST: Add new setting for maximum API page size

2018-08-08 Thread Stewart Smith
Daniel Axtens writes: > FWIW we now have this applied on patchwork.ozlabs.org and it appears to > be working. Would like some more input as to what an appropriate default > limit is. My completely fact-free feeling/opinion is that if it takes more than ~1sec to run on

Re: [RFC PATCH] REST: Add new setting for maximum API page size

2018-08-08 Thread Stewart Smith
Andrew Donnellan writes: > On 26/07/18 23:24, Daniel Axtens wrote: >> Andrew Donnellan writes: >> >>> On 24/07/18 15:10, Andrew Donnellan wrote: In 41790caf59ad ("REST: Limit max page size") we limited the maximum page size to the default page size in the settings. This

Re: [RFC PATCH] REST: Add new setting for maximum API page size

2018-08-08 Thread Stewart Smith
Andrew Donnellan writes: > On 24/07/18 15:10, Andrew Donnellan wrote: >> In 41790caf59ad ("REST: Limit max page size") we limited the maximum page >> size to the default page size in the settings. >> >> This turns out to be rather restrictive, as we usually want to keep the >> default page size