Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-29 Thread Aaron Conole
Ilya Maximets writes: > On 1/24/24 13:13, Aaron Conole wrote: >> Ilya Maximets writes: >> >>> On 1/24/24 01:14, Jeremy Kerr wrote: Hi Ilya, > Jeremy, could you, please, try to unban the robot? Sure thing, done! >>> >>> Thanks! >>> Let me know how it goes on

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-24 Thread Aaron Conole
Ilya Maximets writes: > On 1/24/24 01:14, Jeremy Kerr wrote: >> Hi Ilya, >> >>> Jeremy, could you, please, try to unban the robot? >> >> Sure thing, done! > > Thanks! > >> >> Let me know how it goes on your side, I'll monitor things here. > > We'll re-enable the robot later today and will

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-24 Thread Aaron Conole
Jeremy Kerr writes: > Hi Ilya, > >> Jeremy, could you, please, try to unban the robot? > > Sure thing, done! > > Let me know how it goes on your side, I'll monitor things here. Thanks. I've just re-enabled it. The first run will be noisy (because it will requery the open checks). After that,

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-23 Thread Jeremy Kerr
Hi Ilya, > Jeremy, could you, please, try to unban the robot? Sure thing, done! Let me know how it goes on your side, I'll monitor things here. > Note:  Just to clarify, all the UAs should have '(pw-ci)' prefix > now with additional identifications for the project/actions they > are related

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-23 Thread Aaron Conole
Jeremy Kerr writes: > Hi Ilya, > >> Ugh.  Yeah, the checks/ requests are definitely something we can >> improve. Aaron is working on removing vast majority of this type of >> requests as we speak.  Hopefully, that will be done soon. > > OK, sounds good! > >> Do you think it'll be fine to unban

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-22 Thread Aaron Conole
Jeremy Kerr writes: > Hi Ilya, > >> Ugh.  Yeah, the checks/ requests are definitely something we can >> improve. Aaron is working on removing vast majority of this type of >> requests as we speak.  Hopefully, that will be done soon. > > OK, sounds good! > >> Do you think it'll be fine to unban

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-22 Thread Aaron Conole
Ilya Maximets writes: > On 1/22/24 14:59, Jeremy Kerr wrote: >> Hi Ilya, >> >>> We have a robot running in the RH network that pushes checks and >>> updates statuses for openvswitch and ovn projects, but it shouldn't >>> really make any "global patches view" types of requests.  All the >>>

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-22 Thread Stephen Finucane
On Mon, 2024-01-22 at 15:52 +0800, Jeremy Kerr wrote: > Hi all, > > I'll try and get to that over the weekend. > > Looks like the heaviest database load is due to API request to the > global patches view, which is a bit of an odd use-case; that all > appears to be mostly spider traffic. Just as

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-22 Thread Jeremy Kerr
Hi Ilya, > Ugh.  Yeah, the checks/ requests are definitely something we can > improve. Aaron is working on removing vast majority of this type of > requests as we speak.  Hopefully, that will be done soon. OK, sounds good! > Do you think it'll be fine to unban the robot once it doesn't run that

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-22 Thread Jeremy Kerr
Hi Ilya, > We have a robot running in the RH network that pushes checks and > updates statuses for openvswitch and ovn projects, but it shouldn't > really make any "global patches view" types of requests.  All the > patches it looks at supposed to be only from these two projects.  And > it should

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-21 Thread Jeremy Kerr
Hi all, > I'll try and get to that over the weekend. Looks like the heaviest database load is due to API request to the global patches view, which is a bit of an odd use-case; that all appears to be mostly spider traffic. Konstantin: I'm not sure your new index would help in that case, we're not

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-18 Thread Jeremy Kerr
Hi Rob, (adding Ilya, who has done some very handy logging too) > I have a job that runs every 20min checking PW. I added timestamps > and > here's all the 503 failures in the last 22 hours: > > Wed Jan 17 10:12:29 AM CST 2024 > Wed Jan 17 12:16:38 PM CST 2024 > Wed Jan 17 01:37:01 PM CST 2024

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-18 Thread Rob Herring
On Wed, Jan 17, 2024 at 10:36 AM Jeremy Kerr wrote: > > Hi Rob, > > > When was recent? The 503s have been happening for probably a month or > > 2 now, but seem to be increasing in frequency to the point of PW > > being unusable now. > > That was early Jan, so you've been seeing those since before

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-17 Thread Stephen Rothwell
Hi all, On Thu, 18 Jan 2024 00:36:39 +0800 Jeremy Kerr wrote: > > > When and how long do those run? I'm pretty much working your > > overnight. > > I'd have to defer to Stephen on that, but I had been assuming 4am / > 17:00 UTC. Yeah, 4am to 6am our time (currently +1100) ... it takes a long

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-17 Thread Jeremy Kerr
Hi Rob, > When was recent? The 503s have been happening for probably a month or > 2 now, but seem to be increasing in frequency to the point of PW > being unusable now. That was early Jan, so you've been seeing those since before the upgrade. I do see quite a bit of spider traffic now though,

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-17 Thread Rob Herring
On Tue, Jan 16, 2024 at 6:56 PM Jeremy Kerr wrote: > > Hi Rob & Konstantin, > > > > > > I'm still seeing this slowness and now to add to that intermittent > > > 503 errors. > > OK, I've heard the same yesterday from the ovn folks. Sounds like this > is continuing after a recent hardware upgrade

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-16 Thread Jeremy Kerr
Hi Rob & Konstantin, > > > I'm still seeing this slowness and now to add to that intermittent > > 503 errors. OK, I've heard the same yesterday from the ovn folks. Sounds like this is continuing after a recent hardware upgrade too. So far it does seem time of day related, so possibly

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-16 Thread Konstantin Ryabitsev
On Tue, Jan 16, 2024 at 11:05:42AM -0600, Rob Herring wrote: > > This improved things, but it seems like the DT PW has gotten slower > > again in the last few weeks. It's taking ~12sec to load the patch > > list. > > I'm still seeing this slowness and now to add to that intermittent 503 errors.

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2024-01-16 Thread Rob Herring
Hi Jeremy, On Thu, Jun 29, 2023 at 11:55 AM Rob Herring wrote: > > Hi Jeremy, > > On Wed, Aug 17, 2022 at 6:19 PM Jeremy Kerr wrote: > > > > Hi Rob, > > > > > > I'll do a bit of experimentation here to see what's up, will keep > > > > you > > > > posted. > > > > > > Actually, it is worse than

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2023-06-29 Thread Rob Herring
Hi Jeremy, On Wed, Aug 17, 2022 at 6:19 PM Jeremy Kerr wrote: > > Hi Rob, > > > > I'll do a bit of experimentation here to see what's up, will keep > > > you > > > posted. > > > > Actually, it is worse than just slow. New patches aren't showing up > > since before the move. > > Looks like I

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2022-08-17 Thread Jeremy Kerr
Hi Rob, > > I'll do a bit of experimentation here to see what's up, will keep > > you > > posted. > > Actually, it is worse than just slow. New patches aren't showing up > since before the move. Looks like I missed a dependency for the custom filter for the DT feed; apologies for that. I've

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2022-08-17 Thread Rob Herring
On Tue, Aug 16, 2022 at 8:26 PM Jeremy Kerr wrote: > > Hi Rob, > > > > Since the move, the web interface seems slow for some operations. > > Loading the list view takes ~30s. Loading a patch seems fine, and I'm > > not seeing any slowness with the CLI. > > OK, thanks for letting me know. It looks

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2022-08-16 Thread Jeremy Kerr
Hi Rob, > Since the move, the web interface seems slow for some operations. > Loading the list view takes ~30s. Loading a patch seems fine, and I'm > not seeing any slowness with the CLI. OK, thanks for letting me know. It looks like this is reliably reproducible, but only affects some of the

Re: [Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

2022-08-16 Thread Rob Herring
On Mon, Aug 15, 2022 at 1:06 AM Jeremy Kerr wrote: > > Hi patchworkers, > > Stephen and I will be moving the patchwork.ozlabs.org instance to a new > server this week. For this, we'll need about two hours of downtime, > starting at: > > Australia East: midday Tuesday 16 Aug > Australia West: