Re: Testing of patches on this list
On 06/02/18 22:00, Daniel Axtens wrote: For those of you who don't know, snowpatch is a cool project from Andrew and Russell that takes series from the list using the API, applies them with git, and passes the repository off to Jenkins for testing, and reports back with the checks API. It's a cool project, and an excellent use of the API - all made even cooler by being written in Rust :P I should add that there's a tonne of code we haven't merged quite yet to make it work with the current version of the API... we've been fitting in work on this around our many other commitments and progress has admittedly been slower than we've hoped... But we'll clean all that up and tag a release Real Soon Now(tm). Andrew https://developer.ibm.com/code/2016/06/14/meet-snowpatch-continuous-patch-integration/ https://github.com/ruscur/snowpatch Test results will be posted to Patchwork and visible on the patch list and patch details pages at https://patchwork.ozlabs.org/project/patchwork/list/, along with a link to the build log. We're running the --quick-tox test suite, so no Selenium tests (they seem to be a bit flaky, and they add significantly to test run time). Right now, we only report success or failure based on whether the patch passes or fails the entire quick-tox suite - we don't break it down by configuration. However the log should give enough information to work that out. At some point we'll figure out how to do pep8 reporting as well. At the moment the only reporting is via the patchwork interface, no email alerts or anything. I will - of course - be checking this before I merge patches! Regards, Daniel This is experimental for the time being, we'll likely run into a few problems along the way. Comments and feedback welcome! Patchwork is a nice low traffic project for us to dogfood our own API on before we switch this on for LinuxPPC... Apologies for the noise on the list as we tested this setup! Thanks to Joel Stanley for helping us with Jenkins setup, and a certain bookstore-turned-cloud-provider for having a free tier of VMs for us to run our tests with... Andrew -- Andrew Donnellan OzLabs, ADL Canberra andrew.donnel...@au1.ibm.com IBM Australia Limited ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork -- Andrew Donnellan OzLabs, ADL Canberra andrew.donnel...@au1.ibm.com IBM Australia Limited ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: [PATCH] Fix CRLF newlines upon submission changes
Hi Veronika,
> After changing submission via admin interface, CRLF newlines are
> suddenly present in the body. Replace them back to '\n'.
>
> The issue was found after modifying submission via admin interface
> using Python 2 and downloading the respective mbox file (git choked on
> downloaded patch because of malformed line endings). Python 3's mail
> module uses '\n' internally so the problem doesn't manifest there, but
> the content received by Django/JS is still saved in the database with
> CRLF line endings which shouldn't be there.
Huh, weird. I can't say modifying a sumbission through the admin
interface is recommended behaviour, but still worth fixing.
Please could you add a comment or docstring to the save function
that explains why this is necessary - an abbreviated version of your
commit message would be fine.
Would you like this queued up for the 2.0.2 stable release?
Regards,
Daniel
> Signed-off-by: Veronika Kabatova
> ---
> patchwork/models.py | 4
> 1 file changed, 4 insertions(+)
>
> diff --git a/patchwork/models.py b/patchwork/models.py
> index 3bf7c72..411af63 100644
> --- a/patchwork/models.py
> +++ b/patchwork/models.py
> @@ -328,6 +328,10 @@ class EmailMixin(models.Model):
> return ''.join([match.group(0) + '\n' for match in
> self.response_re.finditer(self.content)])
>
> +def save(self, *args, **kwargs):
> +self.content = self.content.replace('\r\n', '\n')
> +super(EmailMixin, self).save(*args, **kwargs)
> +
> class Meta:
> abstract = True
>
> --
> 2.13.6
>
> ___
> Patchwork mailing list
> Patchwork@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/patchwork
___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork
Re: [RFC 1/4] Automatically remove old events
On Thu, 2018-01-25 at 16:21 +1100, Daniel Axtens wrote: > Hi Stephen, > > This is a good idea and I'm happy for it to go in as-is, before the > rest > of the series. > > Reviewed-by: Daniel Axtens I've been thinking a little more on this and I don't know if I actually want to do this. If we manage to solve the performance impact, we can use events as an auditor and display things like state changes on the patch detail page. I might leave this one for another while. Would still appreciate reviews on the rest of this series though. Stephen > > Regards, > Daniel ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
[PATCH] Fix CRLF newlines upon submission changes
From: Veronika Kabatova
After changing submission via admin interface, CRLF newlines are
suddenly present in the body. Replace them back to '\n'.
The issue was found after modifying submission via admin interface
using Python 2 and downloading the respective mbox file (git choked on
downloaded patch because of malformed line endings). Python 3's mail
module uses '\n' internally so the problem doesn't manifest there, but
the content received by Django/JS is still saved in the database with
CRLF line endings which shouldn't be there.
Signed-off-by: Veronika Kabatova
---
patchwork/models.py | 4
1 file changed, 4 insertions(+)
diff --git a/patchwork/models.py b/patchwork/models.py
index 3bf7c72..411af63 100644
--- a/patchwork/models.py
+++ b/patchwork/models.py
@@ -328,6 +328,10 @@ class EmailMixin(models.Model):
return ''.join([match.group(0) + '\n' for match in
self.response_re.finditer(self.content)])
+def save(self, *args, **kwargs):
+self.content = self.content.replace('\r\n', '\n')
+super(EmailMixin, self).save(*args, **kwargs)
+
class Meta:
abstract = True
--
2.13.6
___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork
Re: Testing of patches on this list
Andrew Donnellan writes: > Greetings, > > Russell and I, with some help from Daniel, are now testing patches on > this list as they come in using snowpatch. Hopefully, this will allow us > to catch test failures before Travis CI (which only triggers after > patches are merged). Congratulations Andrew and Russell!!! For those of you who don't know, snowpatch is a cool project from Andrew and Russell that takes series from the list using the API, applies them with git, and passes the repository off to Jenkins for testing, and reports back with the checks API. It's a cool project, and an excellent use of the API - all made even cooler by being written in Rust :P https://developer.ibm.com/code/2016/06/14/meet-snowpatch-continuous-patch-integration/ https://github.com/ruscur/snowpatch > > Test results will be posted to Patchwork and visible on the patch list > and patch details pages at > https://patchwork.ozlabs.org/project/patchwork/list/, along with a link > to the build log. > > We're running the --quick-tox test suite, so no Selenium tests (they > seem to be a bit flaky, and they add significantly to test run time). > > Right now, we only report success or failure based on whether the patch > passes or fails the entire quick-tox suite - we don't break it down by > configuration. However the log should give enough information to work > that out. At some point we'll figure out how to do pep8 reporting as well. > > At the moment the only reporting is via the patchwork interface, no > email alerts or anything. I will - of course - be checking this before I merge patches! Regards, Daniel > > This is experimental for the time being, we'll likely run into a few > problems along the way. Comments and feedback welcome! Patchwork is a > nice low traffic project for us to dogfood our own API on before we > switch this on for LinuxPPC... > > Apologies for the noise on the list as we tested this setup! > > Thanks to Joel Stanley for helping us with Jenkins setup, and a certain > bookstore-turned-cloud-provider for having a free tier of VMs for us to > run our tests with... > > > Andrew > > -- > Andrew Donnellan OzLabs, ADL Canberra > andrew.donnel...@au1.ibm.com IBM Australia Limited ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: patchwork.ozlabs.org down, and e-mails not recorded
Jeremy Kerr writes: > Hi Andrew, > >> jk, any idea whether there's something particular about the ozlabs.org >> instance that could be causing it to drop these patches? > > We're running plain upstream with regards to the parsing code; so if it > parses we should be fine there. > > Assume the parsing is OK, there are a couple of reasons for potential > drops: > > - the database server was down at the time of parse. This is quite > infrequent, but can happen during database upgrades. The last one of > those was at 2018-01-27 05:08:30 UTC, so doesn't look like the case > here > > - after the update to 2.x, database load has significantly increased; > there have been occasions where db connections are rejected due to > this. The recent patchwork update should address some issues there, > but I'm not sure whether we're back to optimal db behaviour now... (FYI and slightly off-topic:) Not yet - I'm tracking 2 distinct things: - a join is required to do any operation at all on patches, even listing them (join of sumbission and patch tables is required to get the project of a patch, as the project field belongs to the submission table). - massive api slowness, tracked on github and with some proposed patches on list. Regards, Daniel > > Cheers, > > > Jeremy > ___ > Patchwork mailing list > Patchwork@lists.ozlabs.org > https://lists.ozlabs.org/listinfo/patchwork ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: patchwork.ozlabs.org down, and e-mails not recorded
Hello, On Tue, 6 Feb 2018 17:03:50 +0800, Jeremy Kerr wrote: > We're running plain upstream with regards to the parsing code; so if it > parses we should be fine there. > > Assume the parsing is OK, there are a couple of reasons for potential > drops: > > - the database server was down at the time of parse. This is quite > infrequent, but can happen during database upgrades. The last one of > those was at 2018-01-27 05:08:30 UTC, so doesn't look like the case > here > > - after the update to 2.x, database load has significantly increased; > there have been occasions where db connections are rejected due to > this. The recent patchwork update should address some issues there, > but I'm not sure whether we're back to optimal db behaviour now... Could it be that the e-mail hasn't been received? Could you check on the server the logs, and see if the mail has been received at all? Perhaps it was dropped due to spam detection or something like that. It would be useful to understand if those e-mails that made it to the mailing list were received by the patchwork server. We are really seeing lots of patches being not recorded by patchwork, which makes it a lot less usable :-/ Thanks! Thomas -- Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) Embedded Linux and Kernel engineering http://bootlin.com ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: patchwork.ozlabs.org down, and e-mails not recorded
Hi Andrew, jk, any idea whether there's something particular about the ozlabs.org instance that could be causing it to drop these patches? We're running plain upstream with regards to the parsing code; so if it parses we should be fine there. Assume the parsing is OK, there are a couple of reasons for potential drops: - the database server was down at the time of parse. This is quite infrequent, but can happen during database upgrades. The last one of those was at 2018-01-27 05:08:30 UTC, so doesn't look like the case here - after the update to 2.x, database load has significantly increased; there have been occasions where db connections are rejected due to this. The recent patchwork update should address some issues there, but I'm not sure whether we're back to optimal db behaviour now... Cheers, Jeremy ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Testing of patches on this list
Greetings, Russell and I, with some help from Daniel, are now testing patches on this list as they come in using snowpatch. Hopefully, this will allow us to catch test failures before Travis CI (which only triggers after patches are merged). Test results will be posted to Patchwork and visible on the patch list and patch details pages at https://patchwork.ozlabs.org/project/patchwork/list/, along with a link to the build log. We're running the --quick-tox test suite, so no Selenium tests (they seem to be a bit flaky, and they add significantly to test run time). Right now, we only report success or failure based on whether the patch passes or fails the entire quick-tox suite - we don't break it down by configuration. However the log should give enough information to work that out. At some point we'll figure out how to do pep8 reporting as well. At the moment the only reporting is via the patchwork interface, no email alerts or anything. This is experimental for the time being, we'll likely run into a few problems along the way. Comments and feedback welcome! Patchwork is a nice low traffic project for us to dogfood our own API on before we switch this on for LinuxPPC... Apologies for the noise on the list as we tested this setup! Thanks to Joel Stanley for helping us with Jenkins setup, and a certain bookstore-turned-cloud-provider for having a free tier of VMs for us to run our tests with... Andrew -- Andrew Donnellan OzLabs, ADL Canberra andrew.donnel...@au1.ibm.com IBM Australia Limited ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: patchwork.ozlabs.org down, and e-mails not recorded
On 06/02/18 08:45, Thomas Petazzoni wrote: Is there anything that can be done about this ? An example of Message-Id that was not recorded is: Message-Id: <8027bae45d8e041c8a1e0bc714ab378ff984ded3.1517820133.git.yann.morin.1...@free.fr> I've scraped the buildroot archives to see if there's anything obviously wrong with that particular message ID - it appears to parse fine. jk, any idea whether there's something particular about the ozlabs.org instance that could be causing it to drop these patches? -- Andrew Donnellan OzLabs, ADL Canberra andrew.donnel...@au1.ibm.com IBM Australia Limited ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
