Re: [PATCH] parser: Fix parsing of pull request emails with CRLF line endings on Python 2

2018-01-07 Thread Andrew Donnellan 

On 08/01/18 18:49, Andrew Donnellan wrote:

On 08/01/18 18:46, Andrew Donnellan wrote:

When using Python 2, an incoming email that uses CRLF line endings won't
have the line endings converted by the email parser. This means that the
regex to detect pull request emails will fail to match.

Clean up the line endings when we extract the email body to fix this. (On
Python 3, the email parser policy fixes this for us at the initial email
parsing stage.)

Add a test pull request mbox with CRLF line endings to ensure we don't
regress.


Hmm, sudden realisation that git handles line endings for us, so perhaps 
we don't have an easy way of including a test here.





Also this should probably be applied to stable too...




Closes: #148 ("Parsing pull request emails broken on Python 2")
Signed-off-by: Andrew Donnellan 
---
  patchwork/parser.py    |   4 +
  .../mail/0017-git-pull-request-crlf-newlines.mbox  | 348 
+

  patchwork/tests/test_parser.py |   3 +
  3 files changed, 355 insertions(+)
  create mode 100644 
patchwork/tests/mail/0017-git-pull-request-crlf-newlines.mbox


diff --git a/patchwork/parser.py b/patchwork/parser.py
index 1568bc4..7c677db 100644
--- a/patchwork/parser.py
+++ b/patchwork/parser.py
@@ -666,9 +666,13 @@ def clean_content(content):
  """Remove cruft from the email message.
  Catch signature (-- ) and list footer (_) cruft.
+
+    Change to Unix line endings (the Python 3 email module does this 
for us,

+    but not Python 2).
  """
  sig_re = re.compile(r'^(-- |_+)\n.*', re.S | re.M)
  content = sig_re.sub('', content)
+    content = content.replace('\r\n', '\n')
  return content.strip()
diff --git 
a/patchwork/tests/mail/0017-git-pull-request-crlf-newlines.mbox 
b/patchwork/tests/mail/0017-git-pull-request-crlf-newlines.mbox

new file mode 100644
index 000..bad78ae
--- /dev/null
+++ b/patchwork/tests/mail/0017-git-pull-request-crlf-newlines.mbox
@@ -0,0 +1,348 @@
+From b...@kernel.crashing.org Fri Oct 22 11:51:02 2010
+Return-Path: 
+X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on 
bilbo.ozlabs.org

+X-Spam-Level:
+X-Spam-Status: No, score=0.0 required=3.0 tests=none autolearn=disabled
+    version=3.3.1
+X-Original-To: j...@ozlabs.org
+Delivered-To: j...@ozlabs.org
+Received: from bilbo.ozlabs.org (localhost [127.0.0.1])
+    by ozlabs.org (Postfix) with ESMTP id ED4B3100937
+    for ; Fri, 22 Oct 2010 14:51:54 +1100 (EST)
+Received: by ozlabs.org (Postfix)
+    id BF799B70CB; Fri, 22 Oct 2010 14:51:50 +1100 (EST)
+Delivered-To: linuxppc-...@ozlabs.org
+Received: from gate.crashing.org (gate.crashing.org [63.228.1.57])
+    (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
+    (Client did not present a certificate)
+    by ozlabs.org (Postfix) with ESMTPS id 94629B7043
+    for ; Fri, 22 Oct 2010 14:51:49 +1100 (EST)
+Received: from [IPv6:::1] (localhost.localdomain [127.0.0.1])
+    by gate.crashing.org (8.14.1/8.13.8) with ESMTP id o9M3p3SP018234;
+    Thu, 21 Oct 2010 22:51:04 -0500
+Subject: [git pull] Please pull powerpc.git next branch
+From: Benjamin Herrenschmidt 
+To: Linus Torvalds 
+Date: Fri, 22 Oct 2010 14:51:02 +1100
+Message-ID: <1287719462.2198.37.camel@pasglop>
+Mime-Version: 1.0
+X-Mailer: Evolution 2.30.3
+Cc: linuxppc-dev list ,
+ Andrew Morton ,
+ Linux Kernel list 
+X-BeenThere: linuxppc-...@lists.ozlabs.org
+X-Mailman-Version: 2.1.13
+Precedence: list
+List-Id: Linux on PowerPC Developers Mail List 
+List-Unsubscribe: ,
+    
+List-Archive: 
+List-Post: 
+List-Help: 
+List-Subscribe: ,
+    
+Content-Type: text/plain;
+  charset="us-ascii"
+Content-Transfer-Encoding: 7bit
+Sender: linuxppc-dev-bounces+jk=ozlabs@lists.ozlabs.org
+Errors-To: linuxppc-dev-bounces+jk=ozlabs@lists.ozlabs.org
+X-UID: 11446
+X-Length: 16781
+Status: R
+X-Status: N
+X-KMail-EncryptionState:
+X-KMail-SignatureState:
+X-KMail-MDN-Sent:
+
+Hi Linus !
+
+Here's powerpc's batch for this merge window. Mostly bits and pieces,
+such as Anton doing some performance tuning left and right, and the
+usual churn. One hilight is the support for the new Freescale e5500 core
+(64-bit BookE). Another one is that we now wire up the whole lot of
+socket calls as direct syscalls in addition to the old style indirect
+method.
+
+Cheers,
+Ben.
+
+The

Re: [PATCH] docs/development: Fix tox invocation for listing targets

2018-01-07 Thread Daniel Axtens 
Andrew Donnellan  writes:

> Signed-off-by: Andrew Donnellan 
> ---
>  docs/development/contributing.rst | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/docs/development/contributing.rst 
> b/docs/development/contributing.rst
> index 0cab79e..8988c9f 100644
> --- a/docs/development/contributing.rst
> +++ b/docs/development/contributing.rst
> @@ -45,7 +45,7 @@ to do. To start, you can show the default targets like so:
>  
>  .. code-block:: shell
>  
> -   $ tox --list
> +   $ tox -l

Reviewed-by: Daniel Axtens 

It turns out what we wanted was --listenvs, not list.

Regards,
Daniel

>  
>  You'll see that this includes a number of targets to run unit tests against 
> the
>  different versions of Django supported, along with some other targets related
> -- 
> 2.11.0
>
> ___
> Patchwork mailing list
> Patchwork@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/patchwork
___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork