We have 4 new tests: - One with a header with a character that is neither 7-bit ASCII nor UTF-8. - One with UTF-8 in the body. - One with un-encoded UTF-8 in the headers. - One with a header that claims to be UTF-8 but isn't.
This also requires us to change how we open files on Python 3. Use the binary file and message_from_binary_file setup under Python 3. CC: Thomas Monjalon <[email protected]> # for the original header email CC: Johannes Berg <[email protected]> # for the UTF-8 body email Signed-off-by: Daniel Axtens <[email protected]> --- patchwork/tests/mail/0012-invalid-header-char.mbox | 72 ++++++++++++++++++++++ patchwork/tests/mail/0013-with-utf8-body.mbox | 46 ++++++++++++++ .../mail/0014-with-unencoded-utf8-headers.mbox | 46 ++++++++++++++ .../tests/mail/0015-with-invalid-utf8-headers.mbox | 46 ++++++++++++++ patchwork/tests/test_parser.py | 39 +++++++++++- 5 files changed, 247 insertions(+), 2 deletions(-) create mode 100644 patchwork/tests/mail/0012-invalid-header-char.mbox create mode 100644 patchwork/tests/mail/0013-with-utf8-body.mbox create mode 100644 patchwork/tests/mail/0014-with-unencoded-utf8-headers.mbox create mode 100644 patchwork/tests/mail/0015-with-invalid-utf8-headers.mbox diff --git a/patchwork/tests/mail/0012-invalid-header-char.mbox b/patchwork/tests/mail/0012-invalid-header-char.mbox new file mode 100644 index 000000000000..3412ca6da81a --- /dev/null +++ b/patchwork/tests/mail/0012-invalid-header-char.mbox @@ -0,0 +1,72 @@ +From [email protected] Thu Sep 15 14:06:56 2016 +Return-Path: <[email protected]> +Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) + by dpdk.org (Postfix) with ESMTP id E9946530F + for <[email protected]>; Thu, 15 Sep 2016 14:06:55 +0200 (CEST) +Received: from fmsmga001.fm.intel.com ([10.253.24.23]) + by fmsmga101.fm.intel.com with ESMTP; 15 Sep 2016 05:06:54 -0700 +X-ExtLoop1: 1 +X-IronPort-AV: E=Sophos;i="5.30,339,1470726000"; d="scan'208";a="1040330672" +Received: from irvmail001.ir.intel.com ([163.33.26.43]) + by fmsmga001.fm.intel.com with ESMTP; 15 Sep 2016 05:06:54 -0700 +Received: from sivswdev02.ir.intel.com (sivswdev02.ir.intel.com + [10.237.217.46]) + by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id + u8FC6rSL030312; Thu, 15 Sep 2016 13:06:53 +0100 +Received: from sivswdev02.ir.intel.com (localhost [127.0.0.1]) + by sivswdev02.ir.intel.com with ESMTP id u8FC6rwP015039; + Thu, 15 Sep 2016 13:06:53 +0100 +Received: (from fyigit@localhost) + by sivswdev02.ir.intel.com with � id u8FC6rZs015035; + Thu, 15 Sep 2016 13:06:53 +0100 +X-Authentication-Warning: sivswdev02.ir.intel.com: fyigit set sender to + [email protected] using -f +From: Ferruh Yigit <[email protected]> +To: [email protected] +Cc: Ferruh Yigit <[email protected]> +Date: Thu, 15 Sep 2016 13:06:44 +0100 +Message-Id: <[email protected]> +X-Mailer: git-send-email 1.7.0.7 +Subject: [dpdk-dev] [PATCH] kni: fix compilation error when debug enabled +X-BeenThere: [email protected] +X-Mailman-Version: 2.1.15 +Precedence: list +List-Id: patches and discussions about DPDK <dev.dpdk.org> +List-Unsubscribe: <http://dpdk.org/ml/options/dev>, + <mailto:[email protected]?subject=unsubscribe> +List-Archive: <http://dpdk.org/ml/archives/dev/> +List-Post: <mailto:[email protected]> +List-Help: <mailto:[email protected]?subject=help> +List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, + <mailto:[email protected]?subject=subscribe> +X-List-Received-Date: Thu, 15 Sep 2016 12:06:56 -0000 + +Fix build error with Linux kernel >= v4.7 + +Fix compile error because of Linux API change, 'trans_start' field +removed from 'struct net_device'. + +Linux: 9b36627acecd ("net: remove dev->trans_start") + +Signed-off-by: Ferruh Yigit <[email protected]> +--- + lib/librte_eal/linuxapp/kni/kni_net.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/lib/librte_eal/linuxapp/kni/kni_net.c b/lib/librte_eal/linuxapp/kni/kni_net.c +index fc82193..371bbfa 100644 +--- a/lib/librte_eal/linuxapp/kni/kni_net.c ++++ b/lib/librte_eal/linuxapp/kni/kni_net.c +@@ -508,7 +508,7 @@ kni_net_tx_timeout (struct net_device *dev) + struct kni_dev *kni = netdev_priv(dev); + + KNI_DBG("Transmit timeout at %ld, latency %ld\n", jiffies, +- jiffies - dev->trans_start); ++ jiffies - dev_trans_start(dev)); + + kni->stats.tx_errors++; + netif_wake_queue(dev); +-- +2.7.4 + + diff --git a/patchwork/tests/mail/0013-with-utf8-body.mbox b/patchwork/tests/mail/0013-with-utf8-body.mbox new file mode 100644 index 000000000000..c7319f240b5a --- /dev/null +++ b/patchwork/tests/mail/0013-with-utf8-body.mbox @@ -0,0 +1,46 @@ +From patchwork Wed Jun 1 20:00:54 2016 +Content-Type: text/plain; charset="utf-8" +MIME-Version: 1.0 +Content-Transfer-Encoding: 8bit +Subject: MAINTAINERS: Update entry for BCM5301X ARM +From: =?utf-8?b?UmFmYcWCIE1pxYJlY2tp?= <[email protected]> +X-Patchwork-Id: 9148093 +Message-Id: <[email protected]> +To: Florian Fainelli <[email protected]> +Cc: [email protected], + =?UTF-8?q?Rafa=C5=82=20Mi=C5=82ecki?= <[email protected]>, + Andrew Morton <[email protected]>, + "David S. Miller" <[email protected]>, + Greg Kroah-Hartman <[email protected]>, + Mauro Carvalho Chehab <[email protected]>, + Guenter Roeck <[email protected]>, Jiri Slaby <[email protected]>, + [email protected] (open list) +Date: Wed, 1 Jun 2016 22:00:54 +0200 + +Add myself as I contribute to it. Include Broadcom's feedback ML as +suggested by Florian. Finally modify file rule to match +bcm5301x-nand-cs0-bch8.dtsi. + +Signed-off-by: Rafał Miłecki <[email protected]> +--- + MAINTAINERS | 4 +++- + 1 file changed, 3 insertions(+), 1 deletion(-) + +diff --git a/MAINTAINERS b/MAINTAINERS +index 7304d2e..a5352d8 100644 +--- a/MAINTAINERS ++++ b/MAINTAINERS +@@ -2511,10 +2511,12 @@ F: arch/mips/include/asm/mach-bcm47xx/* + + BROADCOM BCM5301X ARM ARCHITECTURE + M: Hauke Mehrtens <[email protected]> ++M: Rafał Miłecki <[email protected]> + L: [email protected] ++L: [email protected] + S: Maintained + F: arch/arm/mach-bcm/bcm_5301x.c +-F: arch/arm/boot/dts/bcm5301x.dtsi ++F: arch/arm/boot/dts/bcm5301x*.dtsi + F: arch/arm/boot/dts/bcm470* + + BROADCOM BCM63XX ARM ARCHITECTURE diff --git a/patchwork/tests/mail/0014-with-unencoded-utf8-headers.mbox b/patchwork/tests/mail/0014-with-unencoded-utf8-headers.mbox new file mode 100644 index 000000000000..43c8c45a0756 --- /dev/null +++ b/patchwork/tests/mail/0014-with-unencoded-utf8-headers.mbox @@ -0,0 +1,46 @@ +From patchwork Wed Jun 1 20:00:54 2016 +Content-Type: text/plain; charset="utf-8" +MIME-Version: 1.0 +Content-Transfer-Encoding: 8bit +Subject: MAINTAINERS: Update entry for BCM5301X ARM to include Rafał Miłecki +From: Rafał Miłecki <[email protected]> +X-Patchwork-Id: 9148093 +Message-Id: <[email protected]> +To: Florian Fainelli <[email protected]> +Cc: [email protected], + Rafał Miłecki <[email protected]>, + Andrew Morton <[email protected]>, + "David S. Miller" <[email protected]>, + Greg Kroah-Hartman <[email protected]>, + Mauro Carvalho Chehab <[email protected]>, + Guenter Roeck <[email protected]>, Jiri Slaby <[email protected]>, + [email protected] (open list) +Date: Wed, 1 Jun 2016 22:00:54 +0200 + +Add myself as I contribute to it. Include Broadcom's feedback ML as +suggested by Florian. Finally modify file rule to match +bcm5301x-nand-cs0-bch8.dtsi. + +Signed-off-by: Rafał Miłecki <[email protected]> +--- + MAINTAINERS | 4 +++- + 1 file changed, 3 insertions(+), 1 deletion(-) + +diff --git a/MAINTAINERS b/MAINTAINERS +index 7304d2e..a5352d8 100644 +--- a/MAINTAINERS ++++ b/MAINTAINERS +@@ -2511,10 +2511,12 @@ F: arch/mips/include/asm/mach-bcm47xx/* + + BROADCOM BCM5301X ARM ARCHITECTURE + M: Hauke Mehrtens <[email protected]> ++M: Rafał Miłecki <[email protected]> + L: [email protected] ++L: [email protected] + S: Maintained + F: arch/arm/mach-bcm/bcm_5301x.c +-F: arch/arm/boot/dts/bcm5301x.dtsi ++F: arch/arm/boot/dts/bcm5301x*.dtsi + F: arch/arm/boot/dts/bcm470* + + BROADCOM BCM63XX ARM ARCHITECTURE diff --git a/patchwork/tests/mail/0015-with-invalid-utf8-headers.mbox b/patchwork/tests/mail/0015-with-invalid-utf8-headers.mbox new file mode 100644 index 000000000000..57a1b40c3b99 --- /dev/null +++ b/patchwork/tests/mail/0015-with-invalid-utf8-headers.mbox @@ -0,0 +1,46 @@ +From patchwork Wed Jun 1 20:00:54 2016 +Content-Type: text/plain; charset="utf-8" +MIME-Version: 1.0 +Content-Transfer-Encoding: 8bit +Subject: MAINTAINERS: Update entry for BCM5301X ARM +From: =?utf-8?b?UmFmYcWCIE1pxYJlY2tp?= <[email protected]> +X-Patchwork-Id: 9148093 +Message-Id: <[email protected]> +To: Florian Fainelli <[email protected]> +Cc: [email protected], + =?UTF-8?q?Rafa=FF=FF=20Mi=FF=FFecki?= <[email protected]>, + Andrew Morton <[email protected]>, + "David S. Miller" <[email protected]>, + Greg Kroah-Hartman <[email protected]>, + Mauro Carvalho Chehab <[email protected]>, + Guenter Roeck <[email protected]>, Jiri Slaby <[email protected]>, + [email protected] (open list) +Date: Wed, 1 Jun 2016 22:00:54 +0200 + +Add myself as I contribute to it. Include Broadcom's feedback ML as +suggested by Florian. Finally modify file rule to match +bcm5301x-nand-cs0-bch8.dtsi. + +Signed-off-by: Rafał Miłecki <[email protected]> +--- + MAINTAINERS | 4 +++- + 1 file changed, 3 insertions(+), 1 deletion(-) + +diff --git a/MAINTAINERS b/MAINTAINERS +index 7304d2e..a5352d8 100644 +--- a/MAINTAINERS ++++ b/MAINTAINERS +@@ -2511,10 +2511,12 @@ F: arch/mips/include/asm/mach-bcm47xx/* + + BROADCOM BCM5301X ARM ARCHITECTURE + M: Hauke Mehrtens <[email protected]> ++M: Rafał Miłecki <[email protected]> + L: [email protected] ++L: [email protected] + S: Maintained + F: arch/arm/mach-bcm/bcm_5301x.c +-F: arch/arm/boot/dts/bcm5301x.dtsi ++F: arch/arm/boot/dts/bcm5301x*.dtsi + F: arch/arm/boot/dts/bcm470* + + BROADCOM BCM63XX ARM ARCHITECTURE diff --git a/patchwork/tests/test_parser.py b/patchwork/tests/test_parser.py index 2a9a72b7df04..3ec7790450c4 100644 --- a/patchwork/tests/test_parser.py +++ b/patchwork/tests/test_parser.py @@ -17,7 +17,7 @@ # along with Patchwork; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA -from email import message_from_file +import email from email import message_from_string from email.mime.multipart import MIMEMultipart from email.mime.text import MIMEText @@ -25,6 +25,7 @@ from email.utils import make_msgid import os from django.test import TestCase +from django.utils import six from patchwork.models import Comment from patchwork.models import Patch @@ -52,7 +53,12 @@ TEST_MAIL_DIR = os.path.join(os.path.dirname(__file__), 'mail') def read_mail(filename, project=None): """Read a mail from a file.""" file_path = os.path.join(TEST_MAIL_DIR, filename) - mail = message_from_file(open(file_path)) + if six.PY3: + with open(file_path, 'rb') as f: + mail = email.message_from_binary_file(f) + else: + with open(file_path) as f: + mail = email.message_from_file(f) if 'Message-Id' not in mail: mail['Message-Id'] = make_msgid() if project is not None: @@ -536,6 +542,35 @@ class PatchParseTest(PatchTest): # Confirm we got both markers self.assertEqual(2, diff.count('\ No newline at end of file')) +class MailParsingTest(TestCase): + def setUp(self): + self.project = create_project() + + def test_invalid_header_char(self): + """Validate behaviour when an invalid character is in a header.""" + mail = read_mail('0012-invalid-header-char.mbox', self.project) + parse_mail(mail, list_id=self.project.listid) + self.assertEqual(Patch.objects.all().count(), 1) + + def test_utf8_mail(self): + """Validate behaviour when a UTF-8 char is in a message.""" + mail = read_mail('0013-with-utf8-body.mbox') + parse_mail(mail, list_id=self.project.listid) + self.assertEqual(Patch.objects.all().count(), 1) + + def test_utf8_unencoded_headers(self): + """Validate behaviour when unencoded UTF-8 is in headers, + including subject and from.""" + mail = read_mail('0014-with-unencoded-utf8-headers.mbox') + parse_mail(mail, list_id=self.project.listid) + self.assertEqual(Patch.objects.all().count(), 1) + + def test_invalid_utf8_headers(self): + """Validate behaviour when invalid encoded UTF-8 is in headers.""" + mail = read_mail('0015-with-invalid-utf8-headers.mbox') + parse_mail(mail, list_id=self.project.listid) + self.assertEqual(Patch.objects.all().count(), 1) + class HeaderTest(TestCase): def test_header_preservation(self): -- 2.7.4
_______________________________________________ Patchwork mailing list [email protected] https://lists.ozlabs.org/listinfo/patchwork
