[PATCH 1/3] REST: Integrate django-filter support

[Stephen Finucane]

This mostly works out of the box, thanks to Django REST Framework.
Mostly unique fields, like name or email, are excluded as these will be
handled separately.

Signed-off-by: Stephen Finucane 
---
 patchwork/api/check.py |  2 ++
 patchwork/api/cover.py |  2 ++
 patchwork/api/filters.py   | 61 ++
 patchwork/api/patch.py |  2 ++
 patchwork/api/series.py|  3 ++-
 patchwork/settings/base.py |  6 -
 requirements-test.txt  |  2 +-
 7 files changed, 75 insertions(+), 3 deletions(-)
 create mode 100644 patchwork/api/filters.py

diff --git a/patchwork/api/check.py b/patchwork/api/check.py
index 43463fe..9a5c22a 100644
--- a/patchwork/api/check.py
+++ b/patchwork/api/check.py
@@ -26,6 +26,7 @@ from rest_framework.serializers import HiddenField
 from rest_framework.serializers import HyperlinkedModelSerializer
 from rest_framework.serializers import HyperlinkedIdentityField
 
+from patchwork.api.filters import CheckFilter
 from patchwork.api import MultipleFieldLookupMixin
 from patchwork.models import Check
 from patchwork.models import Patch
@@ -89,6 +90,7 @@ class CheckMixin(object):
 
 queryset = Check.objects.prefetch_related('patch', 'user')
 serializer_class = CheckSerializer
+filter_class = CheckFilter
 
 
 class CheckListCreate(CheckMixin, ListCreateAPIView):
diff --git a/patchwork/api/cover.py b/patchwork/api/cover.py
index 2674be3..153176c 100644
--- a/patchwork/api/cover.py
+++ b/patchwork/api/cover.py
@@ -25,6 +25,7 @@ from rest_framework.serializers import 
HyperlinkedModelSerializer
 from rest_framework.serializers import HyperlinkedRelatedField
 from rest_framework.serializers import SerializerMethodField
 
+from patchwork.api.filters import CoverLetterFilter
 from patchwork.models import CoverLetter
 
 
@@ -67,6 +68,7 @@ class CoverLetterList(ListAPIView):
 queryset = CoverLetter.objects.all().prefetch_related(
 'series').select_related('submitter').defer('content', 'headers')
 serializer_class = CoverLetterListSerializer
+filter_class = CoverLetterFilter
 
 
 class CoverLetterDetail(RetrieveAPIView):
diff --git a/patchwork/api/filters.py b/patchwork/api/filters.py
new file mode 100644
index 000..a07f464
--- /dev/null
+++ b/patchwork/api/filters.py
@@ -0,0 +1,61 @@
+# Patchwork - automated patch tracking system
+# Copyright (C) 2016 Stephen Finucane 
+#
+# This file is part of the Patchwork package.
+#
+# Patchwork is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# Patchwork is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Patchwork; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+
+from django_filters import FilterSet
+from django_filters import IsoDateTimeFilter
+
+from patchwork.models import Check
+from patchwork.models import CoverLetter
+from patchwork.models import Patch
+from patchwork.models import Series
+
+
+class TimestampMixin(object):
+
+# TODO(stephenfin): These should filter on a 'updated_at' field instead
+before = IsoDateTimeFilter(name='date', lookup_expr='lt')
+since = IsoDateTimeFilter(name='date', lookup_expr='gte')
+
+
+class SeriesFilter(TimestampMixin, FilterSet):
+
+class Meta:
+model = Series
+fields = ['submitter']
+
+
+class CoverLetterFilter(TimestampMixin, FilterSet):
+
+class Meta:
+model = CoverLetter
+fields = ['series', 'submitter']
+
+
+class PatchFilter(FilterSet):
+
+class Meta:
+model = Patch
+fields = ['series', 'submitter', 'delegate', 'state', 'archived']
+
+
+class CheckFilter(TimestampMixin, FilterSet):
+
+class Meta:
+model = Check
+fields = ['user', 'state', 'context']
diff --git a/patchwork/api/patch.py b/patchwork/api/patch.py
index 8d308e8..c1c5030 100644
--- a/patchwork/api/patch.py
+++ b/patchwork/api/patch.py
@@ -27,6 +27,7 @@ from rest_framework.serializers import ChoiceField
 from rest_framework.serializers import HyperlinkedModelSerializer
 from rest_framework.serializers import SerializerMethodField
 
+from patchwork.api.filters import PatchFilter
 from patchwork.api import PatchworkPermission
 from patchwork.api import STATE_CHOICES
 from patchwork.models import Patch
@@ -118,6 +119,7 @@ class PatchList(ListAPIView):
 'content', 'diff', 'headers')
 permission_classes = (PatchworkPermission,)
 serializer_class = PatchListSerializer
+filter_class = PatchFilter
 
 
 class 

Re: [PATCH v2] Fix parsing of interesing series reply structures

[Stephen Fincane]

On Thu, 2016-11-17 at 16:11 +1100, Daniel Axtens wrote:
> There are some things you probably shouldn't do on public
> mailing lists, but which people do anyway.
> 
> The first, and most understandable, is this:
> 
>   - [PATCH 1/2] test: Add some lorem ipsum
> - [PATCH 2/2] test: Convert to Markdown
>   - [PATCH v2 1/2] test: Add some lorem ipsum
> - [PATCH v2 2/2] test: Convert to Markdown
> 
> We should correctly parse these by:
>  - creating a new series if the version number changes
>  - when deciding whether to create a SeriesReference, search by
>    message-id alone, not the message-id/series pair. (Otherwise,
>    we try to create a series ref for v1 2/2 in the series for v2,
>    which breaks a uniqueness constraint.
> 
> The second, and less excusable, is this:
> 
>   - [PATCH 1/2] test: Add some lorem ipsum
> - [PATCH 2/2] test: Convert to Markdown
>   - [PATCH 1/2] test: Add some lorem ipsum
> - [PATCH 2/2] test: Convert to Markdown
> 
> With this patch:
>  - if we get a x/n for a series that already has an x/n, create a
>    new series for it.
> 
> Signed-off-by: Daniel Axtens 

I've got two suggestions for improvement that I'd like to make when
merging, but they're non-blocking. Lovely, tidy fixes.

Reviewed-by: Stephen Finucane 

> ---
> 
> v2: - drop a hunk that I created during debugging but which was no
>   longer required.
> - better support for the second scenario - I had mistakenly
> thought
>   it wouldn't work because of an earier typo.
> ---
>  patchwork/parser.py|  22 +-
>  patchwork/tests/series/bugs-nocover-noversion.mbox | 222
> +
>  patchwork/tests/series/bugs-nocover.mbox   | 222
> +
>  patchwork/tests/test_series.py |  43 
>  4 files changed, 507 insertions(+), 2 deletions(-)
>  create mode 100644 patchwork/tests/series/bugs-nocover-
> noversion.mbox
>  create mode 100644 patchwork/tests/series/bugs-nocover.mbox
> 
> diff --git a/patchwork/parser.py b/patchwork/parser.py
> index b1544c951a3f..c7890ea18c2d 100644
> --- a/patchwork/parser.py
> +++ b/patchwork/parser.py
> @@ -41,6 +41,7 @@ from patchwork.models import Person
>  from patchwork.models import Project
>  from patchwork.models import Series
>  from patchwork.models import SeriesReference
> +from patchwork.models import SeriesPatch
>  from patchwork.models import State
>  from patchwork.models import Submission
>  
> @@ -789,7 +790,16 @@ def parse_mail(mail, list_id=None):
>  delegate = auto_delegate(project, filenames)
>  
>  series = find_series(mail)
> -if not series and n:  # the series markers indicates a
> series
> +# We will create a new series if:
> +# - we have a patch number (x of n), and
> +# - either:
> +#* there is no series, or
> +#* the version doesn't match
> +#* we have a patch with this number already
> +if n and ((not series) or
> +  (series.version != version) or
> +  (SeriesPatch.objects.filter(series=series,
> number=x).count()

Instead of importing SeriesPatch, could we do:

    series.patches.filter(number=x).exists()

Seems a little cleaner?

> +   )):
>  series = Series(date=date,
>  submitter=author,
>  version=version,
> @@ -803,7 +813,15 @@ def parse_mail(mail, list_id=None):
>  # as the earlier patch does not reference the later one.
>  for ref in refs + [msgid]:
>  # we don't want duplicates
> -SeriesReference.objects.get_or_create(series=series,
> msgid=ref)
> +try:
> +# we could have a ref to a previous series. (For
> +# example, a series sent in reply to another
> +# series.) That should not create a series ref
> +# for this series, so check for the msg-id only,
> +# not the msg-id/series pair.
> +SeriesReference.objects.get(msgid=ref)
> +except SeriesReference.DoesNotExist:
> +SeriesReference.objects.create(series=series,
> msgid=ref)

I imagine that as soon as we meet a conflicting SeriesReference we
could (should?) exit. Python has a fancy try-except-else syntax for
exactly this use case:

    try:
        something
    except Exception:
        do something
    else:
        or something else
___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork

Re: [PATCH] Fix parsing of un-numbered messages in series

[Stephen Fincane]

On Wed, 2016-11-16 at 16:58 +1100, Daniel Axtens wrote:
> Say we are sent the following:
> 
>   - [PATCH 0/2] A sample series
> - [PATCH 1/2] test: Add some lorem ipsum
> - Random message with diff
> 
> We expect that:
>  1) we parse normally without errors
>  2) we get a series with a cover letter and a patch
>  3) the random message is orphaned
> 
> What happens is that we get an integrity error, boiling down to:
> 
> (1048, "Column 'number' cannot be null")
> 
> This is caused because we believe that the random message belongs
> to the series because of the headers, but because there are no
> numbers in the Subject, we pass "None" into the number field of
> SeriesPatch. That turns into a null, and rightly hits an integrity
> error.
> 
> Fix this by requring that a message has a series _and_ a number
> before we try to add it to the series.
> 
> Add a test to verify correctness.
> 
> Reported-by: Daniel Wagner 
> Signed-off-by: Daniel Axtens 

Looks good to me. Appreciate the sanity check test.

Reviewed-by: Stephen Finucane 

...and applied. 
___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork

[PATCH 2/5] management: Enable configurable logging levels

[Stephen Finucane]

If we're hashing patches we don't want to out logging information. This
requires configurable log levels.

Signed-off-by: Stephen Finucane 
Cc: Paul Jakma 
---
 patchwork/management/commands/__init__.py | 34 +++
 patchwork/management/commands/parsearchive.py |  8 ++-
 patchwork/management/commands/parsemail.py|  4 
 patchwork/settings/base.py|  2 +-
 4 files changed, 46 insertions(+), 2 deletions(-)

diff --git a/patchwork/management/commands/__init__.py 
b/patchwork/management/commands/__init__.py
index e69de29..fa9c174 100644
--- a/patchwork/management/commands/__init__.py
+++ b/patchwork/management/commands/__init__.py
@@ -0,0 +1,34 @@
+# Patchwork - automated patch tracking system
+# Copyright (C) 2016 Stephen Finucane 
+#
+# This file is part of the Patchwork package.
+#
+# Patchwork is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# Patchwork is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Patchwork; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+
+import logging
+
+
+def configure_logging(loggers, verbosity):
+def set_loggers(loggers, level):
+for logger in loggers:
+logging.getLogger(logger).setLevel(level)
+
+# rely on standard configuration for verbosity 1 (normal)
+if verbosity == 0:
+set_loggers(loggers, logging.ERROR)
+elif verbosity == 1:
+set_loggers(loggers, logging.INFO)
+elif verbosity >= 2:
+set_loggers(loggers, logging.DEBUG)
diff --git a/patchwork/management/commands/parsearchive.py 
b/patchwork/management/commands/parsearchive.py
index 40b2cc0..5f3a11a 100644
--- a/patchwork/management/commands/parsearchive.py
+++ b/patchwork/management/commands/parsearchive.py
@@ -26,6 +26,7 @@ import sys
 import django
 from django.core.management.base import BaseCommand
 
+from patchwork.management.commands import configure_logging
 from patchwork import models
 from patchwork.parser import parse_mail
 
@@ -63,6 +64,9 @@ class Command(BaseCommand):
 dropped = 0
 errors = 0
 
+verbosity = int(options['verbosity'])
+configure_logging(['patchwork.parser', __name__], verbosity)
+
 # TODO(stephenfin): Support passing via stdin
 path = args and args[0] or options['infile']
 if not os.path.exists(path):
@@ -87,7 +91,9 @@ class Command(BaseCommand):
 # somewhere for future reference?
 errors += 1
 
-if (i % 10) == 0:
+# we don't output this when using higher verbosities - logging
+# provides enough information
+if verbosity < 2 and (i % 10) == 0:
 self.stdout.write('%06d/%06d\r' % (i, count), ending='')
 self.stdout.flush()
 
diff --git a/patchwork/management/commands/parsemail.py 
b/patchwork/management/commands/parsemail.py
index 9adfb25..fd85763 100644
--- a/patchwork/management/commands/parsemail.py
+++ b/patchwork/management/commands/parsemail.py
@@ -27,6 +27,7 @@ from django.core.management import base
 from django.utils import six
 
 from patchwork.parser import parse_mail
+from patchwork.management.commands import configure_logging
 
 logger = logging.getLogger(__name__)
 
@@ -58,6 +59,9 @@ class Command(base.BaseCommand):
 def handle(self, *args, **options):
 infile = args[0] if args else options['infile']
 
+configure_logging(['patchwork.parser', __name__],
+  int(options['verbosity']))
+
 if infile:
 logger.info('Parsing mail loaded by filename')
 if six.PY3:
diff --git a/patchwork/settings/base.py b/patchwork/settings/base.py
index a32710f..d2b996b 100644
--- a/patchwork/settings/base.py
+++ b/patchwork/settings/base.py
@@ -185,7 +185,7 @@ LOGGING = {
 },
 'patchwork.parser': {
 'handlers': ['console'],
-'level': 'DEBUG',
+'level': 'INFO',
 'propagate': False,
 },
 'patchwork.management.commands': {
-- 
2.7.4

___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork

[PATCH 5/5] tools: Update to use 'parsemail --hash'

[Stephen Finucane]

This replaces the older 'parser' main function.

Signed-off-by: Stephen Finucane 
Cc: Paul Jakma 
---
 tools/patchwork-update-commits | 18 ++
 tools/post-receive.hook| 21 +++--
 2 files changed, 29 insertions(+), 10 deletions(-)

diff --git a/tools/patchwork-update-commits b/tools/patchwork-update-commits
index d0f63a9..c5e8b15 100755
--- a/tools/patchwork-update-commits
+++ b/tools/patchwork-update-commits
@@ -19,8 +19,16 @@
 # along with Patchwork; if not, write to the Free Software
 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 
-toolsdir="$(dirname "$0")"
-pwpath="${toolsdir}"/../patchwork
+BASE_DIR=`dirname $0`
+PW_DIR=`readlink -e $BASE_DIR/..`
+
+if [ -z $PW_PYTHON ]; then
+PW_PYTHON=python2
+fi
+
+if [ -z $DJANGO_SETTINGS_MODULE ]; then
+DJANGO_SETTINGS_MODULE=patchwork.settings.production
+fi
 
 if [ "$#" -lt 1 ]; then
 echo "usage: $0 " >&2
@@ -29,6 +37,8 @@ fi
 
 git rev-list --reverse "$@" |
 while read commit; do
-hash=$(git show "$commit" | python $pwpath/parser.py -#)
-$pwpath/bin/pwclient update -s Accepted -c "$commit" -h "$hash"
+hash=$(git show "$commit" | \
+   DJANGO_SETTINGS_MODULE="$DJANGO_SETTINGS_MODULE" \
+   $PW_PYTHON "$PW_DIR/manage.py" parsemail --hash --verbosity=0)
+$PW_DIR/patchwork/bin/pwclient update -s Accepted -c "$commit" -h "$hash"
 done
diff --git a/tools/post-receive.hook b/tools/post-receive.hook
index a19e1b2..aba7b8a 100755
--- a/tools/post-receive.hook
+++ b/tools/post-receive.hook
@@ -30,29 +30,38 @@ STATE_MAP="refs/heads/master:Accepted"
 #
 EXCLUDE=""
 
-PWDIR=/opt/patchwork/patchwork
+PW_DIR=/opt/patchwork
 
 do_exit=0
 trap "do_exit=1" INT
 
-get_patchwork_hash()
-{
+if [ -z $PW_PYTHON ]; then
+PW_PYTHON=python2
+fi
+
+if [ -z $DJANGO_SETTINGS_MODULE ]; then
+DJANGO_SETTINGS_MODULE=patchwork.settings.production
+fi
+
+get_patchwork_hash() {
 local hash
-hash=$(git show -C $1 | python $PWDIR/parser.py --hash)
+hash=$(git show -C $1 | \
+   DJANGO_SETTINGS_MODULE="$DJANGO_SETTINGS_MODULE" \
+   $PW_PYTHON "$PW_DIR/manage.py" parsemail --hash --verbosity 0)
 echo $hash
 test -n "$hash"
 }
 
 get_patch_id() {
 local id
-id=$($PWDIR/bin/pwclient info -h $1 2>/dev/null \
+id=$($PW_DIR/patchwork/bin/pwclient info -h $1 2>/dev/null \
  | sed -rne 's,- id[[:space:]]*: ,,p')
 echo $id
 test -n "$id"
 }
 
 set_patch_state() {
-$PWDIR/bin/pwclient update -s $2 -c $3 $1 2>&1
+$PW_DIR/patchwork/bin/pwclient update -s $2 -c $3 $1 2>&1
 }
 
 update_patches() {
-- 
2.7.4

___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork

[PATCH 3/5] parsemail: Add '--hash' argument

[Stephen Finucane]

This allows us to replace the functionality previously provided by the
'parser' executable.

Signed-off-by: Stephen Finucane 
Cc: Paul Jakma 
---
 patchwork/management/commands/parsemail.py | 14 ++
 patchwork/parser.py| 17 +
 patchwork/tests/test_management.py |  9 +
 3 files changed, 40 insertions(+)

diff --git a/patchwork/management/commands/parsemail.py 
b/patchwork/management/commands/parsemail.py
index fd85763..02a84e7 100644
--- a/patchwork/management/commands/parsemail.py
+++ b/patchwork/management/commands/parsemail.py
@@ -26,6 +26,7 @@ import django
 from django.core.management import base
 from django.utils import six
 
+from patchwork.parser import hash_mail
 from patchwork.parser import parse_mail
 from patchwork.management.commands import configure_logging
 
@@ -42,6 +43,10 @@ class Command(base.BaseCommand):
 '--list-id',
 help='mailing list ID. If not supplied, this will be '
 'extracted from the mail headers.'),
+make_option(
+'--hash',
+action='store_true',
+help='show hash for parsed patch without storing it.'),
 )
 else:
 def add_arguments(self, parser):
@@ -55,6 +60,10 @@ class Command(base.BaseCommand):
 '--list-id',
 help='mailing list ID. If not supplied, this will be '
 'extracted from the mail headers.')
+parser.add_argument(
+'--hash',
+action='store_true',
+help='show hash for parsed patch without storing it.')
 
 def handle(self, *args, **options):
 infile = args[0] if args else options['infile']
@@ -76,6 +85,11 @@ class Command(base.BaseCommand):
 mail = email.message_from_binary_file(sys.stdin.buffer)
 else:
 mail = email.message_from_file(sys.stdin)
+
+if options['hash']:
+self.stdout.write(hash_mail(mail))
+sys.exit(0)
+
 try:
 result = parse_mail(mail, options['list_id'])
 if result:
diff --git a/patchwork/parser.py b/patchwork/parser.py
index fd7ec82..0432b52 100644
--- a/patchwork/parser.py
+++ b/patchwork/parser.py
@@ -907,6 +907,23 @@ def parse_mail(mail, list_id=None):
 return comment
 
 
+def hash_mail(mail):
+"""Parse a mail and generate the hash for it.
+
+Args:
+mail (`mbox.Mail`): Mail to parse and add.
+
+Returns:
+None
+"""
+diff, content = find_content(mail)
+
+if not diff:
+return  # nothing to work with
+
+return Patch.hash_diff(diff).hexdigest()
+
+
 def find_filenames(diff):
 """Find files changes in a given diff."""
 # normalise spaces
diff --git a/patchwork/tests/test_management.py 
b/patchwork/tests/test_management.py
index f916673..d61c5b1 100644
--- a/patchwork/tests/test_management.py
+++ b/patchwork/tests/test_management.py
@@ -112,6 +112,15 @@ class ParsemailTest(TestCase):
 count = models.Patch.objects.filter(project=project.id).count()
 self.assertEqual(count, 1)
 
+def test_hash(self):
+out = StringIO()
+path = os.path.join(TEST_MAIL_DIR, '0013-with-utf8-body.mbox')
+with self.assertRaises(SystemExit) as exc:
+call_command('parsemail', infile=path, hash=True, stdout=out)
+
+self.assertEqual(exc.exception.code, 0)
+self.assertIn('4df3c62e1', out.getvalue())
+
 
 class ParsearchiveTest(TestCase):
 def test_invalid_path(self):
-- 
2.7.4

___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork

[PATCH 0/5] Resolve issues with 'tools' scripts

[Stephen Finucane]

There are two scripts provided in tools that reference the removed
hash function of the 'parser' script. Update these to use one of the
management commands, with some cleanup along the way.

Stephen Finucane (5):
  parser: Remove unused parameter
  management: Enable configurable logging levels
  parsemail: Add '--hash' argument
  tools: Trivial formatting fixes
  tools: Update to use 'parsemail --hash'

 patchwork/management/commands/__init__.py |  34 +++
 patchwork/management/commands/parsearchive.py |   8 +-
 patchwork/management/commands/parsemail.py|  18 
 patchwork/parser.py   |  21 -
 patchwork/settings/base.py|   2 +-
 patchwork/tests/test_management.py|   9 ++
 patchwork/tests/test_parser.py|  31 ++
 tools/patchwork-update-commits|  47 +++--
 tools/post-receive.hook   | 131 --
 9 files changed, 216 insertions(+), 85 deletions(-)

-- 
2.7.4

___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork

[PATCH 1/5] parser: Remove unused parameter

[Stephen Finucane]

The find_content function expected a 'project' parameter but never
actually used it. Remove it and clean up tests accordingly.

Signed-off-by: Stephen Finucane 
Cc: Paul Jakma 
---
 patchwork/parser.py|  4 ++--
 patchwork/tests/test_parser.py | 31 ++-
 2 files changed, 12 insertions(+), 23 deletions(-)

diff --git a/patchwork/parser.py b/patchwork/parser.py
index a4036ff..fd7ec82 100644
--- a/patchwork/parser.py
+++ b/patchwork/parser.py
@@ -341,7 +341,7 @@ def parse_version(subject, subject_prefixes):
 return 1
 
 
-def find_content(project, mail):
+def find_content(mail):
 """Extract a comment and potential diff from a mail."""
 patchbuf = None
 commentbuf = ''
@@ -760,7 +760,7 @@ def parse_mail(mail, list_id=None):
 
 # parse content
 
-diff, message = find_content(project, mail)
+diff, message = find_content(mail)
 
 if not (diff or message):
 return  # nothing to work with
diff --git a/patchwork/tests/test_parser.py b/patchwork/tests/test_parser.py
index 96166ad..e4a379d 100644
--- a/patchwork/tests/test_parser.py
+++ b/patchwork/tests/test_parser.py
@@ -62,7 +62,7 @@ def read_mail(filename, project=None):
 mail = email.message_from_file(f)
 if 'Message-Id' not in mail:
 mail['Message-Id'] = make_msgid()
-if project is not None:
+if project:
 mail['List-Id'] = project.listid
 return mail
 
@@ -89,12 +89,9 @@ def parse_mail(*args, **kwargs):
 
 class PatchTest(TestCase):
 
-def setUp(self):
-self.project = create_project()
-
 def _find_content(self, mbox_filename):
-mail = read_mail(mbox_filename, project=self.project)
-diff, message = find_content(self.project, mail)
+mail = read_mail(mbox_filename)
+diff, message = find_content(mail)
 
 return diff, message
 
@@ -106,9 +103,7 @@ class InlinePatchTest(PatchTest):
 
 def setUp(self):
 email = create_email(self.orig_content + '\n' + self.orig_diff)
-
-self.project = create_project()
-self.diff, self.content = find_content(self.project, email)
+self.diff, self.content = find_content(email)
 
 def test_patch_content(self):
 self.assertEqual(self.diff, self.orig_diff)
@@ -130,8 +125,7 @@ class AttachmentPatchTest(InlinePatchTest):
 msg.attach(attachment)
 email = _create_email(msg)
 
-self.project = create_project()
-self.diff, self.content = find_content(self.project, email)
+self.diff, self.content = find_content(email)
 
 
 class AttachmentXDiffPatchTest(AttachmentPatchTest):
@@ -148,8 +142,7 @@ class UTF8InlinePatchTest(InlinePatchTest):
_charset='utf-8')
 email = _create_email(msg)
 
-self.project = create_project()
-self.diff, self.content = find_content(self.project, email)
+self.diff, self.content = find_content(email)
 
 
 class NoCharsetInlinePatchTest(InlinePatchTest):
@@ -160,8 +153,7 @@ class NoCharsetInlinePatchTest(InlinePatchTest):
 del email['Content-Type']
 del email['Content-Transfer-Encoding']
 
-self.project = create_project()
-self.diff, self.content = find_content(self.project, email)
+self.diff, self.content = find_content(email)
 
 
 class SignatureCommentTest(InlinePatchTest):
@@ -172,8 +164,7 @@ class SignatureCommentTest(InlinePatchTest):
 email = create_email(self.orig_content + '\n-- \nsig\n' +
  self.orig_diff)
 
-self.project = create_project()
-self.diff, self.content = find_content(self.project, email)
+self.diff, self.content = find_content(email)
 
 
 class UpdateSigCommentTest(SignatureCommentTest):
@@ -194,8 +185,7 @@ class ListFooterTest(InlinePatchTest):
 'Linuxppc-dev mailing list',
 self.orig_diff]))
 
-self.project = create_project()
-self.diff, self.content = find_content(self.project, email)
+self.diff, self.content = find_content(email)
 
 
 class DiffWordInCommentTest(InlinePatchTest):
@@ -571,8 +561,7 @@ class PatchParseTest(PatchTest):
 self.assertEqual(diff.count("\nrename to "), 2)
 
 def test_git_rename_with_diff(self):
-diff, message = self._find_content(
-'0009-git-rename-with-diff.mbox')
+diff, message = self._find_content('0009-git-rename-with-diff.mbox')
 self.assertTrue(diff is not None)
 self.assertTrue(message is not None)
 self.assertEqual(diff.count("\nrename from "), 2)
-- 
2.7.4

___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork

Re: patchwork "parser: Remove 'main' function"

[Stephen Fincane]

On Thu, 2016-11-17 at 19:30 +, Paul Jakma wrote:

CCing the list.

> Hi Stephen,
> 
> I was using trying to setup patchwork, using latest git. I ran into
> an 
> issue following the instructions to use the scripts in tools/ to
> update 
> the status of existing commits and new commits. The 
> tools/{patchwork-update-commits,ost-receive.hook} scripts both use 
> parser.py to calculate the patch hash:
> 
> $ grep parser.py tools/*
> grep: tools/docker: Is a directory
> tools/patchwork-update-commits: hash=$(git show "$commit" | python 
> $pwpath/parser.py -#)
> tools/post-receive.hook:  hash=$(git show -C $1 | python 
> $PWDIR/parser.py --hash)
> 
> Except of course, the main function is now gone and parser.py doesn't
> do 
> anything anymore of itself. Also, the next commit removes the
> hash_diff 
> functions and stuff.
> 
> I don't see an equivalent command added for manage.py, that could
> obtain 
> the hash, for these scripts to work?
> 
> Any ideas on what to do there?

Ohhh, I never saw those calls. That's a bug. I'll add the missing
functionality back into the parsemail script post haste.

> Reverting those 2 commits, and some tweaking (e.g. I needed to add 
> 'import django\ndjango.setup()", and also got a "AttributeError:
> 'str' 
> object has no attribute 'decode'" error cause of my python version -
> i 
> just removed the '.decode') sort of got it working it seems.

Let me know if this is still an issue once you apply the fix(es).

Stephen
___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork