At present, the 'parsearchive' command only supports parsing of mboxes. Expand this to support maildirs. This allows us to rewrite the 'parsemail-bulk' script to deliver much improved performance.
Signed-off-by: Stephen Finucane <[email protected]> Suggested-by: Daniel Axtens <[email protected]> --- This is an alternative to [1] that avoids us having to add yet another management command. [1] https://patchwork.ozlabs.org/patch/731892/ --- patchwork/bin/parsemail-batch.sh | 27 +++++++++------------------ patchwork/management/commands/parsearchive.py | 7 ++++++- 2 files changed, 15 insertions(+), 19 deletions(-) diff --git a/patchwork/bin/parsemail-batch.sh b/patchwork/bin/parsemail-batch.sh index d42712e..3d3725c 100755 --- a/patchwork/bin/parsemail-batch.sh +++ b/patchwork/bin/parsemail-batch.sh @@ -1,7 +1,7 @@ #!/bin/sh # # Patchwork - automated patch tracking system -# Copyright (C) 2008 Jeremy Kerr <[email protected]> +# Copyright (C) 2017 Stephen Finucane <[email protected]> # # This file is part of the Patchwork package. # @@ -20,25 +20,16 @@ # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA BIN_DIR=$(dirname "$0") +PATCHWORK_BASE=$(readlink -e "$BIN_DIR/../..") -if [ $# -lt 1 ]; then - echo "usage: $0 <dir> [options]" >&2 - exit 1 +if [ -z "$PW_PYTHON" ]; then + PW_PYTHON=python2 fi -mail_dir="$1" - -echo "dir: $mail_dir" - -if [ ! -d "$mail_dir" ]; then - echo "$mail_dir should be a directory"? >&2 - exit 1 +if [ -z "$DJANGO_SETTINGS_MODULE" ]; then + DJANGO_SETTINGS_MODULE=patchwork.settings.production fi -shift - -find "$mail_dir" -maxdepth 1 | -while read -r line; do - echo "$line" - "$BIN_DIR/parsemail.sh" "$@" < "$line" -done +PYTHONPATH="${PATCHWORK_BASE}:${PATCHWORK_BASE}/lib/python:$PYTHONPATH" \ + DJANGO_SETTINGS_MODULE="$DJANGO_SETTINGS_MODULE" \ + "$PW_PYTHON" "$PATCHWORK_BASE/manage.py" parsearchive "$@" diff --git a/patchwork/management/commands/parsearchive.py b/patchwork/management/commands/parsearchive.py index 40b2cc0..a3c8360 100644 --- a/patchwork/management/commands/parsearchive.py +++ b/patchwork/management/commands/parsearchive.py @@ -69,7 +69,12 @@ class Command(BaseCommand): self.stdout.write('Invalid path: %s' % path) sys.exit(1) - mbox = mailbox.mbox(path) + # assume if <infile> is a directory, then we're passing a maildir + if os.path.isfile(path): + mbox = mailbox.mbox(path) + else: + mbox = mailbox.Maildir(path) + count = len(mbox) logger.info('Parsing %d mails', count) -- 2.9.3 _______________________________________________ Patchwork mailing list [email protected] https://lists.ozlabs.org/listinfo/patchwork
