Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package urlwatch for openSUSE:Factory checked in at 2021-04-14 10:11:13 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/urlwatch (Old) and /work/SRC/openSUSE:Factory/.urlwatch.new.2401 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "urlwatch" Wed Apr 14 10:11:13 2021 rev:21 rq:884942 version:2.23 Changes: -------- --- /work/SRC/openSUSE:Factory/urlwatch/urlwatch.changes 2021-01-30 13:58:04.326427885 +0100 +++ /work/SRC/openSUSE:Factory/.urlwatch.new.2401/urlwatch.changes 2021-04-14 10:11:45.585572782 +0200 @@ -1,0 +2,12 @@ +Tue Apr 13 12:37:34 UTC 2021 - Michael Vetter <mvet...@suse.com> + +- Update to 2.23: + Added: + * New filter: pretty-xml to indent/pretty-print XML documents + * New filter: jq to parse, transform, and extract JSON data + * New reporter: prowl + Fixed: + * Proper multi-line highlighting for wdiff (PR#615) + * Fix command-line generation for html2text (PR#619) + +------------------------------------------------------------------- Old: ---- urlwatch-2.22.tar.gz New: ---- urlwatch-2.23.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ urlwatch.spec ++++++ --- /var/tmp/diff_new_pack.WY4upi/_old 2021-04-14 10:11:46.065573592 +0200 +++ /var/tmp/diff_new_pack.WY4upi/_new 2021-04-14 10:11:46.069573599 +0200 @@ -17,7 +17,7 @@ Name: urlwatch -Version: 2.22 +Version: 2.23 Release: 0 Summary: A tool for monitoring webpages for updates License: BSD-3-Clause ++++++ urlwatch-2.22.tar.gz -> urlwatch-2.23.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/.gitignore new/urlwatch-2.23/.gitignore --- old/urlwatch-2.22/.gitignore 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/.gitignore 2021-04-10 10:48:22.000000000 +0200 @@ -2,3 +2,6 @@ .idea build *.egg-info/ +venv/ +dist/ +*.egg \ No newline at end of file diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/.travis.yml new/urlwatch-2.23/.travis.yml --- old/urlwatch-2.22/.travis.yml 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/.travis.yml 2021-04-10 10:48:22.000000000 +0200 @@ -8,5 +8,5 @@ - pip install -r requirements.txt - pip install pycodestyle==2.6.0 pytest - sudo apt-get install -y build-essential libpoppler-cpp-dev pkg-config python-dev tesseract-ocr - - pip install pdftotext docutils pygments pytesseract pillow + - pip install pdftotext docutils pygments pytesseract pillow jq script: pytest -v diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/CHANGELOG.md new/urlwatch-2.23/CHANGELOG.md --- old/urlwatch-2.22/CHANGELOG.md 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/CHANGELOG.md 2021-04-10 10:48:22.000000000 +0200 @@ -4,6 +4,19 @@ The format mostly follows [Keep a Changelog](http://keepachangelog.com/en/1.0.0/). +## [2.23] -- 2021-04-10 + +### Added + +- New filter: `pretty-xml` to indent/pretty-print XML documents +- New filter: `jq` to parse, transform, and extract JSON data +- New reporter: `prowl` (by nitz) + +### Fixed + +- Proper multi-line highlighting for wdiff (PR#615, by kongomongo) +- Fix command-line generation for html2text (PR#619, by Eloy Paris) + ## [2.22] -- 2020-12-19 ### Added diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/COPYING new/urlwatch-2.23/COPYING --- old/urlwatch-2.22/COPYING 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/COPYING 2021-04-10 10:48:22.000000000 +0200 @@ -1,4 +1,4 @@ -Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> All rights reserved. Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/docs/source/conf.py new/urlwatch-2.23/docs/source/conf.py --- old/urlwatch-2.22/docs/source/conf.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/docs/source/conf.py 2021-04-10 10:48:22.000000000 +0200 @@ -18,11 +18,11 @@ # -- Project information ----------------------------------------------------- project = 'urlwatch' -copyright = '2020, Thomas Perl' +copyright = '2021, Thomas Perl' author = 'Thomas Perl' # The full version, including alpha/beta/rc tags -release = '2.22' +release = '2.23' # -- General configuration --------------------------------------------------- diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/docs/source/dependencies.rst new/urlwatch-2.23/docs/source/dependencies.rst --- old/urlwatch-2.22/docs/source/dependencies.rst 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/docs/source/dependencies.rst 2021-04-10 10:48:22.000000000 +0200 @@ -68,4 +68,5 @@ +-------------------------+---------------------------------------------------------------------+ | XMPP reporter |`aioxmpp <https://github.com/horazont/aioxmpp>`__ | +-------------------------+---------------------------------------------------------------------+ - +| `jq` filter | `jq <https://github.com/mwilliamson/jq.py>`__ | ++-------------------------+---------------------------------------------------------------------+ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/docs/source/deprecated.rst new/urlwatch-2.23/docs/source/deprecated.rst --- old/urlwatch-2.22/docs/source/deprecated.rst 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/docs/source/deprecated.rst 2021-04-10 10:48:22.000000000 +0200 @@ -5,7 +5,7 @@ update your configuration to use the replacements (if any). -Filters without subfilters (UNRELEASED) +Filters without subfilters (since 2.22) --------------------------------------- In older urlwatch versions, it was possible to write custom diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/docs/source/filters.rst new/urlwatch-2.23/docs/source/filters.rst --- old/urlwatch-2.22/docs/source/filters.rst 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/docs/source/filters.rst 2021-04-10 10:48:22.000000000 +0200 @@ -61,6 +61,7 @@ - **hexdump**: Convert binary data to hex dump format - **html2text**: Convert HTML to plaintext - **pdf2text**: Convert PDF to plaintext +- **pretty-xml**: Pretty-print XML - **ical2text**: Convert `iCalendar`_ to plaintext - **ocr**: Convert text in images to plaintext using Tesseract OCR - **re.sub**: Replace text with regular expressions using Python's re.sub @@ -70,6 +71,7 @@ - **sort**: Sort input items - **strip**: Strip leading and trailing whitespace - **xpath**: Filter XML/HTML using XPath expressions +- **jq**: Filter, transform and extract values from JSON .. To convert the "urlwatch --features" output, use: sed -e 's/^ \* \(.*\) - \(.*\)$/- **\1**: \2/' @@ -371,8 +373,8 @@ separator: "\n\n" -Watching Github releases ------------------------- +Watching Github releases and Gitlab tags +---------------------------------------- This is an example how to watch the GitHub ???releases??? page for a given project for the latest release version, to be notified of new releases: @@ -385,6 +387,15 @@ - html2text: re - strip +This is the corresponding version for Gitlab tags: + +.. code:: yaml + + url: https://gitlab.com/chinstrap/gammastep/-/tags + filter: + - xpath: (//a[contains(@class,"item-title ref-name")])[1] + - html2text + Remove or replace text using regular expressions ------------------------------------------------ @@ -526,7 +537,33 @@ language: eng - strip -The sub-filters ``timeout`` and ``language`` are optional: +The subfilters ``timeout`` and ``language`` are optional: * ``timeout``: Timeout for the recognition, in seconds (default: 10 seconds) * ``language``: Text language (e.g. ``fra`` or ``eng+fra``, default: ``eng``) + + +Filtering JSON response data using ``jq`` selectors +-------------------------------- + +The ``jq`` filter uses the Python bindings for `jq`_, a lightweight JSON processor. +Use of this filter requires the optional `jq Python module`_ to be installed. + +.. _jq: https://stedolan.github.io/jq/ +.. _jq Python module: https://github.com/mwilliamson/jq.py + +.. code-block:: yaml + + url: https://example.net/jobs.json + filter: + - jq: + query: '.[].title' + +The subfilter ``query`` is optional: + +* ``query``: A valid ``jq`` filter string. + +Supports aggregations, selections, and the built-in operators like ``length``. For +more information on the operations permitted, see the `jq Manual`_. + +.. _jq Manual: https://stedolan.github.io/jq/manual/ \ No newline at end of file diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/docs/source/reporters.rst new/urlwatch-2.23/docs/source/reporters.rst --- old/urlwatch-2.22/docs/source/reporters.rst 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/docs/source/reporters.rst 2021-04-10 10:48:22.000000000 +0200 @@ -55,6 +55,7 @@ - **telegram**: Send a message using Telegram - **ifttt**: Send summary via IFTTT - **xmpp**: Send a message using the XMPP Protocol +- **prowl**: Send a message via prowlapp.com .. To convert the "urlwatch --features" output, use: sed -e 's/^ \* \(.*\) - \(.*\)$/- **\1**: \2/' @@ -365,3 +366,37 @@ you can also set the ``insecure_password`` option in the XMPP config. For more information about the security implications, see :ref:`smtp-login-without-keyring`. + +Prowl +----- + +You can have notifications sent to you through the `Prowl` push +notification service, to recieve the notification on iOS. + +To achieve this, you should register a new Prowl account, and have +the Prowl application installed on your iOS device. + +To create an API key for urlwatch: + +1. Log into the Prowl website at https://prowlapp.com/ +2. Navigate to the ???API Keys??? tab. +3. Scroll to the ???Generate a new API key??? section. +4. Give the key a note that will remind you you've used it for urlwatch. +5. Press ???Generate Key??? +6. Copy the resulting key. + +Here is a sample configuration: + +.. code:: yaml + + prowl: + enabled: true + api_key: '<your api key here>' + priority: 2 + application: 'urlwatch example' + subject: '{count} changes: {jobs}' + +The ???subject" field is similar to the subject field in the email, and +will be used as the name of the Prowl event. The application is prepended +to the event and shown as the source of the event in the Prowl App. + diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/__init__.py new/urlwatch-2.23/lib/urlwatch/__init__.py --- old/urlwatch-2.22/lib/urlwatch/__init__.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/__init__.py 2021-04-10 10:48:22.000000000 +0200 @@ -8,9 +8,9 @@ pkgname = 'urlwatch' -__copyright__ = 'Copyright 2008-2020 Thomas Perl' +__copyright__ = 'Copyright 2008-2021 Thomas Perl' __author__ = 'Thomas Perl <m...@thp.io>' __license__ = 'BSD' __url__ = 'https://thp.io/2008/urlwatch/' -__version__ = '2.22' +__version__ = '2.23' __user_agent__ = '%s/%s (+https://thp.io/2008/urlwatch/info.html)' % (pkgname, __version__) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/browser.py new/urlwatch-2.23/lib/urlwatch/browser.py --- old/urlwatch-2.22/lib/urlwatch/browser.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/browser.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/cli.py new/urlwatch-2.23/lib/urlwatch/cli.py --- old/urlwatch-2.22/lib/urlwatch/cli.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/cli.py 2021-04-10 10:48:22.000000000 +0200 @@ -2,7 +2,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/command.py new/urlwatch-2.23/lib/urlwatch/command.py --- old/urlwatch-2.22/lib/urlwatch/command.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/command.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/config.py new/urlwatch-2.23/lib/urlwatch/config.py --- old/urlwatch-2.22/lib/urlwatch/config.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/config.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/filters.py new/urlwatch-2.23/lib/urlwatch/filters.py --- old/urlwatch-2.22/lib/urlwatch/filters.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/filters.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without @@ -44,6 +44,8 @@ from lxml import etree from lxml.cssselect import CSSSelector +from xml.dom import minidom + from .util import TrackSubClasses, import_module_from_source from .html2txt import html2text @@ -79,6 +81,11 @@ except ImportError: Image = None +try: + import jq +except ImportError: + jq = None + logger = logging.getLogger(__name__) @@ -376,6 +383,22 @@ return json.dumps(parsed_json, ensure_ascii=False, sort_keys=True, indent=indentation, separators=(',', ': ')) +class PrettyXMLFilter(FilterBase): + """Pretty-print XML""" + + __kind__ = 'pretty-xml' + + __supported_subfilters__ = { + 'indentation': 'Indentation level for pretty-printing', + } + + __default_subfilter__ = 'indentation' + + def filter(self, data, subfilter): + indentation = int(subfilter.get('indentation', 2)) + return minidom.parseString(data).toprettyxml(indent=' ' * indentation) + + class GrepFilter(FilterBase): """Filter only lines matching a regular expression""" @@ -855,3 +878,30 @@ raise ImportError('Please install Pillow/PIL') return pytesseract.image_to_string(Image.open(io.BytesIO(data)), lang=language, timeout=timeout) + + +class JQFilter(FilterBase): + """Parse, transform, and extract data from json as text using `jq`""" + + __kind__ = 'jq' + + __supported_subfilters__ = { + 'query': 'jq query function to execute on data', + } + + __default_subfilter__ = 'query' + + def filter(self, data, subfilter): + + try: + jsondata = json.loads(data) + except ValueError: + raise ValueError('The url response contained invalid JSON') + + if 'query' not in subfilter: + raise ValueError('{} filter needs a query'.format(self.__kind__)) + + if jq is None: + raise ImportError('Please install jq') + + return jq.text(subfilter['query'], jsondata) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/handler.py new/urlwatch-2.23/lib/urlwatch/handler.py --- old/urlwatch-2.22/lib/urlwatch/handler.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/handler.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/html2txt.py new/urlwatch-2.23/lib/urlwatch/html2txt.py --- old/urlwatch-2.22/lib/urlwatch/html2txt.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/html2txt.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without @@ -98,7 +98,7 @@ for k, v in options.items(): if v is not None: - cmd.append('-{} {}'.format(k, v)) + cmd.extend(['-{}'.format(k), '{}'.format(v)]) else: cmd.append('-{}'.format(k)) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/ical2txt.py new/urlwatch-2.23/lib/urlwatch/ical2txt.py --- old/urlwatch-2.22/lib/urlwatch/ical2txt.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/ical2txt.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/jobs.py new/urlwatch-2.23/lib/urlwatch/jobs.py --- old/urlwatch-2.22/lib/urlwatch/jobs.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/jobs.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/mailer.py new/urlwatch-2.23/lib/urlwatch/mailer.py --- old/urlwatch-2.22/lib/urlwatch/mailer.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/mailer.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/main.py new/urlwatch-2.23/lib/urlwatch/main.py --- old/urlwatch-2.22/lib/urlwatch/main.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/main.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/migration.py new/urlwatch-2.23/lib/urlwatch/migration.py --- old/urlwatch-2.22/lib/urlwatch/migration.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/migration.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/reporters.py new/urlwatch-2.23/lib/urlwatch/reporters.py --- old/urlwatch-2.22/lib/urlwatch/reporters.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/reporters.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,6 +1,6 @@ # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without @@ -88,6 +88,17 @@ self.job_states = job_states self.duration = duration + def get_signature(self): + return ( + '{pkgname} {version}, {copyright}'.format(pkgname=urlwatch.pkgname, + version=urlwatch.__version__, + copyright=urlwatch.__copyright__), + 'Website: {url}'.format(url=urlwatch.__url__), + 'Buy me a coffee: https://ko-fi.com/thpx86', + 'watched {count} URLs in {duration} seconds'.format(count=len(self.job_states), + duration=self.duration.seconds), + ) + def convert(self, othercls): if hasattr(othercls, '__kind__'): config = self.report.config['report'][othercls.__kind__] @@ -190,29 +201,23 @@ yield SafeHtml('<hr>') - yield SafeHtml(""" - <address> - {pkgname} {version}, {copyright}<br> - Website: {url}<br> - watched {count} URLs in {duration} seconds - </address> + yield SafeHtml('<address>') + for part in self.get_signature(): + yield SafeHtml('{}<br>').format(part) + yield SafeHtml("""</address> </body> </html> - """).format(pkgname=urlwatch.pkgname, version=urlwatch.__version__, copyright=urlwatch.__copyright__, - url=urlwatch.__url__, count=len(self.job_states), duration=self.duration.seconds) + """) def _diff_to_html(self, unified_diff): - for line in unified_diff.splitlines(): - if line.startswith('+'): - yield SafeHtml('<span class="unified_add">{line}</span>').format(line=line) - elif line.startswith('-'): - yield SafeHtml('<span class="unified_sub">{line}</span>').format(line=line) - else: - # Basic colorization for wdiff-style differences - line = SafeHtml('<span class="unified_nor">{line}</span>').format(line=line) - line = re.sub(WDIFF_ADDED_RE, lambda x: '<span class="diff_add">' + x.group(0) + '</span>', line) - line = re.sub(WDIFF_REMOVED_RE, lambda x: '<span class="diff_sub">' + x.group(0) + '</span>', line) - yield line + result = unified_diff + diff_mapping = {'+': 'unified_add', '-': 'unified_sub'} + + result = re.sub(r'^([-+]).*$', lambda x: '<span class="' + diff_mapping[x.group(1)] + '">' + x.group(0) + '</span>', result, flags=re.MULTILINE) + result = re.sub(WDIFF_ADDED_RE, lambda x: '<span class="diff_add">' + x.group(0) + '</span>', result, flags=re.MULTILINE + re.DOTALL) + result = re.sub(WDIFF_REMOVED_RE, lambda x: '<span class="diff_sub">' + x.group(0) + '</span>', result, flags=re.MULTILINE + re.DOTALL) + + return str(SafeHtml('<span class="unified_nor">' + result + '</span>')).splitlines() def _format_content(self, job_state, difftype): if job_state.verb == 'error': @@ -276,10 +281,8 @@ yield from details if summary and show_footer: - yield from ('-- ', - '%s %s, %s' % (urlwatch.pkgname, urlwatch.__version__, urlwatch.__copyright__), - 'Website: %s' % (urlwatch.__url__,), - 'watched %d URLs in %d seconds' % (len(self.job_states), self.duration.seconds)) + yield '-- ' + yield from self.get_signature() def _format_content(self, job_state): if job_state.verb == 'error': @@ -746,10 +749,7 @@ details.extend(details_part) if summary and show_footer: - footer = ('--- ', - '%s %s, %s ' % (urlwatch.pkgname, urlwatch.__version__, urlwatch.__copyright__), - 'Website: %s ' % (urlwatch.__url__,), - 'watched %d URLs in %d seconds' % (len(self.job_states), self.duration.seconds)) + footer = ('--- ',) + self.get_signature() else: footer = None @@ -978,3 +978,63 @@ for chunk in chunkstring(text, self.MAX_LENGTH, numbering=True): asyncio.run(xmpp.send(chunk)) + + +class ProwlReporter(TextReporter): + """Send a detailed notification via prowlapp.com""" + + __kind__ = 'prowl' + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + + def submit(self): + api_add = 'https://api.prowlapp.com/publicapi/add' + + text = '\n'.join(super().submit()) + + if not text: + logger.debug('Not calling Prowl API (no changes)') + return + + filtered_job_states = list(self.report.get_filtered_job_states(self.job_states)) + subject_args = { + 'count': len(filtered_job_states), + 'jobs': ', '.join(job_state.job.pretty_name() for job_state in filtered_job_states), + } + + # 'subject' used in the config file, but the API + # uses what might be called the subject as the 'event' + event = self.config['subject'].format(**subject_args) + + # 'application' is prepended to the message in prowl, + # to show the source of the notification. this too, + # is user configurable, and may reference subject args + application = self.config.get('application') + if application is not None: + application = application.format(**subject_args) + else: + application = '{0} v{1}'.format(urlwatch.pkgname, urlwatch.__version__) + + # build the data to post + post_data = { + 'event': event[:1024].encode('utf8'), + 'description': text[:10000].encode('utf8'), + 'application': application[:256].encode('utf8'), + 'apikey': self.config['api_key'], + 'priority': self.config['priority'] + } + + # all set up, add the notification! + result = requests.post(api_add, data=post_data) + + try: + if result.status_code in (requests.codes.ok, requests.codes.no_content): + logger.info("Prowl response: ok") + else: + logger.error("Prowl error: {0}".format(result.text)) + except ValueError: + logger.error("Failed to parse Prowl response. HTTP status code: {0}, content: {1}".format( + result.status_code, result.content)) + + return result diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/storage.py new/urlwatch-2.23/lib/urlwatch/storage.py --- old/urlwatch-2.22/lib/urlwatch/storage.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/storage.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without @@ -170,6 +170,13 @@ 'sender': '', 'recipient': '', }, + 'prowl': { + 'enabled': False, + 'api_key': '', + 'priority': 0, + 'application': '', + 'subject': '{count} changes: {jobs}' + }, }, 'job_defaults': { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/tests/data/filter_documentation_testdata.yaml new/urlwatch-2.23/lib/urlwatch/tests/data/filter_documentation_testdata.yaml --- old/urlwatch-2.22/lib/urlwatch/tests/data/filter_documentation_testdata.yaml 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/tests/data/filter_documentation_testdata.yaml 2021-04-10 10:48:22.000000000 +0200 @@ -262,6 +262,11 @@ </div> output: |- 2.18 +https://gitlab.com/chinstrap/gammastep/-/tags: + input: |- + <a class="item-title ref-name" href="/chinstrap/gammastep/-/tags/v2.0.7">v2.0.7</a> + output: |- + v2.0.7 https://example.com/regex-substitute.html: input: |- <div> @@ -347,3 +352,23 @@ <div class="cpu">Pentium</div> <div class="cpu">Pentium MMX</div> +https://example.net/jobs.json: + input: | + [ + { + "title": "Senior Data Scientist", + "company": "Boogle" + }, + { + "title": "Python Developer", + "company": "Tack-trest" + }, + { + "title": "TPS Report Analyst", + "company": "Initech" + } + ] + output: | + "Senior Data Scientist" + "Python Developer" + "TPS Report Analyst" \ No newline at end of file diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/util.py new/urlwatch-2.23/lib/urlwatch/util.py --- old/urlwatch-2.22/lib/urlwatch/util.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/util.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/worker.py new/urlwatch-2.23/lib/urlwatch/worker.py --- old/urlwatch-2.22/lib/urlwatch/worker.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/worker.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/lib/urlwatch/xmpp.py new/urlwatch-2.23/lib/urlwatch/xmpp.py --- old/urlwatch-2.22/lib/urlwatch/xmpp.py 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/lib/urlwatch/xmpp.py 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of urlwatch (https://thp.io/2008/urlwatch/). -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlwatch-2.22/share/urlwatch/examples/hooks.py.example new/urlwatch-2.23/share/urlwatch/examples/hooks.py.example --- old/urlwatch-2.22/share/urlwatch/examples/hooks.py.example 2020-12-19 12:27:43.000000000 +0100 +++ new/urlwatch-2.23/share/urlwatch/examples/hooks.py.example 2021-04-10 10:48:22.000000000 +0200 @@ -1,7 +1,7 @@ # # Example hooks file for urlwatch # -# Copyright (c) 2008-2020 Thomas Perl <m...@thp.io> +# Copyright (c) 2008-2021 Thomas Perl <m...@thp.io> # All rights reserved. # # Redistribution and use in source and binary forms, with or without
