Hello community, here is the log from the commit of package urlscan for openSUSE:Factory checked in at 2020-03-20 23:58:39 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/urlscan (Old) and /work/SRC/openSUSE:Factory/.urlscan.new.3160 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "urlscan" Fri Mar 20 23:58:39 2020 rev:6 rq:786748 version:0.9.4 Changes: -------- --- /work/SRC/openSUSE:Factory/urlscan/urlscan.changes 2019-02-08 12:13:45.873461149 +0100 +++ /work/SRC/openSUSE:Factory/.urlscan.new.3160/urlscan.changes 2020-03-21 00:02:31.449136276 +0100 @@ -1,0 +2,15 @@ +Fri Mar 13 06:10:44 UTC 2020 - Dr. Werner Fink <[email protected]> + +- Update to version 0.9.4 + * Remove Python 2 compatibility +- Update to version 0.9.3 + * Cycle through opening links with webbrowser module, xdg-open or --run argument + * Add option to copy to primary selection or clipboard. Fix #87 + * Generate new config file using command line switch instead of keybinding + * Allow remapping a key to open url in addition to space and enter + * Show help menu with F1 and show dynamic keybindings + * Allow editing key bindings in config.json. Fix #72. +- Set python flavour spec file macro to get the rpm pythin macros defined +- Remove egg information tree as we require python3-urwid + +------------------------------------------------------------------- Old: ---- urlscan-0.9.2.tar.gz New: ---- urlscan-0.9.4.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ urlscan.spec ++++++ --- /var/tmp/diff_new_pack.gKYhZO/_old 2020-03-21 00:02:32.881137060 +0100 +++ /var/tmp/diff_new_pack.gKYhZO/_new 2020-03-21 00:02:32.909137075 +0100 @@ -1,7 +1,7 @@ # # spec file for package urlscan # -# Copyright (c) 2019 SUSE LINUX GmbH, Nuernberg, Germany. +# Copyright (c) 2020 SUSE LLC # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -12,25 +12,29 @@ # license that conforms to the Open Source Definition (Version 1.9) # published by the Open Source Initiative. -# Please submit bugfixes or comments via http://bugs.opensuse.org/ +# Please submit bugfixes or comments via https://bugs.opensuse.org/ # Name: urlscan -Version: 0.9.2 +Version: 0.9.4 Release: 0 Summary: An other URL extractor/viewer License: GPL-2.0-or-later Group: Productivity/Networking/Web/Browsers -Url: https://github.com/firecat53/urlscan +URL: https://github.com/firecat53/urlscan Source0: https://github.com/firecat53/urlscan/archive/%{version}.tar.gz#/%{name}-%{version}.tar.gz Source1: muttrc +Requires: python3 +Requires: python3-base Requires: python3-urwid +BuildRequires: python3-base BuildRequires: python3-devel +BuildRequires: python3-rpm-macros BuildRequires: python3-setuptools BuildRoot: %{_tmppath}/%{name}-%{version}-build BuildArch: noarch -%{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")} +%define python_flavor python3 %description The urlscan utility displays URLs found in an email message with @@ -49,13 +53,14 @@ rm -rf %{buildroot}/usr/share/doc/%{name}* mkdir -p %{buildroot}%{_defaultdocdir}/%{name} install -m 0644 %{S:1} %{buildroot}%{_defaultdocdir}/%{name} +rm -rvf %{buildroot}%{python_sitelib}/%{name}-%{version}-*-info %files %defattr(-,root,root) %license COPYING %doc README.rst %{_bindir}/%{name} -%{python_sitelib}/* +%{python_sitelib}/%{name} %{_mandir}/man1/%{name}.1.gz %doc %{_defaultdocdir}/%{name}/muttrc ++++++ urlscan-0.9.2.tar.gz -> urlscan-0.9.4.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlscan-0.9.2/.gitignore new/urlscan-0.9.4/.gitignore --- old/urlscan-0.9.2/.gitignore 2019-01-22 00:05:53.000000000 +0100 +++ new/urlscan-0.9.4/.gitignore 2019-08-30 23:40:59.000000000 +0200 @@ -7,4 +7,3 @@ dist test_emails/ MANIFEST -Pipfile.lock diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlscan-0.9.2/Pipfile new/urlscan-0.9.4/Pipfile --- old/urlscan-0.9.2/Pipfile 2019-01-22 00:05:53.000000000 +0100 +++ new/urlscan-0.9.4/Pipfile 1970-01-01 01:00:00.000000000 +0100 @@ -1,10 +0,0 @@ -[[source]] -url = "https://pypi.org/simple"; -verify_ssl = true -name = "pypi" - -[packages] -urwid = ">=1.2.1" -urlscan = {editable = true, path = "."} - -[dev-packages] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlscan-0.9.2/README.rst new/urlscan-0.9.4/README.rst --- old/urlscan-0.9.2/README.rst 2019-01-22 00:05:53.000000000 +0100 +++ new/urlscan-0.9.4/README.rst 2019-08-30 23:40:59.000000000 +0200 @@ -17,7 +17,9 @@ mailreader to allow you to easily launch a Web browser for URLs contained in email messages. It is a replacement for the "urlview" program. -Requires: Python 2.6+ (including Python 3.x) and the python-urwid library +*NOTE* The last version that is Python 2 compatible is 0.9.3. + +Requires: Python 3.3+ and the python-urwid library Features -------- @@ -46,24 +48,37 @@ - Execute an arbitrary function (for example, copy URL to clipboard) instead of opening URL in a browser. -- Configure colors via ~/.config/urlscan/config.json. Generate default config - file for editing with `P`. Cycle through available palettes with `p`. +- Use `l` to cycle through whether URLs are opened using the Python webbrowser + module (default), xdg-open (if installed) or opened by a function passed on + the command line with `--run`. + +- Configure colors and keybindings via ~/.config/urlscan/config.json. Generate + default config file for editing by running `urlscan -g`. Cycle through + available palettes with `p`. -- Copy URL to clipboard (primary) with `C`. Requires xsel or xclip. +- Copy URL to clipboard with `C` or to primary selection with `P`. Requires + xsel or xclip. - Run a command with the selected URL as the argument or pipe the selected URL to a command. +- Show complete help menu with `F1`. Hide header on startup with `--nohelp`. + Installation and setup ---------------------- To install urlscan, install from your distribution repositories (Archlinux), -from Pypi, or install from source using setup.py. +from Pypi, or do a local development install with pip -e:: + + pip install --user urlscan + + OR + + cd <path/to/urlscan> && pip install --user -e . .. NOTE:: - To work with Python 3.x the minimum required version of urwid is 1.2.1. - Python 2.x needs urwid >= 1.1.0 + The minimum required version of urwid is 1.2.1. Once urlscan is installed, add the following lines to your .muttrc: @@ -87,7 +102,7 @@ :: - urlscan [-n, --no-browser] [-c, --compact] [-d, --dedupe] [-r, --run <expression>] [-p, --pipe] <file> + urlscan [-g, --genconf] [-n, --no-browser] [-c, --compact] [-d, --dedupe] [-r, --run <expression>] [-p, --pipe] [-H, --nohelp] <file> Urlscan can extract URLs and email addresses from emails or any text file. Calling with no flags will start the curses browser. Calling with '-n' will just @@ -96,19 +111,52 @@ duplicate URLs. Files can also be piped to urlscan using normal shell pipe mechanisms: `cat <something> | urlscan` or `urlscan < <something>` -Instead of opening a web browser, the selected URL can be passed as the -argument to a command using `--run <command>`. Alternatively, the URL can be -piped to the command using `--run <command> --pipe`. Using --run with --pipe is -preferred if the command supports it, as it is marginally more secure and -tolerant of special characters in the URL. +Instead of opening a web browser, the selected URL can be passed as the argument +to a command using `--run "<command> {}"`. Note the use of `{}` in the command +string to denote the selected URL. Alternatively, the URL can be piped to the +command using `--run <command> --pipe`. Using --run with --pipe is preferred if +the command supports it, as it is marginally more secure and tolerant of special +characters in the URL. Theming ------- -Press 'P' from urlscan to generate ~/.config/urlscan/config.json with the -default color and black & white palettes. This can be edited or added to, as -desired. The first palette in the list will be the default. Configure the -palettes according to the `Urwid display attributes`_. +Run `urlscan -g` to generate ~/.config/urlscan/config.json with the default +color and black & white palettes. This can be edited or added to, as desired. +The first palette in the list will be the default. Configure the palettes +according to the `Urwid display attributes`_. + +Keybindings +----------- + +Run `urlscan -g` to generate ~/.config/urlscan/config.json. All of the keys will +be listed. You can either leave in place or delete any that will not be altered. + +To unset a binding, set it equal to "". For example: `"P": ""` + +The follow actions are supported: + +- `all_escape` -- toggle unescape all URLs (default: `u`) +- `all_shorten` -- toggle shorten all URLs (default: `S`) +- `bottom` -- move cursor to last item (default: `G`) +- `clear_screen` -- redraw screen (default: `Ctrl-l`) +- `clipboard` -- copy highlighted URL to clipboard using xsel/xclip (default: `C`) +- `clipboard_pri` -- copy highlighted URL to primary selection using xsel/xclip (default: `P`) +- `context` -- show/hide context (default: `c`) +- `down` -- cursor down (default: `j`) +- `help_menu` -- show/hide help menu (default: `F1`) +- `link_handler` -- cycle link handling (webbrowser, xdg-open or --run) (default: `l`) +- `open_url` -- open selected URL (default: `space` or `enter`) +- `palette` -- cycle through palettes (default: `p`) +- `quit` -- quit (default: `q` or `Q`) +- `shorten` -- toggle shorten highlighted URL (default: `s`) +- `top` -- move to first list item (default: `g`) +- `up` -- cursor up (default: `k`) + +Update TLD list (for developers, not users) +------------------------------------------- + +`wget https://data.iana.org/TLD/tlds-alpha-by-domain.txt` Known bugs and limitations -------------------------- diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlscan-0.9.2/bin/urlscan new/urlscan-0.9.4/bin/urlscan --- old/urlscan-0.9.2/bin/urlscan 2019-01-22 00:05:53.000000000 +0100 +++ new/urlscan-0.9.4/bin/urlscan 2019-08-30 23:40:59.000000000 +0200 @@ -41,6 +41,9 @@ """ arg_parse = argparse.ArgumentParser(description="Parse and display URLs") + arg_parse.add_argument('--genconf', '-g', + action='store_true', default=False, + help="Generate config file and exit.") arg_parse.add_argument('--compact', '-c', action='store_true', default=False, help="Don't display the context of each URL.") @@ -58,6 +61,9 @@ arg_parse.add_argument('--pipe', '-p', dest='pipe', action='store_true', default=False, help='Pipe URL into the command specified by --run') + arg_parse.add_argument('--nohelp', '-H', dest='nohelp', + action='store_true', default=False, + help='Hide help menu by default') arg_parse.add_argument('message', nargs='?', default=sys.stdin, help="Filename of the message to parse") return arg_parse.parse_args() @@ -172,10 +178,14 @@ """ args = parse_arguments() + if args.genconf is True: + urlchoose.URLChooser([], genconf=True) + return msg = process_input(args.message) if args.nobrowser is False: tui = urlchoose.URLChooser(urlscan.msgurls(msg), compact=args.compact, + nohelp=args.nohelp, dedupe=args.dedupe, run=args.run, pipe=args.pipe) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlscan-0.9.2/conf.py new/urlscan-0.9.4/conf.py --- old/urlscan-0.9.2/conf.py 1970-01-01 01:00:00.000000000 +0100 +++ new/urlscan-0.9.4/conf.py 2019-08-30 23:40:59.000000000 +0200 @@ -0,0 +1,3 @@ +URLINTERNALPATTERN = r'[{}()@\w/\\\-%?!&.=:;+,#~]' +URLTRAILINGPATTERN = r'[{}(@\w/\-%&=+#]' +HTTPURLPATTERN = (r'(?:(https?|file|ftps?)://' + URLINTERNALPATTERN + r'*' + URLTRAILINGPATTERN + r')') diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlscan-0.9.2/setup.py new/urlscan-0.9.4/setup.py --- old/urlscan-0.9.2/setup.py 2019-01-22 00:05:53.000000000 +0100 +++ new/urlscan-0.9.4/setup.py 2019-08-30 23:40:59.000000000 +0200 @@ -3,12 +3,12 @@ from setuptools import setup setup(name="urlscan", - version="0.9.2", + version="0.9.4", description="View/select the URLs in an email message or file", author="Scott Hansen", author_email="[email protected]", url="https://github.com/firecat53/urlscan";, - download_url="https://github.com/firecat53/urlscan/archive/0.9.2.zip";, + download_url="https://github.com/firecat53/urlscan/archive/0.9.4.zip";, packages=['urlscan'], scripts=['bin/urlscan'], package_data={'urlscan': ['assets/*']}, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlscan-0.9.2/urlscan/assets/tlds-alpha-by-domain.txt new/urlscan-0.9.4/urlscan/assets/tlds-alpha-by-domain.txt --- old/urlscan-0.9.2/urlscan/assets/tlds-alpha-by-domain.txt 2019-01-22 00:05:53.000000000 +0100 +++ new/urlscan-0.9.4/urlscan/assets/tlds-alpha-by-domain.txt 2019-08-30 23:40:59.000000000 +0200 @@ -1,4 +1,4 @@ -# Version 2018073000, Last Updated Mon Jul 30 07:07:01 2018 UTC +# Version 2019021501, Last Updated Fri Feb 15 07:07:01 2019 UTC AAA AARP ABARTH @@ -145,7 +145,6 @@ BJ BLACK BLACKFRIDAY -BLANCO BLOCKBUSTER BLOG BLOOMBERG @@ -391,7 +390,6 @@ ENGINEER ENGINEERING ENTERPRISES -EPOST EPSON EQUIPMENT ER @@ -522,7 +520,6 @@ GOLDPOINT GOLF GOO -GOODHANDS GOODYEAR GOOG GOOGLE @@ -651,7 +648,6 @@ JETZT JEWELRY JIO -JLC JLL JM JMP @@ -934,7 +930,6 @@ PA PAGE PANASONIC -PANERAI PARIS PARS PARTNERS @@ -1154,13 +1149,13 @@ SONY SOY SPACE -SPIEGEL SPORT SPOT SPREADBETTING SR SRL SRT +SS ST STADA STAPLES @@ -1168,7 +1163,6 @@ STARHUB STATEBANK STATEFARM -STATOIL STC STCGROUP STOCKHOLM @@ -1214,7 +1208,6 @@ TECH TECHNOLOGY TEL -TELECITY TELEFONICA TEMASEK TENNIS @@ -1309,7 +1302,6 @@ VIRGIN VISA VISION -VISTA VISTAPRINT VIVA VIVO @@ -1457,6 +1449,7 @@ XN--MGBAAKC7DVF XN--MGBAAM7A8H XN--MGBAB2BD +XN--MGBAH1A3HJKRD XN--MGBAI9AZGQP6J XN--MGBAYH7GPA XN--MGBB9FBPOB @@ -1535,7 +1528,6 @@ ZARA ZERO ZIP -ZIPPO ZM ZONE ZUERICH diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlscan-0.9.2/urlscan/urlchoose.py new/urlscan-0.9.4/urlscan/urlchoose.py --- old/urlscan-0.9.2/urlscan/urlchoose.py 2019-01-22 00:05:53.000000000 +0100 +++ new/urlscan-0.9.4/urlscan/urlchoose.py 2019-08-30 23:40:59.000000000 +0200 @@ -24,7 +24,7 @@ from os.path import dirname, exists, expanduser import re import shlex -from subprocess import Popen, PIPE +from subprocess import call, Popen, PIPE, DEVNULL import sys from threading import Thread from time import sleep @@ -34,11 +34,6 @@ import urwid.curses_display import urwid.raw_display -# Python 2 compatibility -try: - FileNotFoundError -except NameError: - FileNotFoundError = IOError def shorten_url(url, cols, shorten): """Shorten long URLs to fit on one line. @@ -91,38 +86,89 @@ class URLChooser: - def __init__(self, extractedurls, compact=False, dedupe=False, shorten=True, - run="", pipe=False): + def __init__(self, extractedurls, compact=False, nohelp=False, dedupe=False, + shorten=True, run="", pipe=False, genconf=False): self.conf = expanduser("~/.config/urlscan/config.json") - self.palettes = [] + self.keys = {'/': self._search_key, + '0': self._digits, + '1': self._digits, + '2': self._digits, + '3': self._digits, + '4': self._digits, + '5': self._digits, + '6': self._digits, + '7': self._digits, + '8': self._digits, + '9': self._digits, + 'C': self._clipboard, + 'c': self._context, + 'ctrl l': self._clear_screen, + 'f1': self._help_menu, + 'G': self._bottom, + 'g': self._top, + 'j': self._down, + 'k': self._up, + 'P': self._clipboard_pri, + 'l': self._link_handler, + 'p': self._palette, + 'Q': self._quit, + 'q': self._quit, + 'S': self._all_shorten, + 's': self._shorten, + 'u': self._all_escape + } + self.palettes = {} + # Default color palette + default = [('header', 'white', 'dark blue', 'standout'), + ('footer', 'white', 'dark red', 'standout'), + ('search', 'white', 'dark green', 'standout'), + ('msgtext', '', ''), + ('msgtext:ellipses', 'light gray', 'black'), + ('urlref:number:braces', 'light gray', 'black'), + ('urlref:number', 'yellow', 'black', 'standout'), + ('urlref:url', 'white', 'black', 'standout'), + ('url:sel', 'white', 'dark blue', 'bold')] + # Default black & white palette + blw = [('header', 'black', 'light gray', 'standout'), + ('footer', 'black', 'light gray', 'standout'), + ('search', 'black', 'light gray', 'standout'), + ('msgtext', '', ''), + ('msgtext:ellipses', 'white', 'black'), + ('urlref:number:braces', 'white', 'black'), + ('urlref:number', 'white', 'black', 'standout'), + ('urlref:url', 'white', 'black', 'standout'), + ('url:sel', 'black', 'light gray', 'bold')] + self.palettes.update([("default", default), ("bw", blw)]) + if genconf is True: + self._config_create() try: with open(self.conf, 'r') as conf_file: data = json.load(conf_file) - for pal in data.values(): - self.palettes.append([tuple(i) for i in pal]) + try: + for pal_name, pal in data['palettes'].items(): + self.palettes.update([(pal_name, [tuple(i) for i in pal])]) + except KeyError: + pass + try: + items = data['keys'].items() + for key, value in items: + if value: + if value == "open_url": + urwid.Button._command_map._command[key] = 'activate' + value = getattr(self, "_{}".format(value)) + else: + del self.keys[key] + continue + self.keys.update([(key, value)]) + except KeyError: + pass except FileNotFoundError: pass - # Default color palette - self.palettes.append([('header', 'white', 'dark blue', 'standout'), - ('footer', 'white', 'dark red', 'standout'), - ('search', 'white', 'dark green', 'standout'), - ('msgtext', '', ''), - ('msgtext:ellipses', 'light gray', 'black'), - ('urlref:number:braces', 'light gray', 'black'), - ('urlref:number', 'yellow', 'black', 'standout'), - ('urlref:url', 'white', 'black', 'standout'), - ('url:sel', 'white', 'dark blue', 'bold')]) - # Default black & white palette - self.palettes.append([('header', 'black', 'light gray', 'standout'), - ('footer', 'black', 'light gray', 'standout'), - ('search', 'black', 'light gray', 'standout'), - ('msgtext', '', ''), - ('msgtext:ellipses', 'white', 'black'), - ('urlref:number:braces', 'white', 'black'), - ('urlref:number', 'white', 'black', 'standout'), - ('urlref:url', 'white', 'black', 'standout'), - ('url:sel', 'black', 'light gray', 'bold')]) - + try: + call(['xdg-open'], stdout=DEVNULL) + self.xdg = True + except OSError: + self.xdg = False self.shorten = shorten self.compact = compact self.run = run @@ -131,6 +177,8 @@ self.search_string = "" self.no_matches = False self.enter = False + self.activate_keys = [i for i, j in urwid.Button._command_map._command.items() + if j == 'activate'] self.items, self.urls = self.process_urls(extractedurls, dedupe=dedupe, shorten=self.shorten) @@ -146,31 +194,34 @@ # Store items grouped into sections self.items_org = grp_list(self.items) listbox = urwid.ListBox(self.items) - header = (":: q - Quit :: " - "/ - search :: " - "c - context :: " - "C - copy to clipboard :: " - "s - URL short :: " - "S - all URL short :: " - "g/G - top/bottom :: " - "<num> - jump to <num> :: " - "p - cycle palettes :: " - "P - create config file ::" - "u - unescape URL ::") - headerwid = urwid.AttrMap(urwid.Text(header), 'header') - self.top = urwid.Frame(listbox, headerwid) + self.header = (":: F1 - help/keybindings :: " + "q - quit :: " + "/ - search :: " + "URL opening mode - {}") + self.link_open_modes = ["Web Browser", "Xdg-Open"] if self.xdg is True else ["Web Browser"] + if self.run: + self.link_open_modes.insert(0, self.run) + self.nohelp = nohelp + if nohelp is False: + self.headerwid = urwid.AttrMap(urwid.Text( + self.header.format(self.link_open_modes[0])), 'header') + else: + self.headerwid = None + self.top = urwid.Frame(listbox, self.headerwid) if self.urls: self.top.body.focus_position = \ (2 if self.compact is False else 0) self.tui = urwid.curses_display.Screen() + self.palette_names = list(self.palettes.keys()) self.palette_idx = 0 self.number = "" + self.help_menu = False def main(self): """Urwid main event loop """ - self.loop = urwid.MainLoop(self.top, self.palettes[0], screen=self.tui, + self.loop = urwid.MainLoop(self.top, self.palettes[self.palette_names[0]], screen=self.tui, handle_mouse=False, input_filter=self.handle_keys, unhandled_input=self.unhandled) self.loop.run() @@ -200,17 +251,20 @@ text = "" footerwid = urwid.AttrMap(urwid.Text(text), footer) self.top.footer = footerwid - elif k == ' ': - self.search_string += " " - footerwid = urwid.AttrMap(urwid.Text(text), 'footer') - self.top.footer = footerwid + elif k in self.activate_keys: + self.search_string += k + self._search() elif k == 'backspace': self.search_string = self.search_string[:-1] self._search() - elif k in ('enter', ' ') and self.urls and self.search is False: - load_text = "Loading URL..." if not self.run else "Executing: {}".format(self.run) - if os.environ.get('BROWSER') not in ['elinks', 'links', 'w3m', 'lynx']: - self._footer_start_thread(load_text, 5) + elif k in self.activate_keys and \ + self.urls and \ + self.search is False and \ + self.help_menu is False: + self._open_url() + elif self.help_menu is True: + self._help_menu() + return [] if k == 'up': # Works around bug where the up arrow goes higher than the top list # item and unintentionally triggers context and palette switches. @@ -224,145 +278,235 @@ return [i for i in keys if i != 'backspace'] def unhandled(self, key): - """Add other keyboard actions (q, j, k, s, S, c, C, g, G) not handled by - the ListBox widget. + """Handle other keyboard actions not handled by the ListBox widget. """ - size = self.tui.get_cols_rows() + self.key = key + self.size = self.tui.get_cols_rows() if self.search is True: if self.enter is False and self.no_matches is False: if len(key) == 1 and key.isprintable(): self.search_string += key self._search() + elif self.enter is True and not self.search_string: + self.search = False + self.enter = False + return + if not self.urls and key not in "Qq": + return # No other actions are useful with no URLs + if self.help_menu is False: + try: + self.keys[key]() + except KeyError: + pass + + def _quit(self): + """q/Q""" + raise urwid.ExitMainLoop() + + def _open_url(self): + """<Enter> or <space>""" + load_text = "Loading URL..." if self.link_open_modes[0] != self.run \ + else "Executing: {}".format(self.run) + if os.environ.get('BROWSER') not in ['elinks', 'links', 'w3m', 'lynx']: + self._footer_start_thread(load_text, 5) + + def _help_menu(self): + """F1""" + if self.help_menu is False: + self.focus_pos_saved = self.top.body.focus_position + help_men = "\n".join(["{} - {}".format(i, j.__name__.strip('_')) + for i, j in self.keys.items() if j.__name__ != + '_digits']) + help_men = "KEYBINDINGS\n" + help_men + "\n<0-9> - Jump to item" + docs = ("OPTIONS\n" + "all_escape -- toggle unescape all URLs\n" + "all_shorten -- toggle shorten all URLs\n" + "bottom -- move cursor to last item\n" + "clear_screen -- redraw screen\n" + "clipboard -- copy highlighted URL to clipboard using xsel/xclip\n" + "clipboard_pri -- copy highlighted URL to primary selection using xsel/xclip\n" + "config_create -- create ~/.config/urlscan/config.json\n" + "context -- show/hide context\n" + "down -- cursor down\n" + "help_menu -- show/hide help menu\n" + "link_handler -- cycle through xdg-open, webbrowser and user-defined function\n" + "open_url -- open selected URL\n" + "palette -- cycle through palettes\n" + "quit -- quit\n" + "shorten -- toggle shorten highlighted URL\n" + "top -- move to first list item\n" + "up -- cursor up\n") + self.top.body = \ + urwid.ListBox(urwid.SimpleListWalker([urwid.Columns([(30, urwid.Text(help_men)), + urwid.Text(docs)])])) + else: + self.top.body = urwid.ListBox(self.items) + self.top.body.focus_position = self.focus_pos_saved + self.help_menu = not self.help_menu + + def _search_key(self): + """ / """ + if self.urls: + self.search = True + if self.compact is True: + self.compact = False + self.items, self.items_com = self.items_com, self.items + else: return - if key in ('q', 'Q'): - raise urwid.ExitMainLoop() - elif not self.urls: - pass # No other actions are useful with no URLs - elif key == '/': - if self.urls: - self.search = True - if self.compact is True: - self.compact = False - self.items, self.items_com = self.items_com, self.items + self.no_matches = False + self.search_string = "" + # Reset the search highlighting + self._search() + footerwid = urwid.AttrMap(urwid.Text("Search: "), 'footer') + self.top.footer = footerwid + self.items = self.items_orig + self.top.body = urwid.ListBox(self.items) + + def _digits(self): + """ 0-9 """ + self.number += self.key + try: + if self.compact is False: + self.top.body.focus_position = \ + self.items.index(self.items_com[max(int(self.number) - 1, 0)]) else: - return - self.no_matches = False - self.search_string = "" - # Reset the search highlighting - self._search() - footerwid = urwid.AttrMap(urwid.Text("Search: "), 'footer') + self.top.body.focus_position = \ + self.items.index(self.items[max(int(self.number) - 1, 0)]) + except IndexError: + self.number = self.number[:-1] + self.top.keypress(self.size, "") # Trick urwid into redisplaying the cursor + if self.number: + self._footer_start_thread("Selection: {}".format(self.number), 1) + + def _clear_screen(self): + """ Ctrl-l """ + self.draw_screen(self.size) + + def _down(self): + """ j """ + self.top.keypress(self.size, "down") + + def _up(self): + """ k """ + self.top.keypress(self.size, "up") + + def _top(self): + """ g """ + # Goto top of the list + self.top.body.focus_position = 2 if self.compact is False else 0 + self.top.keypress(self.size, "") # Trick urwid into redisplaying the cursor + + def _bottom(self): + """ G """ + # Goto bottom of the list + self.top.body.focus_position = len(self.items) - 1 + self.top.keypress(self.size, "") # Trick urwid into redisplaying the cursor + + def _shorten(self): + """ s """ + # Toggle shortened URL for selected item + fpo = self.top.body.focus_position + url_idx = len([i for i in self.items[:fpo + 1] + if isinstance(i, urwid.Columns)]) - 1 + if self.compact is False and fpo <= 1: + return + url = self.urls[url_idx] + short = not "..." in self.items[fpo][1].label + self.items[fpo][1].set_label(shorten_url(url, self.size[0], short)) + + def _all_shorten(self): + """ S """ + # Toggle all shortened URLs + self.shorten = not self.shorten + urls = iter(self.urls) + for item in self.items: + # Each Column has (Text, Button). Update the Button label + if isinstance(item, urwid.Columns): + item[1].set_label(shorten_url(next(urls), + self.size[0], + self.shorten)) + def _all_escape(self): + """ u """ + # Toggle all escaped URLs + self.unesc = not self.unesc + self.urls, self.urls_unesc = self.urls_unesc, self.urls + urls = iter(self.urls) + for item in self.items: + # Each Column has (Text, Button). Update the Button label + if isinstance(item, urwid.Columns): + item[1].set_label(shorten_url(next(urls), + self.size[0], + self.shorten)) + + def _context(self): + """ c """ + # Show/hide context + if self.search_string: + # Reset search when toggling compact mode + footerwid = urwid.AttrMap(urwid.Text(""), 'default') self.top.footer = footerwid + self.search_string = "" self.items = self.items_orig - self.top.body = urwid.ListBox(self.items) - elif key.isdigit(): - self.number += key - try: - if self.compact is False: - self.top.body.focus_position = \ - self.items.index(self.items_com[max(int(self.number) - 1, 0)]) - else: - self.top.body.focus_position = \ - self.items.index(self.items[max(int(self.number) - 1, 0)]) - except IndexError: - self.number = self.number[:-1] - self.top.keypress(size, "") # Trick urwid into redisplaying the cursor - if self.number: - self._footer_start_thread("Selection: {}".format(self.number), 1) - elif key == 'ctrl l': - self.draw_screen(size) - elif key == 'j': - self.top.keypress(size, "down") - elif key == 'k': - self.top.keypress(size, "up") - elif key == 'g': - # Goto top of the list - self.top.body.focus_position = 2 if self.compact is False else 0 - self.top.keypress(size, "") # Trick urwid into redisplaying the cursor - elif key == 'G': - # Goto bottom of the list - self.top.body.focus_position = len(self.items) - 1 - self.top.keypress(size, "") # Trick urwid into redisplaying the cursor - elif key == 's': - # Toggle shortened URL for selected item - fpo = self.top.body.focus_position - url_idx = len([i for i in self.items[:fpo + 1] - if isinstance(i, urwid.Columns)]) - 1 - if self.compact is False and fpo <= 1: - return - url = self.urls[url_idx] - short = False if "..." in self.items[fpo][1].label else True - self.items[fpo][1].set_label(shorten_url(url, size[0], short)) - elif key in ('S', 'u'): - # Toggle all shortened or escaped URLs - if key == 'S': - self.shorten = not self.shorten - if key == 'u': - self.unesc = not self.unesc - self.urls, self.urls_unesc = self.urls_unesc, self.urls - urls = iter(self.urls) - for item in self.items: - # Each Column has (Text, Button). Update the Button label - if isinstance(item, urwid.Columns): - item[1].set_label(shorten_url(next(urls), - size[0], - self.shorten)) - elif key == 'c': - # Show/hide context - if self.search_string: - # Reset search when toggling compact mode - footerwid = urwid.AttrMap(urwid.Text(""), 'default') - self.top.footer = footerwid - self.search_string = "" - self.items = self.items_orig - fpo = self.top.body.focus_position - self.items, self.items_com = self.items_com, self.items - self.top.body = urwid.ListBox(self.items) - self.top.body.focus_position = self._cur_focus(fpo) - self.compact = not self.compact - elif key == 'C': - # Copy highlighted url to clipboard - fpo = self.top.body.focus_position - url_idx = len([i for i in self.items[:fpo + 1] - if isinstance(i, urwid.Columns)]) - 1 - if self.compact is False and fpo <= 1: - return - url = self.urls[url_idx] + fpo = self.top.body.focus_position + self.items, self.items_com = self.items_com, self.items + self.top.body = urwid.ListBox(self.items) + self.top.body.focus_position = self._cur_focus(fpo) + self.compact = not self.compact + + def _clipboard(self, pri=False): + """ C """ + # Copy highlighted url to clipboard + fpo = self.top.body.focus_position + url_idx = len([i for i in self.items[:fpo + 1] + if isinstance(i, urwid.Columns)]) - 1 + if self.compact is False and fpo <= 1: + return + url = self.urls[url_idx] + if pri is True: cmds = ("xsel -i", "xclip -i") - for cmd in cmds: - try: - proc = Popen(shlex.split(cmd), stdin=PIPE) - proc.communicate(input=url.encode(sys.getdefaultencoding())) - self._footer_start_thread("Copied url to primary selection", 5) - except OSError: - continue - break - elif key == 'p': - # Loop through available palettes - self.palette_idx += 1 + else: + cmds = ("xsel -ib", "xclip -i -selection clipboard") + for cmd in cmds: try: - self.loop.screen.register_palette(self.palettes[self.palette_idx]) - except IndexError: - self.loop.screen.register_palette(self.palettes[0]) - self.palette_idx = 0 - self.loop.screen.clear() - elif key == 'P': - # Create ~/.config/urlscan/config.json if if doesn't exist - if not exists(self.conf): - try: - # Python 2/3 compatible recursive directory creation - os.makedirs(dirname(expanduser(self.conf))) - except OSError as err: - if errno.EEXIST != err.errno: - raise - names = ["default", "bw"] - with open(expanduser(self.conf), 'w') as pals: - pals.writelines(json.dumps(dict(zip(names, - self.palettes)), - indent=4)) - self._footer_start_thread("Created ~/.config/urlscan/config.json", 5) - else: - self._footer_start_thread("Config.json already exists", 5) + proc = Popen(shlex.split(cmd), stdin=PIPE) + proc.communicate(input=url.encode(sys.getdefaultencoding())) + self._footer_start_thread("Copied url to {} selection".format( + "primary" if pri is True else "clipboard"), 5) + except OSError: + continue + break + + def _clipboard_pri(self): + """ P """ + # Copy highlighted url to primary selection + self._clipboard(pri=True) + + def _palette(self): + """ p """ + # Loop through available palettes + self.palette_idx += 1 + try: + self.loop.screen.register_palette(self.palettes[self.palette_names[self.palette_idx]]) + except IndexError: + self.loop.screen.register_palette(self.palettes[self.palette_names[0]]) + self.palette_idx = 0 + self.loop.screen.clear() + + def _config_create(self): + """ --genconf """ + # Create ~/.config/urlscan/config.json if if doesn't exist + if not exists(self.conf): + os.makedirs(dirname(expanduser(self.conf)), exist_ok=True) + keys = dict(zip(self.keys.keys(), + [i.__name__.strip('_') for i in self.keys.values()])) + with open(expanduser(self.conf), 'w') as pals: + pals.writelines(json.dumps({"palettes": self.palettes, "keys": keys}, + indent=4)) + print("Created ~/.config/urlscan/config.json") + else: + print("~/.config/urlscan/config.json already exists") + def _footer_start_thread(self, text, time): """Display given text in the footer. Clears after <time> seconds @@ -456,6 +600,18 @@ def _get_search(self): return lambda: self.search, lambda: self.enter + def _link_handler(self): + """Function to cycle through opening links via webbrowser module, + xdg-open or custom expression passed with --run. + + """ + mode = self.link_open_modes.pop() + self.link_open_modes.insert(0, mode) + if self.nohelp is False: + self.headerwid = urwid.AttrMap(urwid.Text( + self.header.format(self.link_open_modes[0])), 'header') + self.top.header = self.headerwid + def mkbrowseto(self, url): """Create the urwid callback function to open the web browser or call another function with the URL. @@ -481,9 +637,12 @@ if self._get_search()[1]() is True: self.search = False self.enter = False - elif not self.run: + elif self.link_open_modes[0] == "Web Browser": webbrowser.open(url) - elif self.run and self.pipe: + elif self.link_open_modes[0] == "Xdg-Open": + run = 'xdg-open "{}"'.format(url) + process = Popen(shlex.split(run), stdout=PIPE, stdin=PIPE) + elif self.link_open_modes[0] == self.run and self.pipe: process = Popen(shlex.split(self.run), stdout=PIPE, stdin=PIPE) process.communicate(input=url.encode(sys.getdefaultencoding())) else: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlscan-0.9.2/urlscan/urlscan.py new/urlscan-0.9.4/urlscan/urlscan.py --- old/urlscan-0.9.2/urlscan/urlscan.py 2019-01-22 00:05:53.000000000 +0100 +++ new/urlscan-0.9.4/urlscan/urlscan.py 2019-08-30 23:40:59.000000000 +0200 @@ -18,13 +18,9 @@ """Contains the backend logic that scans messages for URLs and context.""" -from __future__ import unicode_literals import os import re -try: - from HTMLParser import HTMLParser -except ImportError: - from html.parser import HTMLParser +from html.parser import HTMLParser def get_charset(message, default="utf-8"): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/urlscan-0.9.2/urlscan.1 new/urlscan-0.9.4/urlscan.1 --- old/urlscan-0.9.2/urlscan.1 2019-01-22 00:05:53.000000000 +0100 +++ new/urlscan-0.9.4/urlscan.1 2019-08-30 23:40:59.000000000 +0200 @@ -1,6 +1,6 @@ .\" Hey, EMACS: -*- nroff -*- -.TH URLSCAN 1 "January 14, 2019" +.TH URLSCAN 1 "30 August 2019" .SH NAME urlscan \- browse the URLs in an email message from a terminal @@ -30,35 +30,50 @@ and base64. \fB2.\fR Extraction and display of the context surrounding each URL. Toggle -context view on/off with `c`. +context view on/off with \fBc\fR. -\fB3.\fR Copy current URL to clipboard primary selection with `C`. +\fB3.\fR Copy current URL to primary selection with \fBP\fR or to clipboard with +\fBC\fR. \fB4.\fR URLs are shortened by default to fit on one line. Toggle one or all -shortened URLs with `s` or `S`. +shortened URLs with \fBs\fR or \fBS\fR. -\fB5.\fR Incremental case-insensitive search using `/`. Footer shows current -search term. `/` again resets search. +\fB5.\fR Incremental case-insensitive search using \fB/\fR. Footer shows current +search term. \fB/\fR again resets search. \fB6.\fR Cycle through all available palettes (color and black & white available -by default) using `p`. `P` will generate a ~/.config/urlscan/config.json file -for editing or adding additional pallettes. See +by default) using \fBp\fR. Running \fBurlscan \-g\fR will generate a +~/.config/urlscan/config.json file for editing or adding additional pallettes +and keybindings. See http://urwid.org/manual/displayattributes.html#display-attributes -for options and allowed values. +for color options and allowed values. -\fB7.\fR `u` will unescape the highlighted URL if necessary. +\fB7.\fR \fBu\fR will unescape the highlighted URL if necessary. -\fB8.\fR Run a command with the selected URL as the argument or pipe the selected - URL to a command using the `--run` and `--pipe` arguments. +\fB8.\fR Run a command with the selected URL as the argument or pipe the +selected URL to a command using the \fB--run\fR and \fB--pipe\fR arguments. + +\fB9.\fR Use \fBl\fR to cycle through whether URLs are opened using the Python +webbrowser module (default), xdg-open (if installed) or a function passed on the +command line with \fB--run\fR. The \fB--run\fR function will respect the value +of \fB--pipe\fR. + +\fB10.\fR \fBF1\fR shows the help menu. .SH OPTIONS .TP +.B \-g, \-\-genconf +Generate ~/.config/urlscan/config.json with default options. +.TP .B \-c, \-\-compact Display a simple list of the extracted URLs, instead of showing the context of each URL. Also toggle with `c` from within the viewer. .TP +.B \-H, \-\-nohelp +Start with header menu hidden. +.TP .B \-d, \-\-dedupe Remove duplicated URLs from the list of URLs. .TP @@ -68,9 +83,10 @@ .TP .B \-r, \-\-run \<expression\> Execute \<expression\> in place of opening URL with a browser. Use {} in -\<expression\> to substitute in the URL. Example: +\<expression\> to substitute in the URL. Examples: $ urlscan --run 'echo {} | xclip -i' file.txt + $ urlscan --run 'tmux set buffer {}' .TP .B \-p, \-\-pipe Pipe the selected URL to the command specified by `--run`. This is preferred @@ -99,6 +115,48 @@ Alternately, you can pipe a message into urlscan using the '|' operator. This can be useful for applying a different flag (such as the '-d' or '-c' options). +.SH KEYBINDINGS + +Run \fBurlscan \-g\fR to generate ~/.config/urlscan/config.json. All of the keys +will be listed. You can either leave in place or delete any that will not be +altered. + +To unset a binding, set it equal to "". For example: \fB"P": ""\fR + +The follow actions are supported: +.TP +\fBall_escape\fR \-\- toggle unescape all URLs (Default: \fBu\fR) +.TP +\fBall_shorten\fR \-\- toggle shorten all URLs (Default: \fBS\fR) +.TP +\fBbottom\fR \-\- move cursor to last item (Default: \fBG\fR) +.TP +\fBclear_screen\fR \-\- redraw screen (Default: \fBCtrl-l\fR) +.TP +\fBclipboard\fR \-\- copy highlighted URL to clipboard using xsel/xclip (Default: \fBC\fR) +.TP +\fBclipboard_pri\fR \-\- copy highlighted URL to primary selection using xsel/xclip (Default: \fBP\fR) +.TP +\fBcontext\fR \-\- show/hide context (Default: \fBc\fR) +.TP +\fBdown\fR \-\- cursor down (Default: \fBj\fR) +.TP +\fBhelp_menu\fR \-\- show/hide help menu (Default: \fBF1\fR) +.TP +\fBlink_handler\fR \-\- cycle link handling (webbrowser, xdg-open or custom) (Default: \fBl\fR) +.TP +\fBopen_url\fR \-\- open selected URL (Default: \fBspace\fR or \fBenter\fR) +.TP +\fBpalette\fR \-\- cycle through palettes (Default: \fBp\fR) +.TP +\fBquit\fR \-\- quit (Default: \fBq\fR or \fBQ\fR) +.TP +\fBshorten\fR \-\- toggle shorten highlighted URL (Default: \fBs\fR) +.TP +\fBtop\fR \-\- move to first list item (Default: \fBg\fR) +.TP +\fBup\fR \-\- cursor up (Default: \fBk\fR) + .SH FILES $HOME/.config/urlscan/config.json
