Hello community, here is the log from the commit of package python-pytidylib for openSUSE:Leap:15.2 checked in at 2020-03-02 13:21:48 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.2/python-pytidylib (Old) and /work/SRC/openSUSE:Leap:15.2/.python-pytidylib.new.26092 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-pytidylib" Mon Mar 2 13:21:48 2020 rev:11 rq:777059 version:0.3.2 Changes: -------- --- /work/SRC/openSUSE:Leap:15.2/python-pytidylib/python-pytidylib.changes 2020-01-15 15:52:28.239564452 +0100 +++ /work/SRC/openSUSE:Leap:15.2/.python-pytidylib.new.26092/python-pytidylib.changes 2020-03-02 13:21:49.162214885 +0100 @@ -1,0 +2,8 @@ +Tue May 14 20:22:54 UTC 2019 - John Jolly <[email protected]> + +- Updated to 0.3.2 + + No upstream changelog +- Added %check section + + Excluded the large file check + +------------------------------------------------------------------- Old: ---- pytidylib-0.2.4.tar.gz New: ---- pytidylib-0.3.2.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-pytidylib.spec ++++++ --- /var/tmp/diff_new_pack.narfFc/_old 2020-03-02 13:21:49.490215538 +0100 +++ /var/tmp/diff_new_pack.narfFc/_new 2020-03-02 13:21:49.490215538 +0100 @@ -1,7 +1,7 @@ # # spec file for package python-pytidylib # -# Copyright (c) 2016 SUSE LINUX GmbH, Nuernberg, Germany. +# Copyright (c) 2019 SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -12,27 +12,25 @@ # license that conforms to the Open Source Definition (Version 1.9) # published by the Open Source Initiative. -# Please submit bugfixes or comments via http://bugs.opensuse.org/ +# Please submit bugfixes or comments via https://bugs.opensuse.org/ # -%if 0%{?suse_version} && 0%{?suse_version} <= 1110 -%{!?python_sitelib: %global python_sitelib %(python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")} -%else -BuildArch: noarch -%endif +%{?!python_module:%define python_module() python-%{**} python3-%{**}} Name: python-pytidylib -Version: 0.2.4 +Version: 0.3.2 Release: 0 Summary: Python wrapper for HTML Tidy (tidylib) on Python 2 and 3 License: MIT Group: Development/Languages/Python -Url: http://countergram.com/open-source/pytidylib/ -Source: https://pypi.python.org/packages/b4/a0/b70cf2b7b4ee1f9d8fa0f1b4abbbac081a2638a580dabf29b8fb554d5fc1/pytidylib-%{version}.tar.gz +URL: http://countergram.com/open-source/pytidylib/ +Source: https://files.pythonhosted.org/packages/source/p/pytidylib/pytidylib-%{version}.tar.gz +BuildRequires: %{python_module pytest} +BuildRequires: %{python_module setuptools} +BuildRequires: fdupes BuildRequires: libtidy-devel -BuildRequires: python-devel -Requires: libtidy5 -BuildRoot: %{_tmppath}/%{name}-%{version}-build +BuildArch: noarch +%python_subpackages %description `PyTidyLib`_ is a Python package that wraps the `HTML Tidy`_ library. This @@ -62,14 +60,20 @@ %setup -q -n pytidylib-%{version} %build -python setup.py build +%python_build %install -python setup.py install --prefix=%{_prefix} --root=%{buildroot} +%python_install +%python_expand %fdupes %{buildroot}%{$python_sitelib} -%files -%defattr(-,root,root,-) -%doc LICENSE README +%check +# The large document test is excluded as it produces inconsistent +# results across architectures. +%pytest -k 'not test_large_document' + +%files %{python_files} +%doc README +%license LICENSE %{python_sitelib}/* %changelog ++++++ pytidylib-0.2.4.tar.gz -> pytidylib-0.3.2.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/pytidylib-0.2.4/PKG-INFO new/pytidylib-0.3.2/PKG-INFO --- old/pytidylib-0.2.4/PKG-INFO 2014-12-20 05:22:41.000000000 +0100 +++ new/pytidylib-0.3.2/PKG-INFO 2016-11-16 02:52:52.000000000 +0100 @@ -1,6 +1,6 @@ Metadata-Version: 1.1 Name: pytidylib -Version: 0.2.4 +Version: 0.3.2 Summary: Python wrapper for HTML Tidy (tidylib) on Python 2 and 3 Home-page: http://countergram.com/open-source/pytidylib/ Author: Jason Stitt @@ -18,12 +18,22 @@ * Indent the output, including proper (i.e. no) indenting for ``pre`` elements, which some (X)HTML indenting code overlooks. - Version usage - ============= + Changes + ======= - * Windows: 0.2.0 and later - * Python 3: Tests pass on 0.2.3 - * tidylib itself is not actively updated and may have problems with newer HTML + * 0.3.2: Initialization bug fix + + * 0.3.1: find_library support while still allowing a list of library names + + * 0.3.0: Refactored to use Tidy and PersistentTidy classes while keeping the + functional interface (which will lazily create a global Tidy() object) for + backward compatibility. You can now pass a list of library names and base + options when instantiating Tidy. The keep_doc argument is now deprecated + and does nothing; use PersistentTidy. + + * 0.2.4: Bugfix for a strange memory allocation corner case in Tidy. + + * 0.2.3: Python 3 support (2 + 3 cross compatible) with passing Tox tests. Small example of use ==================== diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/pytidylib-0.2.4/README new/pytidylib-0.3.2/README --- old/pytidylib-0.2.4/README 2014-12-11 19:31:37.000000000 +0100 +++ new/pytidylib-0.3.2/README 2016-09-22 23:42:09.000000000 +0200 @@ -1,14 +1,10 @@ -For documentation, see docs/html/index.html in this distribution, or -http://countergram.com/open-source/pytidylib/ +This is a Python wrapper around the HTML Tidy library. Quick start example: -Small example of use: +from tidylib import Tidy +tidy = Tidy() +document, errors = tidy.tidy_document('<p>fõo <img src="bar.jpg">', + options={'alt-text': 'baz'}) +print(document) +print(errors) -from tidylib import tidy_document -document, errors = tidy_document('''<p>fõo <img src="bar.jpg">''', - options={'numeric-entities':1}) -print document -print errors - -NOTE: HTML Tidy itself has currently not been updated for a long time, and may -not be, and it may have trouble with newer HTML. This is just a thin Python -wrapper around HTML Tidy, which is a separate project. +For full documentation, see the docs/ directory. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/pytidylib-0.2.4/setup.py new/pytidylib-0.3.2/setup.py --- old/pytidylib-0.2.4/setup.py 2014-12-20 05:19:09.000000000 +0100 +++ new/pytidylib-0.3.2/setup.py 2016-11-16 02:52:32.000000000 +0100 @@ -1,4 +1,4 @@ -# Copyright 2009 Jason Stitt +# Copyright 2009-2015 Jason Stitt # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal @@ -33,12 +33,22 @@ * Indent the output, including proper (i.e. no) indenting for ``pre`` elements, which some (X)HTML indenting code overlooks. -Version usage -============= +Changes +======= -* Windows: 0.2.0 and later -* Python 3: Tests pass on 0.2.3 -* tidylib itself is not actively updated and may have problems with newer HTML +* 0.3.2: Initialization bug fix + +* 0.3.1: find_library support while still allowing a list of library names + +* 0.3.0: Refactored to use Tidy and PersistentTidy classes while keeping the +functional interface (which will lazily create a global Tidy() object) for +backward compatibility. You can now pass a list of library names and base +options when instantiating Tidy. The keep_doc argument is now deprecated +and does nothing; use PersistentTidy. + +* 0.2.4: Bugfix for a strange memory allocation corner case in Tidy. + +* 0.2.3: Python 3 support (2 + 3 cross compatible) with passing Tox tests. Small example of use ==================== @@ -61,7 +71,7 @@ .. _`PyTidyLib`: http://countergram.com/open-source/pytidylib/ """ -VERSION = "0.2.4" +VERSION = "0.3.2" setup( name="pytidylib", @@ -73,16 +83,15 @@ url="http://countergram.com/open-source/pytidylib/", packages=['tidylib'], classifiers=[ - 'Development Status :: 5 - Production/Stable', - 'Environment :: Other Environment', - 'Intended Audience :: Developers', - 'License :: OSI Approved :: MIT License', - 'Programming Language :: Python', - 'Programming Language :: Python :: 3', - 'Natural Language :: English', - 'Topic :: Utilities', - 'Topic :: Text Processing :: Markup :: HTML', - 'Topic :: Text Processing :: Markup :: XML', - ], - ) - + 'Development Status :: 5 - Production/Stable', + 'Environment :: Other Environment', + 'Intended Audience :: Developers', + 'License :: OSI Approved :: MIT License', + 'Programming Language :: Python', + 'Programming Language :: Python :: 3', + 'Natural Language :: English', + 'Topic :: Utilities', + 'Topic :: Text Processing :: Markup :: HTML', + 'Topic :: Text Processing :: Markup :: XML', + ], +) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/pytidylib-0.2.4/tests/test_docs.py new/pytidylib-0.3.2/tests/test_docs.py --- old/pytidylib-0.2.4/tests/test_docs.py 2014-12-20 05:05:51.000000000 +0100 +++ new/pytidylib-0.3.2/tests/test_docs.py 2016-09-29 04:23:12.000000000 +0200 @@ -22,7 +22,7 @@ from __future__ import unicode_literals import unittest -from tidylib import tidy_document, release_tidy_doc, thread_local_doc +from tidylib import Tidy, PersistentTidy, tidy_document DOC = u'''<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"> <html> @@ -48,20 +48,20 @@ def test_alt_added_to_img(self): h = "<img src='foo'>" - expected = DOC % '''<img src='foo' alt="">''' - doc, err = tidy_document(h) + expected = DOC % '''<img src='foo' alt="bar">''' + doc, err = tidy_document(h, {'alt-text': 'bar'}) self.assertEqual(doc, expected) def test_entity_preserved_using_bytes(self): h = b"é" expected = (DOC % "é").encode('utf-8') - doc, err = tidy_document(h) + doc, err = tidy_document(h, {'preserve-entities': 1}) self.assertEqual(doc, expected) def test_numeric_entities_using_bytes(self): h = b"é" expected = (DOC % "é").encode('utf-8') - doc, err = tidy_document(h, {'numeric-entities': 1}) + doc, err = tidy_document(h, {'numeric-entities': 1, 'output-encoding': 'ascii'}) self.assertEqual(doc, expected) def test_non_ascii_preserved(self): @@ -76,6 +76,28 @@ doc, err = tidy_document(h) self.assertEqual(doc, expected) + def test_can_use_two_tidy_instances(self): + t1 = Tidy() + t2 = Tidy() + self.assertEqual(t1.tidy_document(DOC % 'a')[0], DOC % 'a') + self.assertEqual(t2.tidy_document(DOC % 'b')[0], DOC % 'b') + + def test_tidy_doesnt_persist_options(self): + tidy = Tidy() + # This option makes it a fragment + doc, err = tidy.tidy_document(DOC % 'a', {'show-body-only': 1}) + self.assertEqual(doc, 'a\n') + doc, err = tidy.tidy_document(DOC % 'a') + self.assertEqual(doc, DOC % 'a') + + def test_persistent_tidy_does_persist_options(self): + tidy = PersistentTidy() + # This option makes it a fragment + doc, err = tidy.tidy_document(DOC % 'a', {'show-body-only': 1}) + self.assertEqual(doc, 'a\n') + doc, err = tidy.tidy_document(DOC % 'a') + self.assertEqual(doc, 'a\n') + def test_xmlns_large_document_xml_corner_case(self): # Test for a super weird edge case in Tidy that can cause it to return # the wrong required buffer size. @@ -84,16 +106,6 @@ doc, err = tidy_document(html, {'output-xml': 1}) self.assertEqual(doc.strip()[-7:], "</html>") - def test_keep_document(self): - h = "hello" - expected = DOC % h - for i in range(4): - doc, err = tidy_document(h, keep_doc=True) - self.assertEqual(doc, expected) - assert hasattr(thread_local_doc, 'doc') - release_tidy_doc() - assert not hasattr(thread_local_doc, 'doc') - if __name__ == '__main__': unittest.main() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/pytidylib-0.2.4/tests/test_fragments.py new/pytidylib-0.3.2/tests/test_fragments.py --- old/pytidylib-0.2.4/tests/test_fragments.py 2014-12-11 19:31:37.000000000 +0100 +++ new/pytidylib-0.3.2/tests/test_fragments.py 2016-09-29 04:23:12.000000000 +0200 @@ -36,20 +36,20 @@ def test_alt_added_to_img(self): h = "<img src='foo'>" - expected = '''<img src='foo' alt="">''' - doc, err = tidy_fragment(h) + expected = '''<img src='foo' alt="bar">''' + doc, err = tidy_fragment(h, {'alt-text': 'bar'}) self.assertEqual(doc, expected) def test_entity_preserved_using_bytes(self): h = b"é" expected = b"é" - doc, err = tidy_fragment(h) + doc, err = tidy_fragment(h, {'preserve-entities': 1}) self.assertEqual(doc, expected) def test_numeric_entities_using_bytes(self): h = b"é" expected = b"é" - doc, err = tidy_fragment(h, {'numeric-entities': 1}) + doc, err = tidy_fragment(h, {'numeric-entities': 1, 'output-encoding': 'ascii'}) self.assertEqual(doc, expected) def test_non_ascii_preserved(self): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/pytidylib-0.2.4/tests/test_init.py new/pytidylib-0.3.2/tests/test_init.py --- old/pytidylib-0.2.4/tests/test_init.py 1970-01-01 01:00:00.000000000 +0100 +++ new/pytidylib-0.3.2/tests/test_init.py 2016-11-16 02:47:54.000000000 +0100 @@ -0,0 +1,32 @@ +# -*- coding: utf-8 -*- +# Copyright 2009-2016 Jason Stitt +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from __future__ import unicode_literals + +import unittest +from tidylib import Tidy, PersistentTidy, tidy_document + + +class TestDocs1(unittest.TestCase): + + def test_not_find_lib(self): + with self.assertRaises(OSError): + tidy = Tidy(lib_names=[]) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/pytidylib-0.2.4/tests/threadsafety.py new/pytidylib-0.3.2/tests/threadsafety.py --- old/pytidylib-0.2.4/tests/threadsafety.py 2014-12-11 19:31:37.000000000 +0100 +++ new/pytidylib-0.3.2/tests/threadsafety.py 2016-09-22 23:42:09.000000000 +0200 @@ -24,9 +24,8 @@ error_queue = Queue() -DOC = '''<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" - "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml"> +DOC = '''<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"> +<html> <head> <title></title> </head> @@ -63,5 +62,5 @@ if __name__ == '__main__': run_test() if not error_queue.empty(): - print "About %s errors out of %s" % (error_queue.qsize(), NUM_THREADS * NUM_TRIES) - print error_queue.get() + print("About %s errors out of %s" % (error_queue.qsize(), NUM_THREADS * NUM_TRIES)) + print(error_queue.get()) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/pytidylib-0.2.4/tidylib/__init__.py new/pytidylib-0.3.2/tidylib/__init__.py --- old/pytidylib-0.2.4/tidylib/__init__.py 2014-12-20 05:21:13.000000000 +0100 +++ new/pytidylib-0.3.2/tidylib/__init__.py 2016-09-22 23:42:09.000000000 +0200 @@ -1,203 +1 @@ -# Copyright 2009-2014 Jason Stitt -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -# THE SOFTWARE. - -import ctypes -import threading -import platform -from tidylib.sink import create_sink, destroy_sink - -__all__ = ['tidy_document', 'tidy_fragment', 'release_tidy_doc'] - -# -------------------------------------------------------------------------- # -# Constants - -LIB_NAMES = ['libtidy', 'libtidy.so', 'libtidy-0.99.so.0', 'cygtidy-0-99-0', - 'tidylib', 'libtidy.dylib', 'tidy'] -ENOMEM = -12 -BASE_OPTIONS = { - "indent": 1, # Pretty; not too much of a performance hit - "tidy-mark": 0, # No tidy meta tag in output - "wrap": 0, # No wrapping - "alt-text": "", # Help ensure validation - "doctype": 'strict', # Little sense in transitional for tool-generated markup... - "force-output": 1, # May not get what you expect but you will get something -} - -# Note: These are meant as sensible defaults. If you don't like these being -# applied by default, just set tidylib.BASE_OPTIONS = {} after importing. -# You can of course override any of these options when you call the -# tidy_document() or tidy_fragment() function - -# -------------------------------------------------------------------------- # -# Globals - -tidy = None -thread_local_doc = threading.local() - -# Fix for Windows b/c tidy uses stdcall on Windows -if "Windows" == platform.system(): - load_library = ctypes.windll.LoadLibrary -else: - load_library = ctypes.cdll.LoadLibrary - -for name in LIB_NAMES: - try: - tidy = load_library(name) - break - except OSError: - pass - -if tidy is None: - raise OSError("Could not load libtidy using any of these names: %s" % (",".join(LIB_NAMES))) - -tidy.tidyCreate.restype = ctypes.POINTER(ctypes.c_void_p) # Fix for 64-bit systems - -# -------------------------------------------------------------------------- # -# 3.x/2.x cross-compatibility - -try: - unicode # 2.x - - def is_unicode(obj): - return isinstance(obj, unicode) - - def encode_key_value(k, v): - return unicode(k).encode('utf-8'), unicode(v).encode('utf-8') -except NameError: - # 3.x - def is_unicode(obj): - return isinstance(obj, str) - - def encode_key_value(k, v): - return str(k).encode('utf-8'), str(v).encode('utf-8') - -# -------------------------------------------------------------------------- # -# Functions - - -def tidy_document(text, options=None, keep_doc=False): - """ Run a string with markup through HTML Tidy; return the corrected one. - - text: The markup, which may be anything from an empty string to a complete - (X)HTML document. If you pass in a unicode type (py3 str, py2 unicode) you - get one back out, and tidy will have some options set that may affect - behavior (e.g. named entities converted to plain unicode characters). If - you pass in a bytes type (py3 bytes, py2 str) you will get one of those - back. - - options (dict): Options passed directly to HTML Tidy; see the HTML Tidy docs - (http://tidy.sourceforge.net/docs/quickref.html) or run tidy -help-config - from the command line. - - keep_doc (boolean): If True, store 1 document object per thread and re-use - it, for a slight performance boost especially when tidying very large numbers - of very short documents. - - returns (str, str): The tidied markup and unparsed warning/error messages. - Warnings and errors are returned just as tidylib returns them. - """ - global tidy, option_names - - # Unicode approach is to encode as string, then decode libtidy output - use_unicode = False - if is_unicode(text): - use_unicode = True - text = text.encode('utf-8') - - # Manage thread-local storage of persistent document object - if keep_doc: - if not hasattr(thread_local_doc, 'doc'): - thread_local_doc.doc = tidy.tidyCreate() - doc = thread_local_doc.doc - else: - doc = tidy.tidyCreate() - - # This is where error messages are sent by libtidy - sink = create_sink() - tidy.tidySetErrorSink(doc, sink) - - try: - # Set options on the document - # If keep_doc=True, options will persist between calls, but they can - # be overridden, and the BASE_OPTIONS will be set each time - tidy_options = dict(BASE_OPTIONS) - if options: - tidy_options.update(options) - if use_unicode: - tidy_options['input-encoding'] = 'utf8' - tidy_options['output-encoding'] = 'utf8' - for key in tidy_options: - value = tidy_options[key] - key = key.replace('_', '-') - if value is None: - value = '' - key, value = encode_key_value(key, value) - tidy.tidyOptParseValue(doc, key, value) - error = str(sink) - if error: - raise ValueError("(tidylib) " + error) - - # The point of the whole thing - tidy.tidyParseString(doc, text) - tidy.tidyCleanAndRepair(doc) - - # Guess at buffer size; tidy returns ENOMEM if the buffer is too - # small and puts the required size into out_length - out_length = ctypes.c_int(8192) - out = ctypes.c_buffer(out_length.value) - while ENOMEM == tidy.tidySaveString(doc, out, ctypes.byref(out_length)): - out = ctypes.c_buffer(out_length.value) - - document = out.value - if use_unicode: - document = document.decode('utf-8') - errors = str(sink) - finally: - destroy_sink(sink) - if not keep_doc: - tidy.tidyRelease(doc) - - return (document, errors) - - -def tidy_fragment(text, options=None, keep_doc=False): - """ Tidy a string with markup and return only the <body> contents. - - HTML Tidy normally returns a full (X)HTML document; this function returns only - the contents of the <body> element and is meant to be used for snippets. - Calling tidy_fragment on elements that don't go in the <body>, like <title>, - will produce incorrect behavior. - - Arguments and return value are the same as tidy_document. Note that HTML - Tidy will always complain about the lack of a doctype and <title> element - in fragments, and these errors are not stripped out for you. """ - options = dict(options) if options else dict() - options["show-body-only"] = 1 - document, errors = tidy_document(text, options, keep_doc) - document = document.strip() - return document, errors - - -def release_tidy_doc(): - """ Release the stored document object in the current thread. Only useful - if you have called tidy_document or tidy_fragament with keep_doc=True. """ - if hasattr(thread_local_doc, 'doc'): - tidy.tidyRelease(thread_local_doc.doc) - del thread_local_doc.doc +from .tidy import Tidy, PersistentTidy, tidy_document, tidy_fragment, release_tidy_doc diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/pytidylib-0.2.4/tidylib/tidy.py new/pytidylib-0.3.2/tidylib/tidy.py --- old/pytidylib-0.2.4/tidylib/tidy.py 1970-01-01 01:00:00.000000000 +0100 +++ new/pytidylib-0.3.2/tidylib/tidy.py 2016-11-16 02:49:58.000000000 +0100 @@ -0,0 +1,239 @@ +# Copyright 2009-2015 Jason Stitt +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import ctypes +import ctypes.util +import threading +import platform +import warnings +from contextlib import contextmanager +from .sink import create_sink, destroy_sink + +__all__ = ['Tidy', 'PersistentTidy'] + +# Default search order for library names if nothing is passed in +LIB_NAMES = ['libtidy', 'libtidy.so', 'libtidy-0.99.so.0', 'cygtidy-0-99-0', + 'tidylib', 'libtidy.dylib', 'tidy'] + +# Error code from library +ENOMEM = -12 + +# Default options; can be overriden with argument to Tidy() +BASE_OPTIONS = { + "indent": 1, # Pretty; not too much of a performance hit + "tidy-mark": 0, # No tidy meta tag in output + "wrap": 0, # No wrapping + "alt-text": "", # Help ensure validation + "doctype": 'strict', # Little sense in transitional for tool-generated markup... + "force-output": 1, # May not get what you expect but you will get something +} + +KEEP_DOC_WARNING = "keep_doc and release_tidy_doc are no longer used. Create a PersistentTidy object instead." + +# Fix for Windows b/c tidy uses stdcall on Windows +if "Windows" == platform.system(): + load_library = ctypes.windll.LoadLibrary +else: + load_library = ctypes.cdll.LoadLibrary + +# -------------------------------------------------------------------------- # +# 3.x/2.x cross-compatibility + +try: + unicode # 2.x + + def is_unicode(obj): + return isinstance(obj, unicode) + + def encode_key_value(k, v): + return unicode(k).encode('utf-8'), unicode(v).encode('utf-8') +except NameError: + # 3.x + def is_unicode(obj): + return isinstance(obj, str) + + def encode_key_value(k, v): + return str(k).encode('utf-8'), str(v).encode('utf-8') + +# -------------------------------------------------------------------------- # +# The main python interface + + +class Tidy(object): + + """ Wrapper around the HTML Tidy library for cleaning up possibly invalid + HTML and XHTML. """ + + def __init__(self, lib_names=None): + self._tidy = None + if lib_names is None: + lib_names = ctypes.util.find_library('tidy') or LIB_NAMES + if isinstance(lib_names, str): + lib_names = [lib_names] + for name in lib_names: + try: + self._tidy = load_library(name) + break + except OSError: + continue + if self._tidy is None: + raise OSError( + "Could not load libtidy using any of these names: " + + ",".join(lib_names)) + self._tidy.tidyCreate.restype = ctypes.POINTER(ctypes.c_void_p) # Fix for 64-bit systems + + @contextmanager + def _doc_and_sink(self): + " Create and cleanup a Tidy document and error sink " + doc = self._tidy.tidyCreate() + sink = create_sink() + self._tidy.tidySetErrorSink(doc, sink) + yield (doc, sink) + destroy_sink(sink) + self._tidy.tidyRelease(doc) + + def tidy_document(self, text, options=None): + """ Run a string with markup through HTML Tidy; return the corrected one + and any error output. + + text: The markup, which may be anything from an empty string to a complete + (X)HTML document. If you pass in a unicode type (py3 str, py2 unicode) you + get one back out, and tidy will have some options set that may affect + behavior (e.g. named entities converted to plain unicode characters). If + you pass in a bytes type (py3 bytes, py2 str) you will get one of those + back. + + options (dict): Options passed directly to HTML Tidy; see the HTML Tidy docs + (http://tidy.sourceforge.net/docs/quickref.html) or run tidy -help-config + from the command line. + + returns (str, str): The tidied markup and unparsed warning/error messages. + Warnings and errors are returned just as tidylib returns them. + """ + + # Unicode approach is to encode as string, then decode libtidy output + use_unicode = False + if is_unicode(text): + use_unicode = True + text = text.encode('utf-8') + + with self._doc_and_sink() as (doc, sink): + tidy_options = dict(BASE_OPTIONS) + if options: + tidy_options.update(options) + if use_unicode: + tidy_options['input-encoding'] = 'utf8' + tidy_options['output-encoding'] = 'utf8' + for key in tidy_options: + value = tidy_options[key] + key = key.replace('_', '-') + if value is None: + value = '' + key, value = encode_key_value(key, value) + self._tidy.tidyOptParseValue(doc, key, value) + error = str(sink) + if error: + raise ValueError("(tidylib) " + error) + + self._tidy.tidyParseString(doc, text) + self._tidy.tidyCleanAndRepair(doc) + + # Guess at buffer size; tidy returns ENOMEM if the buffer is too + # small and puts the required size into out_length + out_length = ctypes.c_int(8192) + out = ctypes.c_buffer(out_length.value) + while ENOMEM == self._tidy.tidySaveString(doc, out, ctypes.byref(out_length)): + out = ctypes.c_buffer(out_length.value) + + document = out.value + if use_unicode: + document = document.decode('utf-8') + errors = str(sink) + + return (document, errors) + + def tidy_fragment(self, text, options=None): + """ Tidy a string with markup and return only the <body> contents. + + HTML Tidy normally returns a full (X)HTML document; this function returns only + the contents of the <body> element and is meant to be used for snippets. + Calling tidy_fragment on elements that don't go in the <body>, like <title>, + will produce incorrect behavior. + + Arguments and return value are the same as tidy_document. Note that HTML + Tidy will always complain about the lack of a doctype and <title> element + in fragments, and these errors are not stripped out for you. """ + options = dict(options) if options else dict() + options["show-body-only"] = 1 + document, errors = self.tidy_document(text, options) + document = document.strip() + return document, errors + + +class PersistentTidy(Tidy): + + """ Functions the same as the Tidy class but keeps a persistent reference + to one Tidy document object. This increases performance slightly when + tidying many documents in a row. It also persists all options (not just + the base options) between runs, which could lead to unexpected behavior. + If you plan to use different options on each run with PersistentTidy, set + all options that could change on every call. Note that passing in unicode + text will result in the input-encoding and output-encoding options being + automatically set. Thread-local storage is used for the document object + (one document per thread). """ + + def __init__(self, lib_names=None): + Tidy.__init__(self, lib_names) + self._local = threading.local() + self._local.doc = self._tidy.tidyCreate() + + def __del__(self): + self._tidy.tidyRelease(self._local.doc) + + @contextmanager + def _doc_and_sink(self): + " Create and cleanup an error sink but use the persistent doc object " + sink = create_sink() + self._tidy.tidySetErrorSink(self._local.doc, sink) + yield (self._local.doc, sink) + destroy_sink(sink) + + +def tidy_document(text, options=None, keep_doc=False): + if keep_doc: + warnings.warn(KEEP_DOC_WARNING, DeprecationWarning, stacklevel=2) + return get_module_tidy().tidy_document(text, options) + + +def tidy_fragment(text, options=None, keep_doc=False): + if keep_doc: + warnings.warn(KEEP_DOC_WARNING, DeprecationWarning, stacklevel=2) + return get_module_tidy().tidy_fragment(text, options) + + +def get_module_tidy(): + global _tidy + if '_tidy' not in globals(): + _tidy = Tidy() + return _tidy + + +def release_tidy_doc(): + warnings.warn(KEEP_DOC_WARNING, DeprecationWarning, stacklevel=2)
