--- Begin Message ---
Package: python-html5lib
Version: 0.999-2
Severity: important
(Filing this as “important” since it breaks the entire package for my
use case and a reverse dependency, feel free to downgrade severity).
python-html5lib seems to have changed its API in the 0.999 upstream
release.
I get the following error when running planet-venus:
ERROR:planet.runner:Error processing
http://blogs.noname-ev.de/commandline-tools/feeds/index.rss2
ERROR:planet.runner:AttributeError: 'module' object has no attribute
'TreeBuilder'
ERROR:planet.runner: File "/usr/lib/pymodules/python2.7/planet/spider.py",
line 472, in spiderPlanet
writeCache(uri, feed_info, data)
ERROR:planet.runner: File "/usr/lib/pymodules/python2.7/planet/spider.py",
line 279, in writeCache
reconstitute.source(xdoc.documentElement,data.feed,data.bozo,data.version)
ERROR:planet.runner: File
"/usr/lib/pymodules/python2.7/planet/reconstitute.py", line 231, in source
content(xsource, 'subtitle', source.get('subtitle_detail',None), bozo)
ERROR:planet.runner: File
"/usr/lib/pymodules/python2.7/planet/reconstitute.py", line 167, in content
parser = html5parser.HTMLParser(tree=dom.TreeBuilder)
This error can be addressed by patching planet-venus as follows:
--- i/planet/reconstitute.py
+++ w/planet/reconstitute.py
@@ -18,6 +18,7 @@ from xml.sax.saxutils import escape
from xml.dom import minidom, Node
from html5lib import html5parser
from html5lib.treebuilders import dom
+from html5lib import treebuilders
import planet, config
try:
@@ -164,7 +165,7 @@ def content(xentry, name, detail, bozo):
bozo=1
if detail.type.find('xhtml')<0 or bozo:
- parser = html5parser.HTMLParser(tree=dom.TreeBuilder)
+ parser =
html5parser.HTMLParser(tree=treebuilders.getTreeBuilder("dom"))
html = parser.parse(xdiv % detail.value, encoding="utf-8")
for body in html.documentElement.childNodes:
if body.nodeType != Node.ELEMENT_NODE: continue
But, even after that is fixed, there are errors which I wasn’t able to
fix:
ERROR:planet.runner:Error processing https://raumzeitlabor.de/feed/
ERROR:planet.runner:AttributeError: 'module' object has no attribute
'XHTMLSerializer'
ERROR:planet.runner: File "/usr/lib/pymodules/python2.7/planet/spider.py",
line 472, in spiderPlanet
writeCache(uri, feed_info, data)
ERROR:planet.runner: File "/usr/lib/pymodules/python2.7/planet/spider.py",
line 166, in writeCache
scrub.scrub(feed_uri, data)
ERROR:planet.runner: File "/usr/lib/pymodules/python2.7/planet/scrub.py", line
137, in scrub
xhtml = serializer.XHTMLSerializer(inject_meta_charset = False)
The XHTMLSerializer seems to be gone entirely. I am not sure whether the
HTMLSerializer is enough for what planet-venus does. Just using that did
not work for me, though, I get exceptions with this patch:
--- i/planet/scrub.py
+++ w/planet/scrub.py
@@ -131,10 +131,11 @@ def scrub(feed_uri, data):
# Run this through HTML5's serializer
from html5lib import html5parser, sanitizer, treebuilders
from html5lib import treewalkers, serializer
+ from html5lib.serializer import HTMLSerializer
p = html5parser.HTMLParser(tokenizer=sanitizer.HTMLSanitizer,
tree=treebuilders.getTreeBuilder('dom'))
doc = p.parseFragment(node.value, encoding='utf-8')
- xhtml = serializer.XHTMLSerializer(inject_meta_charset = False)
+ xhtml = HTMLSerializer(inject_meta_charset = False)
walker = treewalkers.getTreeWalker('dom')
- tree = xhtml.serialize(walker(doc), encoding='utf-8')
+ tree = xhtml.render(walker(doc), encoding='utf-8')
node['value'] = ''.join([str(token) for token in tree])
The error is:
ERROR:planet.runner:Error processing http://soup.wobbl.es/rss
ERROR:planet.runner:TypeError: 'NoneType' object has no attribute '__getitem__'
ERROR:planet.runner: File "/usr/lib/pymodules/python2.7/planet/spider.py",
line 472, in spiderPlanet
writeCache(uri, feed_info, data)
ERROR:planet.runner: File "/usr/lib/pymodules/python2.7/planet/spider.py",
line 166, in writeCache
scrub.scrub(feed_uri, data)
ERROR:planet.runner: File "/usr/lib/pymodules/python2.7/planet/scrub.py", line
140, in scrub
tree = xhtml.render(walker(doc), encoding='utf-8')
ERROR:planet.runner: File
"/usr/lib/python2.7/dist-packages/html5lib/serializer/htmlserializer.py", line
307, in render
return b"".join(list(self.serialize(treewalker, encoding)))
ERROR:planet.runner: File
"/usr/lib/python2.7/dist-packages/html5lib/serializer/htmlserializer.py", line
199, in serialize
for token in treewalker:
ERROR:planet.runner: File
"/usr/lib/python2.7/dist-packages/html5lib/filters/optionaltags.py", line 18,
in __iter__
type = token["type"]
I then tried s/dom/lxml/ and installed python-lxml, but no dice:
ERROR:planet.runner:Error processing http://soup.wobbl.es/rss
ERROR:planet.runner:AttributeError: DocumentFragment instance has no attribute
'tag'
ERROR:planet.runner: File "/usr/lib/pymodules/python2.7/planet/spider.py",
line 472, in spiderPlanet
writeCache(uri, feed_info, data)
ERROR:planet.runner: File "/usr/lib/pymodules/python2.7/planet/spider.py",
line 166, in writeCache
scrub.scrub(feed_uri, data)
ERROR:planet.runner: File "/usr/lib/pymodules/python2.7/planet/scrub.py", line
140, in scrub
tree = xhtml.render(walker(doc), encoding='utf-8')
ERROR:planet.runner: File
"/usr/lib/python2.7/dist-packages/html5lib/serializer/htmlserializer.py", line
307, in render
return b"".join(list(self.serialize(treewalker, encoding)))
ERROR:planet.runner: File
"/usr/lib/python2.7/dist-packages/html5lib/serializer/htmlserializer.py", line
199, in serialize
for token in treewalker:
ERROR:planet.runner: File
"/usr/lib/python2.7/dist-packages/html5lib/filters/optionaltags.py", line 17,
in __iter__
for previous, token, next in self.slider():
ERROR:planet.runner: File
"/usr/lib/python2.7/dist-packages/html5lib/filters/optionaltags.py", line 9, in
slider
for token in self.source:
ERROR:planet.runner: File
"/usr/lib/python2.7/dist-packages/html5lib/treewalkers/_base.py", line 144, in
__iter__
details = self.getNodeDetails(currentNode)
ERROR:planet.runner: File
"/usr/lib/python2.7/dist-packages/html5lib/treewalkers/lxmletree.py", line 145,
in getNodeDetails
elif node.tag == etree.Comment:
Any idea on how to fix planet-venus? Would it make sense to add some
compatibility code?
--- End Message ---
