[Python-modules-team] Bug#922833: lxml breaks python-mechanicalsoup autopkgtest
Package: python-mechanicalsoup
Followup-For: Bug #922833
User: ubuntu-de...@lists.ubuntu.com
Usertags: origin-ubuntu disco ubuntu-patch
In Ubuntu, the attached patch was applied to achieve the following:
* Fix tests to use https for httpbin
* Do not expect a Connection header in test_submit_online()
Thanks for considering the patch.
-- System Information:
Debian Release: buster/sid
APT prefers disco
APT policy: (991, 'disco'), (500, 'disco'), (500, 'cosmic-security')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 4.19.0-13-generic (SMP w/8 CPU cores)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8),
LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en
diff -Nru python-mechanicalsoup-0.10.0/debian/patches/https-httpbin
python-mechanicalsoup-0.10.0/debian/patches/https-httpbin
--- python-mechanicalsoup-0.10.0/debian/patches/https-httpbin 1970-01-01
01:00:00.0 +0100
+++ python-mechanicalsoup-0.10.0/debian/patches/https-httpbin 2019-03-04
11:20:03.0 +0100
@@ -0,0 +1,104 @@
+Description: Fix httpbin tests to use https
+ This is needed because httpbin now does https redirects and the
+ tests fail.
+Author: Julian Andres Klode
+Last-Update: 2019-03-04
+
+--- python-mechanicalsoup-0.10.0.orig/tests/test_browser.py
python-mechanicalsoup-0.10.0/tests/test_browser.py
+@@ -8,7 +8,7 @@ import pytest
+
+
+ def test_submit_online(httpbin):
+-"""Complete and submit the pizza form at http://httpbin.org/forms/post """
++"""Complete and submit the pizza form at https://httpbin.org/forms/post
"""
+ browser = mechanicalsoup.Browser()
+ page = browser.get(httpbin + "/forms/post")
+ form = page.soup.form
+@@ -23,7 +23,7 @@ def test_submit_online(httpbin):
+
+ response = browser.submit(form, page.url)
+
+-# helpfully the form submits to http://httpbin.org/post which simply
++# helpfully the form submits to https://httpbin.org/post which simply
+ # returns the request headers in json format
+ json = response.json()
+ data = json["form"]
+@@ -38,7 +38,7 @@ def test_submit_online(httpbin):
+
+
+ form_html = """
+-http://httpbin.org/post;>
++https://httpbin.org/post;>
+
+
+ freezer
+--- python-mechanicalsoup-0.10.0.orig/tests/test_form.py
python-mechanicalsoup-0.10.0/tests/test_form.py
+@@ -6,7 +6,7 @@ import pytest
+
+
+ def test_submit_online(httpbin):
+-"""Complete and submit the pizza form at http://httpbin.org/forms/post """
++"""Complete and submit the pizza form at https://httpbin.org/forms/post
"""
+ browser = mechanicalsoup.Browser()
+ page = browser.get(httpbin + "/forms/post")
+ form = mechanicalsoup.Form(page.soup.form)
+@@ -25,7 +25,7 @@ def test_submit_online(httpbin):
+
+ response = browser.submit(form, page.url)
+
+-# helpfully the form submits to http://httpbin.org/post which simply
++# helpfully the form submits to https://httpbin.org/post which simply
+ # returns the request headers in json format
+ json = response.json()
+ data = json["form"]
+@@ -37,7 +37,7 @@ def test_submit_online(httpbin):
+
+
+ def test_submit_set(httpbin):
+-"""Complete and submit the pizza form at http://httpbin.org/forms/post """
++"""Complete and submit the pizza form at https://httpbin.org/forms/post
"""
+ browser = mechanicalsoup.Browser()
+ page = browser.get(httpbin + "/forms/post")
+ form = mechanicalsoup.Form(page.soup.form)
+@@ -51,7 +51,7 @@ def test_submit_set(httpbin):
+
+ response = browser.submit(form, page.url)
+
+-# helpfully the form submits to http://httpbin.org/post which simply
++# helpfully the form submits to https://httpbin.org/post which simply
+ # returns the request headers in json format
+ json = response.json()
+ data = json["form"]
+--- python-mechanicalsoup-0.10.0.orig/tests/test_stateful_browser.py
python-mechanicalsoup-0.10.0/tests/test_stateful_browser.py
+@@ -25,7 +25,7 @@ def test_request_forward():
+
+
+ def test_submit_online(httpbin):
+-"""Complete and submit the pizza form at http://httpbin.org/forms/post """
++"""Complete and submit the pizza form at https://httpbin.org/forms/post
"""
+ browser = mechanicalsoup.StatefulBrowser()
+ browser.set_user_agent('testing MechanicalSoup')
+ browser.open(httpbin.url)
+@@ -217,7 +217,7 @@ def test_verbose(capsys):
+
+ def test_new_control():
+ browser = mechanicalsoup.StatefulBrowser()
+-browser.open("http://httpbin.org/forms/post;)
++browser.open("https://httpbin.org/forms/post;)
+ browser.select_form("form")
+ with pytest.raises(mechanicalsoup.LinkNotFoundError):
+ # The control doesn't exist,
[Python-modules-team] Bug#922833: lxml breaks python-mechanicalsoup autopkgtest
Source: lxml, python-mechanicalsoup
Control: found -1 lxml/4.3.1-1
Control: found -1 python-mechanicalsoup/0.10.0-2
Severity: important
X-Debbugs-CC: debian...@lists.debian.org
User: debian...@lists.debian.org
Usertags: breaks needs-update
Dear maintainers,
With a recent upload of lxml the autopkgtest of python-mechanicalsoup
fails in testing when that autopkgtest is run with the binary packages
of lxml from unstable. It passes when run with only packages from
testing. In tabular form:
passfail
lxml from testing4.3.1-1
python-mechanicalsoup from testing0.10.0-2
versioned deps [0] from testingfrom unstable
all others from testingfrom testing
I copied some of the output at the bottom of this report. I can't judge
the first error, the second error smells like the reference data needs
an update in python-mechanicalsoup.
Currently this regression is blocking the migration of lxml to testing
[1]. Due to the nature of this issue, I filed this bug report against
both packages. Can you please investigate the situation and reassign the
bug to the right package? If needed, please change the bug's severity.
More information about this bug and the reason for filing it can be found on
https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation
Paul
[0] You can see what packages were added from the second line of the log
file quoted below. The migration software adds source package from
unstable to the list if they are needed to install packages from
lxml/4.3.1-1. I.e. due to versioned dependencies or breaks/conflicts.
[1] https://qa.debian.org/excuses.php?package=lxml
https://ci.debian.net/data/autopkgtest/testing/amd64/p/python-mechanicalsoup/1969240/log.gz
=== FAILURES
===
__ test_submit_online
__
httpbin =
def test_submit_online(httpbin):
"""Complete and submit the pizza form at
http://httpbin.org/forms/post """
browser = mechanicalsoup.StatefulBrowser()
browser.set_user_agent('testing MechanicalSoup')
browser.open(httpbin.url)
for link in browser.links():
if link["href"] == "/":
browser.follow_link(link)
break
browser.follow_link("forms/post")
assert browser.get_url() == httpbin + "/forms/post"
browser.select_form("form")
browser["custname"] = "Customer Name Here"
browser["size"] = "medium"
browser["topping"] = ("cheese", "bacon")
# Change our mind to make sure old boxes are unticked
browser["topping"] = ("cheese", "onion")
browser["comments"] = "Some comment here"
browser.get_current_form().set("nosuchfield", "new value", True)
response = browser.submit_selected()
json = response.json()
data = json["form"]
assert data["custname"] == "Customer Name Here"
assert data["custtel"] == "" # web browser submits "" for input
left blank
assert data["size"] == "medium"
assert set(data["topping"]) == set(("cheese", "onion"))
assert data["comments"] == "Some comment here"
assert data["nosuchfield"] == "new value"
assert json["headers"]["User-Agent"] == 'testing MechanicalSoup'
# Ensure we haven't blown away any regular headers
expected_headers = ('Content-Length', 'Host', 'Content-Type',
'Connection',
'Accept', 'User-Agent', 'Accept-Encoding')
> assert set(expected_headers).issubset(json["headers"].keys())
E AssertionError: assert False
E+ where False = (dict_keys(['Accept', 'Accept-Encoding',
'Content-Length', 'Content-Type', 'Host', 'Referer', 'User-Agent']))
E+where = {'Accept', 'Accept-Encoding', 'Connection',
'Content-Length', 'Content-Type', 'Host', ...}.issubset
E+ where {'Accept', 'Accept-Encoding', 'Connection',
'Content-Length', 'Content-Type', 'Host', ...} = set(('Content-Length',
'Host', 'Content-Type', 'Connection', 'Accept', 'User-Agent', ...))
E+and dict_keys(['Accept', 'Accept-Encoding',
'Content-Length', 'Content-Type', 'Host', 'Referer', 'User-Agent']) =
()
E+ where = {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate',
'Content-Length': '147', 'Content-Type':
'application/x-www-form-urlencoded', ...}.keys
tests/test_stateful_browser.py:60: AssertionError
__ test_open_relative
__
httpbin =
def test_open_relative(httpbin):
# Open an arbitrary httpbin page to set the current URL
browser = mechanicalsoup.StatefulBrowser()
browser.open(httpbin + "/html")
# Open a relative page and make sure remote host and browser
agree on URL
resp = browser.open_relative("/get")
> assert resp.json()['url'] == httpbin + "/get"
E
