Hello community,
here is the log from the commit of package python-MechanicalSoup for
openSUSE:Leap:15.2 checked in at 2020-03-27 16:43:38
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Leap:15.2/python-MechanicalSoup (Old)
and /work/SRC/openSUSE:Leap:15.2/.python-MechanicalSoup.new.3160 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-MechanicalSoup"
Fri Mar 27 16:43:38 2020 rev:5 rq:785890 version:0.12.0
Changes:
--------
---
/work/SRC/openSUSE:Leap:15.2/python-MechanicalSoup/python-MechanicalSoup.changes
2020-01-15 15:46:06.935346418 +0100
+++
/work/SRC/openSUSE:Leap:15.2/.python-MechanicalSoup.new.3160/python-MechanicalSoup.changes
2020-03-27 16:43:53.739784268 +0100
@@ -1,0 +2,32 @@
+Tue Mar 17 10:58:15 UTC 2020 - Tomáš Chvátal <[email protected]>
+
+- Fix build on Leap
+
+-------------------------------------------------------------------
+Thu Sep 5 15:01:16 UTC 2019 - Todd R <[email protected]>
+
+- Update to 0.12.0:
+ + Main changes:
+ * Changes in official python version support: added 3.7 and dropped 3.4.
+ * Added ability to submit a form without updating ``StatefulBrowser``
internal
+ state: ``submit_selected(..., update_state=False)``. This means you get a
+ response from the form submission, but your browser stays on the same
page.
+ Useful for handling forms that result in a file download or open a new
tab.
+ + Bug fixes
+ * Improve handling of form enctype to behave like a real browser.
+ * HTML ``type`` attributes are no longer required to be lowercase.
+ * Form controls with the ``disabled`` attribute will no longer be submitted
+ to improve compliance with the HTML standard. If you were relying on this
+ bug to submit disabled elements, you can still achieve this by deleting
the
+ ``disabled`` attribute from the element in the
:class:`~mechanicalsoup.Form`
+ object directly.
+ * When a form containing a file input field is submitted without choosing a
+ file, an empty filename & content will be sent just like in a real
browser.
+ * ``<option>`` tags without a ``value`` attribute will now use their text
as
+ the value.
+ * The optional ``url_regex`` argument to ``follow_link`` and
``download_link``
+ was fixed so that it is no longer ignored.
+ * Allow duplicate submit elements instead of raising a LinkNotFoundError.
+- Drop upstream-included bs4-47.patch
+
+-------------------------------------------------------------------
Old:
----
MechanicalSoup-0.11.0.tar.gz
bs4-47.patch
New:
----
MechanicalSoup-0.12.0.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ python-MechanicalSoup.spec ++++++
--- /var/tmp/diff_new_pack.yVw7Yb/_old 2020-03-27 16:43:54.079784468 +0100
+++ /var/tmp/diff_new_pack.yVw7Yb/_new 2020-03-27 16:43:54.079784468 +0100
@@ -1,7 +1,7 @@
#
# spec file for package python-MechanicalSoup
#
-# Copyright (c) 2019 SUSE LINUX GmbH, Nuernberg, Germany.
+# Copyright (c) 2020 SUSE LLC
#
# All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed
@@ -18,15 +18,15 @@
%{?!python_module:%define python_module() python-%{**} python3-%{**}}
Name: python-MechanicalSoup
-Version: 0.11.0
+Version: 0.12.0
Release: 0
Summary: A Python library for automating interaction with websites
License: MIT
-Group: Development/Languages/Python
URL: https://github.com/hickford/MechanicalSoup
Source:
https://files.pythonhosted.org/packages/source/M/MechanicalSoup/MechanicalSoup-%{version}.tar.gz
-Patch0: bs4-47.patch
BuildRequires: %{python_module beautifulsoup4 >= 4.4}
+BuildRequires: %{python_module httpbin}
+BuildRequires: %{python_module jsonschema >= 2.5.1}
BuildRequires: %{python_module lxml}
BuildRequires: %{python_module pytest-httpbin}
BuildRequires: %{python_module pytest-mock}
@@ -41,6 +41,8 @@
Requires: python-lxml
Requires: python-requests >= 2.0
Requires: python-six >= 1.4
+Recommends: python-httpbin
+Recommends: python-jsonschema >= 2.5.1
BuildArch: noarch
%python_subpackages
@@ -57,7 +59,6 @@
%prep
%setup -q -n MechanicalSoup-%{version}
-%patch0 -p1
# do not require cov/xdist/etc
sed -i -e '/addopts/d' setup.cfg
@@ -69,7 +70,8 @@
%python_expand %fdupes %{buildroot}%{$python_sitelib}
%check
-%python_expand PYTHONPATH=%{buildroot}%{$python_sitelib}
py.test-%{$python_bin_suffix} -v
+# https://github.com/MechanicalSoup/MechanicalSoup/issues/299
test_enctype_and_file_submit
+%pytest -k 'not test_enctype_and_file_submit'
%files %{python_files}
%doc README.rst
++++++ MechanicalSoup-0.11.0.tar.gz -> MechanicalSoup-0.12.0.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/MechanicalSoup-0.11.0/MechanicalSoup.egg-info/PKG-INFO
new/MechanicalSoup-0.12.0/MechanicalSoup.egg-info/PKG-INFO
--- old/MechanicalSoup-0.11.0/MechanicalSoup.egg-info/PKG-INFO 2018-09-12
00:56:35.000000000 +0200
+++ new/MechanicalSoup-0.12.0/MechanicalSoup.egg-info/PKG-INFO 2019-08-27
18:44:28.000000000 +0200
@@ -1,14 +1,13 @@
Metadata-Version: 1.1
Name: MechanicalSoup
-Version: 0.11.0
+Version: 0.12.0
Summary: A Python library for automating interaction with websites
Home-page: https://mechanicalsoup.readthedocs.io/
Author: UNKNOWN
Author-email: UNKNOWN
License: MIT
-Description: .. image:: /assets/mechanical-soup-logo.png
- :alt: MechanicalSoup. A Python library for automating website
- interaction.
+Description: .. image::
https://raw.githubusercontent.com/MechanicalSoup/MechanicalSoup/master/assets/mechanical-soup-logo.png
+ :alt: MechanicalSoup. A Python library for automating website
interaction.
Home page
---------
@@ -26,8 +25,8 @@
MechanicalSoup was created by `M Hickford
<https://github.com/hickford/>`__, who was a fond user of the
`Mechanize <https://github.com/jjlee/mechanize>`__ library.
- Unfortunately, Mechanize is `incompatible with Python 3
- <https://github.com/jjlee/mechanize/issues/96>`__ and its development
+ Unfortunately, Mechanize was `incompatible with Python 3 until 2019
+ <https://github.com/python-mechanize/mechanize/issues/9>`__ and its
development
stalled for several years. MechanicalSoup provides a similar API,
built on Python
giants `Requests <http://docs.python-requests.org/en/latest/>`__ (for
HTTP sessions) and `BeautifulSoup
@@ -146,6 +145,6 @@
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
+Classifier: Programming Language :: Python :: 3.7
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/MechanicalSoup-0.11.0/MechanicalSoup.egg-info/requires.txt
new/MechanicalSoup-0.12.0/MechanicalSoup.egg-info/requires.txt
--- old/MechanicalSoup-0.11.0/MechanicalSoup.egg-info/requires.txt
2018-09-12 00:56:35.000000000 +0200
+++ new/MechanicalSoup-0.12.0/MechanicalSoup.egg-info/requires.txt
2019-08-27 18:44:28.000000000 +0200
@@ -1,4 +1,4 @@
requests>=2.0
-beautifulsoup4
+beautifulsoup4>=4.4
six>=1.4
lxml
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/PKG-INFO
new/MechanicalSoup-0.12.0/PKG-INFO
--- old/MechanicalSoup-0.11.0/PKG-INFO 2018-09-12 00:56:35.000000000 +0200
+++ new/MechanicalSoup-0.12.0/PKG-INFO 2019-08-27 18:44:30.000000000 +0200
@@ -1,14 +1,13 @@
Metadata-Version: 1.1
Name: MechanicalSoup
-Version: 0.11.0
+Version: 0.12.0
Summary: A Python library for automating interaction with websites
Home-page: https://mechanicalsoup.readthedocs.io/
Author: UNKNOWN
Author-email: UNKNOWN
License: MIT
-Description: .. image:: /assets/mechanical-soup-logo.png
- :alt: MechanicalSoup. A Python library for automating website
- interaction.
+Description: .. image::
https://raw.githubusercontent.com/MechanicalSoup/MechanicalSoup/master/assets/mechanical-soup-logo.png
+ :alt: MechanicalSoup. A Python library for automating website
interaction.
Home page
---------
@@ -26,8 +25,8 @@
MechanicalSoup was created by `M Hickford
<https://github.com/hickford/>`__, who was a fond user of the
`Mechanize <https://github.com/jjlee/mechanize>`__ library.
- Unfortunately, Mechanize is `incompatible with Python 3
- <https://github.com/jjlee/mechanize/issues/96>`__ and its development
+ Unfortunately, Mechanize was `incompatible with Python 3 until 2019
+ <https://github.com/python-mechanize/mechanize/issues/9>`__ and its
development
stalled for several years. MechanicalSoup provides a similar API,
built on Python
giants `Requests <http://docs.python-requests.org/en/latest/>`__ (for
HTTP sessions) and `BeautifulSoup
@@ -146,6 +145,6 @@
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
+Classifier: Programming Language :: Python :: 3.7
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/README.rst
new/MechanicalSoup-0.12.0/README.rst
--- old/MechanicalSoup-0.11.0/README.rst 2018-09-06 21:42:18.000000000
+0200
+++ new/MechanicalSoup-0.12.0/README.rst 2019-05-29 00:59:49.000000000
+0200
@@ -1,6 +1,5 @@
.. image:: /assets/mechanical-soup-logo.png
- :alt: MechanicalSoup. A Python library for automating website
- interaction.
+ :alt: MechanicalSoup. A Python library for automating website interaction.
Home page
---------
@@ -18,8 +17,8 @@
MechanicalSoup was created by `M Hickford
<https://github.com/hickford/>`__, who was a fond user of the
`Mechanize <https://github.com/jjlee/mechanize>`__ library.
-Unfortunately, Mechanize is `incompatible with Python 3
-<https://github.com/jjlee/mechanize/issues/96>`__ and its development
+Unfortunately, Mechanize was `incompatible with Python 3 until 2019
+<https://github.com/python-mechanize/mechanize/issues/9>`__ and its development
stalled for several years. MechanicalSoup provides a similar API, built on
Python
giants `Requests <http://docs.python-requests.org/en/latest/>`__ (for
HTTP sessions) and `BeautifulSoup
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/docs/ChangeLog.rst
new/MechanicalSoup-0.12.0/docs/ChangeLog.rst
--- old/MechanicalSoup-0.11.0/docs/ChangeLog.rst 2018-09-12
00:40:50.000000000 +0200
+++ new/MechanicalSoup-0.12.0/docs/ChangeLog.rst 2019-08-27
02:05:10.000000000 +0200
@@ -2,6 +2,52 @@
Release Notes
=============
+Version 0.12
+============
+
+Main changes:
+-------------
+
+* Changes in official python version support: added 3.7 and dropped 3.4.
+
+* Added ability to submit a form without updating ``StatefulBrowser`` internal
+ state: ``submit_selected(..., update_state=False)``. This means you get a
+ response from the form submission, but your browser stays on the same page.
+ Useful for handling forms that result in a file download or open a new tab.
+
+Bug fixes
+---------
+
+* Improve handling of form enctype to behave like a real browser.
+ [`#242 <https://github.com/MechanicalSoup/MechanicalSoup/issues/242>`__]
+
+* HTML ``type`` attributes are no longer required to be lowercase.
+ [`#245 <https://github.com/MechanicalSoup/MechanicalSoup/issues/245>`__]
+
+* Form controls with the ``disabled`` attribute will no longer be submitted
+ to improve compliance with the HTML standard. If you were relying on this
+ bug to submit disabled elements, you can still achieve this by deleting the
+ ``disabled`` attribute from the element in the :class:`~mechanicalsoup.Form`
+ object directly.
+ [`#248 <https://github.com/MechanicalSoup/MechanicalSoup/issues/248>`__]
+
+* When a form containing a file input field is submitted without choosing a
+ file, an empty filename & content will be sent just like in a real browser.
+ [`#250 <https://github.com/MechanicalSoup/MechanicalSoup/issues/250>`__]
+
+* ``<option>`` tags without a ``value`` attribute will now use their text as
+ the value.
+ [`#252 <https://github.com/MechanicalSoup/MechanicalSoup/pull/252>`__]
+
+* The optional ``url_regex`` argument to ``follow_link`` and ``download_link``
+ was fixed so that it is no longer ignored.
+ [`#256 <https://github.com/MechanicalSoup/MechanicalSoup/pull/256>`__]
+
+* Allow duplicate submit elements instead of raising a LinkNotFoundError.
+ [`#264 <https://github.com/MechanicalSoup/MechanicalSoup/issues/264>`__]
+
+Our thanks to the many new contributors in this release!
+
Version 0.11
============
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/docs/conf.py
new/MechanicalSoup-0.12.0/docs/conf.py
--- old/MechanicalSoup-0.11.0/docs/conf.py 2017-11-26 13:59:48.000000000
+0100
+++ new/MechanicalSoup-0.12.0/docs/conf.py 2019-04-18 20:10:03.000000000
+0200
@@ -15,6 +15,7 @@
import sys
import os
+from datetime import datetime
# If extensions (or modules to document with autodoc) are in another directory,
@@ -48,7 +49,7 @@
# General information about the project.
project = 'MechanicalSoup'
-copyright = '2014'
+copyright = '2014-{}'.format(datetime.utcnow().year)
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
@@ -140,7 +141,7 @@
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
-#html_last_updated_fmt = '%b %d, %Y'
+html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/docs/faq.rst
new/MechanicalSoup-0.12.0/docs/faq.rst
--- old/MechanicalSoup-0.11.0/docs/faq.rst 2018-09-07 20:05:26.000000000
+0200
+++ new/MechanicalSoup-0.12.0/docs/faq.rst 2019-04-18 20:10:03.000000000
+0200
@@ -77,18 +77,22 @@
* `Mechanize <http://wwwsearch.sourceforge.net/mechanize/>`__ is an
ancestor of MechanicalSoup (getting its name from the Perl mechanize
- module). It was a great tool, but doesn't support Python 3. It was
- unmaintained for several years but got a new maintainer in 2017.
- Note that Mechanize is a much bigger piece of code (around 20 times
- more lines!) than MechanicalSoup, which is small because it
- delegates most of its work to BeautifulSoup and requests.
+ module). It was a great tool, but became unmaintained for several
+ years and didn't support Python 3. Fortunately, Mechanize got a new
+ maintainer in 2017 and completed Python 3 support in 2019. Note that
+ Mechanize is a much bigger piece of code (around 20 times more
+ lines!) than MechanicalSoup, which is small because it delegates
+ most of its work to BeautifulSoup and requests.
* `RoboBrowser <https://github.com/jmcarp/robobrowser>`__ is very
similar to MechanicalSoup. Both are small libraries built on top of
requests and BeautifulSoup. Their APIs are very similar. Both have an
automated testsuite. As of writing, MechanicalSoup is more actively
- maintained (only 1 really active developer and no activity the last
- two years for RoboBrowser).
+ maintained (only 1 really active developer and no activity since
+ 2015 on RoboBrowser). RoboBrowser is `broken on Python 3.7
+ <https://github.com/jmcarp/robobrowser/issues/87>`__, and while
+ there is an easy workaround this is a sign that the lack of activity
+ is due to the project being abandoned more than to its maturity.
* `Selenium <http://selenium-python.readthedocs.io/>`__ is a much
heavier solution: it launches a real web browser (Firefox,
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/docs/tutorial.rst
new/MechanicalSoup-0.12.0/docs/tutorial.rst
--- old/MechanicalSoup-0.11.0/docs/tutorial.rst 2018-09-06 21:42:18.000000000
+0200
+++ new/MechanicalSoup-0.12.0/docs/tutorial.rst 2019-05-29 00:59:39.000000000
+0200
@@ -107,7 +107,7 @@
>>> browser["comments"] = "This pizza looks really good :-)"
For radio buttons, well, it's simple too: radio buttons have several
-``input`` tag with the same ``name`` and different values, just select
+``input`` tags with the same ``name`` and different values, just select
the one you need (``"size"`` is the ``name`` attribute, ``"medium"``
is the ``"value"`` attribute of the element we want to tick)::
@@ -136,7 +136,7 @@
real web browser on the current page visited by our ``browser``
object, including the changes we just made to the form (note that it
does not open the real webpage, but creates a temporary file
-containing the page content, and point your browser to this file). Try
+containing the page content, and points your browser to this file). Try
changing the boxes ticked and the content of the text field, and
re-launch the browser.
@@ -214,7 +214,7 @@
Alternatively, one can use the :class:`~mechanicalsoup.Browser` class,
which doesn't maintain a state from one call to another (i.e. the
Browser itself doesn't remember which page you are visiting and what
-is its content, it's up to the caller to do so). This example is
+its content is, it's up to the caller to do so). This example is
available as `examples/example_manual.py
<https://github.com/MechanicalSoup/MechanicalSoup/blob/master/examples/example_manual.py>`__
in the source:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/mechanicalsoup/__version__.py
new/MechanicalSoup-0.12.0/mechanicalsoup/__version__.py
--- old/MechanicalSoup-0.11.0/mechanicalsoup/__version__.py 2018-09-12
00:40:50.000000000 +0200
+++ new/MechanicalSoup-0.12.0/mechanicalsoup/__version__.py 2019-08-27
02:06:13.000000000 +0200
@@ -2,5 +2,7 @@
__description__ = 'A Python library for automating interaction with websites'
__url__ = 'https://mechanicalsoup.readthedocs.io/'
__github_url__ = 'https://github.com/MechanicalSoup/MechanicalSoup'
-__version__ = '0.11.0'
+__version__ = '0.12.0'
__license__ = 'MIT'
+__github_assets_absoluteURL__ = """\
+https://raw.githubusercontent.com/MechanicalSoup/MechanicalSoup/master""";
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/mechanicalsoup/browser.py
new/MechanicalSoup-0.12.0/mechanicalsoup/browser.py
--- old/MechanicalSoup-0.11.0/mechanicalsoup/browser.py 2018-09-07
23:28:54.000000000 +0200
+++ new/MechanicalSoup-0.12.0/mechanicalsoup/browser.py 2019-08-13
19:42:05.000000000 +0200
@@ -155,6 +155,8 @@
# Requests also retains order when encoding form data in 2-tuple lists.
data = [(k, v) for k, v in data.items()]
+ multipart = form.get("enctype", "") == "multipart/form-data"
+
# Process form tags in the order that they appear on the page,
# skipping those tags that do not have a name-attribute.
selector = ",".join("{}[name]".format(i) for i in
@@ -162,8 +164,12 @@
for tag in form.select(selector):
name = tag.get("name") # name-attribute of tag
+ # Skip disabled elements, since they should not be submitted.
+ if tag.has_attr('disabled'):
+ continue
+
if tag.name == "input":
- if tag.get("type") in ("radio", "checkbox"):
+ if tag.get("type", "").lower() in ("radio", "checkbox"):
if "checked" not in tag.attrs:
continue
value = tag.get("value", "on")
@@ -171,21 +177,23 @@
# browsers use empty string for inputs with missing values
value = tag.get("value", "")
- if tag.get("type") == "file":
- # read http://www.cs.tut.fi/~jkorpela/forms/file.html
- # in browsers, file upload only happens if the form
- # (or submit button) enctype attribute is set to
- # "multipart/form-data". We don't care, simplify.
- if not value:
- continue
- if isinstance(value, string_types):
- value = open(value, "rb")
- files[name] = value
+ # If the enctype is not multipart, the filename is put in
+ # the form as a text input and the file is not sent.
+ if tag.get("type", "").lower() == "file" and multipart:
+ filename = value
+ if filename != "" and isinstance(filename, string_types):
+ content = open(filename, "rb")
+ else:
+ content = ""
+ # If value is the empty string, we still pass it
+ # for consistency with browsers (see
+ #
https://github.com/MechanicalSoup/MechanicalSoup/issues/250).
+ files[name] = (filename, content)
else:
data.append((name, value))
elif tag.name == "button":
- if tag.get("type", "") in ("button", "reset"):
+ if tag.get("type", "").lower() in ("button", "reset"):
continue
else:
data.append((name, tag.get("value", "")))
@@ -194,8 +202,10 @@
data.append((name, tag.text))
elif tag.name == "select":
+ # If the value attribute is not specified, the content will
+ # be passed as a value instead.
options = tag.select("option")
- selected_values = [i.get("value", "") for i in options
+ selected_values = [i.get("value", i.text) for i in options
if "selected" in i.attrs]
if "multiple" in tag.attrs:
for value in selected_values:
@@ -206,13 +216,30 @@
data.append((name, selected_values[-1]))
elif options:
# Selects the first option if none are selected
- data.append((name, options[0].get("value", "")))
+ first_value = options[0].get("value", options[0].text)
+ data.append((name, first_value))
if method.lower() == "get":
kwargs["params"] = data
else:
kwargs["data"] = data
+ # The following part of the function is here to respect the
+ # enctype specified by the form, i.e. force sending multipart
+ # content. Since Requests doesn't have yet a feature to choose
+ # enctype, we have to use tricks to make it behave as we want
+ # This code will be updated if Requests implements it.
+ if multipart and not files:
+ # Requests will switch to "multipart/form-data" only if
+ # files pass the `if files:` test, so in this case we use
+ # a modified dict that passes the if test even if empty.
+ class DictThatReturnsTrue(dict):
+ def __bool__(self):
+ return True
+ __nonzero__ = __bool__
+
+ files = DictThatReturnsTrue()
+
return self.session.request(method, url, files=files, **kwargs)
def submit(self, form, url=None, **kwargs):
@@ -225,7 +252,7 @@
:param form: The filled-out form.
:param url: URL of the page the form is on. If the form action is a
relative path, then this must be specified.
- :param \*\*kwargs: Arguments forwarded to `requests.Session.request
+ :param \\*\\*kwargs: Arguments forwarded to `requests.Session.request
<http://docs.python-requests.org/en/master/api/#requests.Session.request>`__.
:return: `requests.Response
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/mechanicalsoup/form.py
new/MechanicalSoup-0.12.0/mechanicalsoup/form.py
--- old/MechanicalSoup-0.11.0/mechanicalsoup/form.py 2018-09-06
21:42:18.000000000 +0200
+++ new/MechanicalSoup-0.12.0/mechanicalsoup/form.py 2019-08-27
01:04:16.000000000 +0200
@@ -111,8 +111,8 @@
the HTML is served.
"""
for (name, value) in data.items():
- checkboxes = self.form.find_all("input", {"name": name},
- type="checkbox")
+ # Case-insensitive search for type=checkbox
+ checkboxes = self.find_by_type("input", "checkbox", {'name': name})
if not checkboxes:
raise InvalidFormMethod("No input checkbox named " + name)
@@ -155,7 +155,8 @@
Only one radio button in the family can be checked.
"""
for (name, value) in data.items():
- radios = self.form.find_all("input", {"name": name}, type="radio")
+ # Case-insensitive search for type=radio
+ radios = self.find_by_type("input", "radio", {'name': name})
if not radios:
raise InvalidFormMethod("No input radio named " + name)
@@ -193,9 +194,10 @@
:param data: Dict of ``{name: value, ...}``.
Find the select element whose *name*-attribute is ``name``.
Then select from among its children the option element whose
- *value*-attribute is ``value``. If the select element's
- *multiple*-attribute is set, then ``value`` can be a list
- or tuple to select multiple options.
+ *value*-attribute is ``value``. If no matching *value*-attribute
+ is found, this will search for an option whose text matches
+ ``value``. If the select element's *multiple*-attribute is set,
+ then ``value`` can be a list or tuple to select multiple options.
"""
for (name, value) in data.items():
select = self.form.find("select", {"name": name})
@@ -216,6 +218,16 @@
for choice in value:
option = select.find("option", {"value": choice})
+
+ # try to find with text instead of value
+ if not option:
+ option = select.find("option", string=choice)
+
+ if not option:
+ raise LinkNotFoundError(
+ 'Option %s not found for select %s' % (choice, name)
+ )
+
option.attrs["selected"] = "selected"
def __setitem__(self, name, value):
@@ -321,8 +333,10 @@
raise Exception('Submit already chosen. Cannot change submit!')
# All buttons NOT of type (button,reset) are valid submits
- inps = [i for i in self.form.select('input[type="submit"], button')
- if i.get('type', '') not in ('button', 'reset')]
+ inps = (self.find_by_type("input", "submit", dict()) +
+ self.form.find_all("button"))
+ inps = [i for i in inps
+ if i.get('type', '').lower() not in ('button', 'reset')]
# If no submit specified, choose the first one
if submit is None and inps:
@@ -330,13 +344,18 @@
found = False
for inp in inps:
- if inp == submit or (inp.has_attr('name') and
- inp['name'] == submit):
+ if (inp.has_attr('name') and inp['name'] == submit):
if found:
raise LinkNotFoundError(
"Multiple submit elements match: {0}".format(submit)
)
found = True
+ elif inp == submit:
+ if found:
+ # Ignore submit element since it is an exact
+ # duplicate of the one we're looking at.
+ del inp['name']
+ found = True
else:
# Delete any non-matching element's name so that it will be
# omitted from the submitted form data.
@@ -362,3 +381,8 @@
if subtag.string:
subtag.string = subtag.string.strip()
print(input_copy)
+
+ def find_by_type(self, tag_name, type_attr, attrs):
+ attrs_dict = attrs.copy()
+ attrs_dict['type'] = lambda x: x and x.lower() == type_attr
+ return self.form.find_all(tag_name, attrs=attrs_dict)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/MechanicalSoup-0.11.0/mechanicalsoup/stateful_browser.py
new/MechanicalSoup-0.12.0/mechanicalsoup/stateful_browser.py
--- old/MechanicalSoup-0.11.0/mechanicalsoup/stateful_browser.py
2018-09-07 20:25:39.000000000 +0200
+++ new/MechanicalSoup-0.12.0/mechanicalsoup/stateful_browser.py
2019-05-29 00:59:39.000000000 +0200
@@ -78,7 +78,7 @@
* 0 means no verbose output.
* 1 shows one dot per visited page (looks like a progress bar)
- * >= 1 shows each visited URL.
+ * >= 2 shows each visited URL.
"""
self.__verbose = verbose
@@ -209,14 +209,18 @@
return self.get_current_form()
- def submit_selected(self, btnName=None, *args, **kwargs):
+ def submit_selected(self, btnName=None, update_state=True,
+ *args, **kwargs):
"""Submit the form that was selected with :func:`select_form`.
:return: Forwarded from :func:`Browser.submit`.
If there are multiple submit input/button elements, passes ``btnName``
to :func:`Form.choose_submit` on the current form to choose between
- them. All other arguments are forwarded to :func:`Browser.submit`.
+ them. If `update_state` is False, form will be submited but the browser
+ state will remain unchanged. This is useful for forms that result in
+ a download of a file. All other arguments are forwarded to
+ :func:`Browser.submit`.
"""
self.get_current_form().choose_submit(btnName)
@@ -229,8 +233,9 @@
resp = self.submit(self.__state.form, url=self.__state.url,
*args, **kwargs)
- self.__state = _BrowserState(page=resp.soup, url=resp.url,
- request=resp.request)
+ if update_state:
+ self.__state = _BrowserState(page=resp.soup, url=resp.url,
+ request=resp.request)
return resp
def list_links(self, *args, **kwargs):
@@ -292,7 +297,7 @@
raise ValueError('link parameter cannot be treated as '
'url_regex because url_regex is already '
'present in keyword arguments')
- else:
+ elif link:
kwargs['url_regex'] = link
try:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/requirements.txt
new/MechanicalSoup-0.12.0/requirements.txt
--- old/MechanicalSoup-0.11.0/requirements.txt 2017-11-26 13:59:48.000000000
+0100
+++ new/MechanicalSoup-0.12.0/requirements.txt 2019-04-18 20:09:42.000000000
+0200
@@ -1,4 +1,4 @@
requests >= 2.0
-beautifulsoup4
+beautifulsoup4 >= 4.4
six >= 1.4
lxml
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/setup.py
new/MechanicalSoup-0.12.0/setup.py
--- old/MechanicalSoup-0.11.0/setup.py 2018-08-22 17:41:35.000000000 +0200
+++ new/MechanicalSoup-0.12.0/setup.py 2019-08-13 19:45:30.000000000 +0200
@@ -12,7 +12,7 @@
if line.strip() and not line.strip().startswith('--')]
-def read(fname, URL):
+def read(fname, URL, URLImage):
"""Read the content of a file."""
readme = open(path.join(path.dirname(__file__), fname)).read()
if hasattr(readme, 'decode'):
@@ -22,6 +22,8 @@
readme = re.sub(r'`<([^>]*)>`__',
r'`\1 <' + URL + r"/blob/master/\1>`__",
readme)
+ readme = re.sub(r"\.\. image:: /", ".. image:: " + URLImage + "/", readme)
+
return readme
@@ -44,7 +46,8 @@
version=about['__version__'],
description=about['__description__'],
- long_description=read('README.rst', about['__github_url__']),
+ long_description=read('README.rst', about['__github_url__'], about[
+ '__github_assets_absoluteURL__']),
url=about['__url__'],
license=about['__license__'],
@@ -57,9 +60,9 @@
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
- 'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
],
packages=['mechanicalsoup'],
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/tests/test_browser.py
new/MechanicalSoup-0.12.0/tests/test_browser.py
--- old/MechanicalSoup-0.11.0/tests/test_browser.py 2018-08-15
17:58:44.000000000 +0200
+++ new/MechanicalSoup-0.12.0/tests/test_browser.py 2019-08-13
19:42:05.000000000 +0200
@@ -3,6 +3,7 @@
import sys
from bs4 import BeautifulSoup
import tempfile
+import os
from requests.cookies import RequestsCookieJar
import pytest
@@ -45,18 +46,17 @@
<textarea name="comments">freezer</textarea>
<fieldset>
<legend> Pizza Size </legend>
- <p><input type=radio name=size value="small">Small</p>
- <p><input type=radio name=size value="medium" checked>Medium</p>
+ <p><input type=RADIO name=size value="small">Small</p>
+ <p><input type=radiO name=size value="medium" checked>Medium</p>
<p><input type=radio name=size value="large">Large</p>
</fieldset>
<fieldset>
<legend> Pizza Toppings </legend>
- <p><input type=checkbox name="topping" value="bacon" checked>Bacon</p>
- <p><input type=checkbox name="topping" value="cheese">Extra Cheese</p>
+ <p><input type=CHECKBOX name="topping" value="bacon" checked>Bacon</p>
+ <p><input type=checkBox name="topping" value="cheese">Extra Cheese</p>
<p><input type=checkbox name="topping" value="onion" checked>Onion</p>
<p><input type=checkbox name="topping" value="mushroom">Mushroom</p>
</fieldset>
- <input name="pic" type="file">
<select name="shape">
<option value="round">Round</option>
<option value="square" selected>Square</option>
@@ -81,32 +81,96 @@
"Content-Type"]
-def test__request_file(httpbin):
+valid_enctypes_file_submit = {"multipart/form-data": True,
+ "application/x-www-form-urlencoded": False
+ }
+
+default_enctype = "application/x-www-form-urlencoded"
+
+
[email protected]("file_field", [
+ """<input name="pic" type="file" />""",
+ ""])
[email protected]("submit_file", [
+ True,
+ False
+])
[email protected]("enctype", [
+ pytest.param("multipart/form-data"),
+ pytest.param("application/x-www-form-urlencoded"),
+ pytest.param("Invalid enctype")
+])
+def test_enctype_and_file_submit(httpbin, enctype, submit_file, file_field):
+ # test if enctype is respected when specified
+ # and if files are processed correctly
form_html = """
- <form method="post" action="{}/post">
- <input name="pic" type="file" />
+ <form method="post" action="{}/post" enctype="{}">
+ <input name="in" value="test" />
+ {}
</form>
- """.format(httpbin.url)
+ """.format(httpbin.url, enctype, file_field)
form = BeautifulSoup(form_html, "lxml").form
- # create a temporary file for testing file upload
- pic_path = tempfile.mkstemp()[1]
- with open(pic_path, "w") as f:
- f.write(":-)")
-
- form.find("input", {"name": "pic"})["value"] = pic_path
+ valid_enctype = (enctype in valid_enctypes_file_submit and
+ valid_enctypes_file_submit[enctype])
+ expected_content = b"" # default
+ if submit_file and file_field:
+ # create a temporary file for testing file upload
+ file_content = b":-)"
+ pic_filedescriptor, pic_path = tempfile.mkstemp()
+ os.write(pic_filedescriptor, file_content)
+ os.close(pic_filedescriptor)
+ if valid_enctype:
+ # Correct encoding => send the content
+ expected_content = file_content
+ else:
+ # Encoding doesn't allow sending the content, we expect
+ # the filename as a normal text field.
+ expected_content = pic_path.encode()
+ form.find("input", {"name": "pic"})["value"] = pic_path
browser = mechanicalsoup.Browser()
response = browser._request(form)
- # Check that only "files" includes a "pic" keyword in the response
- for key, value in response.json().items():
- if key == "files":
- assert value["pic"] == ":-)"
+ if enctype not in valid_enctypes_file_submit:
+ expected_enctype = default_enctype
+ else:
+ expected_enctype = enctype
+ assert expected_enctype in response.request.headers["Content-Type"]
+
+ resp = response.json()
+ assert resp["form"]["in"] == "test"
+
+ found = False
+ found_in = None
+
+ for key, value in resp.items():
+ if value:
+ if "pic" in value:
+ content = value["pic"].encode()
+ assert not found
+ assert key in ("files", "form")
+ found = True
+ found_in = key
+ if key == "files" and not valid_enctype:
+ assert not value
+
+ assert found == bool(file_field)
+ if file_field:
+ assert content == expected_content
+
+ if valid_enctype:
+ assert found_in == "files"
+ if submit_file:
+ assert ("filename=\"" + pic_path + "\""
+ ).encode() in response.request.body
+ else:
+ assert b"filename=\"\"" in response.request.body
else:
- assert (value is None) or ("pic" not in value)
+ assert found_in == "form"
- assert "multipart/form-data" in response.request.headers["Content-Type"]
+ if submit_file and file_field:
+ os.remove(pic_path)
def test__request_select_none(httpbin):
@@ -126,6 +190,18 @@
assert response.json()['form'] == {'shape': 'round'}
+def test__request_disabled_attr(httpbin):
+ """Make sure that disabled form controls are not submitted."""
+ form_html = """
+ <form method="post" action="{}/post">
+ <input disabled name="nosubmit" value="1" />
+ </form>""".format(httpbin.url)
+
+ browser = mechanicalsoup.Browser()
+ response = browser._request(BeautifulSoup(form_html, "lxml").form)
+ assert response.json()['form'] == {}
+
+
def test_no_404(httpbin):
browser = mechanicalsoup.Browser()
resp = browser.get(httpbin + "/nosuchpage")
@@ -168,7 +244,7 @@
browser = mechanicalsoup.Browser()
data = {'color': 'blue', 'colorblind': 'True'}
resp = browser.post(httpbin + "/post", data)
- assert(resp.status_code == 200 and resp.json()['form'] == data)
+ assert resp.status_code == 200 and resp.json()['form'] == data
if __name__ == '__main__':
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/tests/test_form.py
new/MechanicalSoup-0.12.0/tests/test_form.py
--- old/MechanicalSoup-0.11.0/tests/test_form.py 2018-09-04
17:30:32.000000000 +0200
+++ new/MechanicalSoup-0.12.0/tests/test_form.py 2019-04-18
20:10:03.000000000 +0200
@@ -74,32 +74,32 @@
@pytest.mark.parametrize("expected_post", [
pytest.param(
[
+ ('text', 'Setting some text!'),
('comment', 'Testing preview page'),
('preview', 'Preview Page'),
- ('text', 'Setting some text!')
], id='preview'),
pytest.param(
[
+ ('text', '= Heading =\n\nNew page here!\n'),
('comment', 'Created new page'),
('save', 'Submit changes'),
- ('text', '= Heading =\n\nNew page here!\n')
], id='save'),
pytest.param(
[
+ ('text', '= Heading =\n\nNew page here!\n'),
('comment', 'Testing choosing cancel button'),
('cancel', 'Cancel'),
- ('text', '= Heading =\n\nNew page here!\n')
], id='cancel'),
])
def test_choose_submit(expected_post):
browser, url = setup_mock_browser(expected_post=expected_post)
browser.open(url)
form = browser.select_form('#choose-submit-form')
- browser['text'] = expected_post[2][1]
- browser['comment'] = expected_post[0][1]
- form.choose_submit(expected_post[1][0])
+ browser['text'] = dict(expected_post)['text']
+ browser['comment'] = dict(expected_post)['comment']
+ form.choose_submit(expected_post[2][0])
res = browser.submit_selected()
- assert(res.status_code == 200 and res.text == 'Success!')
+ assert res.status_code == 200 and res.text == 'Success!'
@pytest.mark.parametrize("value", [
@@ -200,7 +200,7 @@
form = browser.select_form('#choose-submit-form')
form['text1'] = 'newText1'
res = browser.submit_selected()
- assert(res.status_code == 200 and browser.get_url() == url)
+ assert res.status_code == 200 and browser.get_url() == url
submit_form_action = '''
@@ -224,7 +224,7 @@
form = browser.select_form('#choose-submit-form')
form['text1'] = 'newText1'
res = browser.submit_selected()
- assert(res.status_code == 200 and browser.get_url() == url)
+ assert res.status_code == 200 and browser.get_url() == url
set_select_form = '''
@@ -256,7 +256,7 @@
if not option['default']:
browser[option['result'][0][0]] = option['result'][0][1]
res = browser.submit_selected()
- assert(res.status_code == 200 and res.text == 'Success!')
+ assert res.status_code == 200 and res.text == 'Success!'
set_select_multiple_form = '''
@@ -290,7 +290,7 @@
form = browser.select_form('form')
form.set_select({'instrument': options})
res = browser.submit_selected()
- assert(res.status_code == 200 and res.text == 'Success!')
+ assert res.status_code == 200 and res.text == 'Success!'
def test_form_not_found():
@@ -313,6 +313,38 @@
form.set_select({'entree': ('no_multiple', 'no_multiple')})
+def test_form_set_radio_checkbox(capsys):
+ browser = mechanicalsoup.StatefulBrowser()
+ browser.open_fake_page(page_with_various_fields,
+ url="http://example.com/invalid/";)
+ form = browser.select_form("form")
+ form.set_radio({"size": "small"})
+ form.set_checkbox({"topping": "cheese"})
+ browser.get_current_form().print_summary()
+ out, err = capsys.readouterr()
+ # Different versions of bs4 show either <input></input> or
+ # <input/>. Normalize before comparing.
+ out = out.replace('></input>', '/>')
+ assert out == """<input name="foo"/>
+<textarea name="bar"></textarea>
+<select name="entree">
+<option selected="selected" value="tofu">Tofu Stir Fry</option>
+<option value="curry">Red Curry</option>
+<option value="tempeh">Tempeh Tacos</option>
+</select>
+<input name="topping" type="checkbox" value="bacon"/>
+<input checked="" name="topping" type="Checkbox" value="cheese"/>
+<input name="topping" type="checkbox" value="onion"/>
+<input name="topping" type="checkbox" value="mushroom"/>
+<input checked="" name="size" type="Radio" value="small"/>
+<input name="size" type="radio" value="medium"/>
+<input name="size" type="radio" value="large"/>
+<button name="action" value="cancel">Cancel</button>
+<input type="submit" value="Select"/>
+"""
+ assert err == ""
+
+
page_with_radio = '''
<html>
<form method="post">
@@ -351,14 +383,14 @@
<legend> Pizza Toppings </legend>
<p><label> <input type=checkbox name="topping"
value="bacon"> Bacon </label></p>
- <p><label> <input type=checkbox name="topping"
+ <p><label> <input type=Checkbox name="topping"
value="cheese" checked>Extra Cheese </label></p>
<p><label> <input type=checkbox name="topping"
value="onion" checked> Onion </label></p>
<p><label> <input type=checkbox name="topping"
value="mushroom"> Mushroom </label></p>
</fieldset>
- <p><input name="size" type=radio value="small">Small</p>
+ <p><input name="size" type=Radio value="small">Small</p>
<p><input name="size" type=radio value="medium">Medium</p>
<p><input name="size" type=radio value="large">Large</p>
<button name="action" value="cancel">Cancel</button>
@@ -386,10 +418,10 @@
<option value="tempeh">Tempeh Tacos</option>
</select>
<input name="topping" type="checkbox" value="bacon"/>
-<input checked="" name="topping" type="checkbox" value="cheese"/>
+<input checked="" name="topping" type="Checkbox" value="cheese"/>
<input checked="" name="topping" type="checkbox" value="onion"/>
<input name="topping" type="checkbox" value="mushroom"/>
-<input name="size" type="radio" value="small"/>
+<input name="size" type="Radio" value="small"/>
<input name="size" type="radio" value="medium"/>
<input name="size" type="radio" value="large"/>
<button name="action" value="cancel">Cancel</button>
@@ -431,7 +463,27 @@
browser.open(url)
browser.select_form()
res = browser.submit_selected()
- assert(res.status_code == 200 and res.text == 'Success!')
+ assert res.status_code == 200 and res.text == 'Success!'
+ browser.close()
+
+
+def test_duplicate_submit_buttons():
+ """Tests that duplicate submits doesn't break form submissions
+ See issue https://github.com/MechanicalSoup/MechanicalSoup/issues/264""";
+ issue264_form = '''
+<form method="post" action="mock://form.com/post">
+ <input name="box" type="hidden" value="1"/>
+ <input name="search" type="submit" value="Search"/>
+ <input name="search" type="submit" value="Search"/>
+</form>
+'''
+ expected_post = [('box', '1'), ('search', 'Search')]
+ browser, url = setup_mock_browser(expected_post=expected_post,
+ text=issue264_form)
+ browser.open(url)
+ browser.select_form()
+ res = browser.submit_selected()
+ assert res.status_code == 200 and res.text == 'Success!'
browser.close()
@@ -444,11 +496,11 @@
"""Buttons of type reset and button are not valid submits"""
text = """
<form method="post" action="mock://form.com/post">
- <button type="button" name="sub1" value="val1">Val1</button>
- <button type="submit" name="sub2" value="val2">Val2</button>
+ <button type="butTon" name="sub1" value="val1">Val1</button>
+ <button type="suBmit" name="sub2" value="val2">Val2</button>
<button type="reset" name="sub3" value="val3">Val3</button>
<button name="sub4" value="val4">Val4</button>
- <input type="submit" name="sub5" value="val5">
+ <input type="subMit" name="sub5" value="val5">
</form>
"""
browser, url = setup_mock_browser(expected_post=expected_post, text=text)
@@ -458,5 +510,40 @@
assert res.status_code == 200 and res.text == 'Success!'
[email protected]("fail, selected, expected_post", [
+ pytest.param(False, 'with_value', [('selector', 'with_value')],
+ id='Option with value'),
+ pytest.param(False, 'Without value', [('selector', 'Without value')],
+ id='Option without value'),
+ pytest.param(False, 'We have a value here', [('selector', 'with_value')],
+ id='Option with value selected by its text'),
+ pytest.param(True, 'Unknown option', None,
+ id='Unknown option, must raise a LinkNotFound exception')
+])
+def test_option_without_value(fail, selected, expected_post):
+ """Option tag in select can have no value option"""
+ text = """
+ <form method="post" action="mock://form.com/post">
+ <select name="selector">
+ <option value="with_value">We have a value here</option>
+ <option>Without value</option>
+ </select>
+ <button type="submit">Submit</button>
+ </form>
+ """
+ browser, url = setup_mock_browser(expected_post=expected_post,
+ text=text)
+ browser.open(url)
+ browser.select_form()
+ if fail:
+ with pytest.raises(mechanicalsoup.utils.LinkNotFoundError):
+ browser['selector'] = selected
+ else:
+ browser['selector'] = selected
+
+ res = browser.submit_selected()
+ assert res.status_code == 200 and res.text == 'Success!'
+
+
if __name__ == '__main__':
pytest.main(sys.argv)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/tests/test_stateful_browser.py
new/MechanicalSoup-0.12.0/tests/test_stateful_browser.py
--- old/MechanicalSoup-0.11.0/tests/test_stateful_browser.py 2018-09-06
21:42:18.000000000 +0200
+++ new/MechanicalSoup-0.12.0/tests/test_stateful_browser.py 2019-08-13
19:42:05.000000000 +0200
@@ -125,15 +125,15 @@
@pytest.mark.parametrize("expected_post", [
pytest.param(
[
+ ('text', 'Setting some text!'),
('comment', 'Selecting an input submit'),
('diff', 'Review Changes'),
- ('text', 'Setting some text!')
], id='input'),
pytest.param(
[
+ ('text', '= Heading =\n\nNew page here!\n'),
('comment', 'Selecting a button submit'),
('cancel', 'Cancel'),
- ('text', '= Heading =\n\nNew page here!\n')
], id='button'),
])
def test_submit_btnName(expected_post):
@@ -141,18 +141,34 @@
browser, url = setup_mock_browser(expected_post=expected_post)
browser.open(url)
browser.select_form('#choose-submit-form')
- browser['text'] = expected_post[2][1]
- browser['comment'] = expected_post[0][1]
- res = browser.submit_selected(btnName=expected_post[1][0])
- assert(res.status_code == 200 and res.text == 'Success!')
+ browser['text'] = dict(expected_post)['text']
+ browser['comment'] = dict(expected_post)['comment']
+ initial_state = browser._StatefulBrowser__state
+ res = browser.submit_selected(btnName=expected_post[2][0])
+ assert res.status_code == 200 and res.text == 'Success!'
+ assert initial_state != browser._StatefulBrowser__state
+
+
+def test_submit_dont_update_state():
+ expected_post = [
+ ('text', 'Bananas are good.'),
+ ('preview', 'Preview Page')]
+ browser, url = setup_mock_browser(expected_post=expected_post)
+ browser.open(url)
+ browser.select_form('#choose-submit-form')
+ browser['text'] = dict(expected_post)['text']
+ initial_state = browser._StatefulBrowser__state
+ res = browser.submit_selected(update_state=False)
+ assert res.status_code == 200 and res.text == 'Success!'
+ assert initial_state == browser._StatefulBrowser__state
def test_get_set_debug():
browser = mechanicalsoup.StatefulBrowser()
# Debug mode is off by default
- assert(not browser.get_debug())
+ assert not browser.get_debug()
browser.set_debug(True)
- assert(browser.get_debug())
+ assert browser.get_debug()
def test_list_links(capsys):
@@ -257,7 +273,7 @@
browser, url = setup_mock_browser()
browser.open_fake_page(submit_form_noaction)
browser.select_form('#choose-submit-form')
- with pytest.raises(ValueError, message="no URL to submit to"):
+ with pytest.raises(ValueError, match="no URL to submit to"):
browser.submit_selected()
@@ -282,7 +298,7 @@
browser.open_fake_page(submit_form_noname, url=url)
browser.select_form('#choose-submit-form')
response = browser.submit_selected()
- assert(response.status_code == 200 and response.text == 'Success!')
+ assert response.status_code == 200 and response.text == 'Success!'
submit_form_multiple = '''
@@ -306,7 +322,7 @@
browser.open_fake_page(submit_form_multiple, url=url)
browser.select_form('#choose-submit-form')
response = browser.submit_selected()
- assert(response.status_code == 200 and response.text == 'Success!')
+ assert response.status_code == 200 and response.text == 'Success!'
def test_upload_file(httpbin):
@@ -323,8 +339,10 @@
("first file content", "second file content"))
# The form doesn't have a type=file field, but the target action
- # does show it => add the fields ourselves.
+ # does show it => add the fields ourselves, and add enctype too.
browser.select_form()
+ browser._StatefulBrowser__state.form.form[
+ "enctype"] = "multipart/form-data"
browser.new_control("file", "first", path1)
browser.new_control("file", "second", "")
browser["second"] = path2
@@ -421,18 +439,17 @@
assert headers['X-Test-Header'] == 'x-test-value'
-def test_link_arg_text(httpbin):
- browser = mechanicalsoup.StatefulBrowser()
- browser.open_fake_page('<a href="/get">Link</a>', httpbin.url)
- browser.follow_link(link_text='Link')
- assert browser.get_url() == httpbin + '/get'
-
-
-def test_link_arg_regex(httpbin):
[email protected]('expected, kwargs', [
+ pytest.param('/foo', {}, id='none'),
+ pytest.param('/get', {'text': 'Link'}, id='text'),
+ pytest.param('/get', {'url_regex': 'get'}, id='regex'),
+])
+def test_follow_link_arg(httpbin, expected, kwargs):
browser = mechanicalsoup.StatefulBrowser()
- browser.open_fake_page('<a href="/get">Link</a>', httpbin.url)
- browser.follow_link(url_regex='.*')
- assert browser.get_url() == httpbin + '/get'
+ html = '<a href="/foo">Bar</a><a href="/get">Link</a>'
+ browser.open_fake_page(html, httpbin.url)
+ browser.follow_link(**kwargs)
+ assert browser.get_url() == httpbin + expected
def test_link_arg_multiregex(httpbin):
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/MechanicalSoup-0.11.0/tests/utils.py
new/MechanicalSoup-0.12.0/tests/utils.py
--- old/MechanicalSoup-0.11.0/tests/utils.py 2018-08-15 17:58:44.000000000
+0200
+++ new/MechanicalSoup-0.12.0/tests/utils.py 2019-08-27 01:03:29.000000000
+0200
@@ -1,5 +1,7 @@
import mechanicalsoup
import requests_mock
+from distutils.version import StrictVersion
+import bs4
try:
from urllib.parse import parse_qsl
except ImportError:
@@ -62,7 +64,12 @@
def text_callback(request, context):
# Python 2's parse_qsl doesn't like None argument
query = parse_qsl(request.text) if request.text else []
- assert (query == expected)
+ # In bs4 4.7.0+, CSS selectors return elements in page order,
+ # but did not in earlier versions.
+ if StrictVersion(bs4.__version__) >= StrictVersion('4.7.0'):
+ assert query == expected
+ else:
+ assert sorted(query) == sorted(expected)
return reply
mocked_adapter.register_uri('POST', url, text=text_callback)
@@ -88,7 +95,7 @@
try:
response = browser.open(httpbin + "/legacy")
if response.status_code == 404:
- # The line above may or may not have raised the expection
+ # The line above may or may not have raised the exception
# depending on raise_on_404. Raise it unconditionally now.
raise mechanicalsoup.LinkNotFoundError()
except mechanicalsoup.LinkNotFoundError:
