Hello community,
here is the log from the commit of package python-cfscrape for openSUSE:Factory
checked in at 2019-03-26 15:43:41
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/python-cfscrape (Old)
and /work/SRC/openSUSE:Factory/.python-cfscrape.new.25356 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-cfscrape"
Tue Mar 26 15:43:41 2019 rev:4 rq:687059 version:1.9.7
Changes:
--------
--- /work/SRC/openSUSE:Factory/python-cfscrape/python-cfscrape.changes
2018-12-13 19:41:15.989253763 +0100
+++
/work/SRC/openSUSE:Factory/.python-cfscrape.new.25356/python-cfscrape.changes
2019-03-26 15:44:13.788163356 +0100
@@ -1,0 +2,13 @@
+Wed Mar 20 21:25:44 UTC 2019 - Martin Herkt <[email protected]>
+
+- Update to 1.9.7:
+ * A hidden input form has been added to the challenge;
+ fix to allow bypassing it
+
+-------------------------------------------------------------------
+Thu Mar 14 14:55:30 UTC 2019 - Tomáš Chvátal <[email protected]>
+
+- Update to 1.9.6:
+ * various readme updates
+
+-------------------------------------------------------------------
Old:
----
cfscrape-1.9.5.tar.gz
New:
----
cfscrape-1.9.7.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ python-cfscrape.spec ++++++
--- /var/tmp/diff_new_pack.rSSWnt/_old 2019-03-26 15:44:14.356162690 +0100
+++ /var/tmp/diff_new_pack.rSSWnt/_new 2019-03-26 15:44:14.356162690 +0100
@@ -1,7 +1,7 @@
#
# spec file for package python-cfscrape
#
-# Copyright (c) 2018 SUSE LINUX GmbH, Nuernberg, Germany.
+# Copyright (c) 2019 SUSE LINUX GmbH, Nuernberg, Germany.
#
# All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed
@@ -18,12 +18,12 @@
%{?!python_module:%define python_module() python-%{**} python3-%{**}}
Name: python-cfscrape
-Version: 1.9.5
+Version: 1.9.7
Release: 0
Summary: Python module to bypass Cloudflare's anti-bot page
License: MIT
Group: Development/Languages/Python
-Url: https://github.com/Anorov/cloudflare-scrape
+URL: https://github.com/Anorov/cloudflare-scrape
Source:
https://files.pythonhosted.org/packages/source/c/cfscrape/cfscrape-%{version}.tar.gz
BuildRequires: %{python_module setuptools}
BuildRequires: fdupes
@@ -61,8 +61,9 @@
%python_install
%python_expand %fdupes %{buildroot}%{$python_sitelib}
+# no tests even in git
+
%files %{python_files}
-%defattr(-,root,root)
%license LICENSE
%doc README.md
%{python_sitelib}/*
++++++ cfscrape-1.9.5.tar.gz -> cfscrape-1.9.7.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/cfscrape-1.9.5/PKG-INFO new/cfscrape-1.9.7/PKG-INFO
--- old/cfscrape-1.9.5/PKG-INFO 2018-04-05 03:21:15.000000000 +0200
+++ new/cfscrape-1.9.7/PKG-INFO 2019-03-20 21:26:23.000000000 +0100
@@ -1,6 +1,6 @@
Metadata-Version: 1.0
Name: cfscrape
-Version: 1.9.5
+Version: 1.9.7
Summary: A simple Python module to bypass Cloudflare's anti-bot page. See
https://github.com/Anorov/cloudflare-scrape for more information.
Home-page: https://github.com/Anorov/cloudflare-scrape
Author: Anorov
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/cfscrape-1.9.5/README.md new/cfscrape-1.9.7/README.md
--- old/cfscrape-1.9.5/README.md 2018-04-04 21:54:02.000000000 +0200
+++ new/cfscrape-1.9.7/README.md 2019-03-20 21:25:54.000000000 +0100
@@ -32,7 +32,7 @@
* Python 2.6 - 3.x
* **[Requests](https://github.com/kennethreitz/requests)** >= 2.0
* **[Node.js](https://nodejs.org/)**
- * Your computer or server may already have it (check with `node -v`). If
not, you can install it with `apt-get install nodejs` on Ubuntu and Debian.
Otherwise, please read [Node's installation
instructions](https://nodejs.org/en/download/package-manager/).
+ * Your computer or server may already have it (check with `node -v`). If
not, you can install it with `apt-get install nodejs` on Ubuntu. Debian
requires `nodejs-legacy`. Otherwise, please read [Node's installation
instructions](https://nodejs.org/en/download/package-manager/).
`python setup.py install` will install the Python dependencies automatically.
Node is the only application you need to install yourself.
@@ -43,11 +43,20 @@
If you notice that the anti-bot page has changed, or if this module suddenly
stops working, please create a GitHub issue so that I can update the code
accordingly.
-In your issue, please include:
+* Many issues are a result of users not updating to the latest release of this
project. Before filing an issue, please run the following command:
+```
+pip show cfscrape
+```
+If the value of the version field is not the latest release, please run the
following to update your package:
+```
+pip install cfscrape -U
+```
+If you are still encountering a problem, open an issue and please include:
* The full exception and stack trace.
* The URL of the Cloudflare-protected page which the script does not work on.
* A Pastebin or Gist containing the HTML source of the protected page.
+* The version number from `pip show cfscrape`.
[This issue comment is a good
example.](https://github.com/Anorov/cloudflare-scrape/issues/3#issuecomment-45827514)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/cfscrape-1.9.5/cfscrape/__init__.py
new/cfscrape-1.9.7/cfscrape/__init__.py
--- old/cfscrape-1.9.5/cfscrape/__init__.py 2018-04-05 03:18:45.000000000
+0200
+++ new/cfscrape-1.9.7/cfscrape/__init__.py 2019-03-20 21:25:54.000000000
+0100
@@ -2,8 +2,8 @@
import random
import re
import subprocess
-from copy import deepcopy
-from time import sleep
+import copy
+import time
from requests.sessions import Session
@@ -12,7 +12,7 @@
except ImportError:
from urllib.parse import urlparse
-__version__ = "1.9.5"
+__version__ = "1.9.7"
DEFAULT_USER_AGENTS = [
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36",
@@ -70,14 +70,14 @@
return resp
def solve_cf_challenge(self, resp, **original_kwargs):
- sleep(self.delay) # Cloudflare requires a delay before solving the
challenge
+ start_time = time.time()
body = resp.text
parsed_url = urlparse(resp.url)
domain = parsed_url.netloc
submit_url = "%s://%s/cdn-cgi/l/chk_jschl" % (parsed_url.scheme,
domain)
- cloudflare_kwargs = deepcopy(original_kwargs)
+ cloudflare_kwargs = copy.deepcopy(original_kwargs)
params = cloudflare_kwargs.setdefault("params", {})
headers = cloudflare_kwargs.setdefault("headers", {})
headers["Referer"] = resp.url
@@ -85,7 +85,7 @@
try:
params["jschl_vc"] = re.search(r'name="jschl_vc" value="(\w+)"',
body).group(1)
params["pass"] = re.search(r'name="pass" value="(.+?)"',
body).group(1)
-
+ params["s"] = re.search(r'name="s"\svalue="(?P<s_value>[^"]+)',
body).group('s_value')
except Exception as e:
# Something is wrong with the page.
# This may indicate Cloudflare has changed their anti-bot
@@ -101,6 +101,10 @@
# performing other types of requests even as the first request.
method = resp.request.method
cloudflare_kwargs["allow_redirects"] = False
+
+ end_time = time.time()
+ time.sleep(self.delay - (end_time - start_time)) # Cloudflare requires
a delay before solving the challenge
+
redirect = self.request(method, submit_url, **cloudflare_kwargs)
redirect_location = urlparse(redirect.headers["Location"])
@@ -116,7 +120,7 @@
except Exception:
raise ValueError("Unable to identify Cloudflare IUAM Javascript on
website. %s" % BUG_REPORT)
- js = re.sub(r"a\.value = (.+ \+ t\.length).+", r"\1", js)
+ js = re.sub(r"a\.value = (.+ \+ t\.length(\).toFixed\(10\))?).+",
r"\1", js)
js = re.sub(r"\s{3,}[a-z](?: = |\.).+", "", js).replace("t.length",
str(len(domain)))
# Strip characters that could be used to exit the string context
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/cfscrape-1.9.5/cfscrape.egg-info/PKG-INFO
new/cfscrape-1.9.7/cfscrape.egg-info/PKG-INFO
--- old/cfscrape-1.9.5/cfscrape.egg-info/PKG-INFO 2018-04-05
03:21:15.000000000 +0200
+++ new/cfscrape-1.9.7/cfscrape.egg-info/PKG-INFO 2019-03-20
21:26:22.000000000 +0100
@@ -1,6 +1,6 @@
Metadata-Version: 1.0
Name: cfscrape
-Version: 1.9.5
+Version: 1.9.7
Summary: A simple Python module to bypass Cloudflare's anti-bot page. See
https://github.com/Anorov/cloudflare-scrape for more information.
Home-page: https://github.com/Anorov/cloudflare-scrape
Author: Anorov
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/cfscrape-1.9.5/cfscrape.egg-info/requires.txt
new/cfscrape-1.9.7/cfscrape.egg-info/requires.txt
--- old/cfscrape-1.9.5/cfscrape.egg-info/requires.txt 2018-04-05
03:21:15.000000000 +0200
+++ new/cfscrape-1.9.7/cfscrape.egg-info/requires.txt 2019-03-20
21:26:22.000000000 +0100
@@ -1 +1 @@
-requests >= 2.0.0
+requests>=2.0.0
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/cfscrape-1.9.5/setup.cfg new/cfscrape-1.9.7/setup.cfg
--- old/cfscrape-1.9.5/setup.cfg 2018-04-05 03:21:15.000000000 +0200
+++ new/cfscrape-1.9.7/setup.cfg 2019-03-20 21:26:23.000000000 +0100
@@ -1,5 +1,4 @@
[egg_info]
-tag_date = 0
-tag_svn_revision = 0
tag_build =
+tag_date = 0
