First off, this *probably* isn't a Cygwin problem ... but it looks like
your environment is confused as it's using BOTH Python2 and Python3 modules
to try to fulfill the requirements (including resources that have already
been cached).
For the most part, pip and pip3 can differentiate, but there's a "cart" and
"horse" problem, as if you install things in a weird order at the system
level, it may or may not do the right thing. But, I'd recommend "dumping"
Python2, if you can at this point (it was EOL'd in December of 2020 and
WILL NOT receive more updates, except for security ... and more-over,
Python 3.7+ (approx) is going to demand newer SSL libraries that will
REALLY confuse earlier versions).
For working with Python (as a Python devops type), I generally recommend
using 'pyenv' and 'pyenv-virtualenv' and trying to do *as little as
possible* to modify the system-level Python ...this gets HARD with a system
like Cygwin, where the generic user can generally overwrite system
binaries, without any real sort of warning (and NO, UAC does NOT adequately
fix anything).
That also said, pyenv kinda really "fights" with Cygwin in some of the
library placement (specifically things like FFI, IIRC, which is stored in a
different library directory than it is, anywhere else I've found).
Where I MIGHT start is to "Force Reinstall" the Python3 stuff from Setup.
Look to see if requests_html is part of the Cygwin-supported modules, and
use THAT... use the hell out of anything you see in the actual Python
packages list, as those will at least be done RIGHT, and will leave you
with more cycles to not worry about too much, except your development.
For "Advanced" handling in Python, you're going to want to use "pyenv" or
some other multi-python managers that are out there (virtualenv and
virtualenv-wrapper are good, but ONLY manage the library path). Pretty much
"pyenv" and "pipenv" are the top two, IIRC. I use pyenv, and haven't dug
too deep in to pipenv, at this point. But, like I said, it's already tough
enough to manage on older systems with older libraries (SSL, specifically,
throws wrenches in to *everything*).
Feel free to hit me up for other ideas... I write too much Python code, as
it is, and on too many different environments (yes, some still do Python2.3
through 2.6, and it makes me want to shoot myself, sometimes... LOL).
Hope that helps -
Russell VT
On Tue, Sep 21, 2021 at 11:38 AM Dennis Putnam <d...@bellsouth.net> wrote:
> I am trying to install 'requests_html' and when it tries to do a compile
> it fails because Python.h is missing. I have python2-devl installed. I
> notice that it is looking for it in /pub which apparently does not
> exist. Can someone help? TIA.
>
> Here is the entire 'pip' output:
>
> $ pip install requests_html
> Collecting requests_html
> Using cached requests_html-0.10.0-py3-none-any.whl (13 kB)
> Collecting requests
> Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
> Collecting w3lib
> Using cached w3lib-1.22.0-py2.py3-none-any.whl (20 kB)
> Collecting parse
> Using cached parse-1.19.0.tar.gz (30 kB)
> Collecting fake-useragent
> Using cached fake-useragent-0.1.11.tar.gz (13 kB)
> Collecting pyquery
> Using cached pyquery-1.4.3-py3-none-any.whl (22 kB)
> Collecting bs4
> Using cached bs4-0.0.1.tar.gz (1.1 kB)
> Collecting pyppeteer>=0.0.14
> Using cached pyppeteer-0.2.6-py3-none-any.whl (83 kB)
> Requirement already satisfied: tqdm<5.0.0,>=4.42.1 in
> /usr/local/lib/python3.8/site-packages (from
> pyppeteer>=0.0.14->requests_html) (4.62.3)
> Requirement already satisfied: urllib3<2.0.0,>=1.25.8 in
> /usr/local/lib/python3.8/site-packages (from
> pyppeteer>=0.0.14->requests_html) (1.26.6)
> Collecting appdirs<2.0.0,>=1.4.3
> Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
> Collecting importlib-metadata>=1.4
> Using cached importlib_metadata-4.8.1-py3-none-any.whl (17 kB)
> Requirement already satisfied: pyee<9.0.0,>=8.1.0 in
> /usr/local/lib/python3.8/site-packages (from
> pyppeteer>=0.0.14->requests_html) (8.2.2)
> Requirement already satisfied: websockets<10.0,>=9.1 in
> /usr/local/lib/python3.8/site-packages (from
> pyppeteer>=0.0.14->requests_html) (9.1)
> Requirement already satisfied: zipp>=0.5 in
> /usr/local/lib/python3.8/site-packages (from
> importlib-metadata>=1.4->pyppeteer>=0.0.14->requests_html) ( 3.5.0)
> Requirement already satisfied: beautifulsoup4 in
> /usr/local/lib/python3.8/site-packages (from bs4->requests_html) (4.10.0)
> Requirement already satisfied: soupsieve>1.2 in
> /usr/local/lib/python3.8/site-packages (from
> beautifulsoup4->bs4->requests_html) (2.2.1)
> Collecting cssselect>0.7.9
> Using cached cssselect-1.1.0-py2.py3-none-any.whl (16 kB)
> Collecting lxml>=2.1
> Using cached lxml-4.6.3.tar.gz (3.2 MB)
> Collecting idna<4,>=2.5
> Using cached idna-3.2-py3-none-any.whl (59 kB)
> Collecting certifi>=2017.4.17
> Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
> Collecting charset-normalizer~=2.0.0
> Using cached charset_normalizer-2.0.6-py3-none-any.whl (37 kB)
> Requirement already satisfied: six>=1.4.1 in
> /usr/local/lib/python3.8/site-packages (from w3lib->requests_html) (1.16.0)
> Using legacy 'setup.py install' for bs4, since package 'wheel' is not
> installed.
> Using legacy 'setup.py install' for fake-useragent, since package
> 'wheel' is not installed.
> Using legacy 'setup.py install' for parse, since package 'wheel' is not
> installed.
> Using legacy 'setup.py install' for lxml, since package 'wheel' is not
> installed.
> Installing collected packages: lxml, importlib-metadata, idna,
> cssselect, charset-normalizer, certifi, appdirs, w3lib, requests,
> pyquery, pyppeteer,
> parse, fake-useragent, bs4, requests-html
> Running setup.py install for lxml ... error
> ERROR: Command errored out with exit status 1:
> command: /usr/bin/python -u -c 'import io, os, sys, setuptools,
> tokenize; sys.argv[0] =
> '"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a
> 7a21e44c273e/setup.py'"'"';
> __file__='"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/setup.py'"'"';f
>
> = getattr(tokenize, '"'"'open'"'"', open)(__file__) if
> os.path.exists(__file__) else io.StringIO('"'"'from setuptools import
> setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"',
> '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))'
> install --record /tmp/pip-record-qafwouur/install-record.txt
> --single-version-externally-managed --compile --install-headers
> /usr/local/include/python3.8/lxml
> cwd:
> /tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/
> Complete output (86 lines):
> Building lxml version 4.6.3.
> Building without Cython.
> Building against libxml2 2.9.10 and libxslt 1.1.29
> running install
> running build
> running build_py
> creating build
> creating build/lib.cygwin-3.2.0-x86_64-3.8
> creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/builder.py -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/cssselect.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/doctestcompare.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/ElementInclude.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/pyclasslookup.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/sax.py -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/usedoctest.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/_elementpath.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/__init__.py -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/__init__.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/builder.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/clean.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/defs.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/diff.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/ElementSoup.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/formfill.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/html5parser.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/soupparser.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/usedoctest.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/_diffcommand.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/_html5builder.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/_setmixin.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> copying src/lxml/html/__init__.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/html
> creating build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron
> copying src/lxml/isoschematron/__init__.py ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron
> copying src/lxml/etree.h -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/etree_api.h -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/lxml.etree.h ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/lxml.etree_api.h ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> copying src/lxml/includes/c14n.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/config.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/dtdvalid.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/etreepublic.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/htmlparser.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/relaxng.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/schematron.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/tree.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/uri.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/xinclude.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/xmlerror.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/xmlparser.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/xmlschema.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/xpath.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/xslt.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/__init__.pxd ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/etree_defs.h ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> copying src/lxml/includes/lxml-version.h ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/includes
> creating
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources
> creating
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/rng
> copying src/lxml/isoschematron/resources/rng/iso-schematron.rng ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/rng
> creating
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl
> copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl
> copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl
> creating
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
>
>
> copying
> src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl
>
> -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron
> /resources/xsl/iso-schematron-xslt1
> copying
> src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl
>
> -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/re
> sources/xsl/iso-schematron-xslt1
> copying
> src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl
>
> -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschemat
> ron/resources/xsl/iso-schematron-xslt1
> copying
> src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl
>
> -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml
> /isoschematron/resources/xsl/iso-schematron-xslt1
> copying
> src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl
>
> -> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/
> resources/xsl/iso-schematron-xslt1
> copying
> src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt ->
> build/lib.cygwin-3.2.0-x86_64-3.8/lxml/isoschematron/resources/xs
> l/iso-schematron-xslt1
> running build_ext
> building 'lxml.etree' extension
> creating build/temp.cygwin-3.2.0-x86_64-3.8
> creating build/temp.cygwin-3.2.0-x86_64-3.8/src
> creating build/temp.cygwin-3.2.0-x86_64-3.8/src/lxml
> gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall
> -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
> -fstack-protector-strong --param=ssp-buffer-size=4 -DOPENSSL_NO_SSL3=1
> -fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/build=/usr/src/debug/python38-3.8.10-1
>
> -fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/src/Python-3.8.10=/usr/src/debug/python38-3.8.10-1
>
> -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
> -fstack-protector-strong --param=ssp-buffer-size=4 -DOPENSSL_NO_SSL3=1
> -fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/build=/usr/src/debug/python38-3.8.10-1
>
> -fdebug-prefix-map=/pub/devel/python/python38/python38-3.8.10-1.x86_64/src/Python-3.8.10=/usr/src/debug/python38-3.8.10-1
>
> -DCYTHON_CLINE_IN_TRACEBACK=0 -I/usr/include/libxml2 -Isrc
> -Isrc/lxml/includes -I/usr/include/python3.8 -c src/lxml/etree.c -o
> build/temp.cygwin-3.2.0-x86_64-3.8/src/lxml/etree.o -w
> src/lxml/etree.c:97:10: fatal error: Python.h: No such file or
> directory
> 97 | #include "Python.h"
> | ^~~~~~~~~~
> compilation terminated.
> Compile failed: command 'gcc' failed with exit status 1
> creating tmp
> cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c
> /tmp/xmlXPathInit4efcky6t.c -o tmp/xmlXPathInit4efcky6t.o
> cc tmp/xmlXPathInit4efcky6t.o -lxml2 -o a.out.exe
> error: command 'gcc' failed with exit status 1
> ----------------------------------------
> ERROR: Command errored out with exit status 1: /usr/bin/python -u -c
> 'import io, os, sys, setuptools, tokenize; sys.argv[0] =
> '"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/setup.py'"'"';
>
> __file__='"'"'/tmp/pip-install-mp64a75l/lxml_61e1128ed27f4063936a7a21e44c273e/setup.py'"'"';f
>
> = getattr(tokenize, '"'"'open'"'"', open)(__file__) if
> os.path.exists(__file__) else io.StringIO('"'"'from setuptools import
> setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"',
> '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))'
> install --record /tmp/pip-record-qafwouur/install-record.txt
> --single-version-externally-managed --compile --install-headers
> /usr/local/include/python3.8/lxml Check the logs for full command output.
>
>
> --
> Problem reports: https://cygwin.com/problems.html
> FAQ: https://cygwin.com/faq/
> Documentation: https://cygwin.com/docs.html
> Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
>
--
Russell M. Van Tassell <russel...@gmail.com>
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation: https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
