Hi, среда, 18 марта 2015 г., 23:52:19 UTC+5 пользователь Anuj Bansal написал: > > Sir, > > I have learned the differences between Python 2 and Python 3. I have > created a google doc ( > https://docs.google.com/document/d/1xf7OtuyB5b6npCOLalZ-yjPZEcoKNb19iimfElyDino/edit) > > in which I have written the common porting errors which I could find after > going through various blogs and projects and there corresponding syntax > corrections. You can add your valuable suggestions or anything that I have > missed out to it by directly going to the link and editing it. Do tell me > if you find something wrong with the approach. > > >> The recommended way is to use "six" Python module. Some parts of Scrapy >> are already ported to Python 3 - see e.g. >> https://travis-ci.org/scrapy/scrapy/jobs/54761340 - 235 tests pass in >> Python 3.3. To get started try cloning Scrapy and running some tests using >> tox (as described in docs). >> > > I got some errors while setting up scrapy and found out that I had to > install libssl-dev, libffi-dev, python-dev and libxml2-dev. As mentioned on > ( > http://stackoverflow.com/questions/17611324/error-when-installing-scrapy-on-ubuntu-13-04 > ). > Shouldn't these be added to the scrapy requirements ? Should I create an > issue relating to this ? I'm currently working on Ubuntu 14.04. >
Scrapy requirements.txt lists Python packages (not system packages). There are some install notes here: http://doc.scrapy.org/en/latest/intro/install.html libffi-dev is a dependency of PyOpenSSL; libxml2-dev is a dependency of lxml. I'm not sure - maybe we can document this all. It would be documenting the requirements of our requirements though. > > >> You can also check >> https://github.com/scrapy/scrapy/blob/master/tests/py3-ignores.txt file >> - try uncommenting something and run tests again to see what's not ported. >> We can't rely only on tests when porting, but they are a good start. >> > > This is great ! Would really help me in planning my strategy. > > >> This URL encoding thing is where we stopped. Without having a solid >> solution we can't port scrapy.Request, and without scrapy.Request most >> other Scrapy components don't work. >> > > Handling binary data is the most trickiest issue that people face in > supporting Python 2 and Python 3. So the first thing to do would be to find > the best solution for URL encoding. Only then we would be able to port > other scrapy components. > So I should first take a look at the w3lib project. > > As quoted in the book ( > http://python3porting.com/strategies.html#python-2-and-python-3-without-conversion > ): > > "My recommendation for the development workflow if you want to support > Python 3 without using 2to3 is to run 2to3 on the code once and then fix > it up until it works on Python 3. Only then introduce Python 2 support into > the Python 3 code, using six where needed. Add support for Python 2.7 > first, and then Python 2.6. Doing it this way can sometimes result in a > very quick and painless process." > > Is this the recommended method ? > Usually I just start with the existing code and add Python 3 support to it using "six" package and a common sense :) The metod from the book sounds OK, but you need to be very careful not to break existing Python 2.x code. __future__ imports can be also helpful (2to3 doesn't add them). We don't need Python 2.6 support. -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
