Hi, On Mon, Nov 19, 2012 at 6:17 PM, Aaron Meurer <[email protected]> wrote: > On Mon, Nov 19, 2012 at 1:11 AM, Matthew Brett <[email protected]> > wrote: >> >> Hi, >> >> On Mon, Nov 19, 2012 at 12:01 AM, Aaron Meurer <[email protected]> wrote: >> > On Sun, Nov 18, 2012 at 11:19 PM, Matthew Brett >> > <[email protected]> >> > wrote: >> >> >> >> Hi, >> >> >> >> On Sun, Nov 18, 2012 at 6:48 PM, Aaron Meurer <[email protected]> >> >> wrote: >> >> > On Sun, Nov 18, 2012 at 3:16 PM, Matthew Brett >> >> > <[email protected]> >> >> > wrote: >> >> >> >> >> >> Hi, >> >> >> >> >> >> On Sun, Nov 18, 2012 at 2:06 PM, Aaron Meurer <[email protected]> >> >> >> wrote: >> >> >> > On Sat, Nov 17, 2012 at 8:46 PM, Matthew Brett >> >> >> > <[email protected]> >> >> >> > wrote: >> >> >> >> >> >> >> >> Hi, >> >> >> >> >> >> >> >> On Sat, Nov 17, 2012 at 6:51 PM, Aaron Meurer >> >> >> >> <[email protected]> >> >> >> >> wrote: >> >> >> >> > On Nov 17, 2012, at 7:39 PM, Matthew Brett >> >> >> >> > <[email protected]> >> >> >> >> > wrote: >> >> >> >> > >> >> >> >> >> Hi, >> >> >> >> >> >> >> >> >> >> I've been running into trouble installing sympy with >> >> >> >> >> easy_install >> >> >> >> >> and >> >> >> >> >> pip on python 2.7 and Python 3.3. >> >> >> >> > >> >> >> >> > What issues with pip are you having? I thought we fixed pip to >> >> >> >> > do >> >> >> >> > the >> >> >> >> > right thing. >> >> >> >> >> >> >> >> Ah - I have a strong memory that I had issues with pip, but not >> >> >> >> now >> >> >> >> I >> >> >> >> test it on this machine. Broken for easy_install (via >> >> >> >> distribute): >> >> >> > >> >> >> > >> >> >> > That's because pip *was* broken, but we figured out how to fix it. >> >> >> > The >> >> >> > trick was adding -py3.2 and so on to the tarball names so that it >> >> >> > was >> >> >> > sure >> >> >> > to download the right one. >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> mb312@mikesilver ~]$ mkvirtualenv py27 --no-site-packages >> >> >> >> New python executable in py27/bin/python >> >> >> >> Installing setuptools.............done. >> >> >> >> Installing pip...............done. >> >> >> >> [mb312@mikesilver ~]$ workon py27 >> >> >> >> (py27)[mb312@mikesilver ~]$ easy_install sympy >> >> >> >> Searching for sympy >> >> >> >> Reading http://pypi.python.org/simple/sympy/ >> >> >> >> Reading http://code.google.com/p/sympy >> >> >> >> Reading http://sympy.org >> >> >> >> Reading http://sympy.org/ >> >> >> >> Reading >> >> >> >> >> >> >> >> >> >> >> >> http://code.google.com/p/sympy/downloads/detail?name=sympy-0.7.0.tar.gz >> >> >> >> Best match: sympy 0.7.2-py3.3 >> >> >> >> Downloading >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> http://pypi.python.org/packages/source/s/sympy/sympy-0.7.2-py3.3.tar.gz#md5=4645f4bd3c50dac7486ccfaedbb29057 >> >> >> >> Processing sympy-0.7.2-py3.3.tar.gz >> >> >> >> Running sympy-0.7.2/setup.py -q bdist_egg --dist-dir >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> /var/folders/YB/YBjJ+eaQGb0EZfa9PJfcSk+++Tg/-Tmp-/easy_install-n5AKga/sympy-0.7.2/egg-dist-tmp-WFXCnm >> >> >> >> Traceback (most recent call last): >> >> >> >> File "/Users/mb312/.virtualenvs/py27/bin/easy_install", line 8, >> >> >> >> in >> >> >> >> <module> >> >> >> >> load_entry_point('setuptools==0.6c11', 'console_scripts', >> >> >> >> 'easy_install')() >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Users/mb312/.virtualenvs/py27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", >> >> >> >> line 1712, in main >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Users/mb312/.virtualenvs/py27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", >> >> >> >> line 1700, in with_ei_usage >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Users/mb312/.virtualenvs/py27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", >> >> >> >> line 1716, in <lambda> >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", >> >> >> >> line 152, in setup >> >> >> >> dist.run_commands() >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", >> >> >> >> line 953, in run_commands >> >> >> >> self.run_command(cmd) >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", >> >> >> >> line 972, in run_command >> >> >> >> cmd_obj.run() >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Users/mb312/.virtualenvs/py27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", >> >> >> >> line 211, in run >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Users/mb312/.virtualenvs/py27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", >> >> >> >> line 446, in easy_install >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Users/mb312/.virtualenvs/py27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", >> >> >> >> line 476, in install_item >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Users/mb312/.virtualenvs/py27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", >> >> >> >> line 655, in install_eggs >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Users/mb312/.virtualenvs/py27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", >> >> >> >> line 930, in build_and_install >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Users/mb312/.virtualenvs/py27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", >> >> >> >> line 919, in run_setup >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Users/mb312/.virtualenvs/py27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/sandbox.py", >> >> >> >> line 62, in run_setup >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Users/mb312/.virtualenvs/py27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/sandbox.py", >> >> >> >> line 105, in run >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/Users/mb312/.virtualenvs/py27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/sandbox.py", >> >> >> >> line 64, in <lambda> >> >> >> >> File "setup.py", line 36, in <module> >> >> >> >> File >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> "/var/folders/YB/YBjJ+eaQGb0EZfa9PJfcSk+++Tg/-Tmp-/easy_install-n5AKga/sympy-0.7.2/sympy/__init__.py", >> >> >> >> line 27, in <module> >> >> >> >> ImportError: It appears 2to3 has been run on the codebase. Use >> >> >> >> Python >> >> >> >> 3 or get the original source code. >> >> >> > >> >> >> > >> >> >> > ARGH! I did not know this. It seems that the same solution for >> >> >> > pip >> >> >> > is >> >> >> > the >> >> >> > problem for easy_install. It thinks that sympy-0.7.2-py3.3 is the >> >> >> > latest >> >> >> > version. >> >> >> > >> >> >> > Any thoughts on how this can be fixed? I've said this before, but >> >> >> > it >> >> >> > deserves reiterating: I as a package maintainer have surprisingly >> >> >> > little >> >> >> > control over what pip or easy_install install. The best I can do >> >> >> > is >> >> >> > to >> >> >> > manipulate what links are on PyPI and to use naming trickery to >> >> >> > make >> >> >> > it >> >> >> > install the "latest" version. >> >> >> > >> >> >> > I created https://code.google.com/p/sympy/issues/detail?id=3511 >> >> >> > for >> >> >> > this. >> >> >> >> >> >> How about having a sympy3 pypi project? >> >> > >> >> > >> >> > I hadn't thought of that. If we did this, then it would never do the >> >> > right >> >> > thing in Python 3, i.e., the users would have to know to use "pip >> >> > install >> >> > sympy3" instead of "pip install sympy" in Python 3. It might also >> >> > complicated stuff for people whose travis.ini (or whatever) would >> >> > otherwise >> >> > just have a single pip install line. >> >> >> >> Maybe it would be OK if there was a suitable error message from an >> >> attempted pip install sympy in python 3? Sorry - I'm not sure that's >> >> the best solution, just throwing it out there. >> > >> > >> > There is, though admittedly it could be better: >> > >> > $python3 setup.py # python 2 setup.py >> > Traceback (most recent call last): >> > File "setup.py", line 36, in <module> >> > import sympy >> > File >> > "/Users/aaronmeurer/Documents/Python/sympy/sympy/sympy/__init__.py", >> > line 35, in <module> >> > raise ImportError("This is the Python 2 version of SymPy. To use >> > SymPy " >> > ImportError: This is the Python 2 version of SymPy. To use SymPy with >> > Python >> > 3, please obtain a Python 3 version from http://sympy.org, or use the >> > bin/use2to3 script if you are using the git version. >> > >> > $python2 setup.py # python2 setup.py >> > Traceback (most recent call last): >> > File "setup.py", line 36, in <module> >> > import sympy >> > File >> > >> > "/Users/aaronmeurer/Documents/Python/sympy/sympy/py3k-sympy/sympy/__init__.py", >> > line 27, in <module> >> > raise ImportError("It appears 2to3 has been run on the codebase. Use >> > " >> > ImportError: It appears 2to3 has been run on the codebase. Use Python 3 >> > or >> > get the original source code. >> > >> > In particular, I don't think users care about use2to3, and it's just >> > confusing. >> > >> >> >> >> >> >> I agree it might be more complicated to sort out in travis, but people >> >> in the position of writing a travis file for multiple pythons are >> >> likely to be up to that task I would have thought. >> >> >> >> >> >> We use easy_install much more than pip because pip will not >> >> >> >> install >> >> >> >> binaries, and so is more or less useless for compiled packages. >> >> >> > >> >> >> > >> >> >> > My best suggestion for this is to use pip for SymPy and >> >> >> > easy_install >> >> >> > for >> >> >> > the >> >> >> > rest. Or you could just pass the url manually to eas_install. If >> >> >> > you >> >> >> > can >> >> >> > figure out how to name the tarballs so that pip *and* easy_install >> >> >> > do >> >> >> > the >> >> >> > right thing, I'd love to hear it. But I'm very doubtful that a >> >> >> > solution >> >> >> > exists. The Python packaging ecosystem is irrevocably broken as >> >> >> > far >> >> >> > as >> >> >> > I'm >> >> >> > concerned. >> >> >> >> >> >> Tell it brother :) And yes of course it throws the burden for the >> >> >> workarounds onto each package maintainer, which is tiring and >> >> >> wasteful. But nevertheless, it can be made to work, by doing 2to3 >> >> >> in >> >> >> the setup.py. >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> The fact that easy_install / pip is unlikely to work means >> >> >> >> >> that >> >> >> >> >> sympy >> >> >> >> >> is a considerably heavier burden for our (nipy.org/nipy) users >> >> >> >> >> to >> >> >> >> >> install, and I'd very much like to help fix that. >> >> >> >> >> >> >> >> >> >> This led me to start editing the sympy setup.py to do the >> >> >> >> >> standard >> >> >> >> >> thing of running 2to3 during setu.py install : (e.g numpy, >> >> >> >> >> scipy, >> >> >> >> >> matplotlib, our projects). >> >> >> >> >> >> >> >> >> >> I immediately found why you haven't done this: you depend on >> >> >> >> >> running >> >> >> >> >> >> >> >> >> >> python setup.py run_tests >> >> >> >> >> >> >> >> >> >> and >> >> >> >> >> >> >> >> >> >> python setup.py run_benchkmarks >> >> >> >> >> >> >> >> >> >> in the source tree. So, there's no way of running these for >> >> >> >> >> python >> >> >> >> >> 2 >> >> >> >> >> and python 3 in the same source tree (unless you go for python >> >> >> >> >> 3 >> >> >> >> >> compatibility in source - a bit much). >> >> >> >> >> >> >> >> >> >> So - I'm writing to ask if you'd consider looking at a >> >> >> >> >> refactor >> >> >> >> >> that >> >> >> >> >> would change these calls above to: >> >> >> >> >> >> >> >> >> >> python setup.py install --user (or whatever) >> >> >> >> >> cd /somewhere >> >> >> >> >> python -c 'import sympy; sympy.test()' >> >> >> >> > >> >> >> >> > The problem is that setup.py test runs tests that aren't >> >> >> >> > included >> >> >> >> > in >> >> >> >> > an install, namely the Sphinx doctests. >> >> >> >> >> >> >> >> Would it be an option to run these separately? For example with >> >> >> >> >> >> >> >> python setup.py run_doctests >> >> >> >> >> >> >> >> ? Or maybe with >> >> >> >> >> >> >> >> python setup.py run_tests >> >> >> >> >> >> >> >> where ``run_tests`` does something like: >> >> >> >> >> >> >> >> mkdir tmp >> >> >> >> cd tmp >> >> >> >> python -c 'import sympy; sympy.test()' >> >> >> >> cd .. >> >> >> >> cd doc >> >> >> >> make.py doctest >> >> >> > >> >> >> > >> >> >> > setup.py test just runs all combinations of tests, which already >> >> >> > have >> >> >> > their >> >> >> > individual scripts. I think right now it's equivalent to something >> >> >> > like >> >> >> > >> >> >> > ./bin/test >> >> >> > ./bin/doctest >> >> >> > <whatever the sage test script is> >> >> >> > ./examples/all.py -q >> >> >> > >> >> >> > Actually, it just runs sympy.utilities.runtests.run_all_tests(), >> >> >> > which >> >> >> > is >> >> >> > itself a wrapper to the above tests. So there is actually no need >> >> >> > for >> >> >> > anything in the setup.py script itself to do tests. It's just a >> >> >> > nice >> >> >> > shortcut for us. >> >> >> >> >> >> RIght - I think I'm asking whether you'd consider moving from: >> >> >> >> >> >> python setup.py run_tests >> >> >> >> >> >> to >> >> >> >> >> >> python setup.py install >> >> >> python setup.py run_tests >> >> >> >> >> >> in order to make it possible to processing the source tree (for >> >> >> compiled code or 2to3) before running? >> >> > >> >> > >> >> > Am I misunderstanding what you're asking for? Would you run the >> >> > tests >> >> > against the installed version or the local version? >> >> >> >> Yes, the point would be to run the tests against the version on the >> >> path, whatever that was, not the local version. So attempting to run >> >> the tests without an install of some sort (including 'develop') would >> >> get a message on the lines of >> >> >> >> "Please install sympy on the python path before running tests; see >> >> http:/blah/blah" >> > >> > >> > OK. Well, like I said, this won't work because we test things that >> > aren't >> > installed, namely, the Sphinx documentation doctests. Actually, the >> > examples are not installed (but they are included in the tarball). >> > Also, it >> > seems like an unnecessary restriction to make people use virtualenv to >> > test >> > SymPy when it's not actually necessary. >> >> OK. I think I'm not seeing why it won't work, because I'm suggesting >> `python setup.py run_tests` from the source tree, the only difference >> being there would have to be some sort of install of the 'sympy' >> library code first. It might be a virtualenv or it might be 'python >> setup.py develop' for python 2, or it might be 'python setup.py >> install --user' or 'python setup.py install >> --prefix=/somewhere/on/pythonpath. >> >> In my experience so far, most people who are up to checking out the >> git repo and running tests are well capable of doing one of these. >> However, the user who gets an error message on easy_install may not >> find it easy to solve. >> >> >> >> >> - if you see what I mean. This would then fail unless there is >> >> >> >> an >> >> >> >> installation somewhere, perhaps with a nice message to run >> >> >> >> ``python >> >> >> >> setup.py install`` first. >> >> >> >> >> >> >> >> I'm sorry for my ignorance - but how have y'all been doing the >> >> >> >> edit >> >> >> >> / >> >> >> >> debug cycle in python 3? >> >> >> > >> >> >> > >> >> >> > I never use virtualenv to test SymPy, unless I am testing >> >> >> > installation. >> >> >> > I >> >> >> > just run ./bin/use2to3 and cd py3k-sympy and work locally. I >> >> >> > guess >> >> >> > it's >> >> >> > easy for us because we don't have any dependencies, especially >> >> >> > dependencies >> >> >> > that don't work unless installed. >> >> >> >> >> >> I probably wasn't clear; I was thinking that it might be tough to >> >> >> have >> >> >> to >> >> >> do: >> >> >> >> >> >> ./bin/use2to3 >> >> >> cd py3k-sympy >> >> >> while not Finished: >> >> >> <edit> >> >> >> <test> >> >> >> and then have to backport the changes in py3k-sympy to the sympy >> >> >> tree >> >> >> - but I probably didn't get the workflow. >> >> > >> >> > >> >> > Oh, we never develop against the Python 3 code (or at least I don't). >> >> > I >> >> > just >> >> > make all changes against Python 2 and then 99% of the time 2to3 does >> >> > the >> >> > right thing and it works in Python 3 as well, and for the remaining >> >> > 1%, >> >> > I >> >> > find out later and fix it using whatever workaround. I guess the take >> >> > away >> >> > is that I never personally use Python 3, except for testing to see if >> >> > it >> >> > work. My day-to-day workflow is still Python 2. I think I wrote one >> >> > script >> >> > for personal use in Python 3. Maybe if some of the other SymPy >> >> > developers >> >> > use Python 3 more they do things differently. >> >> >> >> I guess at some point you will have to switch to a workflow that's >> >> easy for developers on Python 3? I just wondering if doing that now, >> >> along with fixing easy_install, might be worth the pain of losing the >> >> ability to run the tests from the local tree. >> > >> > >> > I don't see people using Python 3 over Python 2 any time soon. I guess >> > if >> > that ever happens, we can figure it out from there. For the most part, >> > we >> > have to worry about the lowest common denominator, i.e., currently >> > Python >> > 2.5, because that's where the fewest Python features are supported. >> >> OK - well - I think I'm getting the message :) The offer is open of >> a pull request if you would like to consider the alternative. > > > Actually, I still get the feeling that I am misunderstanding what you want > to do, so if it's not too much work, go ahead and submit a pull request, and > then I can see exactly what you mean.
This is just to explain my own developing understanding of the problem to see if it helps. I think the current situation is this: In python2 * ``python setup.py run_tests`` runs tests from source tree * tests include sphinx docstring tests, and examples and sage tests * python3 tarball for e.g. pypi generated with `python setup.py sdist` In python3: * ./bin/use2to3 recreates whole source tree in py3-sympy subdirectory, including sphinx doctests and examples * running tests needs: 'cd py3k-sympy ; python setup.py run_tests * python 3 tarball generated from `py3k-sympy` tree with `cd py3k-sympy; python setup.py install` Benefits of current setup: * Users can run all of sympy tests from local tree without doing an install * Complete porting of code / docs to python3 Problems with the current setup: * Need different source (sdist) tarballs for python2 and python3. This seems to make it impossible for both of easy_install and pip to work by default on python2 and python3. * Python3 not attractive as development environment because of need to backport changes from py3k-sympy tree Is this a reasonable summary? See you, Matthew -- You received this message because you are subscribed to the Google Groups "sympy" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sympy?hl=en.
