I was really asking if you knew a better way to handle the problem of getting stuff into the repository in the first place.
btw, for whatever reason the place where urlgrabber is failing is using easy_install to get stuff, so I don't think pip's -f flag is an option there. -d On Mon, Nov 1, 2010 at 3:12 PM, Ariel Nunez <[email protected]>wrote: > Quoting Simon Willinson: > > At work, the first line in our requirements.txt file is "-f > <http://pypi.internal/%22> > http://pypi.internal/"; - which causes pip to use our own internal PyPI > mirror (just an Apache directory listing which pip/setuptools is smart > enough to scrape). > I dig that idea of just using apache and throwing .tar.gz files there. > > Sent from my iPad > > On Nov 1, 2010, at 13:52, David Winslow <[email protected]> wrote: > > Looking into the "mirror pypi" idea, it seems the > Compoze<http://docs.repoze.org/compoze/>project from Repoze is a pretty > simple way to set up a Pypi-compatible > package index with packages from Pypi or elsewhere. I don't think it will > automatically read our pip requirements files, but for now I'll just it to > just set up a repo with these problem packages. > > Does anyone know a better tool for this? (Ariel??) Otherwise I'll take a > look later this week into making a complete mirror with all the Python > packages we use to prevent this sort of problem from cropping up again. > > -- > David Winslow > OpenGeo - <http://opengeo.org/>http://opengeo.org/ > > On Fri, Oct 29, 2010 at 9:20 AM, Ariel Nunez < <[email protected]> > [email protected]> wrote: > >> I had to do the same for my install yesterday, in my case it was >> because the curl version I had was 1.15 and the latest pycurl required >> 1.17. I had to first install a new curl and then run (inside the >> venv): >> >> pip install pycurl >> pip install >> <http://urlgrabber.baseurl.org/download/urlgrabber-3.9.1.tar.gz> >> http://urlgrabber.baseurl.org/download/urlgrabber-3.9.1.tar.gz >> >> I fully support the idea of setting up our own PyPi mirror, hopefully >> avoiding the rewrite of the build script. >> >> Best, >> Ariel. >> >> >> On Fri, Oct 29, 2010 at 8:02 AM, David Winslow < <[email protected]> >> [email protected]> wrote: >> > I haven't looked very hard but I can't find any information about the >> > removal of urlgrabber from the python package repository. I think it >> would >> > probably be a good idea for us to start mirroring our dependencies in a >> > repository of our own to avoid this sort of thing. >> > In the shorter term I would be open to rewriting the build script to >> avoid >> > requiring urlgrabber. >> > -- >> > David Winslow >> > OpenGeo - <http://opengeo.org/>http://opengeo.org/ >> > On Fri, Oct 29, 2010 at 7:56 AM, Christian Spanring <<[email protected]> >> [email protected]> >> > wrote: >> >> >> >> Ole, >> >> >> >> the problem is that easy_install can't find urlgrabber at >> <http://pypi.python.org>pypi.python.org >> >> anymore, so you have to install urlgrabber manually as workaround: >> >> >> >> download and unpack it from <http://urlgrabber.baseurl.org/> >> http://urlgrabber.baseurl.org/ >> >> >> >> $ wget >> <http://urlgrabber.baseurl.org/download/urlgrabber-3.9.1.tar.gz> >> http://urlgrabber.baseurl.org/download/urlgrabber-3.9.1.tar.gz >> >> $ tar xvzf urlgrabber-3.9.1.tar.gz >> >> >> >> install urlgrabber in GeoNode's Virtual Environment: >> >> >> >> $ . bin/activate >> >> (geonode) $ cd urlgrabber-3.9.1 >> >> (geonode) $ python setup.py install >> >> >> >> in my case it asked for pycurl, which I had to install first >> >> (geonode) $ easy_install pycurl >> >> >> >> That should to it. >> >> >> >> Christian >> >> >> >> On Oct 29, 2010, at 1:54 AM, Ole Nielsen wrote: >> >> >> >> > Dear all >> >> > I am trying to install GeoNode according to the instructions at >> >> > <http://github.com/GeoNode/geonode/blob/1.0-RC2/README.rst> >> http://github.com/GeoNode/geonode/blob/1.0-RC2/README.rst and have gotten >> as >> >> > far as paver build. >> >> > >> >> > However, paver build fails with the following error: ImportError: No >> >> > module named urlgrabber.grabber >> >> > However, if I start Python normally (i.e. without virtual >> environment) >> >> > and try it works just fine: >> >> > >> >> > Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18) >> >> > [GCC 4.3.3] on linux2 >> >> > Type "help", "copyright", "credits" or "license" for more >> information. >> >> > >>> from urlgrabber.grabber import urlgrab, URLGrabError >> >> > >>> >> >> > >> >> > I am running Ubuntu 9.04 - 64 bit. >> >> > >> >> > The tail of the paver output follows below. >> >> > Cheers and thanks for any help >> >> > Ole Nielsen >> >> > >> >> > -- output from paver build -- >> >> > >> >> > ########## SUMMARY INFO ######### >> >> > ################################################ >> >> > #Attempting install of _rl_accel, sgmlop & pyHnj >> >> > #extensions from '/tmp/pip-0Buy2p-build/src/rl_addons/rl_accel' >> >> > ################################################ >> >> > ################################################ >> >> > #Attempting install of _renderPM >> >> > #extensions from '/tmp/pip-0Buy2p-build/src/rl_addons/renderPM' >> >> > # installing with freetype version 21 >> >> > ################################################ >> >> > Successfully installed avatar gsconfig.py OWSLib reportlab >> >> > ---> pavement.setup_webapps >> >> > ---> pavement.setup_geoserver >> >> > ---> pavement.setup_gs_data >> >> > Traceback (most recent call last): >> >> > File "/home/nielso/sandpit/geonode/bin/paver", line 8, in <module> >> >> > load_entry_point('Paver==1.0.1', 'console_scripts', 'paver')() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 615, in main >> >> > _launch_pavement(args) >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 598, in _launch_pavement >> >> > _process_commands(args, auto_pending=auto_pending) >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 563, in _process_commands >> >> > task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 241, in __call__ >> >> > retval = environment._run_task( <http://self.name>self.name, >> self.needs, self.func) >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 181, in _run_task >> >> > return do_task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 160, in do_task >> >> > task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 241, in __call__ >> >> > retval = environment._run_task( <http://self.name>self.name, >> self.needs, self.func) >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 181, in _run_task >> >> > return do_task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 160, in do_task >> >> > task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 241, in __call__ >> >> > retval = environment._run_task( <http://self.name>self.name, >> self.needs, self.func) >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 181, in _run_task >> >> > return do_task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 160, in do_task >> >> > task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 241, in __call__ >> >> > retval = environment._run_task( <http://self.name>self.name, >> self.needs, self.func) >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 181, in _run_task >> >> > return do_task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 161, in do_task >> >> > return func(**kw) >> >> > File "pavement.py", line 186, in setup_gs_data >> >> > grab(src_url, dst_url) >> >> > File "pavement.py", line 159, in grab >> >> > from urlgrabber.grabber import urlgrab, URLGrabError >> >> > ImportError: No module named urlgrabber.grabber >> >> > (geonode)nie...@shiva:~/sandpit/geonode$ >> >> > >> >> > >> >> > >> >> > >> >> >> > >> > >> > >
