On Nov 1, 2010, at 14:16, David Winslow <[email protected]> wrote:
I was really asking if you knew a better way to handle the problem of getting stuff into the repository in the first place. Gotcha, in that case i think using the --fetch-site-packages option in compoze looks like the right place to start. I know there are many projects to create PyPi mirrors but have used none of them, Compoze looks good enough to me. btw, for whatever reason the place where urlgrabber is failing is using easy_install to get stuff, so I don't think pip's -f flag is an option there. -d On Mon, Nov 1, 2010 at 3:12 PM, Ariel Nunez <[email protected]>wrote: > Quoting Simon Willinson: > > At work, the first line in our requirements.txt file is "-f > <http://pypi.internal/%22> > http://pypi.internal/"; - which causes pip to use our own internal PyPI > mirror (just an Apache directory listing which pip/setuptools is smart > enough to scrape). > I dig that idea of just using apache and throwing .tar.gz files there. > > Sent from my i > On Nov 1, 2010, at 13:52, David Winslow <[email protected]> wrote: > > Looking into the "mirror pypi" idea, it seems the > Compoze<http://docs.repoze.org/compoze/>project from Repoze is a pretty > simple way to set up a Pypi-compatible > package index with packages from Pypi or elsewhere. I don't think it will > automatically read our pip requirements files, but for now I'll just it to > just set up a repo with these problem packages. > > Does anyone know a better tool for this? (Ariel??) Otherwise I'll take a > look later this week into making a complete mirror with all the Python > packages we use to prevent this sort of problem from cropping up again. > > -- > David Winslow > OpenGeo - <http://opengeo.org/>http://opengeo.org/ > > On Fri, Oct 29, 2010 at 9:20 AM, Ariel Nunez < <[email protected]> > [email protected]> wrote: > >> I had to do the same for my install yesterday, in my case it was >> because the curl version I had was 1.15 and the latest pycurl required >> 1.17. I had to first install a new curl and then run (inside the >> venv): >> >> pip install pycurl >> pip install >> <http://urlgrabber.baseurl.org/download/urlgrabber-3.9.1.tar.gz> >> http://urlgrabber.baseurl.org/download/urlgrabber-3.9.1.tar.gz >> >> I fully support the idea of setting up our own PyPi mirror, hopefully >> avoiding the rewrite of the build script. >> >> Best, >> Ariel. >> >> >> On Fri, Oct 29, 2010 at 8:02 AM, David Winslow < <[email protected]> >> [email protected]> wrote: >> > I haven't looked very hard but I can't find any information about the >> > removal of urlgrabber from the python package repository. I think it >> would >> > probably be a good idea for us to start mirroring our dependencies in a >> > repository of our own to avoid this sort of thing. >> > In the shorter term I would be open to rewriting the build script to >> avoid >> > requiring urlgrabber. >> > -- >> > David Winslow >> > OpenGeo - <http://opengeo.org/>http://opengeo.org/ >> > On Fri, Oct 29, 2010 at 7:56 AM, Christian Spanring <<[email protected]> >> [email protected]> >> > wrote: >> >> >> >> Ole, >> >> >> >> the problem is that easy_install can't find urlgrabber at >> <http://pypi.python.org>pypi.python.org >> >> anymore, so you have to install urlgrabber manually as workaround: >> >> >> >> download and unpack it from <http://urlgrabber.baseurl.org/> >> http://urlgrabber.baseurl.org/ >> >> >> >> $ wget >> <http://urlgrabber.baseurl.org/download/urlgrabber-3.9.1.tar.gz> >> http://urlgrabber.baseurl.org/download/urlgrabber-3.9.1.tar.gz >> >> $ tar xvzf urlgrabber-3.9.1.tar.gz >> >> >> >> install urlgrabber in GeoNode's Virtual Environment: >> >> >> >> $ . bin/activate >> >> (geonode) $ cd urlgrabber-3.9.1 >> >> (geonode) $ python setup.py install >> >> >> >> in my case it asked for pycurl, which I had to install first >> >> (geonode) $ easy_install pycurl >> >> >> >> That should to it. >> >> >> >> Christian >> >> >> >> On Oct 29, 2010, at 1:54 AM, Ole Nielsen wrote: >> >> >> >> > Dear all >> >> > I am trying to install GeoNode according to the instructions at >> >> > <http://github.com/GeoNode/geonode/blob/1.0-RC2/README.rst> >> http://github.com/GeoNode/geonode/blob/1.0-RC2/README.rst and have gotten >> as >> >> > far as paver build. >> >> > >> >> > However, paver build fails with the following error: ImportError: No >> >> > module named urlgrabber.grabber >> >> > However, if I start Python normally (i.e. without virtual >> environment) >> >> > and try it works just fine: >> >> > >> >> > Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18) >> >> > [GCC 4.3.3] on linux2 >> >> > Type "help", "copyright", "credits" or "license" for more >> information. >> >> > >>> from urlgrabber.grabber import urlgrab, URLGrabError >> >> > >>> >> >> > >> >> > I am running Ubuntu 9.04 - 64 bit. >> >> > >> >> > The tail of the paver output follows below. >> >> > Cheers and thanks for any help >> >> > Ole Nielsen >> >> > >> >> > -- output from paver build -- >> >> > >> >> > ########## SUMMARY INFO ######### >> >> > ################################################ >> >> > #Attempting install of _rl_accel, sgmlop & pyHnj >> >> > #extensions from '/tmp/pip-0Buy2p-build/src/rl_addons/rl_accel' >> >> > ################################################ >> >> > ################################################ >> >> > #Attempting install of _renderPM >> >> > #extensions from '/tmp/pip-0Buy2p-build/src/rl_addons/renderPM' >> >> > # installing with freetype version 21 >> >> > ################################################ >> >> > Successfully installed avatar gsconfig.py OWSLib reportlab >> >> > ---> pavement.setup_webapps >> >> > ---> pavement.setup_geoserver >> >> > ---> pavement.setup_gs_data >> >> > Traceback (most recent call last): >> >> > File "/home/nielso/sandpit/geonode/bin/paver", line 8, in <module> >> >> > load_entry_point('Paver==1.0.1', 'console_scripts', 'paver')() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 615, in main >> >> > _launch_pavement(args) >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 598, in _launch_pavement >> >> > _process_commands(args, auto_pending=auto_pending) >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 563, in _process_commands >> >> > task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 241, in __call__ >> >> > retval = environment._run_task( <http://self.name>self.name, >> self.needs, self.func) >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 181, in _run_task >> >> > return do_task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 160, in do_task >> >> > task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 241, in __call__ >> >> > retval = environment._run_task( <http://self.name>self.name, >> self.needs, self.func) >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 181, in _run_task >> >> > return do_task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 160, in do_task >> >> > task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 241, in __call__ >> >> > retval = environment._run_task( <http://self.name>self.name, >> self.needs, self.func) >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 181, in _run_task >> >> > return do_task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 160, in do_task >> >> > task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 241, in __call__ >> >> > retval = environment._run_task( <http://self.name>self.name, >> self.needs, self.func) >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 181, in _run_task >> >> > return do_task() >> >> > File >> >> > >> "/home/nielso/sandpit/geonode/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", >> >> > line 161, in do_task >> >> > return func(**kw) >> >> > File "pavement.py", line 186, in setup_gs_data >> >> > grab(src_url, dst_url) >> >> > File "pavement.py", line 159, in grab >> >> > from urlgrabber.grabber import urlgrab, URLGrabError >> >> > ImportError: No module named urlgrabber.grabber >> >> > (geonode)nie...@shiva:~/sandpit/geonode$ >> >> > >> >> > >> >> > >> >> > >> >> >> > >> > >> > >
