Re: [Discuss-gnuradio] installing a python package without elevated privileges

[U L]

Being all things to everyone is tough.  I recently contributed some code to
support the openSUSE package manager, zypper, in pybombs.  So universal
support is kind of an evolving feature.  Perhaps it would be a start to
simply limit the initial virtualenv support to python2.  I would suggest
that if one were in a virtualenv, you let that handle the pip version
(which should be referenced just by the pip command).  I would also submit
that python packages should not be added to the system via a package
manager if you're in a virtualenv.  This would alleviate some of the
package naming discrepancies between the versions.  I don't think
gr-recipes supports python3 packages right now for any packager.  Good
points nonetheless.

Jared.

On Fri, Mar 31, 2017 at 10:30 AM, Martin Braun 
wrote:

> Please note that there's an open issue on how PyBOMBS and venvs are
> effectively broken (https://github.com/gnuradio/pybombs/issues/363).
> There are also related issues regarding the fact that we don't know if
> we're running Python 2 or 3. So yeah, I know about this, and would love
> to see it fixed, but I'm not doing it myself anytime soon.
>
> For example, what if your system's default is Python 2, but your
> virtualenv is running Python 3. In that case, you would need to make
> sure that you don't accidentally check your system's package manager for
> python2-mako. pybombs itself may be running a different Python than is
> going to be executed in the prefix.
> The same holds for if we need pip, pip2, or pip3.
>
> I'm open for suggestions and pull requests, but keep in mind that I
> can't merge stuff if
>
> - It's based on personal preferences, e.g. the pip vs. apt discussion.
> Note that there's also different cases here, you might prefer apt over
> pip for system-wide installs, and pip over apt for your virtualenv install.
> - It hardcodes stuff where it shouldn't (e.g., having IF PKGR == PIP
> style code in the package_manager.py)
> - It makes assumptions about people's systems.
>
> Also, I should acknowledge that both Jared and Marcus have contributed
> to PyBOMBS, so this is not just an empty discussion here :)
>
> Cheers,
> Martin
>
> On 03/27/2017 08:14 PM, U L wrote:
> > This is a problem I have faced as well.  I have a number of projects at
> > various levels of modification of GR and various 3rd party python
> > packages being supported.  I find having virtualenvs to be very useful
> > in keeping python package versions, GR mods, and my own modules in
> > agreement.  I've been tinkering a bunch with getting venvs and pybombs
> > to work together and I think I've made some headway.  For example, the
> > elevated privileges for pip is (nominally) hardcoded in
> > packagers/pip.py.  I made a modification to condition elevation on the
> > presence of a virtualenv, which is detected in the config_manager.  This
> > is one way to perhaps get pybombs and virtualenvs to play nice.  I'll
> > hopefully be offering a pull request soon that incorporates a number of
> > changes.
> >
> > On Wed, Mar 22, 2017 at 12:30 PM, Naceur  > > wrote:
> >
> > You are right if they had access to it at first place. Anyhow I
> > fixed it.
> >
> >
> >
> > --
> > View this message in context:
> > http://gnuradio.4.n7.nabble.com/installing-a-python-
> package-without-elevated-privileges-tp63183p63258.html
> >  package-without-elevated-privileges-tp63183p63258.html>
> > Sent from the GnuRadio mailing list archive at Nabble.com.
> >
> > ___
> > Discuss-gnuradio mailing list
> > Discuss-gnuradio@gnu.org 
> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> > 
> >
> >
> >
> >
> > ___
> > Discuss-gnuradio mailing list
> > Discuss-gnuradio@gnu.org
> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> >
>
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] installing a python package without elevated privileges

[Martin Braun]

Please note that there's an open issue on how PyBOMBS and venvs are
effectively broken (https://github.com/gnuradio/pybombs/issues/363).
There are also related issues regarding the fact that we don't know if
we're running Python 2 or 3. So yeah, I know about this, and would love
to see it fixed, but I'm not doing it myself anytime soon.

For example, what if your system's default is Python 2, but your
virtualenv is running Python 3. In that case, you would need to make
sure that you don't accidentally check your system's package manager for
python2-mako. pybombs itself may be running a different Python than is
going to be executed in the prefix.
The same holds for if we need pip, pip2, or pip3.

I'm open for suggestions and pull requests, but keep in mind that I
can't merge stuff if

- It's based on personal preferences, e.g. the pip vs. apt discussion.
Note that there's also different cases here, you might prefer apt over
pip for system-wide installs, and pip over apt for your virtualenv install.
- It hardcodes stuff where it shouldn't (e.g., having IF PKGR == PIP
style code in the package_manager.py)
- It makes assumptions about people's systems.

Also, I should acknowledge that both Jared and Marcus have contributed
to PyBOMBS, so this is not just an empty discussion here :)

Cheers,
Martin

On 03/27/2017 08:14 PM, U L wrote:
> This is a problem I have faced as well.  I have a number of projects at
> various levels of modification of GR and various 3rd party python
> packages being supported.  I find having virtualenvs to be very useful
> in keeping python package versions, GR mods, and my own modules in
> agreement.  I've been tinkering a bunch with getting venvs and pybombs
> to work together and I think I've made some headway.  For example, the
> elevated privileges for pip is (nominally) hardcoded in
> packagers/pip.py.  I made a modification to condition elevation on the
> presence of a virtualenv, which is detected in the config_manager.  This
> is one way to perhaps get pybombs and virtualenvs to play nice.  I'll
> hopefully be offering a pull request soon that incorporates a number of
> changes.
> 
> On Wed, Mar 22, 2017 at 12:30 PM, Naceur  > wrote:
> 
> You are right if they had access to it at first place. Anyhow I
> fixed it.
> 
> 
> 
> --
> View this message in context:
> 
> http://gnuradio.4.n7.nabble.com/installing-a-python-package-without-elevated-privileges-tp63183p63258.html
> 
> 
> Sent from the GnuRadio mailing list archive at Nabble.com.
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org 
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> 
> 
> 
> 
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> 


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] installing a python package without elevated privileges

[U L]

This is a problem I have faced as well.  I have a number of projects at
various levels of modification of GR and various 3rd party python packages
being supported.  I find having virtualenvs to be very useful in keeping
python package versions, GR mods, and my own modules in agreement.  I've
been tinkering a bunch with getting venvs and pybombs to work together and
I think I've made some headway.  For example, the elevated privileges for
pip is (nominally) hardcoded in packagers/pip.py.  I made a modification to
condition elevation on the presence of a virtualenv, which is detected in
the config_manager.  This is one way to perhaps get pybombs and virtualenvs
to play nice.  I'll hopefully be offering a pull request soon that
incorporates a number of changes.

On Wed, Mar 22, 2017 at 12:30 PM, Naceur  wrote:

> You are right if they had access to it at first place. Anyhow I fixed it.
>
>
>
> --
> View this message in context: http://gnuradio.4.n7.nabble.
> com/installing-a-python-package-without-elevated-
> privileges-tp63183p63258.html
> Sent from the GnuRadio mailing list archive at Nabble.com.
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] installing a python package without elevated privileges

[Naceur]

You are right if they had access to it at first place. Anyhow I fixed it.



--
View this message in context: 
http://gnuradio.4.n7.nabble.com/installing-a-python-package-without-elevated-privileges-tp63183p63258.html
Sent from the GnuRadio mailing list archive at Nabble.com.

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] installing a python package without elevated privileges

[Marcus Müller]

Huh, that seems a bit *too* permissive – other users might thus have
modified freely a script that you run as your user!


On 21.03.2017 21:39, Naceur wrote:
> Thanks. I just circumvented it by getting rid of pip requirement and asking
> the recipe to get the deb/rpm requirements instead.
> FYI, my /home/nae/.local/bin/pip was already allowed 777.
>
> Regards,
> Naceur
>
>
>
> --
> View this message in context: 
> http://gnuradio.4.n7.nabble.com/installing-a-python-package-without-elevated-privileges-tp63183p63229.html
> Sent from the GnuRadio mailing list archive at Nabble.com.
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] installing a python package without elevated privileges

[Naceur]

Thanks. I just circumvented it by getting rid of pip requirement and asking
the recipe to get the deb/rpm requirements instead.
FYI, my /home/nae/.local/bin/pip was already allowed 777.

Regards,
Naceur



--
View this message in context: 
http://gnuradio.4.n7.nabble.com/installing-a-python-package-without-elevated-privileges-tp63183p63229.html
Sent from the GnuRadio mailing list archive at Nabble.com.

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] installing a python package without elevated privileges

[Marcus Müller]

Personal two cents: On most modern linuxes, you shouldn't be installing
mako via pip, and much less even numpy (because that's a huge compile
including fortran interfaces).

I just go and remove pip from the list of preferred packagers on my
systems¹. Never ran into problems with that. Now, I've never had the
need to install a python package that wasn't there as a distro package
in the last months on my recent systems, and on my retro systems I
typically not use pybombs, but your needs seem to be different. But
that's the problem here: PyBOMBS has to make sume assumptions on how you
want to install packages, and these include that pip needs to run as
root. Now, it seems that your ~/.local/bin/pip doesn't have the file
priorities set to allow others (including root, as it seems) to execute
it. You could check with "stat /home/nae/.local/bin/pip" what the
"Access" rights are. Should be something like 0755 / -rwxr-xr-x; my
guess is the last r-x is missing.

Best regards,

Marcus

¹ I'm even in personal disagreement with the folks maintaining Pybombs
on whether pip should be allowed to install things globally at all if a
package could potentially also be installed via native methods. But
generally, Pybombs is an awesome tool, and so is pip.

On 21.03.2017 19:45, Naceur wrote:
> Thank you for your reply. However, I don't want to have anything done
> manually. 
> Maybe I need rephrasing my initial post: 
>
> It is not that I want PyBOMBS to to do a pip --user but I am more wondering
> why it is saying:
>
>> PyBOMBS.PackageManager - DEBUG - Using packager pip 
>> PyBOMBS.Packager.pip - DEBUG - Calling `/home/nae/.local/bin/pip install
>> pytun' 
>> PyBOMBS.monitor_process() - DEBUG - Running with elevated privileges. 
>> PyBOMBS._process_thread() - DEBUG - Executing command `['sudo', '-H',
>> '/home/nae/.local/bin/pip', 'install', 'pytun']' 
>> sudo: unable to execute /home/nae/.local/bin/pip: Permission denied 
>> PyBOMBS.monitor_process() - DEBUG - Thread signaled termination or
>> returned 
>> PyBOMBS.monitor_process() - DEBUG - Return value: 1 
> why is it going straight to elevated priviliges.
> I looked for a package that needs installation through pip and found mako
> and numpy. 
> I realized that when uhd was installed it needed mako as a prerequesite and
> it did install it without the elevation and error.
>
> Thanks,
>
>
>
> --
> View this message in context: 
> http://gnuradio.4.n7.nabble.com/installing-a-python-package-without-elevated-privileges-tp63183p63226.html
> Sent from the GnuRadio mailing list archive at Nabble.com.
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] installing a python package without elevated privileges

[Naceur]

Thank you for your reply. However, I don't want to have anything done
manually. 
Maybe I need rephrasing my initial post: 

It is not that I want PyBOMBS to to do a pip --user but I am more wondering
why it is saying:

> PyBOMBS.PackageManager - DEBUG - Using packager pip 
> PyBOMBS.Packager.pip - DEBUG - Calling `/home/nae/.local/bin/pip install
> pytun' 
> PyBOMBS.monitor_process() - DEBUG - Running with elevated privileges. 
> PyBOMBS._process_thread() - DEBUG - Executing command `['sudo', '-H',
> '/home/nae/.local/bin/pip', 'install', 'pytun']' 
> sudo: unable to execute /home/nae/.local/bin/pip: Permission denied 
> PyBOMBS.monitor_process() - DEBUG - Thread signaled termination or
> returned 
> PyBOMBS.monitor_process() - DEBUG - Return value: 1 

why is it going straight to elevated priviliges.
I looked for a package that needs installation through pip and found mako
and numpy. 
I realized that when uhd was installed it needed mako as a prerequesite and
it did install it without the elevation and error.

Thanks,



--
View this message in context: 
http://gnuradio.4.n7.nabble.com/installing-a-python-package-without-elevated-privileges-tp63183p63226.html
Sent from the GnuRadio mailing list archive at Nabble.com.

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] installing a python package without elevated privileges

[Martin Braun]

The answer would be virtualenvs, but virtualenvs are not fully supported
in PyBOMBS. You might be able to to this by hand, but I've never tried it:

- Create a prefix using the --virtualenv switch
- Manually pip install the package into the virtualenv
- Maybe you need to manually set the package as installed

When the prefix is enabled, so is the virtualenv, so the package should
get picked up.

-- M

On 03/20/2017 09:02 AM, Naceur wrote:
> Hello GR,
> 
> I am creating a recipe for pytun as a dependency for an OOT module.
> Is there a way I can tell pybombs to install that while passing --user to
> pip because testing the install tells: pip is using elevated privileges 
> .
> .
> .
> PyBOMBS.PackageManager - DEBUG - Checking if package pytun is installed.
> PyBOMBS.install_manager - INFO - Installing package: pytun
> PyBOMBS.PackageManager - DEBUG - install(pytun, static=False)
> PyBOMBS.PackageManager - DEBUG - Using packager pip
> PyBOMBS.Packager.pip - DEBUG - Calling `/home/nae/.local/bin/pip install
> pytun'
> PyBOMBS.monitor_process() - DEBUG - Running with elevated privileges.
> PyBOMBS._process_thread() - DEBUG - Executing command `['sudo', '-H',
> '/home/nae/.local/bin/pip', 'install', 'pytun']'
> sudo: unable to execute /home/nae/.local/bin/pip: Permission denied
> PyBOMBS.monitor_process() - DEBUG - Thread signaled termination or returned
> PyBOMBS.monitor_process() - DEBUG - Return value: 1
> .
> .
> .
> 
> Thanks,
> 
> 
> 
> --
> View this message in context: 
> http://gnuradio.4.n7.nabble.com/installing-a-python-package-without-elevated-privileges-tp63183.html
> Sent from the GnuRadio mailing list archive at Nabble.com.
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> 


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio