Re: [libvirt] [jenkins-ci PATCH] jobs: Set $PYTHONPATH for python-distutils jobs
On Wed, 2018-02-14 at 17:03 +, Daniel P. Berrangé wrote: > Might as well kill PYTHONPATH from the nodes after activating this, to > avoid accidents elswhere. Absolutely. In the slightly longer term, the plan is to move all environment definitions from the nodes to the jobs, so that we can track changes along with everything else. -- Andrea Bolognani / Red Hat / Virtualization -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [jenkins-ci PATCH] jobs: Set $PYTHONPATH for python-distutils jobs
On Wed, Feb 14, 2018 at 05:53:57PM +0100, Andrea Bolognani wrote: > Since we install Python modules under $VIRT_PREFIX, we need to set > $PYTHONPATH or the interpreter won't be able to locate them. This is > currently being done per-worker in the Jenkins Web interface. > > However, now that we've introduced Python 3 builds, depending on the > project and the OS, we might be using any of Python 2.7, 3.4, 3.5 > and 3.6, which means we can't have a single $PYTHONPATH anymore: in > particular, while it's okay to have non-esisting directories in > $PYTHONPATH, we have to make sure that a Python 3 interpreter will > never try to use a Python 2 module and vice versa. > > To solve the issue, we use a fairly large hammer: we set $PYTHONPATH > at the job level, and include all reasonable minor versions for the > Python major version (pyver) the job is using, even if they don't > yet exist. > > Signed-off-by: Andrea Bolognani> --- > Another approach would be to use something like > > T=$VIRT_PREFIX/lib/python{pyver}.4/site-packages > T=$T:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages > T=$T:$VIRT_PREFIX/lib/python{pyver}.5/site-packages > T=$T:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages > T=$T:$VIRT_PREFIX/lib/python{pyver}.6/site-packages > T=$T:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages > T=$T:$VIRT_PREFIX/lib/python{pyver}.7/site-packages > T=$T:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages > export PYTHONPATH=$T > > but that's just a different kind of ugly :/ > > jobs/python-distutils.yaml | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/jobs/python-distutils.yaml b/jobs/python-distutils.yaml > index ff68c29..510769e 100644 > --- a/jobs/python-distutils.yaml > +++ b/jobs/python-distutils.yaml > @@ -42,6 +42,7 @@ >- shell: | >{global_env} >{local_env} > + export > PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages >{command_pre_build} >python{pyver} ./setup.py build >python{pyver} ./setup.py install --prefix=$VIRT_PREFIX > @@ -83,6 +84,7 @@ >- shell: | >{global_env} >{local_env} > + export > PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages >python{pyver} ./setup.py test > publishers: >- email: > @@ -121,6 +123,7 @@ >- shell: | >{global_env} >{local_env} > + export > PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages >sed -i -e 's/BuildRequires: libvirt.*devel.*//' *.spec.in >python{pyver} ./setup.py rpm > publishers: Reviewed-by: Daniel P. Berrangé Might as well kill PYTHONPATH from the nodes after activating this, to avoid accidents elswhere. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [jenkins-ci PATCH] jobs: Set $PYTHONPATH for python-distutils jobs
Since we install Python modules under $VIRT_PREFIX, we need to set $PYTHONPATH or the interpreter won't be able to locate them. This is currently being done per-worker in the Jenkins Web interface. However, now that we've introduced Python 3 builds, depending on the project and the OS, we might be using any of Python 2.7, 3.4, 3.5 and 3.6, which means we can't have a single $PYTHONPATH anymore: in particular, while it's okay to have non-esisting directories in $PYTHONPATH, we have to make sure that a Python 3 interpreter will never try to use a Python 2 module and vice versa. To solve the issue, we use a fairly large hammer: we set $PYTHONPATH at the job level, and include all reasonable minor versions for the Python major version (pyver) the job is using, even if they don't yet exist. Signed-off-by: Andrea Bolognani--- Another approach would be to use something like T=$VIRT_PREFIX/lib/python{pyver}.4/site-packages T=$T:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages T=$T:$VIRT_PREFIX/lib/python{pyver}.5/site-packages T=$T:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages T=$T:$VIRT_PREFIX/lib/python{pyver}.6/site-packages T=$T:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages T=$T:$VIRT_PREFIX/lib/python{pyver}.7/site-packages T=$T:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages export PYTHONPATH=$T but that's just a different kind of ugly :/ jobs/python-distutils.yaml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/jobs/python-distutils.yaml b/jobs/python-distutils.yaml index ff68c29..510769e 100644 --- a/jobs/python-distutils.yaml +++ b/jobs/python-distutils.yaml @@ -42,6 +42,7 @@ - shell: | {global_env} {local_env} + export PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages {command_pre_build} python{pyver} ./setup.py build python{pyver} ./setup.py install --prefix=$VIRT_PREFIX @@ -83,6 +84,7 @@ - shell: | {global_env} {local_env} + export PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages python{pyver} ./setup.py test publishers: - email: @@ -121,6 +123,7 @@ - shell: | {global_env} {local_env} + export PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages sed -i -e 's/BuildRequires: libvirt.*devel.*//' *.spec.in python{pyver} ./setup.py rpm publishers: -- 2.14.3 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list