Hi,
just to elaborate because a few things are missing from Maxime's email.
1. We still need PYTHONPATH to find sitecustomize.py itself. We put
that in the main Python module but alternatively it could be put in
$EBROOTPYTHON/lib/pythonx.y/site-packages in which case you don't need
to set PYTHONPATH at all.
2. postfix = os.path.join('lib', 'python'+sys.version[:3], 'site-packages')
3. it's a little awkward at this point because it depends on a proper
pip2 being in $PATH when you load the Python 3 module. Automatic
installation also does not work yet combining 3.5 and 3.6 for example
(you need conflicting builddepedencies). At runtime it would not be an
issue.
Bart
On 17 November 2017 at 12:44, Maxime Boissonneault
<maxime.boissonnea...@calculquebec.ca> wrote:
> Hi all,
>
> I was at SC last week, and many people shared interests in our setup for
> python, in particular the fact that we manage to support multiple versions
> of Python for the same module. I thought I would share our solution here.
>
> 1) We do not define PYTHONPATH. PYTHONPATH is too intrusive, and it is not
> python-version aware. In your PYTHONPATH contains packages for python 2.7,
> it won't work with python 3.5 and vice versa.
>
> 2) Instead, we define EBPYTHONPREFIXES (our made up environment variable
> name), which points to the prefix directory.
>
> 3) We have a file called sitecustomize.py (that's standard python) which
> contains the following logic (and a bit more that is not relevant for this
> crowd since it is related to our nix layer):
>
> if "EBPYTHONPREFIXES" in os.environ:
> for prefix in os.environ["EBPYTHONPREFIXES"].split(os.pathsep):
> sitedir = os.path.join(prefix, postfix)
> if os.path.isdir(sitedir):
> site.addsitedir(sitedir)
>
>
> 4) In the modules, we install the python packages for all versions of python
> (currently only 2.7 and 3.5, but soon 3.6) under their typical
> "lib/pythonX.Y/site-packages" folders.
>
>
> For examples recipes that use this :
>
> https://github.com/ComputeCanada/easybuild-easyconfigs/blob/computecanada-master/easybuild/easyconfigs/i/igraph/igraph-0.7.1-iomkl-2016.4.11.eb
>
> https://github.com/ComputeCanada/easybuild-easyconfigs/blob/computecanada-master/easybuild/easyconfigs/o/OpenMM/OpenMM-7.1.1-iomkl-2016.4.11.eb
>
>
> Maxime
>
--
Dr. Bart E. Oldeman | bart.olde...@mcgill.ca | bart.olde...@calculquebec.ca
Scientific Computing Analyst / Analyste en calcul scientifique
McGill HPC Centre / Centre de Calcul Haute Performance de McGill |
http://www.hpc.mcgill.ca
Calcul Québec | http://www.calculquebec.ca
Compute/Calcul Canada | http://www.computecanada.ca
Tel/Tél: 514-396-8926 | Fax/Télécopieur: 514-396-8934
