Hi Kenneth,
Here is the structure of my private easyblocks, easyconfigs and toolchains.
└── easybuild
├── build_my_toolchain.sh
├── easyblocks
│ ├── abaqus.py
│ ├── abaqus.pyc
│ ├── __init__.py
│ ├── mcnp5.py
│ ├── mcnp5.pyc
│ ├── mcnp6.py
│ ├── mcnp6.pyc
│ ├── mcnpdata.py
│ ├── mcnpdata.pyc
│ ├── mcnpx.py
│ ├── mcnpx.pyc
│ ├── petsc.py_keep
│ ├── starccm.py
│ ├── starccm.pyc
│ ├── vasp.py
│ └── vasp.pyc
├── easyconfigs
│ ├── ABAQUS-6.13.2-inl.eb
│ ├── apr-1.4.6-GCC-4.7.3-inl.eb
│ ├── apr-util-1.5.1-GCC-4.7.3-inl.eb
│ ├── ATLAS-3.10.1-gmvapich2-1.1.0-inl.eb
│ ├── ATLAS-3.10.1-gmvapich2-1.7.12-inl.eb
│ ├── BLACS-1.1-gmvapich2-1.1.0-inl.eb
│ ├── BLACS-1.1-gmvapich2-1.7.12-inl.eb
│ ├── configure.patch
│ ├── expire.F90.1.40.patch
│ ├── expire.F90.1.60.patch
│ ├── f_check_GCC.patch
│ ├── f_check.patch
│ ├── FFTW-2.1.5-gmvapich2-1.1.0-inl.eb
│ ├── FFTW-2.1.5-gmvapich2-1.7.12-inl.eb
│ ├── FFTW-3.3.3-gmvapich2-1.1.0-inl.eb
│ ├── FFTW-3.3.3-gmvapich2-1.1.0-single-inl.eb
│ ├── FFTW-3.3.3-gmvapich2-1.7.12-inl.eb
│ ├── FFTW-3.3.3-gmvapich2-1.7.12-single-inl.eb
│ ├── GCC-4.7.3-inl.eb
│ ├── git-1.8.3.4-GCC-4.7.3-inl.eb
│ ├── gmvapich2-1.1.0-inl.eb
│ ├── gmvolf-1.7.13-inl.eb
│ ├── gnupack-1.1.1-inl.eb
│ ├── gompi-1.1.0-inl.eb
│ ├── Hypre-2.8.0b-gmvolf-1.7.13-inl.eb
│ ├── Hypre-2.8.0b-gnupack-1.1.1-inl.eb
│ ├── icc-13.1.3-inl.eb
│ ├── icc-2013.5.192-inl.eb
│ ├── iccifort-2013.5.192-inl.eb
│ ├── ictce-5.5.0-inl.eb
│ ├── ifort-2013.5.192-inl.eb
│ ├── imkl-11.0.5.192-inl.eb
│ ├── impi-4.1.1.036-inl.eb
│ ├── intelpack-1.1.2-inl.eb
│ ├── intelpack-1.1.3-inl.eb
│ ├── LAPACK-3.4.2-gmvapich2-1.1.0-OpenBLAS-0.2.8-inl.eb
│ ├── LAPACK-3.4.2-gmvapich2-1.7.12-in.eb
│ ├── Linux.gcf.patch
│ ├── Makefile.in_2.7.0.patch
│ ├── Makefile.in.patch
│ ├── makefile.linux_ifc_P4_mpiifort.patch
│ ├── makefile.linux_ifc_P4_no_ngzhalf_mpiifort.patch
│ ├── makefile.linux_ifc_P4_no_ngzhalf.patch
│ ├── makefile.linux_ifc_P4.patch
│ ├── makefile.linux_ifc_P4_tbdyn_mpiifort.patch
│ ├── makefile.linux_ifc_P4_tbdyn.patch
│ ├── MCNP5-1.40-intelpack-1.1.2-inl.eb
│ ├── MCNP5-1.40-intelpack-1.1.2-serial-inl.eb
│ ├── MCNP5-1.40-intelpack-1.1.3-inl.eb
│ ├── MCNP5-1.40-intelpack-1.1.3-serial-inl.eb
│ ├── MCNP5-1.51-intelpack-1.1.2-inl.eb
│ ├── MCNP5-1.51-intelpack-1.1.2-serial-inl.eb
│ ├── MCNP5-1.51-intelpack-1.1.3-inl.eb
│ ├── MCNP5-1.51-intelpack-1.1.3-serial-inl.eb
│ ├── MCNP5-1.60-intelpack-1.1.2-inl.eb
│ ├── MCNP5-1.60-intelpack-1.1.2-serial-inl.eb
│ ├── MCNP5-1.60-intelpack-1.1.3-inl.eb
│ ├── MCNP5-1.60-intelpack-1.1.3-serial-inl.eb
│ ├── MCNP6-1.0-intelpack-1.1.2-inl.eb
│ ├── MCNP6-1.0-intelpack-1.1.3-inl.eb
│ ├── MCNP6-1.0-intelpack-1.1.3-serial-inl.eb
│ ├── MCNP_DATA-1.0-inl.eb
│ ├── MCNPX-2.6.0-intelpack-1.1.2-inl.eb
│ ├── MCNPX-2.6.0-intelpack-1.1.3-inl.eb
│ ├── MCNPX-2.7.0-intelpack-1.1.2-inl.eb
│ ├── MCNPX-2.7.0-intelpack-1.1.3-inl.eb
│ ├── MVAPICH2-1.7-GCC-4.7.3-inl.eb
│ ├── MVAPICH2-1.7-iccifort-2013.5.192-inl.eb
│ ├── MVAPICH2-1.9-GCC-4.7.3-inl.eb
│ ├── MVAPICH2-1.9-iccifort-2013.5.192-inl.eb
│ ├── nose-1.3.0-gnupack-1.1.1-Python-2.7.5-inl.eb
│ ├── numpy-1.7.1_distutils_multiple-lib-dirs.patch
│ ├── numpy-1.7.1-gnupack-1.1.1-Python-2.7.5-inl.eb
│ ├── numpy-iqacml.patch
│ ├── OpenBLAS-0.2.6-gmvapich2-1.1.0-inl.eb
│ ├── OpenBLAS-0.2.8-gmvapich2-1.1.0-inl.eb
│ ├── OpenBLAS-0.2.8-gmvapich2-1.7.12-LAPACK-3.4.2-inl.eb
│ ├── OpenMPI-1.6.5-GCC-4.7.3-inl.eb
│ ├── OpenMPI-1.6.5-iccifort-2013.5.192-inl.eb
│ ├── PETSc-3.3-p4-gmvolf-1.7.13-inl.eb
│ ├── PETSc-3.3-p4-gnupack-1.1.1-inl.eb
│ ├── Python-2.7.5-gmvolf-1.7.13-inl.eb
│ ├── Python-2.7.5-gnupack-1.1.1-inl.eb
│ ├── Python-2.7.5-gnupack-1.1.1-inl.eb_not_working
│ ├──
ScaLAPACK-2.0.2-gmvapich2-1.1.0-OPENBLAS-0.2.6-LAPACK-3.4.2-BLACS-1.1-inl.eb
│ ├──
ScaLAPACK-2.0.2-gmvapich2-1.1.0-OPENBLAS-0.2.8-LAPACK-3.4.2-inl.eb
│ ├──
ScaLAPACK-2.0.2-gmvapich2-1.7.12-OpenBLAS-0.2.8-LAPACK-3.4.2-inl.eb
│ ├── serf-1.2.1-GCC-4.7.3-inl.eb
│ ├── sqlite-3.7.17-GCC-4.7.3-inl.eb
│ ├── STARCCM+8.04.010-inl.eb
│ ├── sub_abaqus_6.13.2
│ ├── sub_mcnp5_1.40
│ ├── sub_mcnp5_1.40-serial
│ ├── sub_mcnp5_1.51
│ ├── sub_mcnp5_1.51-serial
│ ├── sub_mcnp5_1.60
│ ├── sub_mcnp5_1.60-serial
│ ├── sub_mcnp6_1.0
│ ├── sub_mcnp6_1.0-serial
│ ├── sub_mcnpx_2.6.0
│ ├── sub_mcnpx_2.7.0
│ ├── sub_starccm+_8.04.010
│ ├── sub_vasp_5.3.2
│ ├── Subversion-1.8.1-GCC-4.7.3-inl.eb
│ ├── tbb-4.1.4.192-inl.eb
│ ├── VASP-5.3.2-ictce-5.5.0-inl.eb
│ ├── VASP-5.3.2-ictce-5.5.0-no-ngzhalf-inl.eb
│ ├── VASP-5.3.2-ictce-5.5.0-tbdyn-inl.eb
│ ├── VASP-5.3.2-intelpack-1.1.3-inl.eb
│ └── zlib-1.2.8-GCC-4.7.3-inl.eb
├── __init__.py
└── toolchains
├── compiler
│ └── __init__.py
├── fft
│ └── __init__.py
├── gnupack.py
├── gnupack.pyc
├── __init__.py
├── intelpack.py
├── intelpack.pyc
├── linalg
│ ├── __init__.py
│ ├── myopenblas.py
│ └── myopenblas.pyc
├── mpi
│ └── __init__.py
├── tbb.py
└── tbb.pyc
When I remove __init__.py I can use my private easyblocks.
When I execute "eb --list-easyblocks" I do see my private easyblocks.
garvct@eos:~> eb --list-easyblocks
EasyBlock
|-- EB_ALADIN
|-- ConfigureMake
| |-- EB_ARB
| |-- EB_ATLAS
| |-- CMakeMake
| | |-- EB_Armadillo
| | |-- EB_BamTools
| | |-- EB_CGAL
| | |-- EB_Clang
| | |-- CMakePythonPackage
| | | |-- EB_DOLFIN
| | | |-- EB_UFC
| | |-- EB_Geant4
| | |-- EB_OpenBabel
| | |-- EB_Trilinos
| |-- EB_BLACS
| |-- EB_BWA
| |-- EB_bzip2
| |-- MakeCp
| | |-- EB_BamTools
| | |-- EB_BLAT
| |-- EB_Bowtie2
| |-- EB_R
| |-- EB_Bowtie
| |-- EB_CBLAS
| |-- EB_Chapel
| |-- EB_Cufflinks
| |-- EB_Python
| |-- EB_Doxygen
| |-- EB_DB
| |-- EB_DL_underscore_POLY_underscore_Classic
| |-- EB_ESPResSo
| |-- EB_ESMF
| |-- EB_flex
| |-- EB_Ferret
| |-- EB_freetype
| |-- EB_GHC
| |-- EB_g2clib
| |-- EB_GCC
| |-- EB_g2lib
| |-- EB_HPL
| |-- EB_HDF5
| |-- EB_Hypre
| |-- EB_libxml2
| |-- EB_libint2
| |-- EB_LAPACK
| |-- EB_Perl
| |-- EB_MUMmer
| |-- EB_MrBayes
| |-- EB_MUMPS
| |-- EB_METIS
| |-- EB_Mothur
| |-- EB_MetaVelvet
| |-- EB_MVAPICH2
| |-- EB_ncurses
| |-- EB_netCDF_minus_Fortran
| |-- EB_NEURON
| |-- EB_netCDF
| |-- EB_NWChem
| |-- EB_OpenSSL
| |-- EB_PSI
| |-- EB_PETSc
| |-- EB_Primer3
| |-- EB_Pasha
| |-- EB_Qt
| |-- EB_QuantumESPRESSO
| |-- EB_ROOT
| |-- EB_Score_minus_P
| |-- EB_SAMtools
| |-- EB_SLEPc
| |-- EB_SuiteSparse
| |-- EB_SWIG
| |-- EB_SHRiMP
| |-- EB_SOAPdenovo
| |-- EB_ScaLAPACK
| |-- EB_Scalasca1
| |-- EB_Velvet
| |-- EB_XCrySDen
| |-- EB_MCNP5
| |-- EB_MCNP6
| |-- EB_MCNPDATA
| |-- EB_MCNPX
| |-- EB_VASP
| |-- PerlModule
|-- EB_ant
|-- EB_ACML
|-- Binary
| |-- EB_Allinea
| |-- EB_CPLEX
| |-- EB_CUDA
| |-- EB_EPD
| |-- PackedBinary
| | |-- EB_Java
| | |-- EB_Tornado
| |-- EB_Mathematica
| |-- Rpm
| | |-- EB_QLogicMPI
| |-- EB_ABAQUS
| |-- EB_STARCCM
| |-- JAR
|-- EB_BiSearch
|-- ExtensionEasyBlock
| |-- RPackage
| | |-- EB_Bioconductor
| | |-- EB_Rserve
| | |-- EB_Rmpi
| | |-- EB_XML
| |-- PythonPackage
| | |-- CMakePythonPackage
| | | |-- EB_DOLFIN
| | | |-- EB_UFC
| | |-- EB_EasyBuildMeta
| | |-- EB_libxml2
| | |-- EB_nose
| | |-- FortranPythonPackage
| | | |-- EB_numpy
| | | |-- EB_scipy
| | |-- EB_python_minus_meep
| | |-- EB_PyZMQ
| | |-- EB_PyQuante
| | |-- EB_VSC_minus_tools
| |-- PerlModule
|-- EB_Boost
|-- EB_CP2K
|-- EB_Eigen
|-- EB_FSL
|-- Tarball
| |-- EB_FoldX
| |-- EB_FreeSurfer
| |-- EB_MTL4
| |-- BinariesTarball
|-- EB_FLUENT
|-- EB_FDTD_underscore_Solutions
|-- EB_GenomeAnalysisTK
|-- IntelBase
| |-- EB_icc
| | |-- EB_ifort
| |-- EB_ifort
| |-- EB_itac
| |-- EB_impi
| |-- EB_imkl
| |-- EB_ipp
| |-- EB_Inspector
| |-- EB_tbb
| |-- EB_VTune
|-- PackedBinary
| |-- EB_Java
| |-- EB_Tornado
|-- EB_libsmm
|-- EB_MATLAB
|-- EB_Maple
|-- EB_NCL
|-- EB_OpenFOAM
|-- EB_OpenIFS
|-- EB_picard
|-- EB_ParMETIS
|-- EB_Rosetta
|-- EB_SCOTCH
|-- EB_Trinity
|-- EB_TotalView
|-- EB_WRF
|-- EB_WIEN2k
|-- EB_WPS
|-- Toolchain
Extension
|-- ExtensionEasyBlock
| |-- RPackage
| | |-- EB_Bioconductor
| | |-- EB_Rserve
| | |-- EB_Rmpi
| | |-- EB_XML
| |-- PythonPackage
| | |-- CMakePythonPackage
| | | |-- EB_DOLFIN
| | | |-- EB_UFC
| | |-- EB_EasyBuildMeta
| | |-- EB_libxml2
| | |-- EB_nose
| | |-- FortranPythonPackage
| | | |-- EB_numpy
| | | |-- EB_scipy
| | |-- EB_python_minus_meep
| | |-- EB_PyZMQ
| | |-- EB_PyQuante
| | |-- EB_VSC_minus_tools
| |-- PerlModule
My PYTHONPATH looks like
echo $PYTHONPATH
/hpc-common/software/easybuild/1.9.0/gcc/opt/lib/python2.7/site-packages:/hpc-common/software/easybuild/inl-easyconfigs
If I include the path of the easyconfig files which come with EB, then EB
cannot find my private easyconfig files?
e.g
echo $PYTHONPATH
/hpc-common/software/easybuild/1.9.0/gcc/opt/lib/python2.7/site-packages:/hpc-common/software/easybuild/1.9.0/gcc/opt:/hpc-common/software/easybuild/inl-easyconfigs
eb MVAPICH2-1.7-GCC-4.7.3-inl.eb
== temporary log file in case of crash /tmp/easybuild-1iSTLA.log
== ERROR: Can't find path /home/garvct/MVAPICH2-1.7-GCC-4.7.3-inl.eb
Thanks,
Cormac.
On Mon, Dec 23, 2013 at 10:41 AM, Kenneth Hoste <[email protected]>wrote:
> Hi Cormac,
>
> On 19 Dec 2013, at 23:47, Garvey, Cormac T wrote:
>
> > Hi All,
> > On the Easybuild wiki page "Setting up own easyblock repository", it
> instructs to
> > put __init__.py in the easybuild directory (e.g touch
> $MYEBDIR/easybuild/__init__.py).
> >
> > When I do that I get the following easybuild error.
> >
> > Traceback (most recent call last):
> > File
> "/hpc-common/software/easybuild/1.9.0/gcc/opt/lib/python2.7/site-packages/easybuild/main.py",
> line 80, in <module>
> > from easybuild.tools.build_log import EasyBuildError, print_msg,
> print_error, print_warning
> > ImportError: No module named tools.build_log
> >
> > Removing __init__.py from the $MYEBDIR/easybuild directory works (i.e My
> easyblocks can co-exist with the default Easybuild
> > easyblocks.)
>
> When you say things work when you simply remove the __init__.py, is
> EasyBuils still able to find your 'private' easyblocks?
> I.e., does "eb --list-easyblocks" include your easyblocks as well (they
> really shouldn't, I would be really surprised if they are)?
>
> The __init__.py really needs to be there to declare the 'easybuild'
> namespace, such that 'from easybuild.easyblocks.foo import EB_Foo' can work
> (with Foo a private easyblock).
>
> There is one nasty detail though, and I guess you're being bitten by this.
> The path with your private easyblocks needs to be added *last* to
> $PYTHONPATH, rather than first.
> This is also described on the "Setting up your own easyblock repository"
> wiki page at
> https://github.com/hpcugent/easybuild/wiki/Setting-up-your-own-easyblocks-repository
> :
>
> "Note: You need to add the path to your easyblocks at the end, not at the
> beginning (see below as to why)."
>
> The reason why is also explained on that page, but it boils down to use
> flattening the easyblocks namespace from e.g. 'easybuild.easyblocks.g.gcc'
> to 'easybuid,easyblocks.gcc'.
> This trick only works *once* though, and only when your
> easybuild.easyblocks namespace is first in $PYTHONPATH.
>
> I hope this helps, do let us know how this works out.
>
>
> regards,
>
> Kenneth
--
Cormac Garvey
HPC Software Consultant
Scientific Computing
Idaho National Laboratory
Ph: 208-526-6294
