date:20170510

Re: [easybuild] Mesa with SWR on non-AVX processor

2017-05-10 Thread Bart Oldeman

That phoronix article is quite old. In the end swrast will use llvmpipe if
you also use --enable-gallium-llvm (
http://www.paraview.org/Wiki/ParaView/ParaView_And_Mesa_3D)

Software rendering without either llvmpipe or swr is very slow indeed, but
as you can see on the above page software rendering is sometimes even
faster than a GPU.
Actually I found it also works better for me for remote X, instead of the
more traditional indirect rendering via a GPU on the connecting machine.

On 10 May 2017 at 13:09, Vanzo, Davide  wrote:

> Jack,
>
> Thank you for the feedback.
>
> In my path to enlightenment I stumbled across the following article, which
> seems to suggest that using swrast is not such a great idea:
>
> http://www.phoronix.com/scan.php?page=article=mesa_81_llvmpipe=1
>
> Any comment on that?
>
> --
> Davide Vanzo, PhD
> Application Developer
> Adjunct Assistant Professor of Chemical and Biomolecular Engineering
> Advanced Computing Center for Research and Education (ACCRE)
> Vanderbilt University - Hill Center 201
> (615)-875-9137 <(615)%20875-9137>
> www.accre.vanderbilt.edu
>
> On May 10 2017, at 12:01 pm, Jack Perdue  wrote:
>
>> AFAIK, we haven't had any problems with this change
>> (things seem to be working users aren't yelling):
>>
>> < --with-gallium-drivers='swrast,swr'\
>> ---
>>  > --with-gallium-drivers='swrast'\
>>
>> jack
>>
>>
>> On 05/10/2017 11:45 AM, Vanzo, Davide wrote:
>> > Hello fellow easybuilders,
>> >
>> > I am working on creating a minimal toolchain set of modules for X11
>> > and Qt for GUI-based software in the foss-2016b toolchain. When trying
>> > to install Mesa-12.0.2-GCC-5.4.0-2.26.eb (created by
>> > using Mesa-12.0.2-foss-2016b.eb) it fails when building the libswrAVX2
>> > since the LLVM library has been built on a Nehalem machine which does
>> > not have support for AVX (see log at the end of the email).
>> >
>> > Clearly one solution would be to avoid building SWR as part of Mesa
>> > but I have no idea of the implication of this choice since I am not
>> > very familiar with this type of libraries.
>> >
>> > Any suggestion?
>> >
>> > Thanks!
>> >
>> >
>> > PS: Here is the error log:
>> >
>> > CXX rasterizer/jitter/libswrAVX2_la-blend_jit.lo
>> > CXX rasterizer/jitter/libswrAVX2_la-builder.lo
>> > CXX rasterizer/jitter/libswrAVX2_la-builder_misc.lo
>> > CXX rasterizer/jitter/libswrAVX2_la-fetch_jit.lo
>> > CXX rasterizer/jitter/libswrAVX2_la-JitManager.lo
>> > CXX rasterizer/jitter/libswrAVX2_la-streamout_jit.lo
>> > CXX rasterizer/memory/libswrAVX2_la-ClearTile.lo
>> > CXX rasterizer/memory/libswrAVX2_la-LoadTile.lo
>> > CXX rasterizer/memory/libswrAVX2_la-StoreTile.lo
>> > rasterizer/core/threads.cpp: In function ‘void
>> > WorkOnFifoBE(SWR_CONTEXT*, uint32_t, uint64_t&, TileSet&, uint32_t,
>> > uint32_t)’:
>> > rasterizer/core/threads.cpp:467:26: warning: unused variable
>> > ‘numWorkItems’ [-Wunused-variable]
>> > uint32_t numWorkItems = tile.getNumQueued();
>> > ^
>> > CXX rasterizer/scripts/libswrAVX2_la-gen_knobs.lo
>> > CXX rasterizer/jitter/libswrAVX2_la-builder_x86.lo
>> > CXX rasterizer/jitter/libswrAVX2_la-builder_gen.lo
>> > CXX swr_loader.lo
>> > CXX rasterizer/common/libswrAVX_la-formats.lo
>> > CXX rasterizer/common/libswrAVX_la-rdtsc_buckets.lo
>> > rasterizer/jitter/builder_misc.cpp: In member function ‘llvm::Value*
>> > Builder::PMAXSD(llvm::Value*, llvm::Value*)’:
>> > rasterizer/jitter/builder_misc.cpp:862:77: error: ‘x86_sse41_pmaxsd’
>> > is not a member of ‘llvm::Intrinsic’
>> > Function* pmaxsd =
>> > Intrinsic::getDeclaration(JM()->mpCurrentModule,
>> > Intrinsic::x86_sse41_pmaxsd);
>> > ^
>> > rasterizer/jitter/builder_misc.cpp: In member function ‘llvm::Value*
>> > Builder::PMINSD(llvm::Value*, llvm::Value*)’:
>> > rasterizer/jitter/builder_misc.cpp:891:77: error: ‘x86_sse41_pminsd’
>> > is not a member of ‘llvm::Intrinsic’
>> > Function* pminsd =
>> > Intrinsic::getDeclaration(JM()->mpCurrentModule,
>> > Intrinsic::x86_sse41_pminsd);
>> > ^
>> > make[5]: *** [rasterizer/jitter/libswrAVX2_la-builder_misc.lo] Error 1
>> > make[5]: *** Waiting for unfinished jobs
>> > rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value*
>> > Builder::VPMINSD(llvm::Value*, llvm::Value*)’:
>> > rasterizer/jitter/builder_x86.cpp:85:71: error: ‘x86_avx2_pmins_d’ is
>> > not a member of ‘llvm::Intrinsic’
>> > Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule,
>> > Intrinsic::x86_avx2_pmins_d);
>> > ^
>> > rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value*
>> > Builder::VPMAXSD(llvm::Value*, llvm::Value*)’:
>> > rasterizer/jitter/builder_x86.cpp:92:71: error: ‘x86_avx2_pmaxs_d’ is
>> > not a member of ‘llvm::Intrinsic’
>> > Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule,
>> > Intrinsic::x86_avx2_pmaxs_d);
>> > ^
>> > rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value*
>> > Builder::VPMOVSXBD(llvm::Value*)’:
>> >

[easybuild] Python 3 fails to install setuptools

2017-05-10 Thread Vanzo, Davide

Hello again,

I am facing an issue when installing setuptools-23.1.0 as part of  
Python-3.5.2-foss-2016b.eb. It fails with a permission denied despite it has 
full write permissions on the installation path:

creating build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/dependency_links.txt -> 
build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/entry_points.txt -> 
build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/requires.txt -> 
build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/top_level.txt -> 
build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
creating dist
creating 'dist/setuptools-23.1.0-py3.5.egg' and adding 
'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Creating 
/gpfs22/easybuild/centos6/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/Python/3.5.2/lib/python3.5/site-packages/site.py
Processing setuptools-23.1.0-py3.5.egg
Copying setuptools-23.1.0-py3.5.egg to 
/gpfs22/easybuild/centos6/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/Python/3.5.2/lib/python3.5/site-packages
error: [Errno 13] Permission denied: 
'/gpfs22/easybuild/centos6/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/Python/3.5.2/lib/python3.5/site-packages/setuptools-23.1.0-py3.5.egg'
(at 
easybuild/software/Core/EasyBuild/3.1.2/lib/python2.7/site-packages/easybuild_framework-3.1.2-py2.7.egg/easybuild/tools/run.py:449
 in parse_cmd_output)
== 2017-05-10 11:58:07,532 easyblock.py:2518 WARNING build failed (first 300 
chars): cmd " 
/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/Python/3.5.2/bin/python
 setup.py install 
--prefix=/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/Python/3.5.2 
" exited with exitcode 1 and output:
running install
running bdist_egg
running egg_info
writing top-level names to
== 2017-05-10 11:58:07,532 easyblock.py:276 INFO Closing log for application 
name Python version 3.5.2


According to the setuptools documentation this error can also occur when a 
previous version of setuptools is installed and the egg ant pth files are not 
deleted before installing the new version. I checked by installing a bare 
version of the same Python and, although it has setuptools-20.10.1 installed, 
none of the conflicting files mentioned in the documentation are present:

$ find . -name sys.path
$ find . -name setuptools*
./lib/python3.5/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl
./lib/python3.5/site-packages/pip/utils/setuptools_build.py
./lib/python3.5/site-packages/pip/utils/__pycache__/setuptools_build.cpython-35.pyc
./lib/python3.5/site-packages/setuptools
./lib/python3.5/site-packages/setuptools-20.10.1.dist-info


Anybody has encoutered this issue before?

Thanks!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

Re: [easybuild] Mesa with SWR on non-AVX processor

2017-05-10 Thread Vanzo, Davide

Jack,

Thank you for the feedback.

In my path to enlightenment I stumbled across the following article, which 
seems to suggest that using swrast is not such a great idea:

http://www.phoronix.com/scan.php?page=article=mesa_81_llvmpipe=1

Any comment on that?

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On May 10 2017, at 12:01 pm, Jack Perdue  wrote:

AFAIK, we haven't had any problems with this change
(things seem to be working users aren't yelling):

< --with-gallium-drivers='swrast,swr'\
---
 > --with-gallium-drivers='swrast'\

jack

On 05/10/2017 11:45 AM, Vanzo, Davide wrote:
> Hello fellow easybuilders,
>
> I am working on creating a minimal toolchain set of modules for X11
> and Qt for GUI-based software in the foss-2016b toolchain. When trying
> to install Mesa-12.0.2-GCC-5.4.0-2.26.eb (created by
> using Mesa-12.0.2-foss-2016b.eb) it fails when building the libswrAVX2
> since the LLVM library has been built on a Nehalem machine which does
> not have support for AVX (see log at the end of the email).
>
> Clearly one solution would be to avoid building SWR as part of Mesa
> but I have no idea of the implication of this choice since I am not
> very familiar with this type of libraries.
>
> Any suggestion?
>
> Thanks!
>
>
> PS: Here is the error log:
>
> CXX rasterizer/jitter/libswrAVX2_la-blend_jit.lo
> CXX rasterizer/jitter/libswrAVX2_la-builder.lo
> CXX rasterizer/jitter/libswrAVX2_la-builder_misc.lo
> CXX rasterizer/jitter/libswrAVX2_la-fetch_jit.lo
> CXX rasterizer/jitter/libswrAVX2_la-JitManager.lo
> CXX rasterizer/jitter/libswrAVX2_la-streamout_jit.lo
> CXX rasterizer/memory/libswrAVX2_la-ClearTile.lo
> CXX rasterizer/memory/libswrAVX2_la-LoadTile.lo
> CXX rasterizer/memory/libswrAVX2_la-StoreTile.lo
> rasterizer/core/threads.cpp: In function ‘void
> WorkOnFifoBE(SWR_CONTEXT*, uint32_t, uint64_t&, TileSet&, uint32_t,
> uint32_t)’:
> rasterizer/core/threads.cpp:467:26: warning: unused variable
> ‘numWorkItems’ [-Wunused-variable]
> uint32_t numWorkItems = tile.getNumQueued();
> ^
> CXX rasterizer/scripts/libswrAVX2_la-gen_knobs.lo
> CXX rasterizer/jitter/libswrAVX2_la-builder_x86.lo
> CXX rasterizer/jitter/libswrAVX2_la-builder_gen.lo
> CXX swr_loader.lo
> CXX rasterizer/common/libswrAVX_la-formats.lo
> CXX rasterizer/common/libswrAVX_la-rdtsc_buckets.lo
> rasterizer/jitter/builder_misc.cpp: In member function ‘llvm::Value*
> Builder::PMAXSD(llvm::Value*, llvm::Value*)’:
> rasterizer/jitter/builder_misc.cpp:862:77: error: ‘x86_sse41_pmaxsd’
> is not a member of ‘llvm::Intrinsic’
> Function* pmaxsd =
> Intrinsic::getDeclaration(JM()->mpCurrentModule,
> Intrinsic::x86_sse41_pmaxsd);
> ^
> rasterizer/jitter/builder_misc.cpp: In member function ‘llvm::Value*
> Builder::PMINSD(llvm::Value*, llvm::Value*)’:
> rasterizer/jitter/builder_misc.cpp:891:77: error: ‘x86_sse41_pminsd’
> is not a member of ‘llvm::Intrinsic’
> Function* pminsd =
> Intrinsic::getDeclaration(JM()->mpCurrentModule,
> Intrinsic::x86_sse41_pminsd);
> ^
> make[5]: *** [rasterizer/jitter/libswrAVX2_la-builder_misc.lo] Error 1
> make[5]: *** Waiting for unfinished jobs
> rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value*
> Builder::VPMINSD(llvm::Value*, llvm::Value*)’:
> rasterizer/jitter/builder_x86.cpp:85:71: error: ‘x86_avx2_pmins_d’ is
> not a member of ‘llvm::Intrinsic’
> Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule,
> Intrinsic::x86_avx2_pmins_d);
> ^
> rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value*
> Builder::VPMAXSD(llvm::Value*, llvm::Value*)’:
> rasterizer/jitter/builder_x86.cpp:92:71: error: ‘x86_avx2_pmaxs_d’ is
> not a member of ‘llvm::Intrinsic’
> Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule,
> Intrinsic::x86_avx2_pmaxs_d);
> ^
> rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value*
> Builder::VPMOVSXBD(llvm::Value*)’:
> rasterizer/jitter/builder_x86.cpp:148:71: error: ‘x86_avx2_pmovsxbd’
> is not a member of ‘llvm::Intrinsic’
> Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule,
> Intrinsic::x86_avx2_pmovsxbd);
> ^
> rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value*
> Builder::VPMOVSXWD(llvm::Value*)’:
> rasterizer/jitter/builder_x86.cpp:155:71: error: ‘x86_avx2_pmovsxwd’
> is not a member of ‘llvm::Intrinsic’
> Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule,
> Intrinsic::x86_avx2_pmovsxwd);
> ^
> make[5]: *** [rasterizer/jitter/libswrAVX2_la-builder_x86.lo] Error 1
> make[5]: Leaving directory
> `/tmp/Mesa/12.0.2/GCC-5.4.0-2.26/mesa-12.0.2/src/gallium/drivers/swr'
> make[4]: *** [all] Error 2
> make[4]: Leaving directory
> `/tmp/Mesa/12.0.2/GCC-5.4.0-2.26/mesa-12.0.2/src/gallium/drivers/swr'
> make[3]: *** [all-recursive] Error 1
> make[3]:

Re: [easybuild] Mesa with SWR on non-AVX processor

2017-05-10 Thread Jack Perdue



AFAIK, we haven't had any problems with this change
(things seem to be working users aren't yelling):

< --with-gallium-drivers='swrast,swr'\
---
> --with-gallium-drivers='swrast'\

jack


On 05/10/2017 11:45 AM, Vanzo, Davide wrote:

Hello fellow easybuilders,

I am working on creating a minimal toolchain set of modules for X11 
and Qt for GUI-based software in the foss-2016b toolchain. When trying 
to install Mesa-12.0.2-GCC-5.4.0-2.26.eb (created by 
using Mesa-12.0.2-foss-2016b.eb) it fails when building the libswrAVX2 
since the LLVM library has been built on a Nehalem machine which does 
not have support for AVX (see log at the end of the email).


Clearly one solution would be to avoid building SWR as part of Mesa 
but I have no idea of the implication of this choice since I am not 
very familiar with this type of libraries.


Any suggestion?

Thanks!


PS: Here is the error log:

 CXX  rasterizer/jitter/libswrAVX2_la-blend_jit.lo
 CXX  rasterizer/jitter/libswrAVX2_la-builder.lo
 CXX  rasterizer/jitter/libswrAVX2_la-builder_misc.lo
 CXX  rasterizer/jitter/libswrAVX2_la-fetch_jit.lo
 CXX  rasterizer/jitter/libswrAVX2_la-JitManager.lo
 CXX  rasterizer/jitter/libswrAVX2_la-streamout_jit.lo
 CXX  rasterizer/memory/libswrAVX2_la-ClearTile.lo
 CXX  rasterizer/memory/libswrAVX2_la-LoadTile.lo
 CXX  rasterizer/memory/libswrAVX2_la-StoreTile.lo
rasterizer/core/threads.cpp: In function ‘void 
WorkOnFifoBE(SWR_CONTEXT*, uint32_t, uint64_t&, TileSet&, uint32_t, 
uint32_t)’:
rasterizer/core/threads.cpp:467:26: warning: unused variable 
‘numWorkItems’ [-Wunused-variable]

uint32_t numWorkItems = tile.getNumQueued();
 ^
 CXX  rasterizer/scripts/libswrAVX2_la-gen_knobs.lo
 CXX  rasterizer/jitter/libswrAVX2_la-builder_x86.lo
 CXX  rasterizer/jitter/libswrAVX2_la-builder_gen.lo
 CXX  swr_loader.lo
 CXX  rasterizer/common/libswrAVX_la-formats.lo
 CXX  rasterizer/common/libswrAVX_la-rdtsc_buckets.lo
rasterizer/jitter/builder_misc.cpp: In member function ‘llvm::Value* 
Builder::PMAXSD(llvm::Value*, llvm::Value*)’:
rasterizer/jitter/builder_misc.cpp:862:77: error: ‘x86_sse41_pmaxsd’ 
is not a member of ‘llvm::Intrinsic’
Function* pmaxsd = 
Intrinsic::getDeclaration(JM()->mpCurrentModule, 
Intrinsic::x86_sse41_pmaxsd);

  ^
rasterizer/jitter/builder_misc.cpp: In member function ‘llvm::Value* 
Builder::PMINSD(llvm::Value*, llvm::Value*)’:
rasterizer/jitter/builder_misc.cpp:891:77: error: ‘x86_sse41_pminsd’ 
is not a member of ‘llvm::Intrinsic’
Function* pminsd = 
Intrinsic::getDeclaration(JM()->mpCurrentModule, 
Intrinsic::x86_sse41_pminsd);

  ^
make[5]: *** [rasterizer/jitter/libswrAVX2_la-builder_misc.lo] Error 1
make[5]: *** Waiting for unfinished jobs
rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value* 
Builder::VPMINSD(llvm::Value*, llvm::Value*)’:
rasterizer/jitter/builder_x86.cpp:85:71: error: ‘x86_avx2_pmins_d’ is 
not a member of ‘llvm::Intrinsic’
Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule, 
Intrinsic::x86_avx2_pmins_d);

^
rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value* 
Builder::VPMAXSD(llvm::Value*, llvm::Value*)’:
rasterizer/jitter/builder_x86.cpp:92:71: error: ‘x86_avx2_pmaxs_d’ is 
not a member of ‘llvm::Intrinsic’
Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule, 
Intrinsic::x86_avx2_pmaxs_d);

^
rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value* 
Builder::VPMOVSXBD(llvm::Value*)’:
rasterizer/jitter/builder_x86.cpp:148:71: error: ‘x86_avx2_pmovsxbd’ 
is not a member of ‘llvm::Intrinsic’
Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule, 
Intrinsic::x86_avx2_pmovsxbd);

^
rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value* 
Builder::VPMOVSXWD(llvm::Value*)’:
rasterizer/jitter/builder_x86.cpp:155:71: error: ‘x86_avx2_pmovsxwd’ 
is not a member of ‘llvm::Intrinsic’
Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule, 
Intrinsic::x86_avx2_pmovsxwd);

^
make[5]: *** [rasterizer/jitter/libswrAVX2_la-builder_x86.lo] Error 1
make[5]: Leaving directory 
`/tmp/Mesa/12.0.2/GCC-5.4.0-2.26/mesa-12.0.2/src/gallium/drivers/swr'

make[4]: *** [all] Error 2
make[4]: Leaving directory 
`/tmp/Mesa/12.0.2/GCC-5.4.0-2.26/mesa-12.0.2/src/gallium/drivers/swr'

make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory 
`/tmp/Mesa/12.0.2/GCC-5.4.0-2.26/mesa-12.0.2/src/gallium'

make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory 
`/tmp/Mesa/12.0.2/GCC-5.4.0-2.26/mesa-12.0.2/src'

make[1]: *** [all] Error 2
make[1]: Leaving directory 
`/tmp/Mesa/12.0.2/GCC-5.4.0-2.26/mesa-12.0.2/src'

make: *** [all-recursive] Error 1


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and

Re: [easybuild] eb in singularity

2017-05-10 Thread Kenneth Hoste




On 08/05/2017 20:21, Siddiqui, Shahzeb wrote:


Here is conversation between Greg and I, I have not got any response 
from Kenneth but would like to know if EB will build singularity 
containers or will we need to create singularity definition files and 
run eb inside the definition file.




I have no clear answer here, but I think the 2nd option is a lot easier...

Making EasyBuild spit out Singularity packages may make the integration 
too tight.


If each application is a container, we need a orchestration tool to 
resolve dependencies. A per app container model is difficult and will 
take some time, whereas a software collection model in a container is 
straight forward. Since I build my packages as RPMs, I can install the 
packages as RPMs and also build eb packages as RPMs. This way if I 
need to recreate the container or create custom container environment 
I can do this via yum groups.




I don't think packaging each application/tool in a separate container is 
a good idea, it'll make things more difficult than it has to be.


Of course, sharing dependencies is no longer possible when going with 
'fat' containers for specific purposes.


eb needs a similar feature like  FPM to generate RPM but now write out 
a singularity definition file that is then used to bootstrap the image 
that can then install the eb package as RPM or do a eb build inside 
container.




How would this be better than just using EasyBuild as the installation 
tool in a definition file?


Sure, some other things need to be taken care of (e.g. avoiding the need 
to use modules in the container to make the software available)...



regards,

Kenneth


On Tue, Apr 11, 2017 at 1:04 PM, Siddiqui, Shahzeb 
> wrote:


Hello Gregory,

It was easy for me to create a single singularity container that has 
all my collection of software packages, but it was difficult for me to 
figure out how to create individual containers that can interact together.


Ahh, and there-in lies orchestration of containers.

Initially I would try to setup Lmod in container, I was able to figure 
this out  and then I was able to build EB packages. My concern is a 
single collection scheme seems to be limited in the sense that 
software packages are not truly independent.


I would expect that I  can install Singularity containers like rpm 
which have all the dependencies that pick up other singularity 
containers. Initially I can start out by installing dependency for GCC 
as separate containers but the one thing that caused issues was 
setting up Lmod on each container to allow EB to build properly. In a 
1-1 app to container scenario how will I implement the module behavior 
to pick up paths installed in different containers.


Again, as soon as you start talking about multiple containers 
interacting with each other, and dependencies, we end up with a layer 
of orchestration.


This is both a good and bad thing. Thinking in terms of 
reproducibility in it's simplest form, we have a single Singularity 
image that has within it all of the "bits" necessary for a given 
software workflow. The moment we start requiring other containers, we 
do not have a single point of reproducibility anymore. Not that this 
is a bad thing, but it compounds the problem of complexity exponentially.


Assuming EB installs in some path /usr/local/easybuild and there is a 
module file for GCC. Users can execute commands against the container: 
singularity exec GCC-5.4.0-2.27.img gcc –V  provided that GCC module 
is loaded.  I don’t think this would still work since dependent 
modules need to exist in same container. Single container solution 
would be simple but I only see this useful for managing a group of 
software in a container. There would be no way to interact between two 
containers.


Perhaps a standardized method to interacting with an orchestration 
system like Kubernetes (not HPC ready, but Slurm doesn't have the 
necessary bits to deal with this easily).


Ken, is there anyone from EB working on this, I would like to get 
started on this approach. Looking forward to hearing back from both of 
you.


Thoughts?

Regards,

*From:*Gregory M. Kurtzer [mailto:gmkurt...@gmail.com 
]

*Sent:* Tuesday, April 11, 2017 1:25 PM
*To:* Siddiqui, Shahzeb
*Cc:* Kenneth Hoste
*Subject:* Re: Singularity + EasyBuild

Hello Shahzeb,

I think this is a very interesting idea, but I will have to defer to 
Kenneth to comment as it pertains to EB.


My initial feeling is that this should be possible, and I would love 
to see EB generate the build commands for the package and the 
dependencies such that it can be transposed into a Singularity build 
definition file. That way we can bring the deffile and build it 
anywhere (e.g. Singularity-hub) and create packages from that.


Thoughts?

Greg

Thoughts?

On Mon, Apr 10, 2017 at 8:00 AM, Siddiqui, Shahzeb

Re: [easybuild] eb in singularity

2017-05-10 Thread Kenneth Hoste

Hello Di,

On 08/05/2017 19:43, Di Pe wrote:

Interesting discussion about Singularity, there was an older thread here

https://mail.google.com/mail/u/0/#search/singularity/159045c3913ad572

and it would be great to learn what EB leadership thinks about
integration of containers in EB?

To look a little bit into our requirements we would like to make it
easy to share the software stacks we build using EB with

1) other researchers outside our organization who are also running
things on HPC systems
2) other software platforms in the same organization (outside HPC
clusters)

In both cases we want to support a certain version of a containerized
software stack for a certain period of time. Take for example a data
science stack. We would bundle a certain version of R with a certain
version of Python and some other ancillary tools. We would build a
container that has one version of Python2 and one version of R and
another container that has Python3 with R.
I would not need or even want any Lmod environment inside the
container because we may need to tie R and python together. For
example R argparse is linked to a specific version of Python and
Python rpy2 is linked to a specific version or R. This cannot be
handled well with lmod.

As Jens already asked, can you clarify the problem here with modules?

Am I right that this is a chicken-egg situation (Python depends on R, R
depends on Python)?

You could probably break this by installing rpy2 as a separate module?

Another issue that plays into this is size of the container. We want
to keep the container relatively small to facilitate sharing but the
easybuild environment is actually pretty large with sources and
everything. if there was a good way to remove all the unnecessary
things including compiler after the container is created that would be
great.

Sources is not a problem, those are kept in a different location
(configurable via --sourcepath).

Cleaning up build modules that only serve as build dependencies for
other things shouldn't be too hard either.

Getting rid of the compilers as a whole is a different matter though,
there are runtime dependencies on libraries provided by the compiler...

Yet another issue is that non-HPC people are not used to lmod. So if I
have hadoop or devops folks they need to know a little bit of lmod to
be able to troubleshoot the software stack and because that is not the
case (at least here) they are building stuff by hand or are using
crappy Ubuntu debs even through the HPC person one floor down has
already done the (better) job using EB. In that sense it would be
great if we could use EB to create a container stack that support only
one version of each software and wherethe software inside the
container simply installs to /usr/local to they can use it as a base
and then fuzz with it further if they need to.

I you want to have all the software that is included in the container
available out-of-the-box, you could avoid using modules.

EasyBuild currently doesn't have support for this, but you could just
capture the changes that should be made to the environment and use the
commands that make those changes a part of the container startup script
(or whatever sets up the environment).

Modules are mostly useful to let users juggle with different
versions/builds of software, and make them pick which tools they'd like
to use together.

Another interesting discussion was Singularity vs Docker. After
reading this
https://www.nextplatform.com/2017/04/10/singularity-containers-hpc-reproducibility-mobility/
and trying out singularity I think the answer is that we want to use
both. Docker plain will probably never be usable on HPC systems (See
the story about rejected pull requests) so you have to take a detour
and use Shifter (Shifter needs to be integrated by a HPC sysadmin
requiring root access and a test plan. It is optimized towards and
tested with Slurm.) Singularity works out of the box on HPC and
standalone systems. The benefit of Shifter seems to be that it can use
Docker images natively while singularity requires you to either build
singularity images or import docker images. In my ad-hoc testing I was
having much more problems with building singularity images (e.g FS
corruption ) than importing docker images with singularity. The latter
worked really well and fast.

As far as I know, Shifter is specific to Cray systems. And it also needs
quite a bit of infrastructure to work well, as opposed to Singularity
that doesn't really need anything special.

So perhaps it makes sense to focus on integrating Docker builds into
EB as one could deliver a more comprehensive solution to bigdata,
devops and HPC people.

Interested to hear some thoughts

We're certainly interested to look into integration between EasyBuild
and Singularity, but just don't have the time to dive into it...

A couple of use cases I had in mind:

* use EasyBuild to populate

Re: [easybuild] Mesa with SWR on non-AVX processor

[easybuild] Python 3 fails to install setuptools

Re: [easybuild] Mesa with SWR on non-AVX processor

Re: [easybuild] Mesa with SWR on non-AVX processor

Re: [easybuild] eb in singularity

Re: [easybuild] eb in singularity

6 matches

Site Navigation

Mail list logo

Footer information