Hi Tymofii,
On 08/07/14 21:36, Tymofii Nikolaienko wrote:
> Dear Kenneth ,
>
> here is an important update:
>
> I've tried different solutions (see an unfinished report below), but
then I remembered that Cygwin has been designed to be as much
Linux-compatible
> as possible;
> Therefore, I simply modified get_os_type() as
>
------------------------------------------------------------------------------------------------------------------------------------
> def get_os_type():
> """Determine system type, e.g., 'Linux', 'Darwin', 'Java'."""
> os_type = platform.system()
> if "CYGWIN" in os_type:
> os_type = 'Linux'
> if len(os_type) > 0:
> return os_type
> else:
> raise SystemToolsException("Failed to determine system name
using platform.system().")
That might make a lot of sense indeed... Nice find!
>
------------------------------------------------------------------------------------------------------------------------------------
>
> So, to summarize my findings here is an exhaustive protocol of
patches to get python test/framework/systemtools.py working
> (a have tested this sequence on clean distribution):
>
------------------------------------------------------------------------------------------------------------------------------------
> git clone https://github.com/hpcugent/easybuild-framework
>
> 1) insert at the beginning of
easybuild-framework/test/framework/systemtools.py
> import sys
> import os
> sys.path.append(os.path.abspath("."))
Just set $PYTHONPATH here (see my previous mail, and also below).
>
>
> 2) modify file easybuild-framework/easybuild/tools/systemtools.py as
>
> def get_os_type():
> """Determine system type, e.g., 'Linux', 'Darwin', 'Java'."""
> os_type = platform.system()
> if "CYGWIN" in os_type:
> os_type = 'Linux'
> if len(os_type) > 0:
> return os_type
> else:
> raise SystemToolsException("Failed to determine system name
using platform.system().")
Looks like a proper fix at first sight indeed, although we probably will
need to differentiate between an actual Linux system and Cygwin.
>
>
> 3) replace
> num_cores = int(sum(sched_getaffinity().cpus))
>
> in easybuild-framework/easybuild/tools/systemtools.py with
> import multiprocessing
> num_cores = int(multiprocessing.cpu_count())
OK, rather than replacing we should do something else here, since we
don't want to break what is working now.
You'll probably need to make some exception for Cygwin, something like
"if get_os_type() == CYGWIN: ...". To be discussed later.
>
------------------------------------------------------------------------------------------------------------------------------------
>
> The result is:
>
------------------------------------------------------------------------------------------------------------------------------------
> $ cat go
> #!/bin/bash
> export PYTHONHOME=/usr/
> export PYTHONPATH=/usr/lib/
>
> cd easybuild-framework
> python test/framework/systemtools.py
>
> $ ./go
> ...........
> ----------------------------------------------------------------------
> Ran 11 tests in 31.719s
>
> OK
Nice results!
>
------------------------------------------------------------------------------------------------------------------------------------
>
> Is it possible for you to make the aforementioned changes in the
development version of EasyBuild?
We could, but I don't think you're done. ;-)
Also, a better way of contributing back what you've found out is to
issue a pull request via GitHub.
You'll need to pick up some basic git skills to do that, and create an
account on GitHub.
A detailed guide is available at
https://github.com/hpcugent/easybuild/wiki/Contributing-back .
Those instructions are for the easybuild-easyconfigs repository, but you
can easily map them to easybuild-framework instead.
Maybe that should be your next step, trying to contribute the patches
you did to kickstart EasyBuild on Cygwin?
> What should I do next in order to get EasyBuild installed on my Cygwin ?
For now, you'll probably need to use a development version of EasyBuild,
since you will most likely run into more problems.
To get a full EasyBuild setup, also clone the easybuild-easyblocks and
easybuild-easyconfigs repositories, and simply extend your $PYTHONPATH
to include those paths too:
git clone https://github.com/hpcugent/easybuild-easyblocks
git clone https://github.com/hpcugent/easybuild-easyconfigs
export
PYTHONPATH=$PWD/easybuild-easyblocks:$PWD/easybuild-easyconfigs:$PYTHONPATH
Once you have that covered, you can run a couple of tests like:
* print list of available easyblocks (i.e. which software is
supported): eb --list-easyblocks
* print list of compiler toolchains EasyBuild knows about: eb
--list-toolchains
* perform a dry run of a goolf compiler toolchain: eb
goolf-1.5.14-no-OFED.eb --dry-run --robot
If that works out, I would suggest you continue with playing around with
the various unit tests modules.
See "ls easybuild-framework/test/framework": there are multiple unit
test modules available, one of them is systemtools which you already
handled.
Try running them, and see what kind of problems you run into. The
suite.py will simply run all available unit tests (all test modules, one
by one).
If you can get a setup that manages to pass (most of) the toy_build.py
unit tests, then you're close to doing a first actual software build
using EasyBuild on Cygwin (e.g. GCC, or something simpler like bzip2).
Keep us posted, and if you're up for it, please contribute back your
patches so your efforts are not in vain.
If you would be able to jump in at the #easybuild IRC channel, that
would certainly help...
regards,
Kenneth
