Dear Kenneth,
In fact, my motivation for installing EqsyBuild on Cygwin is very
"prosaic": I'd like to have such quantum-chemical
software as NwChem and PSI4 compiled for Windows.
I've succeeded in doing so with GROMACS and some related tools and with
NwChem 6.1 with Gygwin with some
tricks/workaround. However, many efforts with compiling NwChem 6.3 and
PSI4 beta 5 in a variety of ways failed.
Lately, I noted that on a certain Linux machine it was also VERY
problematic to compile above mentioned versions of
NwChem and PSI4. But when I installed GCC 4.7.2 from EasyBuild's
packages everything gone fine! This story makes
me suspect that if I use some EasyBuild's package of GCC under Cygwin I
may also be succeeded.
I think that it won't be a problem to try to play around with a
development setup of EasyBuild.
Unfortunately, I'm not very familiar with git/GitHub and Python
(regardless of the fact that recently one
of my friends has strongly advised me to get familiar with Python), but
I have some experience with Cygwin.
Here is the log of cloning files from gitHub under Cygwin:
------------------------------------------------
*$ git clone [email protected]:hpcugent/easybuild-framework.git
*Cloning into 'easybuild-framework'...
The authenticity of host 'github.com (192.30.252.128)' can't be established.
RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,192.30.252.128' (RSA) to the list
of known hosts.
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
*$ git clone [email protected]:hpcugent/easybuild-framework.git
*Cloning into 'easybuild-framework'...
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
*$ git clone https://github.com/hpcugent/easybuild-framework
*Cloning into 'easybuild-framework'...
remote: Reusing existing pack: 32648, done.
remote: Counting objects: 1032, done.
remote: Compressing objects: 100% (588/588), done.
remote: Total 33680 (delta 701), reused 649 (delta 430)
Receiving objects: 100% (33680/33680), 6.20 MiB | 465 KiB/s, done.
Resolving deltas: 100% (23696/23696), done.
----------------------------------------
Here is the next step:
---------------------------------------
*$ ls
*easybuild-framework go
*$ cat go
*#!/bin/bash
export PYTHONHOME=/usr/
export PYTHONPATH=/usr/lib/
cd easybuild-framework
python test/framework/systemtools.py
*$ ./go
*Traceback (most recent call last):
File "test/framework/systemtools.py", line 30, in <module>
from test.framework.utilities import EnhancedTestCase
ImportError: No module named test.framework.utilities
----------------------------------------
Then I followed this [
http://stackoverflow.com/questions/17976158/python-unable-to-import-module
] advice and modified easybuild-framework\test\framework\systemtools.py
by including
import sys
print "AAAAA"
sys.path.append("/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework")
After that I got
-------------------------------------
*$ ./go
*/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework
Python 2.7.3
AAAAA
Traceback (most recent call last):
File "test/framework/systemtools.py", line 35, in <module>
from test.framework.utilities import EnhancedTestCase
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/test/framework/utilities.py",
line 39, in <module>
import easybuild.tools.options as eboptions
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/options.py",
line 41, in <module>
from easybuild.framework.easyblock import EasyBlock
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/framework/easyblock.py",
line 55, in <module>
from easybuild.framework.easyconfig.default import
get_easyconfig_parameter_default
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/framework/easyconfig/__init__.py",
line 35, in <module>
from easybuild.framework.easyconfig.easyconfig import EasyConfig
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/framework/easyconfig/easyconfig.py",
line 58, in <module>
from easybuild.framework.easyconfig.format.convert import Dependency
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/framework/easyconfig/format/__init__.py",
line 33, in <module>
from easybuild.framework.easyconfig.format.one import FormatOneZero
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/framework/easyconfig/format/one.py",
line 40, in <module>
from easybuild.framework.easyconfig.format.pyheaderconfigobj import
EasyConfigFormatConfigObj
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/framework/easyconfig/format/pyheaderconfigobj.py",
line 38, in <module>
from easybuild.framework.easyconfig.templates import TEMPLATE_CONSTANTS
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/framework/easyconfig/templates.py",
line 113, in <module>
('SHLIB_EXT', get_shared_lib_ext(), 'extension for shared libraries'),
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/systemtools.py",
line 296, in get_shared_lib_ext
"unknown system name: %s" % os_type)
*easybuild.tools.systemtools.SystemToolsException: Unable to determine
extention for shared libraries,unknown system name: CYGWIN_NT-5.1*
-------------------------------------
I tried resolving this by modifying
easybuild-framework\easybuild\tools\systemtools.py (backslash by Windows
tradition ;) ) by adding
-------------------------
# constants
AMD = 'AMD'
ARM = 'ARM'
INTEL = 'Intel'
LINUX = 'Linux'
DARWIN = 'Darwin'
CYG51NT = 'CYGWIN_NT-5.1' # <---------------
-------------------------
and
-------------------------
def get_shared_lib_ext():
"""Determine extention for shared libraries
Linux: 'so', Darwin: 'dylib'
"""
shared_lib_exts = {
LINUX: 'so',
DARWIN: 'dylib',
CYG51NT: 'dll'
}
-------------------------
After this I got
-------------------------
*$ ./go
*/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework
Python 2.7.3
AAAAA
E.EE..F.EFE
======================================================================
ERROR: test_core_count (__main__.SystemToolsTest)
Test getting core count.
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/framework/systemtools.py", line 50, in test_core_count
for core_count in [get_avail_core_count(), get_core_count()]:
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/systemtools.py",
line 119, in get_avail_core_count
out, _ = run_cmd('sysctl -n hw.ncpu')
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/run.py",
line 80, in inner
return func(cmd, *args, **kwargs)
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/run.py",
line 153, in run_cmd
return parse_cmd_output(cmd, stdouterr, ec, simple, log_all,
log_ok, regexp)
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/run.py",
line 382, in parse_cmd_output
_log.error('cmd "%s" exited with exitcode %s and output:\n%s' %
(cmd, ec, stdouterr))
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/build_log.py",
line 103, in error
raise EasyBuildError(newMsg)
EasyBuildError: 'EasyBuild crashed with an error (at
easybuild/tools/run.py:382 in parse_cmd_output): cmd " sysctl -n
hw.ncpu" exited with exitcode 127 and output:\n/bin/bash: sysctl:
\xd0\xba\xd0\xbe\xd0\xbc\xd0\xb0\xd0\xbd\xd0\xb4\xd0\xb0
\xd0\xbd\xd0\xb5 \xd0\xbd\xd0\xb0\xd0\xb9\xd0\xb4\xd0\xb5\xd0\xbd\xd0\xb0\n'
======================================================================
ERROR: test_cpu_speed (__main__.SystemToolsTest)
Test getting CPU speed.
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/framework/systemtools.py", line 61, in test_cpu_speed
cpu_speed = get_cpu_speed()
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/systemtools.py",
line 257, in get_cpu_speed
raise SystemToolsException("Could not determine CPU clock frequency
(OS: %s)." % os_type)
SystemToolsException: Could not determine CPU clock frequency (OS:
CYGWIN_NT-5.1).
======================================================================
ERROR: test_cpu_vendor (__main__.SystemToolsTest)
Test getting CPU vendor.
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/framework/systemtools.py", line 67, in test_cpu_vendor
cpu_vendor = get_cpu_vendor()
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/systemtools.py",
line 179, in get_cpu_vendor
out, exitcode = run_cmd("sysctl -n hw.model")
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/run.py",
line 80, in inner
return func(cmd, *args, **kwargs)
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/run.py",
line 153, in run_cmd
return parse_cmd_output(cmd, stdouterr, ec, simple, log_all,
log_ok, regexp)
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/run.py",
line 382, in parse_cmd_output
_log.error('cmd "%s" exited with exitcode %s and output:\n%s' %
(cmd, ec, stdouterr))
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/build_log.py",
line 103, in error
raise EasyBuildError(newMsg)
EasyBuildError: 'EasyBuild crashed with an error (at
easybuild/tools/run.py:382 in parse_cmd_output): cmd " sysctl -n
hw.model" exited with exitcode 127 and output:\n/bin/bash: sysctl:
\xd0\xba\xd0\xbe\xd0\xbc\xd0\xb0\xd0\xbd\xd0\xb4\xd0\xb0
\xd0\xbd\xd0\xb5 \xd0\xbd\xd0\xb0\xd0\xb9\xd0\xb4\xd0\xb5\xd0\xbd\xd0\xb0\n'
======================================================================
ERROR: test_platform_name (__main__.SystemToolsTest)
Test getting platform name.
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/framework/systemtools.py", line 82, in test_platform_name
platform_name_nover = get_platform_name()
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/systemtools.py",
line 315, in get_platform_name
raise SystemToolsException("Failed to determine platform name,
unknown system name: %s" % os_type)
SystemToolsException: Failed to determine platform name, unknown system
name: CYGWIN_NT-5.1
======================================================================
ERROR: test_system_info (__main__.SystemToolsTest)
Test getting system info.
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/framework/systemtools.py", line 110, in test_system_info
system_info = get_system_info()
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/systemtools.py",
line 460, in get_system_info
'core_count': get_avail_core_count(),
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/systemtools.py",
line 119, in get_avail_core_count
out, _ = run_cmd('sysctl -n hw.ncpu')
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/run.py",
line 80, in inner
return func(cmd, *args, **kwargs)
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/run.py",
line 153, in run_cmd
return parse_cmd_output(cmd, stdouterr, ec, simple, log_all,
log_ok, regexp)
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/run.py",
line 382, in parse_cmd_output
_log.error('cmd "%s" exited with exitcode %s and output:\n%s' %
(cmd, ec, stdouterr))
File
"/cygdrive/k/cygwin/dev-tools/EasyBuild/Kenneth/easybuild-framework/easybuild/tools/build_log.py",
line 103, in error
raise EasyBuildError(newMsg)
EasyBuildError: 'EasyBuild crashed with an error (at
easybuild/tools/run.py:382 in parse_cmd_output): cmd " sysctl -n
hw.ncpu" exited with exitcode 127 and output:\n/bin/bash: sysctl:
\xd0\xba\xd0\xbe\xd0\xbc\xd0\xb0\xd0\xbd\xd0\xb4\xd0\xb0
\xd0\xbd\xd0\xb5 \xd0\xbd\xd0\xb0\xd0\xb9\xd0\xb4\xd0\xb5\xd0\xbd\xd0\xb0\n'
======================================================================
FAIL: test_os_type (__main__.SystemToolsTest)
Test getting OS type.
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/framework/systemtools.py", line 73, in test_os_type
self.assertTrue(os_type in [DARWIN, LINUX])
AssertionError: False is not true
======================================================================
FAIL: test_shared_lib_ext (__main__.SystemToolsTest)
Test getting extension for shared libraries.
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/framework/systemtools.py", line 78, in test_shared_lib_ext
self.assertTrue(ext in ['dylib', 'so'])
AssertionError: False is not true
----------------------------------------------------------------------
Ran 11 tests in 27.406s
FAILED (failures=2, errors=5)
-------------------------
But what do all these mean? What should I do next?...
P.S. Would you please advise how to convert
\xd0\xba\xd0\xbe\xd0\xbc\xd0\xb0\xd0\xbd\xd0\xb4\xd0\xb0
\xd0\xbd\xd0\xb5
\xd0\xbd\xd0\xb0\xd0\xb9\xd0\xb4\xd0\xb5\xd0\xbd\xd0\xb0 - like
strings into a printable ones (perhaps, there are some problems due to
cyrryllic encoding on my system...)
Thank you for support!
Tymofii
08.07.2014 10:03, Kenneth Hoste wrote:
Hi Tymofii,
Welcome to the wonderful world of EasyBuild!
On 08/07/14 08:24, [email protected] wrote:
Dear developers,
I ran into troubles when trying to install EasyBuild via
bootstraping under
Cygwin.
That's quite brave!
Can you elaborate a bit why you're looking into using EasyBuild on
Cygwin?
As far as I understand the error stems from here:
------------systemtools.py------------
def get_shared_lib_ext():
"""Determine extention for shared libraries
Linux: 'so', Darwin: 'dylib'
"""
shared_lib_exts = {
LINUX: 'so',
DARWIN: 'dylib'
}
os_type = get_os_type()
if os_type in shared_lib_exts.keys():
return shared_lib_exts[os_type]
else:
raise SystemToolsException("Unable to determine extention
for shared
libraries,"
"unknown system name: %s" % os_type)
-------------------------------------------
As the error message suggests, 'CYGWIN_NT-5.1' (or, more generally,
'CYGWIN*')
should be
present in shared_lib_exts array.
Is it possible to overcome this difficulty ?
You're exactly right: as far as I know, nobody has tried using
EasyBuild on Cygwin before (or gave up very soon after running into
this). Congratulations, you're not the EasyBuild maintainer for
Cygwin. ;-)
To start with making it work (or try it), you'll have to make
EasyBuild Cygwin-aware.
Since the bootstrap pulls in the latest EasyBuild release, installing
EasyBuild this way won't work.
We can't include Cygwin support before it has been tested properly, so
if you want to look into this, you'll need to play around with a
development setup of EasyBuild.
Are you up for that?
How familiar are you with git/GitHub, Python, Cygwin?
In short, here's what I recommend to get started:
0) get a copy of the EasyBuild framework code repository, check out
the develop branch:
git clone [email protected]:hpcugent/easybuild-framework.git
git checkout -b develop origin/develop
If you're unfamiliar with git, you can download the current develop
version directly by downloading and unpacking
https://github.com/hpcugent/easybuild-framework/archive/develop.tar.gz
(but you won't have an easy way of staying up to date).
1) Try running the EasyBuild unit tests, and see what kind of trouble
you run into. Start with the unit tests for systemtools.py specifically:
$ cd easybuild-framework
$ python test/framework/systemtools.py
...........
----------------------------------------------------------------------
Ran 11 tests in 3.871s
OK
To add initial Cygwin support, dive into
easybuild/tools/systemtools.py, and try extending the code to make it
Cygwin aware.
Let us know how that turns out, or if you have any further questions.
regards,
Kenneth
