Hi Riccardo,
On 13/08/14 18:10, Riccardo Murri wrote:
I'll see if I can spare some time this week to have a look at the
EasyBuild source code and estimate how much effort would it be to
adapt the current PBS support to use GC3Pie instead.
It looks like it won't be much work until a draft working version.
I've started some coding here:
https://github.com/gc3-uzh-ch/easybuild-framework/tree/gc3pie
Feedback is very welcome! (Although there is no GC3Pie code yet.)
It's looking great so far, after giving it a quick glance, thanks for
looking into this!
Having an open pull request would make it significantly easier to
discuss the particular changes you're making, but the overall approach
looks great.
As soon as your comfortable to be bombarded with remarks w.r.t. your
implementation, please open a PR, even if you're done not 100% done yet
(just mark it as (WIP) in the title or some such).
Ideally, you should retain the current behavior of --job for backward
compatibility (which you seem to be doing), although I'm willing to make
an exception here (since I don't think anyone has ever been able to use
--job other than ourselves at HPC-UGent).
In addition, I already have a few questions:
* I had to do some refactoring on the `parallelbuild` and `pbs_job`
modules; how do I run tests for those?
The entire test suite is run using "python test/framework/suite.py"
(which can take a while, e.g. over half an hour), but you can also run
each of the subsuites separately, e.g. "python test/framework/options.py".
For your purposes, not breaking the tests in parallelbuild.py and
options.py are the most important, although tweaks to the tests will
probably be required, both because of the changes you've made and to
test the new functionality. We can help you there, don't worry.
* the "CONTRIBUTING.md" file mentions a "coding style guide", but the
link is broken. Are there guidelines I should be aware of, besides
PEP8?
It's mostly PEP8, with a slightly relaxed column-width limitation (100
chars soft limit, 120 chars hard limit).
The correct link is
https://github.com/hpcugent/easybuild/wiki/Code-style, I'll fix that in
CONTRIBUTING.md, thanks.
If you follow the existing code, you should be fine (except if you're
touching a part of the codebase that hasn't been touched in a a while).
Try and make sure your code is Python 2.4 compatible, even though it's
deprecated (see
https://github.com/hpcugent/easybuild/wiki/Deprecated-functionality#python-compatibility).
We'll be moving on to Python 2.6 (in preparation of Python 3.x support)
with EasyBuild v2.0, but that's easily still months away (we want to see
a couple of major features finished first, e.g. support for easyconfig
format v2).
* AFAICS, `eb --job` submits the jobs to PBS and then exits. I presume
there is no interest in collecting the job output or checking if
they are successful as the admins will figure that out?
For now, just being able to submit a set of jobs and exit is fine.
Further extending that functionality can be done later in a follow-up PR
is deemed useful. Open issues related to this include:
https://github.com/hpcugent/easybuild-framework/issues/152,
https://github.com/hpcugent/easybuild-framework/issues/113.
Typically, simply submitting a bunch of jobs before lunch using --robot
--job, and then checking up afterwards using --dry-run and loading the
modules suffices.
I haven't really felt the need for anything more (but that might be
because I'm used to the current behavior of --job).
regards,
Kenneth
