On 12/02/14 08:41, Atgeirr Rasmussen wrote:
Hi James (and others),
I share your worry that practical (and principal) hurdles make it hard to
sustain and grow a
sound user and developer community. However I do not quite understand all your
points. I'll deal
with each below.
The current binary-build model is sufficient?
No I do not think it is working well enough, because we should have more
frequent releases
than we have had (this year). However I think binary packages are a must for
many people,
because it makes it easy and quick to test the software.
I agree that distributing binaries periodically is a great idea. If one
of the modules also wants to support it's software in a stand alone form
too, then they should. If somebody wants to build the latest of each of
the modules, aka like a "rolling release" then there should be a concise
semantic that guides downloading the require modules and then
a listing of optional modules to that the current work can be compile
frequently and tested. How the modules fit together, which modules to
use, when to substitute different mathematical codes into a given
module, etc etc, are not very clear decisions, due to the complexity and
size of the the problems that OPM proposes to solve. So the less
frequent to test the latest codes, the further the divergence from the
end goal will be. Another way of stating this is that taking a bit of
weekly (organizational) pain is much better (akin to athletic training)
can trying to fix it all up every 6-18 months (akin to an office working
training for one week before a 15K run).
This aforementioned perspective only gets worse, when you plan to run
these codes on HPC, Cloud, Cluster, or most forms of specialize hardware
or massively parallel system.
Strangely, this list has grown quite. If I upset anyone, I am apologizing now.
However, I have
seen many and been part of a few projects that have fallen apart, because, open
source was
not "conveniently_open". We can also move to a model where codes are developed
however the
core developers want them to be; even separately if desired. We can then copy
codes to a
central location(s) where anyone can download, compile, test and validate the
codes
separately or as part of a "release", a "version number" or simple as
./configure and make
makefile types of scenarios.
We do have a central location: the github repos are open and anyone can
download any module
in its current state, or older releases. What is it you are looking for that is
not provided
by the current repos?
OK, OS's that use a rolling update cycle (meaning you can update any
software package on any given day) depend on the package build system.
These sorts of distros are *excellent* (some would say quintessentially
important) for developing and testing the latest source code offerings.
Gentoo and Arch linux are a couple of prominent rolling distro examples.
I use Gentoo.
So my ebuild for OPM core looks like this:
<snip>
# Copyright 1999-2014 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header$
EAPI=5
inherit cmake-utils
SRC_URI="https://github.com/OPM/${PN}/archive/release/${PV}/final.tar.gz
-> ${P}.tar.gz"
LICENSE="GPL-2+"
SLOT="0"
KEYWORDS="amd64"
S=${WORKDIR}/${PN}-release-${PV}-final
RDEPEND="
dev-libs/boost
virtual/blas
virtual/lapack
sci-libs/dune-common
sci-libs/dune-istl[superlu,umfpack]
sci-libs/umfpack
>=sci-libs/superlu-4.3"
DEPEND="${RDEPEND}"
<end/snip>
So my question not just about this module, but all the modules, will
I get the latest codes available for compiling and testing?
For discussion sake, let's say were working with 7 packages (modules). 3
have updated packages available. Will I get the latest 3 packages and
those new packages will compile and run against the other 4 packages.
How will I pick up the inter packages dependency changes?
You see what I want, is what everyone needs. That is the ability at any
given point in time to be able to download the latest sources, compile
and test. So a few hours after a dev uploads any packages with updates,
changes, new codes etc etc, I'm testing it and providing feedback, just
a couple of clicks off real time?
That way coding development moves forward at light speed. There is not
bickering and arguing at "release time". Problems are
fixed as they appear and when they are small.
Make sense?
EAPI 6 will readily support user patches into the codes without dev
intervention. If it works a user can the post it to the github for the
wider community to evaluate. Very new and very slick feature coming to
Gentoo EAPI 6.
The syntax {SLOT="0"} from above allow me to use slot numbers to keep
the different versions of code separate. That is a very useful construct
when a modules (package) takes on different tracks, or different version
work with different versions of the other modules (packages).
Ebuilds can be quite complex, depending on compile time and run time
dependencies, initial configurations options and the mix of other
packages (modules) those various setting between the interactions
of the packages. I used a simple example. When we fully support multiple
linear solvers and such, the ebuilds will become the best mechanism to
formally state the package interactions by passing arguments (setting
flags) on the modules at compile time.
Likewise the link provides the Arch linux link to the EPANet surface
flow simulation:
https://aur.archlinux.org/packages/ep/epanet2-git/PKGBUILD
The point is the modules should all have code that is downloadable and
compiles and can be readily tested against all the other core modules.
Hopefully, you "release schedule" will be nothing more than a list of
modules-versions that are know to work, reasonable well with one another
to tag it as a "stable release". Other newer works are "testing"
labeled. These two system complement each other and make a regular
release cycle, a trivial effort. Where is the stable release for 2014?
I can say that we should probably tag the repos more often. When we have
reached a useful
milestone (such as when the SPE cases were running successfully last winter) we
should
have tagged the repos to be able to refer to it easily later, even if we did
not make a
release as such. I think we should also be open to accepting tags from
the general community
for a variety of reasons, for example to mark the exact versions used in the
publishing of a paper.
YES! This becomes a trivial task, if your codes (critical modules) all
compile and work on an ongoing basis. At first it's a bit of a
challenge. In the long run we'll get better, more focused codes and
manynew complementary codes, besides a very active community porting
this work to every from of massive computation resources that folks can
dream up.
I am not trying to harm this OPM-project. I'm trying to instil some
structure into the daily methodologies, whilst encourage each dev team
to build the fiefdom they like, that will make this project something
many what to test and use. Only having stable release every 18 months
is pitiful, imho. Ensuring that a particular module compiles against
the other core modules, should be the responsibility of that dev team.
Testing, regression testing, vector testing and porting to a myriad
of hardware is both the responsibility and the joy of a vivacious user
community.
Atgeirr
James
_______________________________________________
Opm mailing list
[email protected]
http://www.opm-project.org/mailman/listinfo/opm
