Re: [Plplot-general] Nightly testing?

[Alan W. Irwin]

On 2016-09-23 12:30-0700 Alan W. Irwin wrote:

> Note, a decade-old build system normally accumulates a lot of cruft
> and PLplot is no exception in that regard.  So reducing that cruft and
> also learning to take advantage of modern CMake facilities requires a
> substantial amount of on-going maintenance.  But I am happy to do that
> maintenance because the result is a build system that is much more
> sophisticated and useful than what we created a decade ago which in
> turn was already a bit more sophisticated than what we could achieve
> with autotools at that time.

I want to correct a slightly wrong impression I gave there.  I am
positive that with enough care and learning experience anything we do
with CMake could also be replicated by autotools and vice versa so
fundamentally there is no limit on the sophistication of build systems
implemented with _either_ CMake or autotools.

However, that said, it is also fair to say that once someone uses
CMake seriously for a short time (it was just a few hours for me and
other PLplot developers who wanted to change the PLplot build system
in some way have had similar quick-learning experiences), they will
find most build-system and test-system components so easy to implement
with CMake that they by far prefer to use CMake rather than autotools
for that task from then on.  Which is why we dropped our
autotools-based build system so quickly a decade ago.

In sum, we were pretty heavy autotools users, but we tried CMake and
never looked back.  And I expect most autotools users that try CMake
will also have similar positive experiences with it.

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__

--
___
Plplot-general mailing list
Plplot-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-general

Re: [Plplot-general] Nightly testing?

[Alan W. Irwin]

On 2016-09-23 15:30+0100 Tom Schoonjans wrote:

> Hello once again,
>
> Travis-CI and Appveyor are not part of Github, but are separate companies in 
> their own right. They are just (popular) examples of companies that make use 
> of the webhook and integration possibilities that are offered by Github. A 
> list of similar companies can be found here: https://github.com/integrations 
> 
>
> To the best of my knowledge, most of these companies operate under a 
> financial model that includes free usage by open source projects, but paid 
> services for closed source projects (usually these are private repositories 
> on Github, but private hosting is also possible). Either way, their usage is 
> completely optional, though highly recommended. Even if one of these 
> continuous integration providers would go broke or cease to provide free 
> services for open source projects, there are alternative providers available, 
> or they can just be turned off and other testing options could be found.
>

> If you would get [a cdash] server up and running, I am sure
there are ways to have a webhook following a commit trigger a build.

Good.

> [out of order] I have to admit that my knowledge of CMake, CTest and CDash is 
> next
to nothing (I am more of autotools kind of guy myself), but it looks
promising.

We are strong advocates for CMake and friends ever since we moved to
CMake a decade ago from a pretty sophisticated autotools approach.
However, I will attempt to avoid over-advocating CMake to you by
confining myself to making the suggestion that you give it a serious
try for one of your software projects to see how you like it for your
build-system and test system needs.

Note, with a bit of care in choosing template file names for
configuration both CMake-based and autotools-based build systems can
coexist for the same software project so you can directly compare the
results of the two. When we did that for PLplot we started by simply
building one of our minor libraries with CMake as a proof of concept
and when that proved to be trivial everyone pitched in so that very
quickly (a month or so in our spare time) we had implemented a
complete CMake-based build system for all the large number of
components of PLplot.  And ctest soon followed.  We ended up liking
that build and test system so much that we quickly (within 6 months or
so) abandoned our autotools-based build system as too much trouble to
maintain.  After that positive experience I have personally moved to
CMake for all my other software projects as well, and I assume other
PLplot developers have mostly done the same.

Note, a decade-old build system normally accumulates a lot of cruft
and PLplot is no exception in that regard.  So reducing that cruft and
also learning to take advantage of modern CMake facilities requires a
substantial amount of on-going maintenance.  But I am happy to do that
maintenance because the result is a build system that is much more
sophisticated and useful than what we created a decade ago which in
turn was already a bit more sophisticated than what we could achieve
with autotools at that time.

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__

--
___
Plplot-general mailing list
Plplot-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-general

Re: [Plplot-general] Possible move to github?

[Alan W. Irwin]

On 2016-09-23 11:29+0100 Tom Schoonjans wrote:

> Hi Alan,
>
>
>
> A move to Github would certainly be advantageous for a number of reasons:
>
> - It would make receiving contributions and patches a lot easier (git-patch 
> is not that cool). The fork-pullrequest system is phenomenally good.
> - Github provides a more pleasant user experience on its website: no one is 
> confronted with annoying ads when downloading tarballs
> - Github is a more stable and reliable host: it has a very impressive uptime, 
> and I think we all remember SF’s week long outage last year… As another 
> example, it took me yesterday three attempts before I could successfully 
> clone the PLplot repository.

> - Github is considerably less evil than SF 
> https://en.wikipedia.org/wiki/SourceForge#Controversies 
> 

They are both big companies trying to figure out a way to monetize
free software so in my view some stupidities are inevitable with both,
and the question is when such stupidities occur how well do they
ultimately respond to the concerns of the free software community they
are hosting. I also think use of true free software products for their
hosting environment does provide the free software community with some
options to curb the worst excesses of the big companies in the long
term.  So the fact that Allura is completely free software that allows
"SF clone" alternatives to SF to exist that use their identical
hosting environment is reassuring to me.  So I lean toward sticking
with SF or one of its clones.  But I might encourage someone else to
move our git repository to github if git outages become the norm at SF
rather than the one or two isolated incidents we have encountered so
far.

>
> Moving the git repository to Github itself would be trivial. Just use 
> https://github.com/new/import , but I do 
> recommend using the PLplot organization as owner. Just add yourself to it 
> first and get sufficient privileges. Hazen I think you would be able to help 
> out here since you seem to be in charge of this organization.
>
> The git history would be identical to the current one as it is a direct copy. 
> Afterwards you can enforce whatever workflow you are comfortable with 
> regarding contributions/patches.
>
> The website you host at plplot.sourceforge.net 
>  is another matter. Github does allow you to 
> host websites (e.g. plplot.github.io), but they have to be static: no PHP! 
> However there is nothing from switching the git repo to github while keeping 
> the website where it is right now. This is actually pretty common.
>
> Same for downloads: either they can stay where they are or they can be moved 
> to github.

You have answered Hazen and my concerns about the negatives well. And
it does makes sense to look at each SF service we currently use (e.g.,
git repository, website, file release area, wiki, and mailing lists)
in turn to see which we might want to migrate to github and which not.
So if some PLplot developer wanted to start that process with just the
git repository, I would not stand in their way. However, that person
would have to take full responsibility for moving our git repository
to github which would include figuring out how to enforce our current
workflow (and also our future workflow if we ever change to a
different enforced workflow) at github in that git pull environment.
However, that work would not be enough.  That same person would also
have to take the responsibility for responding to git pull requests,
and when they discovered a pull violated our workflow rules, they
would have to advise the requester what they have to change in their
own workflow to make their pull request a valid one.  And I am pretty
sure a request simply to rebase on master tip will not be enough for
topic branches with a seriously complex history that includes merge
commits.  So this is a non-trivial responsibility which I would not
want to personally take on.

Of course, another alternative (which my impression is a lot of naive
git projects use) is not to worry in the slightest about enforced
workflow and just accept the consequences of a seriously complex
history and git log results that are impossible for a human to
decipher.  But I think that would be a serious mistake for PLplot to
move in that naive direction so someone really does have to take the
responsibilities that I have outlined above if they are determined to
move our git repository from SF to github.  However, I would also be
content if the PLplot developers decided as a group to stick with our
current simple procedure of using the SF git repository and git
format-patch/am when we wish to collaborate on non-public topic
branches.

Note the git format-patch/am method worked very well for Arjen and me
when we collaborated on the new Fortran binding before we jointly
matured that private topic enough so that we could push that whole

Re: [Plplot-general] Nightly testing?

[Tom Schoonjans]

Hello once again,

Travis-CI and Appveyor are not part of Github, but are separate companies in 
their own right. They are just (popular) examples of companies that make use of 
the webhook and integration possibilities that are offered by Github. A list of 
similar companies can be found here: https://github.com/integrations 


To the best of my knowledge, most of these companies operate under a financial 
model that includes free usage by open source projects, but paid services for 
closed source projects (usually these are private repositories on Github, but 
private hosting is also possible). Either way, their usage is completely 
optional, though highly recommended. Even if one of these continuous 
integration providers would go broke or cease to provide free services for open 
source projects, there are alternative providers available, or they can just be 
turned off and other testing options could be found.

I have to admit that my knowledge of CMake, CTest and CDash is next to nothing 
(I am more of autotools kind of guy myself), but it looks promising. If you 
would get such a server up and running, I am sure there are ways to have a 
webhook following a commit trigger a build.

Best,

Tom

> On 22 Sep 2016, at 20:06, Alan W. Irwin  wrote:
> 
> Hi Tom:
> 
> On 2016-09-22 13:54+0100 Tom Schoonjans wrote:
> 
>> 
> [Hazen said]
>>> I am not familiar with Travis-CI or AppVeyor but they sound interesting.
>> 
>> Nowadays Continuous Integration is a standard tool for automated testing and 
>> building nightly releases. With Travis-CI and AppVeyor you can easily 
>> configure that each commit sets of a number of builds on Linux, Mac OS X and 
>> Windows using any combination of options and compilers you want. Plus it’s 
>> completely free for open-source projects!
>> 
>> It is also possible to require PRs to successfully build and survive testing 
>> on Travis-CI and AppVeyor as a precondition for being merged in.
>> 
>> For an example of Travis-CI and AppVeyor, have a look at for example 
>> https://github.com/tschoonj/easyRNG , 
>> scroll to the end of the page and click on the green badges.
> 
> Even if we move to github I would advise whoever is in charge of such
> a move to not rely on semi-proprietary or proprietary products this
> way since "free of cost" can change to "costly" with one corporate
> decision at github.
> 
> I do agree that nightly testing of PLplot would be worthwhile.  In
> fact, we are already almost completely set up to do that with ctest,
> and very little more is required for anybody interested to publish
> their nightly PLplot ctest results in cdash client mode. So all that
> is essentially required is a cdash (see ) server to
> collect and display those results in convenient form on a website.
> KitWare already provides such a free service (see
> ), but I recall there are some limitations
> (e.g., only a limited number of clients are allowed for a given
> project such as PLplot). But note that cdash is a completely
> open-source product so it should be straightforward to download that
> source and build a cdash server. So ultimately if the my.cdash.org
> limitations became an issue we would need to find a volunteer
> (presumably someone who is already running their own website) who is
> willing to build and host a cdash server for PLplot.
> 
> Alan
> __
> Alan W. Irwin
> 
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
> 
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __
> 
> Linux-powered Science
> __

--
___
Plplot-general mailing list
Plplot-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-general

Re: [Plplot-general] New release?

[Tom Schoonjans]

Hi Alan,


Releasing regularly new versions of the package is always good: it instills 
confidence in users that they are using a software package that is being 
actively developed and maintained.

However, I do think that the approach of having bug-fix branches is essential 
when making regular releases (3-6 months). Implementing new features (like the 
new Fortran bindings you mentioned) can take a long time, and it is difficult 
to predict when a new feature is ready to ship. After all, I am assuming none 
of the core developers are working full-time on PLplot right? This way of 
working unfortunately leads to large gaps between releases, which has led to 
the current situation where PLplot cannot be compiled with the latest version 
of CMake. Having a bug fix branch, and making releases off it, would ensure 
that PLplot can quickly address incompatible changes in its dependencies, or 
just fix important bugs, while relieving the developers from the pressure of 
having to produce a new release that will be a mixture of (hopefully properly 
working) new features and bug fixes.

Best,

Tom


> On 22 Sep 2016, at 20:01, Alan W. Irwin  wrote:
> 
> Hi Tom:
> 
> On 2016-09-22 13:54+0100 Tom Schoonjans wrote:
> 
>> In the scenario I am suggesting, you would just end up with a new
> branch that starts at a tag and will contain bug fixes only. It will
> never need to be merged into master as it will consist of commits that
> were cherrypicked from master.
> 
>> This kind of workflow is used for example by all GNOME projects such
> as glib and gtk+: development occurs on master, when a new major
> release is ready, a tag is created (e.g. 3.22.0), as well as a new
> branch with the name of that major release (gtk-3-22), which will then
> receive bug fixes that were cherrypicked from master, but not its new
> features! Every now and then maintenance releases will be created off
> this branch like 3.22.1, 3.22.2 etc.
> 
> This is certainly a possible approach, but in my view an even better
> (and simpler) approach would be to get out our releases (whether
> maintenance releases or otherwise) in a timely manner, i.e., every 3-6
> months.  I have obviously fallen short of that goal for this release,
> but I also hope to do a lot better moving forward!
> 
> Alan
> __
> Alan W. Irwin
> 
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
> 
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __
> 
> Linux-powered Science
> __


--
___
Plplot-general mailing list
Plplot-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-general

Re: [Plplot-general] Possible move to github?

[Tom Schoonjans]

Hi Alan,



 A move to Github would certainly be advantageous for a number of reasons:

- It would make receiving contributions and patches a lot easier (git-patch is 
not that cool). The fork-pullrequest system is phenomenally good.
- Github provides a more pleasant user experience on its website: no one is 
confronted with annoying ads when downloading tarballs
- Github is a more stable and reliable host: it has a very impressive uptime, 
and I think we all remember SF’s week long outage last year… As another 
example, it took me yesterday three attempts before I could successfully clone 
the PLplot repository.
- Github is considerably less evil than SF 
https://en.wikipedia.org/wiki/SourceForge#Controversies 


Moving the git repository to Github itself would be trivial. Just use 
https://github.com/new/import , but I do 
recommend using the PLplot organization as owner. Just add yourself to it first 
and get sufficient privileges. Hazen I think you would be able to help out here 
since you seem to be in charge of this organization.

The git history would be identical to the current one as it is a direct copy. 
Afterwards you can enforce whatever workflow you are comfortable with regarding 
contributions/patches.

The website you host at plplot.sourceforge.net  
is another matter. Github does allow you to host websites (e.g. 
plplot.github.io), but they have to be static: no PHP! However there is nothing 
from switching the git repo to github while keeping the website where it is 
right now. This is actually pretty common.

Same for downloads: either they can stay where they are or they can be moved to 
github.

On to the next email!

Tom


> On 22 Sep 2016, at 19:50, Alan W. Irwin  wrote:
> 
> Hi Tom:
> 
> On 2016-09-22 13:54+0100 Tom Schoonjans wrote:
> 
>> Hello again,
>> 
> [Hazen asked]
>>> We have been trying to maintain a linear history with git following
> the process explained in the README.developers file. I think that the
> fork and pull-request system would break this?
> 
>> Not necessarily, you can ask people submitting a PR that they rebase
> against master. You can even enforce as a required check before
> merging that new branches are up to date with the latest changes in
> master.
> 
> @Tom:
> 
> Keeping both a github and SF repository adds quite a bit of work so I
> think what we are really discussing is whether we want to move
> completely to github.  But that move is a lot of work as well. I (as
> current de facto maintainer of our official git repository at
> SourceForge) would not want to do such work.  However, if someone else
> wanted to take up the responsibility of maintaining our official git
> repository, I would be happy to hand over that responsibility. Furthermore, 
> if they decided it would be a good idea to move to
> github, I would not get in the way of such a move so long as such a
> move did not dictate our git workflow and our git history (currently
> stretching back to the early 90's) was preserved.  More later about our 
> workflow
> under a different subject line
> 
> Alan
> __
> Alan W. Irwin
> 
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
> 
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __
> 
> Linux-powered Science
> __

--
___
Plplot-general mailing list
Plplot-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-general