Hi Tim,
with regards to the point you and others raised with regards to funding
provided by industry users:
From my point of view as a Structural Biology group leader responsible for
crystallographic software budget decisions in a major Pharma franchise, I very
much appreciate the effort and commitment by the academic software developers
to make their programs available in either source code or binary distributions.
We heavily depend on these programs to conduct our day-today work, which
ultimately helps patients across the globe. The modest amounts charged by most
academic groups are small price to pay and we are more than happy to do this to
support the effort and would have absolutely no intention to pull out of this
in case everyone switched to an open source model. Prices for "professional"
packages with sometimes questionable value propositions are another matter...
Still flabbergasted by the fact that Paul is not charging anything, arguably
the most valuable and unique package out there, way to go!
Have to agree with Takanori on his point of reproducibility in the context of
Open Source and might add that opening the software up to other committed
developers might lead to improvements not anticipated by a "lone wolf
developer" closed source model.
My 2 cents worth
Regards
Carsten
-----Original Message-----
From: CCP4 bulletin board [mailto:[email protected]] On Behalf Of Tim Gruene
Sent: Wednesday, October 21, 2015 9:24 AM
To: [email protected]
Subject: Re: [ccp4bb] new version of (academic user) XDS package
Dear Takanori Nakane,
the link to the dials github repository underlines very nicely the main
argument for closed source software. It appears that the person writing that
comment would like to mimic what XDS does. Imagine the source code were
available. Someone could write a program that reproduces very closely XDS'
results. If that program were made available to everyone, why should companies
buy a licence?
I also can confirm that the statement
'I also note that authors can simply state "earlier versions are provided for
reproducibility.'
is not as simply as you imply. I have received users requests about odd results
from some shelx program and only after a while it turned out that this was a
very old version where that feature was under development. Redirecting the
person to the latest version made the problem disappear. It took me quite some
time to realise that the reason was a too old version of shelx, so I could not
have replied "is not supported anymore" at first sight of the email.
And I only receive a very, very small fraction of shelx user requests.
Multiply it by one hundred and you get a quite a lot of time that gets lost for
the program authors to actually improve their software, as Kay already pointed
out.
I actually don't understand why a closed-source program affects reproducibility
and transparency. The programs we are talking about are very well tested by
thousands if not millions of runs. The fact that people report bugs underlines
that these are not black-box programs. How would the algorithm how XDS
determines e.g. the REFLECTING_RANGE affects your results or the transparency
of your research? The REFLECTING_RANGE is not even part of the data you deposit
in the end, and this is true for nearly all parameters XDS uses.
And again, if you really were serious about your concerns, you would not be
using XDS at all, but one of the alternatives that match your standards.
Best regards,
Tim
On Wednesday, October 21, 2015 09:25:02 PM Takanori Nakane wrote:
> Dear all,
>
> Thank you very much for your comments on-list and off-list.
> All of them were thought provoking.
>
> First of all, I would like to emphasize that this is not an issue of
> my personal preference as Tim wrote, but that of scientific
> reproducibility and transparency as Graeme pointed out.
>
> > The truth is that several different models exist, and they all have
> > their role.
>
> Yes, but higher level of openness and reproducibility is required for
> scientific programs than Mac OS and Word. The most direct way is to
> make source codes public. This is what this Nature article argues.
>
> Ince, Darrel C., Leslie Hatton, and John Graham-Cumming.
> "The case for open computer programs." Nature 482.7386 (2012): 485-488.
> http://www.nature.com/nature/journal/v482/n7386/full/nature10836.html
>
> > If the XDS source code were made freely available, even this revenue
> > would not be available;
>
> I disagree. As Marcin pointed out, transparency and revenue are
> compatible. XDS source code can be made public under a restrictive
> license, where we can look at codes, but are not allowed to
> redistribute them. MPI can still charge industrial users. This is
> exactly how
> CCP4 and PHENIX are distributed.
>
> > just reset the clock of your computer
>
> This is a surprising remark. I thought that it constitutes a license
> violation, as XDS prints out "Copy licensed until DD-MM-YYYY"!
>
> I also note that authors can simply state "earlier versions are
> provided for reproducibility. They are not supported any more,"
> instead of imposing time limits and deleting older binaries from the server.
> Thus, the burden of support is not an excuse.
>
> > The working principles of XDS are published in sufficient detail to
> > allow their coding by experienced developers;
>
> I highly doubt it. Finer details are missing from the paper.
> For example, how REFLECTING_RANGE is determined based on
> REFLECTING_RANGE_E.S.D? The paper [1] says "typically chosen to be
> 6–10 times larger than σM and σD", which is vague [2].
>
> > Moreover, coding details have been exposed via e-mail discussions to
> > ultimately specify open points.
>
> I wish every detail be open (ideally as source codes, otherwise as a
> paper or Web page), rather than exposed in private communication. In
> addition, I repeat that description of algorithms in natural language
> never substitutes source code. See "The failure of code descriptions"
> section in the above article.
>
> > Not all software development is done in the author's "spare time,"
> > it is sometimes supported by grants.
>
> In addition to David's point, academic research projects are usually
> supported by public grants. So not only software developers, but also
> users hold accountability for methods.
>
> It is true that there are many other "black boxes" in this field, such
> as proprietary culture medium. But we, as scientists, are striving
> after transparency and spread of knowledge, aren't we? Would you
> accept a paper if the crystallization condition was kept secret?
>
> Of course, the required level of reproducibility and transparency
> might differ among scientific fields. Since many of the community
> members seem comfortable with the wild spread use of black-box
> programs like XDS and SHELX [3], I will not argue any more. I might
> have been too dogmatic and idealist for this field...
>
> [1] Kabsch, Wolfgang
> "Integration, scaling, space-group assignment and post-refinement."
> Acta Crystallographica Section D 66.2 (2010): 133-144.
>
> [2] Incidentally, DIALS developers seem to have same question.
> https://github.com/dials/dials/blob/master/algorithms/profile_model/ga
> ussian
> _rs/calculator.py#L13
>
> [3] I know the source code was available in the past, but I cannot find
> it now.
>
> Best regards,
>
> Takanori Nakane
>
> On 2015/10/20 22:32, Kay Diederichs wrote:
> > Dear Takanori,
> >
> > the time-limited versions exist for the sole purpose of keeping the
> > package maintainable, not for annoying users. There is only Wolfgang
> > Kabsch (semi-retired) and me (in my spare time) who maintain the
> > package, and there is no funding for its development. The small
> > revenue that is made from the industrial versions of XDS (which BTW
> > are not time-limited) goes to the Max Planck
> > Verwertungsgesellschaft, and thus helps to fund
> > (general) research. If the XDS source code were made freely
> > available, even this revenue would not be available; at the time
> > when XDS was started (coinciding with the start of the GNU project),
> > the term "open source" with all its different license schemes (that
> > may even lead to
> > payments) was not even invented.
> >
> > To give you an idea of what I mean by "maintainable": just a few
> > minutes ago I sent an email that started with "...please upgrade
> > your XDSGUI! The Rd plot in the latest version would tell you ..."
> > and I am considering to put an expiration date on future versions of
> > XDSGUI, because it is in everybody's interest to get the best result
> > from a program. It is really the other way around: _you_ should have
> > an interest in getting the best results, by using the latest
> > version. It is unlikely that there will ever be a need to reproduce
> > the errors from the old program. And if you are in a situation that
> > you want to reproduce those old results: just reset the clock of
> > your computer. This is a very small effort, and a very small price to pay.
> >
> > Furthermore you seem to believe that the one model of software
> > development that you like is evidently the only one that everybody
> > else should use. I do not agree - although I also like Open Source
> > in general. The truth is that several different models exist, and
> > they all have their role. Is OSX open source, is Windows, is Word? -
> > still everybody uses them, and people pay for it. So why complain about
> > software you don't pay for?
> > Effectively you are saying "the software does not cost money, so its
> > source code should also be free".
> >
> > The bugs fixed in this release are associated with highly technical
> > details having to do with floating overflow occurring in corner
> > cases. If you want to more specifically know what Wolfgang Kabsch
> > changed, ask him by email; this is nothing for this BB as nobody
> > will understand it. If many people write to him requesting 32bit
> > Linux or OSX 10.5 binaries, he might change his mind. But to be
> > honest, it is very little effort to install 64bit Linux, and
> > 32bit-only Intel-compatible processors for notebooks and PCs have
> > not been manufactured (I guess) for 10 years or so. Apple hardware
> > that cannot be upgraded to OSX 10.6 must be more than
> > 6 years old. Yes I know one person still using OSX 10.4, and he may
> > have his reasons - but then, he's also not a XDS user.
> >
> > best wishes,
> >
> > Kay
> >
> > On Tue, 20 Oct 2015 21:35:14 +0900, Takanori Nakane
<[email protected]> wrote:
> >> Dear XDS developers,
> >>
> >> I object to the decision to stop the distribution of
> >> 32 bit binary. I know many people who still use 32-bit Linux.
> >> I don't mean that the new version should be built for 32-bit.
> >> I know a huge burden associated with the maintenance of building
> >> and testing environment. Still, I believe that the last 32-bit
> >> build (version June 17, 2015) be available without time limit. For
> >> example, although CCP4 dropped support of 32-bit Mac OS, we can
> >> still download older versions or build 32-bit binary from source codes.
> >> Both options are unavailable for XDS.
> >>
> >> Honestly speaking, the way XDS is distributed is frustrating, where
> >> only the latest binary with time limit is available. How can we
> >> guarantee reproducibility of our research, if the exact version of
> >> a scientific program used for the research is no longer available [1]?
> >> Even worse, since XDS is a closed-source, black-box program, we do
> >> not know what actually changed in each update. For example, what
> >> does "simplification in the method for calculating the initial
> >> value of the mosaicity" [2] mean? How does it differ from the
> >> previous, published method? If the authors insist on keeping XDS
> >> source code confidential, they should at least provide detailed
> >> explanations of the algorithms in natural language, although it never
> >> substitutes source codes [3].
> >> It is understandable that some authors want to charge non-academic
> >> users to secure funding. But it does not justify secrecy of
> >> internals and unavailability of earlier versions.
> >>
> >> [1] See discussion on ccp4bb last May
> >> https://www.mail-archive.com/[email protected]/msg38928.html
> >> [2] XDS Release notes
> >> http://xds.mpimf-heidelberg.mpg.de/html_doc/Release_Notes.html
> >> [3] Ince, Darrel C., Leslie Hatton, and John Graham-Cumming.
> >> "The case for open computer programs." Nature 482.7386 (2012): 485-488.
> >> http://www.nature.com/nature/journal/v482/n7386/full/nature10836.ht
> >> ml
> >>
> >> Best regards,
> >>
> >> Takanori Nakane
> >>
> >> (This is my personal opinion, and is not necessarily that of my
> >> employer or projects I am/was involved with.)
> >>
> >> On 2015/10/19 23:07, Kay Diederichs wrote:
> >>> Dear XDS users,
> >>>
> >>> a new package has been released for academic users; the download
> >>> link is at http://xds.mpimf-heidelberg.mpg.de/ .
> >>>
> >>> There is no longer a 32bit version for Linux, and there is only a
> >>> single package for OSX 10.6 and up. Those of you who still use
> >>> 32bit Linux or OSX 10.5 thus should consider upgrading their
> >>> operating system; the current (academic user) XDS package expires
> >>> in less than 6 months so there is still some time left!
> >>>
> >>> As always, if there are bugs/crashes/problems then pls tell
> >>> Wolfgang Kabsch or me about them; we can only fix bugs if we can
> >>> reproduce them so pls be prepared to share (confidentially) the
> >>> minimal amount of data with us that allows to reproduce the problem.
> >>>
> >>> best wishes,
> >>>
> >>> Kay
--
--
Paul Scherrer Institut
Dr. Tim Gruene
- persoenlich -
OFLC/102
CH-5232 Villigen PSI
phone: +41 (0)56 310 5297
GPG Key ID = A46BEE1A
