Hi Phil,
I agree that the issue that you raise (about the need to define the data
items and categories propery) is an important one that needs proper
consideration. However, your mail could be read to suggest that correct
parsing of CIF-format data is a secondary issue that doesn't deserve the
same attention from developers.
I hope that this isn't quite what you meant.... There are already
mutually-incompatible CIF dialects out there that have been created by
developers coding to their own understanding and interpretations of the
CIF/STAR format. I am sure that you would not want to be the creator of yet
another one :-) Correct tokenising is a necessary (but not sufficient)
condition for preventing the problem getting worse.
In practice, the code and applications that I have seen, and the discussions
about this that I have had, all suggest that developers find it more
difficult to write code that tokenises CIF/STAR-format data correctly than
code that handles other text formats that they have to deal with in this
field. My experience suggests that this is an important practical issue with
real-world ramifications, and it is worthwhile devoting some effort to it.
Regards,
Peter.
On Wed, 18 Sep 2013, Phil Evans wrote:
Date: Wed, 18 Sep 2013 13:38:07 +0100
From: Phil Evans <[email protected]>
To: [email protected]
Subject: Re: [ccp4bb] Code to handle the syntax of (mm)CIF data correctly.
As a novice looking at mmCIF from a developers point of view, for
reflection data, the complication is not so much tokenising (parsing), but
what items to write or to expect to read. For example as far as I can see
an observed intensity may be encoded in a reflection loop (merged or
unmerged) as any one of the following, and there seem to be similar
choices for other items:-
_refln_intensity_meas
_refln.F_squared_meas
_refln.pdbx_I_plus, _refln.pdbx_I_minus
_diffrn_refln.counts_net
_diffrn_refln.intensity_net
If I'm writing a file, which should I use, and if I'm reading one which ones
should I expect? And is there a distinction between merged and unmerged data?
confused (easily)
Phil
On 17 Sep 2013, at 15:30, Peter Keller <[email protected]> wrote:
Dear all,
At Global Phasing, we have seen that there are still issues with the way that
different applications deal with mmCIF-format data, and this continues to cause
problems for users. I believe that part of the reason for this is that the
underlying syntax (the STAR format) is not universally understood, and that a
common and complete understanding of the full STAR syntax amongst programmers
who deal with the format will help with some of the existing problems.
I wrote some code for low-level handling of the STAR format a while ago that I
have been meaning to release for over a year. Garry Battle's announcement on 23
August about the mmCIF/PDBx workshop at the EBI has prompted me into action: I
have written a short article that discusses some examples of the issues that we
have encountered, and made my code available for download. The references in
the article are given primarily as web links: more conventional citations can
usually be found in the pages that I link to. This code has not been used in
any released products, but it has had some internal use at Global Phasing.
There is an MX bias in the article's discussion, but the issues are not
restricted to MX.
As I explain in the article, the handling of the input data is based on an
enourmous regular expression that matches STAR data, with only a little logic
in the code itself. The regular expression should be usable with a variety of
other languages, not only in Java (which I have used in this case). The code,
or the regular expression on its own, may be freely used in other projects: see
the included licencing for details, but basically you should: (i) give credit
for using it, and (ii) if you choose to modify the regular expression, state
that you have done so in that credit.
The article, which contains links to a tar file containing the code, and the
documentation, is here:
<http://www.globalphasing.com/startools/>
Hoping that others will find this useful and/or help to resolve or clarify
outstanding questions,
Peter.
--
Peter Keller Tel.: +44 (0)1223 353033
Global Phasing Ltd., Fax.: +44 (0)1223 366889
Sheraton House,
Castle Park,
Cambridge CB3 0AX
United Kingdom
--
Peter Keller Tel.: +44 (0)1223 353033
Global Phasing Ltd., Fax.: +44 (0)1223 366889
Sheraton House,
Castle Park,
Cambridge CB3 0AX
United Kingdom
