yes - prob. converted from lcf to mtz at some point.. Maybe on a VAX / E
On Wed, 14 Nov 2018 at 10:46, Harry Powell < 0000193323b1e616-dmarc-requ...@jiscmail.ac.uk> wrote: > Hi > > Okay, I can read that (sort of) with errors using a copy of mtzdmp from > ccp4 v 5.0, dated 23/01/04; I can tell that Ben Luisi's name is in there > (hello Ben!) and that the file was "converted to MTZ file mark 1 on 15/ > 5/92", inter alia. I can even tell that there are five columns and 17218 > reflections... > > Unfortunately, I get an error in reading the reflections themselves - > > LIST OF REFLECTIONS > =================== > > >>>>>> CCP4 library signal library_file:Bad mode (Error) > raised in ccp4_file_readfloat <<<<<< > >>>>>> System signal 22:Invalid argument (Error) > raised in ccp4_file_read <<<<<< > <B><FONT COLOR="#FF0000"><!--SUMMARY_BEGIN--> > MTZDUMP: Normal termination of mtzdump > Times: User: 0.0s System: 0.0s Elapsed: 0:00 > > > > My feeling is that all the discussion about VAXes etc may be irrelevant - > in my view it's more likely that the programs for reading and writing the > MTZ format changed (and the MTZ format itself changed as well) and are no > longer capable of dealing with "MTZ file mark 1"... > > If I can actually get the VAXes to which I referred to read an old DAT > tape, I may be able to build a copy of CCP4 from 1993... but that's a > problem for another day (anyone got a SCSI DDS drive I could plug into a > VAX?). > > On 14 Nov 2018, at 10:25, Eleanor Dodson wrote: > > Here is the file I was trying to read - please feel free to play with it!! > Eleanor > > On Wed, 14 Nov 2018 at 10:17, Harry Powell < > 0000193323b1e616-dmarc-requ...@jiscmail.ac.uk> wrote: > >> Hi >> >> Weren't the CCP4 base-level routines re-written from FORTRAN to C >> sometime in the late 1990's? Very occasionally I used to find bugs that had >> been introduced in this process (or possibly not corrected...) so it's >> possible that Eleanor's file might be readable with a really old code base. >> >> BTW, I have recently had cause to search out a VMS system and have found >> someone with a whole farm of running VAXes (or VAXen if you're geeky). >> >> Harry >> >> On 14 Nov 2018, at 09:46, Ian Tickle wrote: >> >> The CCP4 routines for MTZ and map files are written in C and thus do not >> use a Fortran unformatted OPEN statement, they use C-style block read & >> write. >> >> Cheers >> >> -- Ian >> >> >> On Wed, 14 Nov 2018 at 08:59, Kay Diederichs < >> kay.diederi...@uni-konstanz.de> wrote: >> >>> It is not necessary to do error-prone conversions manually: the ifort >>> Compiler understands the convert='VAXD' Option in its OPEN statement - see >>> >>> https://software.intel.com/en-us/fortran-compiler-developer-guide-and-reference-open-convert-specifier >>> >>> Thus one could just write a tiny read-write loop. >>> >>> HTH >>> Kay >>> >>> On Wed, 14 Nov 2018 00:51:02 +0000, Zhijie Li <zhijie...@utoronto.ca> >>> wrote: >>> >>> >It's also said here, at the end of file : >>> > >>> >https://www.cs.auckland.ac.nz/~patrice/210LN/DR4.pdf >>> > >>> >"add 1 to the left, with the binary point" >>> > >>> >0.10000..... >>> > >>> > >>> > >>> >________________________________ >>> >From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> on behalf of Zhijie >>> Li <zhijie...@utoronto.ca> >>> >Sent: Tuesday, November 13, 2018 7:43 PM >>> >To: CCP4BB@JISCMAIL.AC.UK >>> >Subject: Re: [ccp4bb] VERY old mtz file.. >>> > >>> > >>> >Hi all, >>> > >>> > >>> >I think I know why it is a division of 4 instead of 2 is involved in >>> conversion from VAX to IEEE now. Short answer: a 2 is in the exponent bits >>> (bias of 128 instead of 127, visible), another 2 is hidden in the >>> scientific notation. >>> > >>> > >>> >I found this explanation+example on VAX F-float: >>> > >>> >http://courseweb.stthomas.edu/tpsturm/private/notes/qm300/FLOATPT.html >>> > >>> > >>> >So for IEEE754 float32, if we want to represent a same 12.75 (1100.11) >>> in the above example, we would first conceptually write it in scientific >>> notation as 1.10011 x 1000 in binary. Then the mantissa part is the part >>> after the dot filled with zero to 23 bits: '10011000000000000000000', the >>> exponent part is 3+127=130 (dec)=10000010(bin). Then the binary IEEE754 >>> float32 number is 0[10000010][10011000000000000000000]. (You can check it >>> here: https://www.h-schmidt.net/FloatConverter/IEEE754.html) >>> > >>> > >>> >Now compare this with the VAX 12.75 in the linked example, we can find >>> that besides the bias becoming 128, the conceptual binary scientific >>> notation is actually 0.110011 x 10000, instead of 1.10011 x 1000. So the >>> exponent needed is 4 instead of 3. Then the exponent bits are >>> 4+128=132=10000100 and the VAX float32 becomes >>> > >>> >0[10000100][10011000000000000000000] ---if we write in a IEEE-style >>> order. Note that the mantissa appears to be same as the ieee mantissa, and >>> the exponent to be applied is 132-128=4. If this number is interpreted as >>> IEEE754, then it will be 1.10011 x 2exp(132-127)=1.10011 x 100000, four >>> times of what it should be. >>> > >>> > >>> >So, for normalised values, rearranging the VAX F-float bytes, reading >>> as IEEE, then dividing by 4 gives the correct value. (The C[0]-1 treatment >>> in the ccp4 lib is neat.) >>> > >>> > >>> >In this link describing VAX floats, it is unfortunate that it only >>> states that the bias for F-float is 128, but not that the mantissa starts >>> from 0.01 instead of 0.1. Therefore the confusion. >>> > >>> >https://nssdc.gsfc.nasa.gov/nssdc/formats/VAXFloatingPoint.htm >>> > >>> > >>> > >>> >Thanks to all responded! >>> > >>> > >>> >Zhijie >>> > >>> > >>> >________________________________ >>> >From: Ian Tickle <ianj...@gmail.com> >>> >Sent: Tuesday, November 13, 2018 4:54 PM >>> >To: Zhijie Li >>> >Cc: CCP4BB@JISCMAIL.AC.UK >>> >Subject: Re: [ccp4bb] VERY old mtz file.. >>> > >>> > >>> >Hi Zhijie >>> > >>> >It's definitely a factor 4. The code is in subroutine QTIEEE in the >>> Fortran source I mentioned previously at this line: >>> > >>> >See line: >>> > >>> > A(I)=((A(I)+SIGN(2,A(I)))/4.AND..NOT.MNAN).OR.MDN2 >>> > >>> >If you prefer it in C code it's in function vaxF2ieeeF in: >>> > >>> >ccp4-7.0-src/checkout/libccp4/ccp4/vmslibrary.c >>> > >>> >See line: >>> > >>> >out.c[0] = buffer[i].c[1] - (uint8)1; /* subtracts 2 from exponent */ >>> > >>> >i.e. subtract 2 from exponent -> division by 4. >>> > >>> >Cheers >>> > >>> >-- Ian >>> > >>> > >>> >On Tue, 13 Nov 2018 at 19:52, Zhijie Li <zhijie...@utoronto.ca<mailto: >>> zhijie...@utoronto.ca>> wrote: >>> >If somebody is going to send these files by email, please send one to >>> me too. Thanks in advance. I actually prefer to get a MTZ file because the >>> miller indices would serve as good clues for understanding the encodings. >>> Even the first 1024 bytes of an MTZ would do (data array starts at byte 80 >>> in MTZ). >>> > >>> >In my life I had only seen ieee754. According to what I can find, VAX >>> has an exponent bias of 128 (ieee754 uses 127). Then it seems to me that >>> when converting from vax to ieee a division of 2 is involved. However all >>> procedures I have seen use a division of 4, which is quite puzzling to me. >>> A real data file containing meaningful numbers (eg., HKL indices) would be >>> very helpful. Thanks in advance. >>> > >>> >Zhijie >>> > >>> >> On Nov 13, 2018, at 2:21 PM, Johan Hattne <hat...@ucla.edu<mailto: >>> hat...@ucla.edu>> wrote: >>> >> >>> >> Related by not exactly on topic: would anybody on the list be able to >>> share old map files (not MTZ:s) with Convex, Cray, Fujitsu, or VAX >>> reals/strings? I�d be interested to see what those files actually look(ed) >>> like. >>> >> >>> >> // Best wishes; Johan >>> >> >>> >>> On Nov 9, 2018, at 18:38, Zhijie Li <zhijie...@utoronto.ca<mailto: >>> zhijie...@utoronto.ca>> wrote: >>> >>> >>> >>> Hi all, >>> >>> >>> >>> On linux there are a few good GUI HEX editors. Here I�d like to >>> recommend BLESS, which conveniently displays all possible numerical >>> interpretations of the four bytes under cursor. It also allows the user to >>> switch between big endian or little endian through a checkbox. >>> Unfortunately all floats are assumed to be IEEE754, therefore VAX floats >>> won�t be interpreted correctly. ( The simplest way to convert vax to ieee >>> float would be to write a little program to do some bit operations. I�d be >>> happy to take that as my weekend project) >>> >>> >>> >>> >>> >>> BTW, along the line of space efficiency, I can�t help noticing that >>> the miller indices are saved as float32 in mtz, as all other numbers in >>> mtz. This certainly have made mtz format a beautiful homogeneous data >>> format ;). In this particular case, if we have doubts about the >>> reliability of the machine stamp, trying to restore the miller indices >>> would be a good way to test hypotheses. >>> >>> >>> >>> Zhijie >>> >>> >>> >>>> On Nov 9, 2018, at 9:04 PM, James Holton < >>> 0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk<mailto: >>> 0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk>> wrote: >>> >>>> >>> >>>> As a beamline scientist I must say I am glad that diffraction image >>> data is not usually stored as ASCII text. In fact, I am slowly warming to >>> the idea of storing it as not just binary, but compressed formats. >>> Problem, I'm sure will be that it won't be long before we forget how to >>> decompress them, as most of the algorithms we are using aren't all that >>> widespread. Probably around the same time future generations will curse us >>> for using ASCII instead of unicode, which is a 16-bit standard. I'm sure we >>> will be reviled for limiting ourselves so, just to save a factor of two in >>> disk space. >>> >>>> In situations like this I always use the unix "od" command. It >>> makes everything "human readable" by converting the bytes into strings you >>> can read. Then it is just a matter of figuring out what the bytes are. >>> >>>> Unfortunately, "od" only decodes floats on the native platform, so >>> if the mtz is from another platform (Windows vs Linux, for example), then >>> you might need to do some swapping. Thus far, I have encountered files >>> that require one of a few swapping strategies in order to make them work: >>> >>>> >>> >>>> 1 2 3 4 - no swapping >>> >>>> >>> >>>> 4 3 2 1 - reverse all bytes >>> >>>> >>> >>>> 3 4 1 2 - swap words and swap bytes within the words >>> >>>> 2 1 4 3 - reverse of previous >>> >>>> >>> >>>> 2-1 1 4 3 - same as last, but if not all zero, decrement byte #2 >>> before swapping >>> >>>> 3 4 1 2+1 - same as 3412, but if not all zero increment byte #2 >>> before swapping >>> >>>> I'm sure there are other combinations, but the oldest MTZ I have is >>> only from 1996. >>> >>>> >>> >>>> -James Holton >>> >>>> MAD Scientist >>> >>>> >>> >>>> >>> >>>>> On 11/9/2018 4:47 AM, Eleanor Dodson wrote: >>> >>>>> Anyone any idea what to do about this?? Created in 1992!! >>> >>>>> Seems unreadable.. >>> >>>>> >>> >>>>> No CTYP lines input for file: 1 >>> >>>>> Indices output even if all data items flagged "missing" >>> >>>>> Warning, NOT all LABOUT data lines given >>> >>>>> Warning: Machine stamp corrupted? Assuming native format. >>> >>>>>>>>>>> CCP4 library signal library_file:End of File (Error) >>> >>>>> >>> >>>>> >>> >>>>> To unsubscribe from the CCP4BB list, click the following link: >>> >>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>> >>>>> >>> >>>> >>> >>>> >>> >>>> To unsubscribe from the CCP4BB list, click the following link: >>> >>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>> >>>> >>> >>> >>> >>> To unsubscribe from the CCP4BB list, click the following link: >>> >>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>> >>> >>> >> >>> >> Research Specialist @ Gonen Lab >>> >> ____________________________________________________ >>> >> UCLA * 615 Charles E. Young Drive South >>> >> BSRB #347 * Los Angeles, CA 90095 >>> >> >>> >> >>> ######################################################################## >>> >> >>> >> To unsubscribe from the CCP4BB list, click the following link: >>> >> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>> > >>> >######################################################################## >>> > >>> >To unsubscribe from the CCP4BB list, click the following link: >>> >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>> > >>> >________________________________ >>> > >>> >To unsubscribe from the CCP4BB list, click the following link: >>> >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>> > >>> >######################################################################## >>> > >>> >To unsubscribe from the CCP4BB list, click the following link: >>> >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>> > >>> >>> ######################################################################## >>> >>> To unsubscribe from the CCP4BB list, click the following link: >>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>> >> >> ------------------------------ >> >> To unsubscribe from the CCP4BB list, click the following link: >> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> >> >> >> ------------------------------ >> >> To unsubscribe from the CCP4BB list, click the following link: >> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >> > > ------------------------------ > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > <semioxyhb_tstate_BetaCobalt.mtz> > > > > ------------------------------ > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
