MTZ was always 32 bit floats for the main data, with ASCII headers at the end
Sent from my iPhone > On 13 Nov 2018, at 21:29, Ethan A Merritt <merr...@u.washington.edu> wrote: > >> On Tuesday, November 13, 2018 1:06:01 PM PST Zhijie Li wrote: >> Hi Ethan, >> Thanks for the information. My guess is that in MTZ only F-float is >> expected, because it is the only 32bit form? > > I do not remember exactly what was used for mtz files at that time. > It might have been REAL*4 or it might have been REAL*8. > <\me rummages along the shelf above my desk> > Looking here at my trusty VAX-11 Fortran manual from April 1982, > D_floating was the default for REAL*8. > > F_floating: 1 bit sign, 8 bit exponent, 23 bit mantissa > D_floating: 1 bit sign, 8 bit exponent, 55 bit mantissa > G_floating: 1 bit sign, 11 bit exponent, 52 bit mantissa > > Later on they introduced H_floating, S_floating, T_floating and probably an > entire > zoo that went extinct shortly after. > > Bit assignments: > https://nssdc.gsfc.nasa.gov/nssdc/formats/VAXFloatingPoint.htm > > > Ethan > > >> Zhijie >> >>>> On Nov 13, 2018, at 3:44 PM, Ethan A Merritt <merr...@u.washington.edu> >>>> wrote: >>>> >>>> On Tuesday, November 13, 2018 11:51:55 AM PST Zhijie Li wrote: >>>> If somebody is going to send these files by email, please send one to me >>>> too. Thanks in advance. I actually prefer to get a MTZ file because the >>>> miller indices would serve as good clues for understanding the encodings. >>>> Even the first 1024 bytes of an MTZ would do (data array starts at byte 80 >>>> in MTZ). >>>> >>>> In my life I had only seen ieee754. According to what I can find, VAX has >>>> an exponent bias of 128 (ieee754 uses 127). Then it seems to me that when >>>> converting from vax to ieee a division of 2 is involved. >>> >>> It's more complicated than that. VAXen supported multiple floating point >>> formats, >>> F-floating G-floating and H-floating. >>> They had differed by how many bits were used for the exponent, and hence how >>> many bits were left for the mantissa. >>> I can pull out the architecture manuals if necessary. >>> >>> ah, nostalgia >>> >>> Ethan >>> >>> >>>> However all procedures I have seen use a division of 4, which is quite >>>> puzzling to me. A real data file containing meaningful numbers (eg., HKL >>>> indices) would be very helpful. Thanks in advance. >>>> >>>> Zhijie >>>> >>>>> On Nov 13, 2018, at 2:21 PM, Johan Hattne <hat...@ucla.edu> wrote: >>>>> >>>>> Related by not exactly on topic: would anybody on the list be able to >>>>> share old map files (not MTZ:s) with Convex, Cray, Fujitsu, or VAX >>>>> reals/strings? I’d be interested to see what those files actually >>>>> look(ed) like. >>>>> >>>>> // Best wishes; Johan >>>>> >>>>>> On Nov 9, 2018, at 18:38, Zhijie Li <zhijie...@utoronto.ca> wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> On linux there are a few good GUI HEX editors. Here I’d like to >>>>>> recommend BLESS, which conveniently displays all possible numerical >>>>>> interpretations of the four bytes under cursor. It also allows the user >>>>>> to switch between big endian or little endian through a checkbox. >>>>>> Unfortunately all floats are assumed to be IEEE754, therefore VAX floats >>>>>> won’t be interpreted correctly. ( The simplest way to convert vax to >>>>>> ieee float would be to write a little program to do some bit operations. >>>>>> I’d be happy to take that as my weekend project) >>>>>> >>>>>> >>>>>> BTW, along the line of space efficiency, I can’t help noticing that the >>>>>> miller indices are saved as float32 in mtz, as all other numbers in mtz. >>>>>> This certainly have made mtz format a beautiful homogeneous data format >>>>>> ;). In this particular case, if we have doubts about the reliability of >>>>>> the machine stamp, trying to restore the miller indices would be a good >>>>>> way to test hypotheses. >>>>>> >>>>>> Zhijie >>>>>> >>>>>>> On Nov 9, 2018, at 9:04 PM, James Holton >>>>>>> <0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk> wrote: >>>>>>> >>>>>>> As a beamline scientist I must say I am glad that diffraction image >>>>>>> data is not usually stored as ASCII text. In fact, I am slowly warming >>>>>>> to the idea of storing it as not just binary, but compressed formats. >>>>>>> Problem, I'm sure will be that it won't be long before we forget how >>>>>>> to decompress them, as most of the algorithms we are using aren't all >>>>>>> that widespread. Probably around the same time future generations will >>>>>>> curse us for using ASCII instead of unicode, which is a 16-bit >>>>>>> standard. I'm sure we will be reviled for limiting ourselves so, just >>>>>>> to save a factor of two in disk space. >>>>>>> In situations like this I always use the unix "od" command. It makes >>>>>>> everything "human readable" by converting the bytes into strings you >>>>>>> can read. Then it is just a matter of figuring out what the bytes are. >>>>>>> Unfortunately, "od" only decodes floats on the native platform, so if >>>>>>> the mtz is from another platform (Windows vs Linux, for example), then >>>>>>> you might need to do some swapping. Thus far, I have encountered files >>>>>>> that require one of a few swapping strategies in order to make them >>>>>>> work: >>>>>>> >>>>>>> 1 2 3 4 - no swapping >>>>>>> >>>>>>> 4 3 2 1 - reverse all bytes >>>>>>> >>>>>>> 3 4 1 2 - swap words and swap bytes within the words >>>>>>> 2 1 4 3 - reverse of previous >>>>>>> >>>>>>> 2-1 1 4 3 - same as last, but if not all zero, decrement byte #2 before >>>>>>> swapping >>>>>>> 3 4 1 2+1 - same as 3412, but if not all zero increment byte #2 before >>>>>>> swapping >>>>>>> I'm sure there are other combinations, but the oldest MTZ I have is >>>>>>> only from 1996. >>>>>>> >>>>>>> -James Holton >>>>>>> MAD Scientist >>>>>>> >>>>>>> >>>>>>>> On 11/9/2018 4:47 AM, Eleanor Dodson wrote: >>>>>>>> Anyone any idea what to do about this?? Created in 1992!! >>>>>>>> Seems unreadable.. >>>>>>>> >>>>>>>> No CTYP lines input for file: 1 >>>>>>>> Indices output even if all data items flagged "missing" >>>>>>>> Warning, NOT all LABOUT data lines given >>>>>>>> Warning: Machine stamp corrupted? Assuming native format. >>>>>>>>>>>>>> CCP4 library signal library_file:End of File (Error) >>>>>>>> >>>>>>>> >>>>>>>> To unsubscribe from the CCP4BB list, click the following link: >>>>>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> To unsubscribe from the CCP4BB list, click the following link: >>>>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>>>>>> >>>>>> >>>>>> To unsubscribe from the CCP4BB list, click the following link: >>>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>>>>> >>>>> >>>>> Research Specialist @ Gonen Lab >>>>> ____________________________________________________ >>>>> UCLA * 615 Charles E. Young Drive South >>>>> BSRB #347 * Los Angeles, CA 90095 >>>>> >>>>> ######################################################################## >>>>> >>>>> To unsubscribe from the CCP4BB list, click the following link: >>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>>> >>>> ######################################################################## >>>> >>>> To unsubscribe from the CCP4BB list, click the following link: >>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 >>>> >>> >>> >>> -- >>> Ethan A Merritt >>> Biomolecular Structure Center, K-428 Health Sciences Bldg >>> MS 357742, University of Washington, Seattle 98195-7742 >>> >>> >>> >> > > > -- > Ethan A Merritt > Biomolecular Structure Center, K-428 Health Sciences Bldg > MS 357742, University of Washington, Seattle 98195-7742 > > ######################################################################## > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
