Re: [ccp4bb] processing hd5 files from Dectris detector

[Folmer Fredslund]

Dear all,

With respect to the usage of the XDSAPP program, it depends on the
generate_XDS.INP script. For our local installation, I have edited the
script to include the "LIB=/path/to/dectrisneggia.so" in the resulting XDS
input file and this works as expected without any problems.
This does depend on knowing the location of the file on the filesystem, and
hence not really easily portable.

Nonetheless and easy and quick workaround for a local system.

Best regards


Folmer Fredslund
Staff scientist
DTU Biosustain


fre. 1. jun. 2018 11.04 skrev Harry Powell <
193323b1e616-dmarc-requ...@jiscmail.ac.uk>:

> Hi
>
> Laurent has contacted me off-board about this and sent me an example of a
> CBF file extracted from the HDF5.
>
> As far as I can see, there is no problem with the CBF file created by
> eiger2cbf - it can be processed correctly by iMosflm, and the miniCBF
> header is not missing any of the information that Mosflm requires.
>
> From this, I think that there's an issue with XDSAPP - so probably
> something for Manfred to have a look at!
>
> In answer to Kay's questions -
>
> (1) Massif-1
> (2) That would be my first port of call...
> (3) not tried yet...
> (4) the exact error messages start with what Laurent gave originally, i.e.
>  -
>
> Some important experimental parameters are missing from the images header
>
>
> which is not so useful, but there is an additional traceback -
>
> Some important experimental parameters are missing from the images header
> Traceback (most recent call last):
>  File "/mnt/maredsous/programs/bin/xdsit.py", line 4018, in 
>manual_process()
>  File "/mnt/maredsous/programs/bin/xdsit.py", line 3897, in manual_process
>manual_index()
>  File "/mnt/maredsous/programs/bin/xdsit.py", line 3885, in manual_index
>print(params['detector'])
> KeyError: ‘detector'
>
>
> Harry
> --
> Dr Harry Powell
> Chairman of European Crystallographic Association SIG9 (Crystallographic
> Computing)
>
>
>
>
> On 1 Jun 2018, at 07:21, Kay Diederichs wrote:
>
> Hi Laurent,
>
> this prompte a couple of questions -
> 1. where did you collect the data?
> 2. can this not be answered by the beamline staff? After all they should
> know their detector, and procedures for use of its data.
> 3. what are the exact error messages when using the neggia plugin? I guess
> Dectris would care if their plugin cannot read "their" data.
> 4. what are the exact error messages / missing items when using eiger2cbf
> / hdf2cbf-mini?
>
> best,
> Kay
>
> On Thu, 31 May 2018 14:58:52 +0200, maveyrau 
> wrote:
>
> Hi CCP4ers
>
>
> we recently collected many datasets on dectris detectors producing hd5
> files. I would like to use some auto processing tools to process them
> (xdsapp, xdsme…). As far as I can say, xdsapp or xdsme cannot process hd5
> natively. I tried to convert them to cbf format (eiger2cbf,
> hdf2mini-cbf,...), but then it seams that the header of the caf files are
> lacking some required informations…
>
>
> Any idea how to convert hd5 files to complete caf files ? Are there any
> plans for xdsapp to be able to work on hd5 files ?
>
>
> thanks for your help
>
>
> Laurent
>
> --
>
> Laurent Maveyraud laurent.maveyraud AT ipbs DOT fr
>
> P I C T   ---  Plateforme Intégrée de Criblage de Toulouse
>
> Université  Paul  Sabatier /  CNRS  /  I.P.B.S.  UMR  5089
>
> Département BiologieStructurale   et   Biophysique
>
> http://cribligand.ipbs.fr   http://www.ipbs.fr
>
> 205 route de Narbonne  31077 TOULOUSE Cedex FRANCE
>
> Tél: +33 (0)561 175 435   Mob.: +33 (0)646 042 111
>
> --
>
>
>
>
>
>
> To unsubscribe from the CCP4BB list, click the following link:
>
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>
>
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>
>
>
> --
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] processing hd5 files from Dectris detector

[Andreas Förster]

Dear Laurent,

sorry for getting into this discussion a bit late, but I was traveling.
Many good points have been raised in response to your question.  I'd like
to summarize and add to them.

1.  Your beamline scientist WILL know how to process the data, and if the
software she recommends is not what you're familiar with, she will also be
able to help you with your favorite software.  This should always be your
first port of call.

2.  HDF5 in itself doesn't pose problems to XDS.  If the HDF read library
Neggia that was mentioned in a few of the responses doesn't help XDS read
HDF5 files, let me and Kay Diederichs know.  We'll try to figure out the
problem.

3.  XDS pipelines (autoProc, xdsme, xdsgui, xds-app, etc) should be able to
process HDF5 data as long as they know where Neggia is located.

4.  HDF5 files written by the detector should all look the same (files
written when the detector was introduced looked slightly different).  Many
beamlines are now streaming the data off the detector and write the files
themselves.  Here, considerable variety in the metadata is possible.  We
are working on a validator for HDF5 metadata, but that's still work in
progress.

5.  Besides XDS, DIALS and HKL-2000 should process HDF5 files without
complaints.

6.  Conversion of HDF5 files into CBF is still required for processing with
iMosflm, but that's likely to change before too long.

7.  Our tool H5ToXds is technically not a converter but an extractor.  It
writes the images in headerless CBF, which is not a standard.  H5ToXds was
written with the singular purpose of helping XDS process HDF5 data by
presenting individual images as temporary CBFs upon request.  Since the
introduction of the plugin mechanism and Neggia, H5ToXds is obsolete.  If
you do need to create CBFs, eiger2cbf or hdf2mini-cbf should be used.  They
write miniCBF headers.  Best is to work with HDF5, though.

8.  The source code for eiger2cbf (https://github.com/biochem-fan/eiger2cbf)
doesn't just contain a CBF converter but also a plugin for XDS.  This
plugin does the same thing as Neggia but works even better under extreme
load (data from NSLS-II).

I'm happy to answer any questions that might remain.  Thanks to everyone
for answering.

All best.


Andreas




On Thu, May 31, 2018 at 8:58 AM, maveyrau  wrote:

> Hi CCP4ers
>
> we recently collected many datasets on dectris detectors producing hd5
> files. I would like to use some auto processing tools to process them
> (xdsapp, xdsme…). As far as I can say, xdsapp or xdsme cannot process hd5
> natively. I tried to convert them to cbf format (eiger2cbf,
> hdf2mini-cbf,...), but then it seams that the header of the caf files are
> lacking some required informations…
>
> Any idea how to convert hd5 files to complete caf files ? Are there any
> plans for xdsapp to be able to work on hd5 files ?
>
> thanks for your help
>
> Laurent
> --
> Laurent Maveyraud laurent.maveyraud AT ipbs DOT fr
> P I C T   ---  Plateforme Intégrée de Criblage de Toulouse
> Université  Paul  Sabatier /  CNRS  /  I.P.B.S.  UMR  5089
> Département BiologieStructurale   et   Biophysique
> http://cribligand.ipbs.fr   http://www.ipbs.fr
> 205 route de Narbonne  31077 TOULOUSE Cedex FRANCE
> Tél: +33 (0)561 175 435   Mob.: +33 (0)646 042 111
> --
>
>
>
>
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>



-- 

Andreas Förster, Ph.D.
Application Scientist Crystallography, Scientific Sales
Phone: +41 56 500 21 00 | Direct: +41 56 500 21 76 | Email:
andreas.foers...@dectris.com
DECTRIS Ltd. | Taefernweg 1 | 5405 Baden-Daettwil | Switzerland |
www.dectris.com

[image: LinkedIn] 
 [image: facebook]







Confidentiality Note: This message is intended only for the use of the
named
recipient(s) and may contain confidential and/or privileged information. If
you
are not the intended recipient, please contact the sender and delete the
message.
Any unauthorized use of the information contained in this message is
prohibited.



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] processing hd5 files from Dectris detector

[Didier Nurizzo]

Hi All,

I doubt it is on MASSIF-1, we still have a Pilatus3 2M, but I guess more 
on Massif-3 if collected at ESRF. People very often confuse between the 
two. Massif-1 is the fully automatic beamline and Massif-3 is the 
microfocus.


Cheers.

D


On 01/06/2018 11:03, Harry Powell wrote:

Hi

Laurent has contacted me off-board about this and sent me an example 
of a CBF file extracted from the HDF5.


As far as I can see, there is no problem with the CBF file created by 
eiger2cbf - it can be processed correctly by iMosflm, and the miniCBF 
header is not missing any of the information that Mosflm requires.


From this, I think that there's an issue with XDSAPP - so probably 
something for Manfred to have a look at!


In answer to Kay's questions -

(1) Massif-1
(2) That would be my first port of call...
(3) not tried yet...
(4) the exact error messages start with what Laurent gave originally, 
i.e.  -



Some important experimental parameters are missing from the images header


which is not so useful, but there is an additional traceback -


Some important experimental parameters are missing from the images header
Traceback (most recent call last):
 File "/mnt/maredsous/programs/bin/xdsit.py", line 4018, in 
   manual_process()
 File "/mnt/maredsous/programs/bin/xdsit.py", line 3897, in 
manual_process

   manual_index()
 File "/mnt/maredsous/programs/bin/xdsit.py", line 3885, in manual_index
   print(params['detector'])
KeyError: ‘detector'


Harry
--
Dr Harry Powell
Chairman of European Crystallographic Association SIG9 
(Crystallographic Computing)





On 1 Jun 2018, at 07:21, Kay Diederichs wrote:


Hi Laurent,

this prompte a couple of questions -
1. where did you collect the data?
2. can this not be answered by the beamline staff? After all they 
should know their detector, and procedures for use of its data.
3. what are the exact error messages when using the neggia plugin? I 
guess Dectris would care if their plugin cannot read "their" data.
4. what are the exact error messages / missing items when using 
eiger2cbf / hdf2cbf-mini?


best,
Kay

On Thu, 31 May 2018 14:58:52 +0200, maveyrau 
mailto:laurent.maveyr...@ipbs.fr>> wrote:



Hi CCP4ers

we recently collected many datasets on dectris detectors producing 
hd5 files. I would like to use some auto processing tools to process 
them (xdsapp, xdsme…). As far as I can say, xdsapp or xdsme cannot 
process hd5 natively. I tried to convert them to cbf format 
(eiger2cbf, hdf2mini-cbf,...), but then it seams that the header of 
the caf files are lacking some required informations…


Any idea how to convert hd5 files to complete caf files ? Are there 
any plans for xdsapp to be able to work on hd5 files ?


thanks for your help

Laurent
--
Laurent Maveyraud laurent.maveyraud AT ipbs DOT fr
P I C T   ---  Plateforme Intégrée de Criblage de Toulouse
Université  Paul  Sabatier /  CNRS  /  I.P.B.S.  UMR  5089
Département Biologie    Structurale   et   Biophysique
http://cribligand.ipbs.fr http://www.ipbs.fr
205 route de Narbonne  31077 TOULOUSE Cedex FRANCE
Tél: +33 (0)561 175 435   Mob.: +33 (0)646 042 111
--





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] processing hd5 files from Dectris detector

[Harry Powell]

Hi Didier

Sorry about that - my mistake - it's an Eiger 4M detector, so I guess that 
would be MASSIF-3.

Harry
--
Dr Harry Powell
Chairman of European Crystallographic Association SIG9 (Crystallographic 
Computing) 




On 1 Jun 2018, at 13:09, Didier NURIZZO wrote:

> Hi All,
> 
> I doubt it is on MASSIF-1, we still have a Pilatus3 2M, but I guess more on 
> Massif-3 if collected at ESRF. People very often confuse between the two 
> Massif-1 is the fully automatic beamline and Massif-3 is the microfocus.
> Cheers.
> 
> D
> 
> On 01/06/2018 11:03, Harry Powell wrote:
>> Hi
>> 
>> Laurent has contacted me off-board about this and sent me an example of a 
>> CBF file extracted from the HDF5.
>> 
>> As far as I can see, there is no problem with the CBF file created by 
>> eiger2cbf - it can be processed correctly by iMosflm, and the miniCBF header 
>> is not missing any of the information that Mosflm requires.
>> 
>> From this, I think that there's an issue with XDSAPP - so probably something 
>> for Manfred to have a look at!
>> 
>> In answer to Kay's questions - 
>> 
>> (1) Massif-1
>> (2) That would be my first port of call...
>> (3) not tried yet...
>> (4) the exact error messages start with what Laurent gave originally, i.e.  
>> - 
>> 
>>> Some important experimental parameters are missing from the images header
>> 
>> 
>> which is not so useful, but there is an additional traceback - 
>> 
>>> Some important experimental parameters are missing from the images header
>>> Traceback (most recent call last):
>>>  File "/mnt/maredsous/programs/bin/xdsit.py", line 4018, in 
>>>manual_process()
>>>  File "/mnt/maredsous/programs/bin/xdsit.py", line 3897, in manual_process
>>>manual_index()
>>>  File "/mnt/maredsous/programs/bin/xdsit.py", line 3885, in manual_index
>>>print(params['detector'])
>>> KeyError: ‘detector'
>> 
>> Harry
>> --
>> Dr Harry Powell
>> Chairman of European Crystallographic Association SIG9 (Crystallographic 
>> Computing) 
>> 
>> 
>> 
>> 
>> On 1 Jun 2018, at 07:21, Kay Diederichs wrote:
>> 
>>> Hi Laurent,
>>> 
>>> this prompte a couple of questions - 
>>> 1. where did you collect the data?
>>> 2. can this not be answered by the beamline staff? After all they should 
>>> know their detector, and procedures for use of its data.
>>> 3. what are the exact error messages when using the neggia plugin? I guess 
>>> Dectris would care if their plugin cannot read "their" data.
>>> 4. what are the exact error messages / missing items when using eiger2cbf / 
>>> hdf2cbf-mini?
>>> 
>>> best,
>>> Kay
>>> 
>>> On Thu, 31 May 2018 14:58:52 +0200, maveyrau  
>>> wrote:
>>> 
 Hi CCP4ers
 
 we recently collected many datasets on dectris detectors producing hd5 
 files. I would like to use some auto processing tools to process them 
 (xdsapp, xdsme…). As far as I can say, xdsapp or xdsme cannot process hd5 
 natively. I tried to convert them to cbf format (eiger2cbf, 
 hdf2mini-cbf,...), but then it seams that the header of the caf files are 
 lacking some required informations…
 
 Any idea how to convert hd5 files to complete caf files ? Are there any 
 plans for xdsapp to be able to work on hd5 files ?
 
 thanks for your help
 
 Laurent
 --
 Laurent Maveyraud laurent.maveyraud AT ipbs DOT fr
 P I C T   ---  Plateforme Intégrée de Criblage de Toulouse
 Université  Paul  Sabatier /  CNRS  /  I.P.B.S.  UMR  5089
 Département BiologieStructurale   et   Biophysique
 http://cribligand.ipbs.fr   http://www.ipbs.fr
 205 route de Narbonne  31077 TOULOUSE Cedex FRANCE
 Tél: +33 (0)561 175 435   Mob.: +33 (0)646 042 111
 --
 
 
 
 
 
 To unsubscribe from the CCP4BB list, click the following link:
 https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
 
>>> 
>>> To unsubscribe from the CCP4BB list, click the following link:
>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>> 
>> 
>> To unsubscribe from the CCP4BB list, click the following link:
>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>> 
> 

Re: [ccp4bb] processing hd5 files from Dectris detector

[Harry Powell]

Hi

Laurent has contacted me off-board about this and sent me an example of a CBF 
file extracted from the HDF5.

As far as I can see, there is no problem with the CBF file created by eiger2cbf 
- it can be processed correctly by iMosflm, and the miniCBF header is not 
missing any of the information that Mosflm requires.

From this, I think that there's an issue with XDSAPP - so probably something 
for Manfred to have a look at!

In answer to Kay's questions - 

(1) Massif-1
(2) That would be my first port of call...
(3) not tried yet...
(4) the exact error messages start with what Laurent gave originally, i.e.  - 

> Some important experimental parameters are missing from the images header


which is not so useful, but there is an additional traceback - 

> Some important experimental parameters are missing from the images header
> Traceback (most recent call last):
>  File "/mnt/maredsous/programs/bin/xdsit.py", line 4018, in 
>manual_process()
>  File "/mnt/maredsous/programs/bin/xdsit.py", line 3897, in manual_process
>manual_index()
>  File "/mnt/maredsous/programs/bin/xdsit.py", line 3885, in manual_index
>print(params['detector'])
> KeyError: ‘detector'

Harry
--
Dr Harry Powell
Chairman of European Crystallographic Association SIG9 (Crystallographic 
Computing) 




On 1 Jun 2018, at 07:21, Kay Diederichs wrote:

> Hi Laurent,
> 
> this prompte a couple of questions - 
> 1. where did you collect the data?
> 2. can this not be answered by the beamline staff? After all they should know 
> their detector, and procedures for use of its data.
> 3. what are the exact error messages when using the neggia plugin? I guess 
> Dectris would care if their plugin cannot read "their" data.
> 4. what are the exact error messages / missing items when using eiger2cbf / 
> hdf2cbf-mini?
> 
> best,
> Kay
> 
> On Thu, 31 May 2018 14:58:52 +0200, maveyrau  
> wrote:
> 
>> Hi CCP4ers
>> 
>> we recently collected many datasets on dectris detectors producing hd5 
>> files. I would like to use some auto processing tools to process them 
>> (xdsapp, xdsme…). As far as I can say, xdsapp or xdsme cannot process hd5 
>> natively. I tried to convert them to cbf format (eiger2cbf, 
>> hdf2mini-cbf,...), but then it seams that the header of the caf files are 
>> lacking some required informations…
>> 
>> Any idea how to convert hd5 files to complete caf files ? Are there any 
>> plans for xdsapp to be able to work on hd5 files ?
>> 
>> thanks for your help
>> 
>> Laurent
>> --
>> Laurent Maveyraud laurent.maveyraud AT ipbs DOT fr
>> P I C T   ---  Plateforme Intégrée de Criblage de Toulouse
>> Université  Paul  Sabatier /  CNRS  /  I.P.B.S.  UMR  5089
>> Département BiologieStructurale   et   Biophysique
>> http://cribligand.ipbs.fr   http://www.ipbs.fr
>> 205 route de Narbonne  31077 TOULOUSE Cedex FRANCE
>> Tél: +33 (0)561 175 435   Mob.: +33 (0)646 042 111
>> --
>> 
>> 
>> 
>> 
>> 
>> To unsubscribe from the CCP4BB list, click the following link:
>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1


To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] processing hd5 files from Dectris detector

[Kay Diederichs]

Hi Laurent,

this prompte a couple of questions - 
1. where did you collect the data?
2. can this not be answered by the beamline staff? After all they should know 
their detector, and procedures for use of its data.
3. what are the exact error messages when using the neggia plugin? I guess 
Dectris would care if their plugin cannot read "their" data.
4. what are the exact error messages / missing items when using eiger2cbf / 
hdf2cbf-mini?

best,
Kay

On Thu, 31 May 2018 14:58:52 +0200, maveyrau  wrote:

>Hi CCP4ers
>
>we recently collected many datasets on dectris detectors producing hd5 files. 
>I would like to use some auto processing tools to process them (xdsapp, 
>xdsme…). As far as I can say, xdsapp or xdsme cannot process hd5 natively. I 
>tried to convert them to cbf format (eiger2cbf, hdf2mini-cbf,...), but then it 
>seams that the header of the caf files are lacking some required informations…
>
>Any idea how to convert hd5 files to complete caf files ? Are there any plans 
>for xdsapp to be able to work on hd5 files ?
>
>thanks for your help
>
>Laurent
>--
>Laurent Maveyraud laurent.maveyraud AT ipbs DOT fr
>P I C T   ---  Plateforme Intégrée de Criblage de Toulouse
>Université  Paul  Sabatier /  CNRS  /  I.P.B.S.  UMR  5089
>Département BiologieStructurale   et   Biophysique
>http://cribligand.ipbs.fr   http://www.ipbs.fr
>205 route de Narbonne  31077 TOULOUSE Cedex FRANCE
>Tél: +33 (0)561 175 435   Mob.: +33 (0)646 042 111
>--
>
>
>
>
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>

To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] processing hd5 files from Dectris detector

[Aaron Finke]

Sometimes neggia has hiccups with uncompressed h5 images, in which case I 
recommend using eiger2cbf via H5ToXDS (instructions here: 
https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Eiger#A_script_for_faster_XDS_processing_of_CBF-converted_Eiger_data)

I have found that in most cases, using the above approach has comparable 
performance to neggia and does not have the compression compatibility issue.

Aaron

--
Aaron Finke
Staff Scientist, MacCHESS
Cornell University
e-mail: af...@cornell.edu<mailto:af...@cornell.edu>

On May 31, 2018, at 3:20 PM, SAVKO Martin 
mailto:martin.sa...@synchrotron-soleil.fr>> 
wrote:

Dear Laurent,

you can enable xdsme to process Eiger data in hdf5 format by giving it the path 
to neggia plugin. For example

xdsme -i "LIB= /path/to/dectris_neggia.so" collect_master.h5

This will work on both Linux and OSX/macOS provided the plugin has been 
compiled for the given platform.

You may get the dectris neggia plugin compiled for Linux and OSX on the links 
below

http://bit.ly/neggia_plugin_linux

http://bit.ly/neggia_plugin_osx

Source code is available at https://github.com/dectris/neggia.git

Kind regards,
Martin

--
Martin Savko

Synchrotron SOLEIL
PROXIMA 2A
L'Orme des Merisiers
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex
France

Office: +33 1 69 35 81 76
Beamline:  +33 1 69 35 81 80

martin.sa...@synchrotron-soleil.fr


De : CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] de la part de maveyrau 
[laurent.maveyr...@ipbs.fr]
Envoyé : jeudi 31 mai 2018 14:58
À : CCP4BB@JISCMAIL.AC.UK
Objet : [ccp4bb] processing hd5 files from Dectris detector

Hi CCP4ers

we recently collected many datasets on dectris detectors producing hd5 files. I 
would like to use some auto processing tools to process them (xdsapp, xdsme…). 
As far as I can say, xdsapp or xdsme cannot process hd5 natively. I tried to 
convert them to cbf format (eiger2cbf, hdf2mini-cbf,...), but then it seams 
that the header of the caf files are lacking some required informations…

Any idea how to convert hd5 files to complete caf files ? Are there any plans 
for xdsapp to be able to work on hd5 files ?

thanks for your help

Laurent
--
Laurent Maveyraud laurent.maveyraud AT ipbs DOT fr
P I C T   ---  Plateforme Intégrée de Criblage de Toulouse
Université  Paul  Sabatier /  CNRS  /  I.P.B.S.  UMR  5089
Département BiologieStructurale   et   Biophysique
http://cribligand.ipbs.fr   http://www.ipbs.fr
205 route de Narbonne  31077 TOULOUSE Cedex FRANCE
Tél: +33 (0)561 175 435   Mob.: +33 (0)646 042 111
--





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1


To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] processing hd5 files from Dectris detector

[SAVKO Martin]

Dear Laurent,

you can enable xdsme to process Eiger data in hdf5 format by giving it the path 
to neggia plugin. For example

xdsme -i "LIB= /path/to/dectris_neggia.so" collect_master.h5

This will work on both Linux and OSX/macOS provided the plugin has been 
compiled for the given platform.

You may get the dectris neggia plugin compiled for Linux and OSX on the links 
below

http://bit.ly/neggia_plugin_linux

http://bit.ly/neggia_plugin_osx

Source code is available at https://github.com/dectris/neggia.git

Kind regards,
Martin

--
Martin Savko

Synchrotron SOLEIL
PROXIMA 2A
L'Orme des Merisiers
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex
France

Office: +33 1 69 35 81 76
Beamline:  +33 1 69 35 81 80

martin.sa...@synchrotron-soleil.fr


De : CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] de la part de maveyrau 
[laurent.maveyr...@ipbs.fr]
Envoyé : jeudi 31 mai 2018 14:58
À : CCP4BB@JISCMAIL.AC.UK
Objet : [ccp4bb] processing hd5 files from Dectris detector

Hi CCP4ers

we recently collected many datasets on dectris detectors producing hd5 files. I 
would like to use some auto processing tools to process them (xdsapp, xdsme…). 
As far as I can say, xdsapp or xdsme cannot process hd5 natively. I tried to 
convert them to cbf format (eiger2cbf, hdf2mini-cbf,...), but then it seams 
that the header of the caf files are lacking some required informations…

Any idea how to convert hd5 files to complete caf files ? Are there any plans 
for xdsapp to be able to work on hd5 files ?

thanks for your help

Laurent
--
Laurent Maveyraud laurent.maveyraud AT ipbs DOT fr
P I C T   ---  Plateforme Intégrée de Criblage de Toulouse
Université  Paul  Sabatier /  CNRS  /  I.P.B.S.  UMR  5089
Département BiologieStructurale   et   Biophysique
http://cribligand.ipbs.fr   http://www.ipbs.fr
205 route de Narbonne  31077 TOULOUSE Cedex FRANCE
Tél: +33 (0)561 175 435   Mob.: +33 (0)646 042 111
--





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] processing hd5 files from Dectris detector

[Raghurama P Hegde]

Hi, 

Dectris has a plugin, neggia, which they say allows the use of HDF5 format 
files directly in XDS without interconversion: 
https://www.dectris.com/company/news/newsroom/news-details/process-eiger-data-with-xds-fast
 

Cheers, 
Raghurama 

-Original Message-
From: CCP4 bulletin board  On Behalf Of maveyrau
Sent: Thursday, May 31, 2018 18:29
To: CCP4BB@JISCMAIL.AC.UK
Subject: [ccp4bb] processing hd5 files from Dectris detector

Hi CCP4ers

we recently collected many datasets on dectris detectors producing hd5 files. I 
would like to use some auto processing tools to process them (xdsapp, xdsme…). 
As far as I can say, xdsapp or xdsme cannot process hd5 natively. I tried to 
convert them to cbf format (eiger2cbf, hdf2mini-cbf,...), but then it seams 
that the header of the caf files are lacking some required informations…

Any idea how to convert hd5 files to complete caf files ? Are there any plans 
for xdsapp to be able to work on hd5 files ?

thanks for your help

Laurent
--
Laurent Maveyraud laurent.maveyraud AT ipbs DOT fr
P I C T   ---  Plateforme Intégrée de Criblage de Toulouse
Université  Paul  Sabatier /  CNRS  /  I.P.B.S.  UMR  5089
Département BiologieStructurale   et   Biophysique
http://cribligand.ipbs.fr   http://www.ipbs.fr
205 route de Narbonne  31077 TOULOUSE Cedex FRANCE
Tél: +33 (0)561 175 435   Mob.: +33 (0)646 042 111
--





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1


To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] processing hd5 files from Dectris detector

[Clemens Vonrhein]

Dear Laurent,

On Thu, May 31, 2018 at 02:58:52PM +0200, maveyrau wrote:
> we recently collected many datasets on dectris detectors producing
> hd5 files. I would like to use some auto processing tools to process
> them (xdsapp, xdsme…). As far as I can say, xdsapp or xdsme cannot
> process hd5 natively. I tried to convert them to cbf format
> (eiger2cbf, hdf2mini-cbf,...), but then it seams that the header of
> the caf files are lacking some required informations…

Since you already mention "hdf2mini-cbf" from autoPROC, why not use
autoPROC itself for doing the actual processing:

  process -h5 /where/ever/your_master.h5 ...

See also [1] and [2] for more information.

> Any idea how to convert hd5 files to complete cdf files?

hdf2mini-cbf [3] should create CBF files that are as compatible with
original Pilatus mini-CBF files as possible: at least we tried very
hard to achieve this in it's original design. If this doesn't work in
your particular case we would be very interested to have access to
some example files from the beamline(s) you collected on - maybe there
are changes in the HDF5 file(s) we aren't aware of (yet). Keeping up
with changes to detectors or beamlines is always tricky, unless
developers are being made aware of this ;-)

Cheers

Clemens

[1] www.globalphasing.com/autoproc/wiki/index.cgi?DataProcessingHdf5
[2] http://www.globalphasing.com/autoproc/manual/autoproc_reference_card.pdf
[3] 
www.globalphasing.com/autoproc/wiki/index.cgi?DataProcessingHdf5#hdf5converter

To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] processing hd5 files from Dectris detector

[Harry Powell]

Hi

The first thing I would try is to process the HDF5 files directly with DIALS, 
which reads several different "dialects" of HDF5 (the wonderful thing about a 
standard format is that everyone implements their own idea of a standard...).

If that doesn't work, I think you would need to provide more information about 
what is actually missing from the cbf files produced from the HDF5 files - 
"lacking some required information" doesn't really tell us anything very useful!

Where were the HDF5 files collected? It could be that the beamline staff need 
to make sure that the HDF5 files contain all the information so that the 
conversion programs can find it!

Harry
--
Dr Harry Powell
Chairman of European Crystallographic Association SIG9 (Crystallographic 
Computing) 




On 31 May 2018, at 13:58, maveyrau wrote:

> Hi CCP4ers
> 
> we recently collected many datasets on dectris detectors producing hd5 files. 
> I would like to use some auto processing tools to process them (xdsapp, 
> xdsme…). As far as I can say, xdsapp or xdsme cannot process hd5 natively. I 
> tried to convert them to cbf format (eiger2cbf, hdf2mini-cbf,...), but then 
> it seams that the header of the caf files are lacking some required 
> informations…
> 
> Any idea how to convert hd5 files to complete caf files ? Are there any plans 
> for xdsapp to be able to work on hd5 files ?
> 
> thanks for your help
> 
> Laurent
> --
> Laurent Maveyraud laurent.maveyraud AT ipbs DOT fr
> P I C T   ---  Plateforme Intégrée de Criblage de Toulouse
> Université  Paul  Sabatier /  CNRS  /  I.P.B.S.  UMR  5089
> Département BiologieStructurale   et   Biophysique
> http://cribligand.ipbs.fr   http://www.ipbs.fr
> 205 route de Narbonne  31077 TOULOUSE Cedex FRANCE
> Tél: +33 (0)561 175 435   Mob.: +33 (0)646 042 111
> --
> 
> 
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1


To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

[ccp4bb] processing hd5 files from Dectris detector

[maveyrau]

Hi CCP4ers

we recently collected many datasets on dectris detectors producing hd5 files. I 
would like to use some auto processing tools to process them (xdsapp, xdsme…). 
As far as I can say, xdsapp or xdsme cannot process hd5 natively. I tried to 
convert them to cbf format (eiger2cbf, hdf2mini-cbf,...), but then it seams 
that the header of the caf files are lacking some required informations…

Any idea how to convert hd5 files to complete caf files ? Are there any plans 
for xdsapp to be able to work on hd5 files ?

thanks for your help

Laurent
--
Laurent Maveyraud laurent.maveyraud AT ipbs DOT fr
P I C T   ---  Plateforme Intégrée de Criblage de Toulouse
Université  Paul  Sabatier /  CNRS  /  I.P.B.S.  UMR  5089
Département BiologieStructurale   et   Biophysique
http://cribligand.ipbs.fr   http://www.ipbs.fr
205 route de Narbonne  31077 TOULOUSE Cedex FRANCE
Tél: +33 (0)561 175 435   Mob.: +33 (0)646 042 111
--





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1


smime.p7s
Description: S/MIME cryptographic signature