Processing Crystallography Data
Fully Automated Data Processing
fast_dp is a program which uses XDS, CCP4, and CCTBX to deliver data processing results very quickly. This will be especially useful for users who are looking for fast processing results to monitor how to use their beam time most effectively. However, it is also robust and intelligent enough to deliver publication-quality reduced data for most datasets.
fast_dp uses XDS for indexing and integration, then runs Pointless for space group determination. XDS is then rerun with the chosen space group, and the data scaled with Aimless.
ADX will run fast_dp on our dedicated processing server immediately after a run if Auto-process is turned on in the Run(s) window (see below). fast_dp output will be saved in the 'fast_dp' folder of the data directory. A typical run with fast_dp delivers reduced data in CCP4 .mtz format in three minutes or less.
Running fast_dp manually
fast_dp can be run manually from a terminal window. To see options, type:
$ fast_dp --help
xia2: Automated Processing with DIALS
xia2 is another automated processing routine which uses DIALS, an open-source processing package based on cctbx. The best aspect of xia2 is XTriage, which can identify potential problems in the dataset such as twinning, reflection outliers, missed symmetry, or missed anomalous signal.
xia2 is especially recommended for problematic data sets that may fail with other methods.
xia2 can be accessed on CHESS computers by entering `xia2` on a terminal window, which will return a list of options and parameters. Make sure to set reverse_phi=True when running xia2 with data collected at ID7B2.
A typical xia2 run on data from ID7B2 look something like this:
$ xia2 pipeline=dials-aimless reverse_phi=True /path/to/images
XDS is the preferred data reduction package for crystallographic data collected at CHESS.
The best method for generating the XDS.INP file is to use the program generate_XDS.INP, which is available on the CHESS network. From a terminal window, type:
$ generate_XDS.INP /path/to/file_master.h5 # for EIGER HDF5 data $ generate_XDS.INP /path/to/file.cbf # for PILATUS or other detector data
The program will automatically fill in the data collection parameters from the detector metadata, along with the best default settings.
XDS input file templates for MacCHESS detectors can be found here: XDS Input Files
Processing EIGER Data With XDS: Durin Plugin
In order to process EIGER HDF5 data with XDS, a library that can read the data into XDS is required. The Durin plugin for XDS reads EIGER HDF5 data efficiently and is recommended. The plugin is available for Linux and Mac. To use the library, add the following line to your XDS.INP file:
generate_XDS.INP and fast_dp insert the correct LIB= path for Durin for processing on the CHESS network automatically. You can download the Durin plugin for Mac and Linux for personal use here.
Please note that the NEGGIA plugin provided by Dectris no longer works with EIGER images taken after Dec 2020. Durin can process all EIGER data from any detector.
XDSGUI is a nice GUI for processing XDS data. It is especially useful for looking at plots of data statistics. It is available on the terminal by typing
HKL2000 and HKL (xdisp/denzo/scalepack)
HKL2000 and HKL (xdisp/denzo/scalepack), a graphical interface to Denzo, XdisplayF, Scalepack, and associated routines, is available, through a collaborative arrangement with HKL Research, on all MacCHESS computers. All detectors in use on MacCHESS beamlines are supported.
Experimenters from academic and non-commercial institutions may use the HKL programs for data evaluation and processing, while collecting data at the beamline. Users, holding a non-profit HKL license, can obtain, from HKL Research, Inc., the program upgrade to process data collected at CHESS.
Users who do not have a permanent HKL license can obtain, free of charge, a temporary license from HKL Research, Inc. to (re)process data collected at CHESS.
Users from commercial and for-profit institutions must have an appropriate commercial HKL license to use the program for data evaluation and processing, both when collecting data at the beamline and when processing data at home. The license can be obtained from HKL Research, Inc.
To launch HKL2000 from a terminal window:
A few copies of the HKL Manual are on hand at CHESS. An on-line version is also available.
CCP4/Mosflm Data Processing Suite
For interface to CCP4 programs & file display, from a terminal window:
$ ccp4i # for the 'classic' GUI $ ccp4i2 # for the 'new' GUI
iMosflm is the graphical user interface for the data reduction program Mosflm, and a part of the CCP4 suite of programs.
To launch iMosflm independently of CCP4 from a terminal window:
Phenix: Phasing and Refinement
phenix is a crystallographic program suite based on Python. It has powerful tools for both crystallographic and cryo-EM data, with an especially nice GUI.
For connecting to the phenix GUI, from a terminal window:
Laue processing software
coot is a popular model building and refinement tool. From a terminal window:
Fancy graphics: movies, rendering, and so much more..., type: