Over the past 12 months, there has been a major effort to re-write much of the CCP4 library. The aims are:
The "mmdb" library provides various high-level tools for working with coordinate files, which include not only reading and writing, but also orthogonal-fractional coordinate transforms, generation of symmetry mates, editing the molecular structure and some others. More information can be found on the mmdb project pages (http://msd.ebi.ac.uk/~keb/cldoc/).
File -> Crystal -> Dataset -> [ Datalist ] -> Column
A `Crystal' is essentially a single crystal form, a `Dataset' is a set of observations on a crystal, and a `Datalist' (not yet implemented) is a grouping of associated columns. Note that the `Project' used in Data Harvesting (Newsletter #37) is now simply an attribute of the crystal.
The MTZ file format has been extended slightly to record this heirarchy. I have written a C function library to read/write these extended MTZ files, and to manipulate a data structure representing the above data model. Some functions are derived from Jan-Pieter Abraham's solomon code, though they have been substantially altered (and therefore any problems are of my creation!).
One consequence of this formulation is that columns can now be identified in terms of their crystal or dataset, e.g.
LABIN FP=NATIVE/FTOXD3 SIGFP=NATIVE/SIGFTOXD3
I have also written a Fortran API to the C library, which mimics the existing mtzlib.f. It should be possible to migrate existing Fortran programs to use this API with few/no changes. It is expected however, that future applications will use the core C functions which give better access to the data structure.
Some information on cmtzlib is available from http://www.ccp4.ac.uk/martyn/cmtz.html, but further developments will be integrated into the CCP4 library development.
In a parallel development, there have been recent efforts to bring the MRC format for three-dimensional electron microscopy maps in to line with the closely-related CCP4 format map. For our part, we are considering suggested changes to the CCP4 map format which will be useful for EM applications. Some details can be found on the IIMS home page ( http://msd.ebi.ac.uk/iims.html).
In the new formulation, symop.lib is replaced by another data file which is automatically generated. This is currently done using a short program which uses functions from sgtbx (part of the Computational Crystallography Toolbox, http://cctbx.sourceforge.net) . The new data file is more likely to be error-free, and is also more complete, in that many non-standard settings can be included easily. The new data file contains most quantities of interest, and only a few pieces of tabulated data are retained in the code (e.g. specifications of centric and epsilon zones).
I have written a new C library of functions to manipulate this symmetry information. When a spacegroup is identified by its name, number or operators, all the information connected with that spacegroup is loaded into memory, where it can be accessed easily. Wrapper functions mimic the old symlib.f routines.
library_file.c ) library_err.c ) replacing library.c library_utils.c ) ccp4_general.c ) to give CCP4 look-and-feel ccp4_program.c ) ccp4_parser.c parser functions from Pete Briggs ccp4_diskio.c replacing diskio.fThese functions mostly support other code, but may be of use to application developers.
cmtzlib_python_wrap.c cmtzlib_perl_wrap.c cmtzlib_tcl_wrap.cprovide cmtzlib functions which can be called from python, perl and tcl respectively. These wrapper functions, together with the main library code, are compiled into shared libraries which can be loaded by the scripting language:
ccp4module.so loaded by from ccp4 import * ccp4_pl.so loaded by package ccp4_pl; ccp4_tcl.so loaded by load ./ccp4_tcl.so ccp4_tclMuch of the library functionality can be made available to the scripting language with virtually no effort. Access to complex datatypes requires a little more effort, as does wrapping C++ code, and this will be provided as required.
The formulation of this library has benefited from many discussions with Kevin Cowtan (who also provided some core functions), Eugene Krissinel, Airlie McCoy, and all members of the Daresbury team (Alun Ashton, Peter Briggs, Charles Ballard).