But Don't Panic!. All your favourite programs, and others, will continue to be supported. However, with the new framework in place, you will increasingly find new tools, more automation and better data management.
A preliminary account of the new software library was given in Newsletter 40. Therefore, this article will concentrate on recent progress and plans for the next CCP4 release.
A Fortran-callable interface, built on MMDB, replicates the functionality of the old rwbrook.f library file. MMDB also provides model handling for the CCP4 Molecular Graphics project (E. Potterton, S. McNicholas, E. Krissinel, K. Cowtan and M. Noble, Acta Cryst. D58, 1955-1957 (2002)).
More information can be found on the MMDB project pages (http://msd.ebi.ac.uk/~keb/cldoc/).
File -> Crystal -> Dataset -> Column -> Reflection dataA `Crystal' is essentially a single crystal form, while a `Dataset' is a set of observations on a crystal. Note that the `Project' used in Data Harvesting (Newsletter #37) is now simply an attribute of the crystal.
The MTZ file format has been extended slightly to record this heirarchy. CMTZ is a C function library to read/write these extended MTZ files, and to manipulate a data structure representing the above data model. A Fortran-callable interface to CMTZ replicates the functionality of the old mtzlib.f library file.
Older MTZ files will lack the Crystal level of the hierarchy. The new library will assume that each project consists of a single crystal, unless different cell dimensions (recorded for each dataset) indicate the presence of different crystals. As with the earlier introduction of dataset information, it is important that the user establish the correct data model at an early stage, for example by the correct labelling of datasets in MOSFLM. Given a correct data model, software downstream can infer appropriate relationships and thus work in a more automated manner.
The CMTZ library can now work in two modes, one which holds all reflection data in memory, and one which leaves the reflection data on disk for sequential processing. The latter is the traditional method used by Fortran programs, but the former is likely to be more useful for newer applications. The mode can be selected by an environment variable CMTZ_IN_MEMORY.
Recent work has concentrated on testing the Fortran interface and ensuring robust support for existing programs. When this work is completed, our attention will turn to providing new and improved tools. An early target is MTZ file handling in ccp4i which is currently done by interpreting MTZDUMP output. The tcl interface to the new library enables direct access to the MTZ data structure. This is both more robust and allows more advanced graphical manipulation of MTZ files.
In the new formulation, symop.lib is replaced by another data file syminfo.lib which is automatically generated. This is currently done using a short program which uses functions from sgtbx (part of the Computational Crystallography Toolbox, http://cctbx.sourceforge.net) . The new data file is more likely to be error-free, and is also more complete, in that many non-standard settings can be included easily. The new data file contains most quantities of interest, and only a few pieces of tabulated data are retained in the code (e.g. specifications of centric and epsilon zones).
The new CCP4 library contains C functions to manipulate this symmetry information. When a spacegroup is identified by its name, number or operators, all the information connected with that spacegroup is loaded into memory, where it can be accessed easily. Wrapper functions mimic the old symlib.f routines. Recently added functionality includes:
The library also retains various Fortran subroutine libraries where conversion is not appropriate or has not yet been attempted, e.g. certain routines in ccplib.f and all of plot84driver.f
On the other hand, for developers CCP4 5.0 will provide a more powerful environment for writing applications and complex tasks. The new libraries, together with Clipper and FFTw, provide functionality for writing applications in C++, C or Fortran. In addition, much of this functionality is available to scripts written in python, tcl or perl using the SWIG-generated programming interfaces. Makefiles to be distributed with CCP4 allow the generation of shared libraries which then form loadable modules for a scripting environment.
At the moment, ccp4i executes a job as a separate process running a wish script. From CCP4 5.0, ccp4i will also be able to execute python scripts, with job parameters being saved in the database as usual. With the object-orientated capabilities of python, this allows the creation of more sophisticated, data-orientated tasks within the familiar user environment of ccp4i.