New Developments in CCP4i: December 2007 CCP4 Newsletter 47 Winter 2007

New Developments in CCP4i: December 2007

Peter Briggs

CCP4, CSE Department, STFC Daresbury Laboratory, Warrington WA4 4AD

1 Introduction

CCP4i [Potterton03] is the CCP4 graphical user interface. This article is intended to give an overview of some of the developments in CCP4i which will be available in the next release of the CCP4 software suite (version 6.1). The article is divided into sections outlining the new task interfaces, new utilities and the other major and minor new features scheduled for the next version of CCP4i.

Details of features in the current version of CCP4i can be found in a previous article in newsletter 42.

2 New Task Interfaces

There are a number of new task interfaces scheduled for the next release of CCP4i. Many of these correspond to the introduction of major new packages in the main suite, while others provide access to programs in the suite that have previously not had a graphical interface. A number of these interfaces are shown in figure 1.

Figure 1: CCP4i surrounded by some of the new task interfaces (XIA2, Pointless, MrBUMP and PISA)

Other new tasks for existing programs

A number of new tasks have been added to provide access to existing programs. These include interfaces for the CHOOCH and DYNDOM programs (which previously haven't had interfaces at all) and for the new experimental phasing functionality in Phaser 2.1.1.

3 New Utilities

Two new utilities are planned for inclusion in the next version of CCP4i.

3.1 idiffdisp: diffraction image viewer

idiffdisp is a standalone viewer for raw diffraction images (figure 2). It has been developed by Francois Remacle and which is intended as a replacement for the old IPDISP program. idiffdisp is more fully-featured than IPDISP (for example it includes options to "play" a set of images like a movie, which useful for rapid visual inspection to find anomalous images) and also runs on Windows as well as Linux and UNIX systems.

idiffdisp has been described in a previous newsletter article:

Figure 2: Screenshot showing the idiffdisp viewer displaying an X-ray diffraction image

3.2 dbviewer: a visualiser for CCP4i project history

dbviewer provides graphical views of jobs within a CCP4i project, giving a perspective on the project history and the flow of data (see figure 3). It is a component of the database handler system (see section 4.2 below) and may already be familiar to people who have used more recent versions of MrBUMP. Within the next version of CCP4i the viewer will provide views of any of the user's projects as a complement to the existing joblist view in the main CCP4i window.

The viewer has previously been described in a recent newsletter article:

Figure 3: The dbviewer showing a view of the jobs in a CCP4i project. Each box represents a job, with the arrows linking jobs where the output of one is used as the input for another.

4 Major New Features

A number of new features have been implemented in the next version of CCP4i.

4.1 Better integration with Coot and CCP4mg ("plugins")

Although increasingly powerful tools have become available over the last few years, users of CCP4i have increasingly suffered from the lack of integration between CCP4i and many of these tools - most significantly, when wishing to access model building tools via Coot [Emsley04] and visualisation facilities via CCP4mg [Potterton02].

To begin to address this, in the next release of CCP4i the "View Files from Job" menu will include new options from certain tasks (most notably Refmac5 [Murshudov96]) to view the results in Coot or CCP4mg (provided that the programs themselves are installed and available). An example of how these shortcuts appear in the menu is shown in figure 4.

Figure 4: "View Files from Job" menu offering options to view the results of a Refmac5 run in Coot and CCP4mg directly.

Selecting these options will launch the chosen application with the files preloaded - for example, from Refmac5 Coot or CCP4mg will be started with both the appropriate MTZ and coordinate files already selected.

Although still fairly limited, the aim of this integration is to make it much easier to move from CCP4i into the appropriate tool. Feedback or suggestions on other tools or tasks where this mechanism could be added would be gratefully received.

4.2 Integration with iMOSFLM

iMOSFLM is the new interface for the MOSFLM data processing and integration program [Leslie92], which improves vastly over the old X-windows based interface. The CCP4 6.1 will include iMOSFLM and CCP4i will provide a button to launch it.

The development of iMOSFLM will ultimately make the current MOSFLM-in-batch CCP4i task interface obsolete. However the older interface will be retained for now and should still be available in the next CCP4i alongside the iMOSFLM option.

More information and downloads of iMosflm can be found at The program is also described in an earlier newsletter article.

4.3 Improved presentation of tasklists and modules

In previous (and current) versions of CCP4i, the only forms of organisation for the tasklists (the sets of tasks available in the menus on the left hand side of the main CCP4i window) have been a division into separate modules, and the ordering of tasks within those modules.

Traditionally, tasks appropriate for a particular part of the structure determination are grouped together in a module, for example "data reduction" or "molecular replacement". But an increasingly frequent criticism of this presentation is that it can difficult for novice (and sometimes for experts) to see which tasks are related, what order they could be used in, and which tasks are alternatives to each other. In an attempt to mitigate this criticism, the next version of CCP4i will see the incorporation of an extra level of organisation within modules, which allows tasks to also be grouped into "subfolders".

Some examples of the new arrangements are shown in figure 5. Although the changes to allow subfolders have been made to the CCP4i code, some work still remains to be done on the actual reorganisation itself and feedback from users would be very valuable.

Figure 5: examples of the suggested reorganised tasklists for (from left to right respectively) "Experimental Phasing", "Molecular Replacement" and "Refinement" modules, with open and closed subfolders.

4.4 Integration with database handler system "dbccp4i"

Since 2004 there has been an ongoing project to develop the job database in CCP4i under the auspices of the European Union BIOXHIT project. The project includes a "database handler" program called dbccp4i, that manages the job database and provides access to the jobs by other programs. The dbviewer visualisation tool (discussed earlier in section 3.2) is also part of this system.

While an earlier version of dbccp4i has successfully been incorporated into the more recent versions of MrBUMP, as of the next release of the CCP4 suite, the database handler will be incorporated directly into CCP4i. This work is currently on-going.

Most of the changes should happen "under the hood" and initially there should be few perceptible differences in usage. However in the longer term the incorporation of dbccp4i should allow other programs such as Coot, CCP4mg and iMOSFLM to communicate directly with the CCP4i project database and help provide a more integrated CCP4 environment.

5 Other new features

In addition to the major new features outlined in the previous section there are a number of minor new features that are intended to improve the general ease of use of CCP4i.

5.1 Task importer

The task importer is a function within CCP4i that allows the reinstallation of "3rd party" interfaces from an old installation of CCP4 into a newer installation. For example: if you have installed the ARP/wARP CCP4i interface into your CCP4 6.0.2 installation, then this function makes it possible to automatically copy that interface to a new CCP4 6.1 installation.

Figure 6: example of the task importer interface showing the packages already installed in a different CCP4 installation that can be copied to the current installation.

An example of the interface window is shown in figure 6. The function will be accessed from the "System Administration->Install/uninstall tasks" menu option. It can also be activated from the command line using:

ccp4i -import_packages /home/pjx/CCP4_CVS/ccp4-6.0.2

Note that only the task interfaces will be imported using this function - any associated programs will (if necessary) need to be moved manually to your new installation.

5.2 Improvements to interactions with the Job Database List

In an attempt to improve usability of the job database list (in the centre of CCP4i's main window), a number of improvements have been implemented.

Figure 7: job list menu for a single selected job, offering various options including viewing files, launching plugins and rerunning the job.

Figure 8: job list menu for a several selected jobs.

5.3 Loggraph support for Scala xmgr-formatted ROGUES, CORRELPLOT and NORMPLOT files

The Scala program (accessed in CCP4i through the "Scale and Merge" task in the "Data Reduction" module) generates a number of different output files which contain useful analyses of the data. Examples of such files include the NORMPLOT (normal probability distribution from the merge stage), CORRELPLOT (a scatter plot of pairs of anomalous differences from random half-datasets) and ROGUES (outliers on the detector).

Inspection of these plots is useful as a way of assessing the quality of the scaling and merging steps. However as the files are formatted for display using the XMGR or XMGRace programs, this presents a problem for users on systems where these programs are not available (particularly Microsoft Windows). However modifications to Loggraph mean that it can now also be used to display these files as a last resort if XMGR is not found on the system. Some examples of Loggraph's display can be seen in figure 9.

Figure 9: examples of loggraph displaying the XMGR-formatted NORMPLOT, CORRELPLOT and ROGUES output from Scala

5.4 Colourisation of the job list

This is actually an old feature that has been available since CCP4 6.0, which provides the ability to customise the colours of jobs displayed in the main window according to user-defined criteria (for example, based on the job status). Some examples are shown in figure 10 b) and c). The purpose of providing the colourisation is to allow users to define their own colour scheme which helps with comprehending the status of the jobs in their projects.

In the current version of CCP4i this functionality is accessed via the "Configure Interface" window under the "System Administration" menu, so it's not very obvious. In the next version of CCP4i however the colourisation functionality will be more easily accessible via direct links to a dedicated interface from the "System Administration" menu and from a right-mouse click on the job database.


Figure 10: The interface for applying custom colours to the job database list displayed in the main CCP4i window (figure a) and two examples of custom colour schemes applied to the job list (b and c).


5.5 Incorporation of "baubles"

The baubles program is a reformatter for logfiles from CCP4i and CCP4 programs, which generates an "annotated" HTML version of standard logfiles that includes "inline" graphs using the JLogGraph applet (see figure 11). It will be incorporated into CCP4i to give the option of viewing the logfile in this format.

See the article elsewhere in this newsletter for more information about baubles and integration with CCP4i.

Figure 11: Annotated logfile produced by baubles with inline loggraphs and viewed in a web browser.

6 Current Status and Availability

Although the majority of the changes and updates described in this article are already implemented, they will unfortunately not be generally available until the release of CCP4 6.1. In the meantime if you are particularly interested in trying out a developmental version of CCP4i then please contact me for information on how to get hold of the updated code.

7 Acknowledgements

MrBUMP and its interface has been developed by Ronan Keegan and Martyn Winn. The interface to XIA2 was developed by Graeme Winter and Peter Briggs. Martyn Winn developed the interface to the local version of the PISA program. The original pre-1.2.11 interfaces to Pointless were developed by Peter Briggs; the latest version has been developed by Phil Evans with input from Peter Briggs. The interfaces to CHOOCH and DYNDOM have been developed by Francois Remacle, and the Phaser interfaces are developed and maintained by the Phaser developers Randy Read and Airlie McCoy. The RAPPER interface has been developed by Nick Furnham. The BUCCANEER and SEQUINS interfaces have been developed by Kevin Cowtan.

idiffdisp has been developed by Francois Remacle. Francois also integrated iMOSFLM into CCP4i and implemented the joblist-colourisation functionality. iMosflm has been developed by Geoff Battye, Harry Powell and Luke Kontogiannis.

The dbviewer and dbccp4i programs were developed by Wanjuan Yang in collaboration with Peter Briggs as part of CCP4's contribution to the BIOXHIT project, which is funded by the European Commission within its FP6 Programme under the thematic area "Life sciences, genomics and biotechnology for health", contract number LHSG-CT-2003-503420. Additional funds are provided from CCLRC Daresbury Laboratory via the CCP4 project.

Various other modifications and changes were made by Peter Briggs with useful input from CCP4 staff and collaborators. The remodelled modules and tasklists were developed from original suggestions by Charlie Bond.

CCP4i was originally developed by Liz Potterton, and is now maintained and developed by the Daresbury CCP4 staff (Peter Briggs, Martyn Winn, Charles Ballard, Francois Remacle, Norman Stein and Ronan Keegan) who contributed other fixes and developments. Please send questions, requests and bug reports to us at

The figures in this article were prepared using the Gimp and ImageMagick programs.

8 References