Installation of CCP4 Version 6.3

Current version CCP4 6.3 May 2012

Contents

CCP4 Packages
Requirements
Downloading the Software
Manually Building the Suite on Unix/Linux
Manually Installing Pre-Built CCP4
Automated installation of CCP4 on UNIX/Linux
Installing CCP4 on Microsoft Windows
Installing CCP4 on Macintosh OS-X

Appendices:

  1. Directory Structure
  2. Alternative Compression Formats
  3. Introduction to compilers, configure and make
  4. Configure system
  5. Configure Options
  6. X-Windows (X11) Graphics Programs
  7. Multiple Installations
  8. Reducing Disk Space Requirements for Executables
  9. Testing the Suite
  10. Problems Building the Suite
  11. Scratch Directories
  12. Installing Only the Object Libraries
  13. Installing Updates to Individual Programs
  14. Installing Updates to the Whole Suite
  15. Uninstalling the Suite
  16. Contacting CCP4

Also:

Installation of CCP4 Interface.

CCP4 Packages

The CCP4 suite is divided up into a number of separate components (referred to as packages) in order to deal with user preferences, portability and installation issues. The packages are:

CCP4 Program Suite
The minimal CCP4 suite containing the core CCP4 libraries plus the standard programs, the user interface CCP4i and the Clipper software.
CCTBX and Phaser
Phaser is a powerful package for phasing macromolecular crystal structures with maximum likelihood techniques. Phaser requires the CCTBX (Computational Crystallography Toolbox) libraries.
CHOOCH
Program to perform calculation of anomalous scattering factors from X-ray fluorescence data.
CCP4mg
CCP4 molecular graphics package with emphasis on structure analysis and publication-quality presentation graphics.
Coot
Model-building tools in a molecular graphics framework.
POINTLESS
Pointless and other tools depending on clipper and cctbx.
RAPPER
The Rapper suite of model building programs.
BALBES databse
The database for the BALBES auto-MR pipeline.

Requirements

Building the Suite
A number of tools are required in order to be able to build the suite from the source code, and you should check that these are available before attempting to install.

Compilers
Fortran, C and C++ compilers are needed to build the suite. A FORTRAN 90 compliant compiler is required for building Refmac5.4 and molrep.
Build tools
m4 utility (a macro processor)
make utility
curl and bison (required for pdb_extract)
xmkmf (required if you wish to build the X-windows (X11) graphics programs)
fsplit is also required but this is now included in the suite
Libraries and headers
A number of non-standard libraries and headers may also be required in order to build the X-windows (X11) graphics programs, for example the ncurses library.
fftw: clipper requires fftw version 2 compiled with the float API. A version of fftw may be present on your system, please confirm that that this is the correct version, and has the correct interface. Note that the API of FFTW 3.x is compatible with the API of FFTW 2.x If in doubt, use fftw version 2.1.5 which is included in CCP4.
python: Installing CCTBX and Phaser from source code requires that the Python scripting language is already installed on your system.
Tcl and Tk: Building the Mapslicer command library ccp4mapwish requires Tcl and Tk libraries and the corresponding C language header files tcl.h and tk.h to be installed on your system. Note that some default Linux installations may include Tcl/Tk but without the Tcl/Tk developers package (which contains the header files). Please check that you have also installed the Tcl/Tk developers package.

Systems which supply a "developer's toolkit" or similar will normally include the appropriate compilers, tools and libraries.
Note however that on some systems these toolkits are no longer supplied or installed by default.
Please check with your vendor - or alternatively if you are installing the operating system yourself (for example on a Linux box), then ensure that you install the appropriate packages so that you have access to these tools, as it is possible that some or all will be missing from a "default" or "standard" installation.

Disk Space

The CCP4 software is available as a set of discrete packages in a number of different forms (see Downloading the Software. The table below gives a representative set of approximate sizes for the downloaded packages:

Component Compressed archive (Mb) Uncompressed archive (Mb) Total
CCP4 core suite (source code) 56.2 197.6 253.8
CCP4 core suite (Linux binaries) 89.7 248.9 338.6
Phaser and CCTBX (source code) 16.5 84.9 101.4
Phaser binary only (for Linux) 2.0 5.4 7.4
CHOOCH (source code) 2.9 16.5 19.4
CCP4mg (Linux executables) 43.8 153.6 197.4
Coot (Linux executables) 109.2 338.6 447.8
Balbes database 450 1800 2250

The required disk space is obviously dependent on which packages are downloaded. For example, to uncompress a ``full'' distribution containing source code for CCP4, CCTBX, Phaser and Chooch plus binaries for CCP4mg and Coot would require ~1070Mbytes (~230Mbytes for the compressed archive file plus ~790Mbytes for the unpacked source code). A more modest download of executables for CCP4 and Phaser would require ~350Mbytes (~95Mbytes for the compressed archive plus ~255Mbytes).

Note that the download pages report the uncompressed sizes for each package in addition to the total uncompressed and compressed sizes for the whole set of packages.

Disk requirements for the fully built and installed suite depends further on your system and particularly if you use shared libraries. In addition performing make realclean (see "Reducing Disk Space Requirements for Executables") can further reduce the size of the final installation (in the past reductions of ~30% have been observed).

Please note also that some programs use a lot of temporary scratch space to execute - you will need 10s of MBytes free in the directory pointed at by the environment variable $CCP4_SCR, see Scratch Directories.

Other dependencies

The CCP4 graphical user interface CCP4i requires the Tcl/Tk and BLT packages to be installed on your system before it can be run. (See the separate CCP4i Installation page for more details).

Imosflm, the new mosflm interface requires various other tcl-tk extensions. These are

MrBUMP, XIA2 and the cctbx dispatchers require that the Python scripting language is installed on your system.

Downloading the Software

The CCP4 packages can be obtained directly from the CCP4 FTP site, or via a set of interactive download pages. The pages are the recommended route for selecting the appropriate files based on the target system and the user's requirements.

Using the interactive download pages

The interactive pages guide you interactively through the following process:

Using the FTP site

The pages have an associated FAQ page which is recommended for more detailed information.

The CCP4 FTP site is at ftp.ccp4.ac.uk and arranges the downloadable files (which are mostly in the form of tar archive files compressed with gzip) in a browsable hierarchy. At the top level are each of the available packages, followed by the package version, within which are a number of subdirectories:

---ccp4---6.3.0-----|--binaries--|--ccp4-6.3.0-linux-i386.tar.gz
                    |            |--(plaform specific binaries)
                    |--packed     --ccp4-6.3.0-core-src.tar.gz (source code)
                    |--patches      (patches to source code release)
                    |--prerelease   (prerelease software)

---cctbx_based ---|--binaries-------(platform specific binaries)
                  |--packed  -------(source code archive)

---ccp4mg-----2.5.2-----binaries-----(platform specific binaries)

---coot  -----0.6.2-----binaries-----(platform specific binaries)

---chooch-----5.0.2--|--binaries----(platform specific binaries)
                     |--packed  ----(source code archive)
---balbesdb (balbes database)

If you are installing on a Unix- or Linux-type system then first you need to decide where you want to put the software: for example this might be /usr/local (following usual Unix convention). (The chosen directory is later referred to by the environment variable $CCP4_MASTER.)

FTP the compressed tar files for each of the packages that you wish to install (for example ccp4-6.3.0-core-src.tar.gz, ccp4-6.3.0-phaser-cctbx.tar.gz, chooch.tar.gz etc) to this directory (or see Alternative Compression Formats). See CCP4 Packages for information about each of the available packages.

Manually Building the Suite on Unix/Linux

This section describes the process for manually installing the suite from source code on Unix and Linux platforms. These steps are performed automatically by the customised install.sh script generated by the interactive download pages, see Automated installation of CCP4 on UNIX/Linux.

The following sections outline the procedures for the following packages:

Manual build and installation of core CCP4 and Phaser/CCTBX

Manual installation of the core CCP4 and Phaser/CCTBX packages are described below:

Unpacking the Archives

Uncompress the gzipped files:

> gzip -dc ccp4-6.3.0-core-src.tar.gz | tar xvf -
> gzip -dc ccp4-6.3.0-phaser-cctbx.tar.gz | tar xvf -

This will create a new ccp4-6.3.0 directory, in turn containing a set of new directories (see Directory Structure for details).

Provided that the Phaser and CCTBX source code are present in the correct locations within the CCP4 directory tree (see Directory Structure) there are no other special actions to take - Phaser and CCTBX will be built as part of the standard installation. Conversely the configure options --disable-phaser and --disable-cctbx will suppress the installation of the two packages - see Configure Options.

Precompiled binaries of the program executables are available for a number of Unix and Linux-based platforms. These can be installed using the custom install.sh script created by the interactive download pages, otherwise see the section on Manually Installing Pre-Built CCP4.

Editing the Setup File

The CCP4 distribution comes with 4 example setup files in $CCP4/include: ccp4.setup-csh, ccp4.setup-bash, ccp4.setup-sh, ccp4.setup-zsh. These are appropriate to different Unix shells, as indicated by the name (N.B. ccp4.setup-csh is for (t)csh). If you are unsure which shell you are using, then use the following command:

> echo $SHELL

which will return something like /bin/bash. The following assumes you are working in (t)csh, but applies to other shells.

If you have a previously installed CCP4 Suite it is worthwhile looking at the old $CCP4/include/ccp4.setup as it may save re-editing the new file (but do a comparison to check there are no new features).

The setup file is sourced by users (typically from their .cshrc file) to create the environment to run CCP4 programs. You will need to customise this file for your site.

> cd ccp4-6.3.0/include
> cp ccp4.setup-csh ccp4.setup          (to make two copies of the file -
one is left unchanged for reference).

Edit ccp4.setup following the instructions in the comments - the minimum that you will need to do is:

Source this file - the setting of the environment variables is necessary for the rest of the installation:

> source ccp4.setup

This script also places CCP4 executables and scripts on the user's path, places any shared libraries in LD_LIBRARY_PATH, and sets some other paths. ccp4.setup has an option ccp4_first_in_path to determine the priority of CCP4 programs. Be aware that there may be name conflicts with non-CCP4 programs.

Configure and Make

See Introduction to compilers, configure and make if you are a novice installer.

Make sure that you are in the top CCP4 directory and look at a listing of currently supported systems using the configure help option (see Configure Systems for more information - essential if you have multiple operating systems). Then start the configure process proper:

> cd $CCP4
> ./configure help
> ./configure system options

where the system is one of those listed by configure help, and must be supplied. There are options for setting non-default directories, for controlling whether or not certain parts of the Suite are built, and for overcoming possible problems. The options that you are most likely use are --with-shared-libs and --with-x. See Configure Options for details and a full list of options.

If you are running configure for the first time then before the configuration starts a copy of the current CCP4 licensing conditions will be displayed and you will be asked whether you agree to them. For academic/non-profit users of the software, you should return a completed and signed copy of the licence, if this has not already been done.

Copies of the academic licence agreement can be found in various formats in the $CCP4 directory:

Commercial users of the software should contact CCP4 directly to obtain a commercial licence; see Contacting CCP4. Please note that all users are required to return a new copy of the licence agreement, even if a copy of the previous license has been already been sent to CCP4.

To agree to the licence conditions and continue with the configuration, enter "y" at the prompt.

It is possible to tweak the installation of some programs before starting the compilation:

When the configuration is complete be sure that you are in the $CCP4 directory and then run make (to build the executables) and make install to move executables from the src to the bin directory (unless an alternative target directory was specified with the configure --bindir option).

A reminder for a few people using the configure --bindir option: if you intend to reuse the same executables directory as for a previous CCP4 installation then remove the installed executables with make uninstall.
> cd $CCP4
> make (or make > make.log to keep a log file)
> make install (or make install >> make.log to append to the log file)
> make runtest (optional test of the most important programs)

It is generally unnecessary but if you want to test the suite before installation look at Testing.

You can save disk space by removing many files which have been moved elsewhere:

> make realclean

or, rather than completely deleting files you can leave dummy copies of the executables so that if the sources get updated, make will only rebuild the relevant ones:

> make clean empty-targets

If you have problems see Problems.

Configuring the CCP4 Interface CCP4i

If you have Tcl/Tk/BLT installed, and have set the environment variables in ccp4.setup correctly, then you should be ready to go. Just type ccp4i. Information on Tcl/Tk/BLT and on configuring the interface for local preferences can be found in Installation of CCP4 Interface.

Manual build and installation of CHOOCH

Uncompress the gzipped file :

> tar zxf chooch.tar.gz

This will create a new chooch directory, in turn containing a set of new directories plus a README document:

chooch-5.0.2
The CHOOCH source code
gsl-1.4
The source code for the GNU Scientific Libraries (GSL)
cgraph-2.04
The Cgraph Postscript plotting library

CHOOCH does not have to be unpacked in any special place relative to the rest of the CCP4 suite.

Building CHOOCH requires linking against the GSL and Cgraph libraries, which may already be on your system or else (in the case of Linux systems) easily installable from the installation disks. Otherwise the versions supplied with the CHOOCH distribution should be built and installed first.

Hint: By default both GSL and Cgraph will attempt to install themselves in /usr/local. If you don't have write permission to this area then you can reset the install location using the --prefix=PATH option of the GSL and Cgraph configure scripts to install in a different place. Be aware that in this case you will need to tell the CHOOCH configure script where to find GSL using the --with-gsl-prefix=PATH, and where to find Cgraph using --with-cgraph-prefix=PATH.

Other options that are useful if not installing GSL and Cgraph in system-wide locations are --enable-static and --disable-shared, which will improve the portability of the resulting CHOOCH binary.

A recipe for installing CHOOCH from source code might look like:

# GSL build and install
cd chooch/gsl-1.4/
./configure --prefix=mylocal --enable-static --disable-shared
make
make install
# Cgraph build and install
cd ../../chooch/cgraph-2.04/
./configure --prefix=mylocal --enable-static --disable-shared
make
make install
# CHOOCH build and install
cd ../../chooch/chooch-5.0.2/
./configure --with-cgraph-prefix=mylocal --with-gsl-prefix=mylocal --prefix=$CBIN
make
make install
cd ../..

Manual installation of CCP4mg

CCP4mg is currently only available as precompiled executables and the CCP4mg archive file contains everything that is required to run the software.

Once you have downloaded the CCP4mg gzipped archive file, uncompress it using e.g. :

> gzip -dc ccp4mg-0.12-Linux-SuSE-9.0.tar.gz | tar xvf -

This will create a new ccp4mg-0.12 directory in the current directory. Note that this does not have to be unpacked in any special location relative to the CCP4 suite. To start CCP4mg, execute the command e.g. :

> ./ccp4mg-0.12/bin/ccp4mg

CCP4mg does not require ccp4.setup to be sourced in order to operate. However adding the CCP4mg "bin" directory to your PATH in ccp4.setup will mean that you do not have to type the full pathname in order to invoke the program from the command line.

More information on CCP4mg can be found at the project website: http://www.ysbl.york.ac.uk/~ccp4mg.

Manual installation of Coot

Coot is currently only available from CCP4 as precompiled executables and the Coot archive file contains everything that is required to run the software.

If necessary, uncompress the gzipped file, e.g.:

> gzip -dc coot-0.0.33-binary-Linux-generic.tar.gz | tar xvf -

This will create a new Coot-0.0.33 directory in the current directory (note that this does not have to be unpacked in any special location relative to the CCP4 suite). If you have used the new CCP4 download mechanism, then this directory is created by untarring the download file.

The subdirectory bin contains template startup files for Coot, namely coot.sh and coot.csh (there are also versions in the setup subdirectory). Choose the one that is appropriate for your shell and edit the line

set COOT_PREFIX = /h/paule/autobuild/Linux-bubbles

to point to the location of the top-level Coot directory (that is, directory Coot-0.0.33 including the full path). To use Coot you should then do:

> source ccp4.setup
> source Coot-0.0.33/setup/coot.csh
> coot

More information on Coot can be found at the project website: http://www.ysbl.york.ac.uk/~emsley/coot. A number of installation and configuration problems are addressed in the Coot FAQ: http://www.ysbl.york.ac.uk/~emsley/coot/coot-faq.html.

Manually Installing Pre-Built CCP4 on UNIX/Linux

Pre-compiled binary versions of the programs are available for some platforms. It is no longer necessary to obtain the source code distribution, as the various data and source code files required by the programs at run-time are bundled with the executables.

If you are not using the custom install.sh script provided by the interactive download pages then the following steps must be performed manually in order to complete the installation.

  1. Download and unpack the appropriate binary distribution, see "Downloading the Software". The binary distributions contain all the files necessary to install the CCP4 suite.
  2. Edit and source the ccp4.setup file, see "Editing the Setup File"
  3. Run the BINARY.setup script in $CCP4 to create binary data file and complete the installation.
    NB If you are installing for the first time then you will be asked to agree to the current CCP4 licence before the configuration proceeds.

The $CEXAM/unix/runnable/run-all script can be run to test the installation, see Testing the Suite.

Automated installation of CCP4 on UNIX/Linux

The interactive download pages generate a single tar file customised to your platform and choice of packages. This file can be unpacked in the place where you wish to install CCP4 using the command:

tar xvf <tar file>

You will get either 3 or 4 files as a result:

README A text file containing instructions similar to these
install.sh A customised shell script for automatic installation
packages.tar.gz A compressed archive file containing the chosen packages
tools.tar.gz A compressed archive file containing the `external tools' (if chosen) e.g. Tcl/Tk and/or Python

To perform an automated installation, run the install.sh script:

./install.sh

You will be asked:

Do you want to keep all default parameters for installation (yes/no):

If you answer "yes" (the default) then the packages will be built (if necessary) and installed in subdirectories of the current working directory, with different subdirectories for CCP4, CCP4mg, Coot and CHOOCH. (Note that some packages are amalgamated e.g. Phaser exists within the CCP4 subdirectory.)

For each package, the stdout is redirected to a file <package>.log while the stderr is redirected to a file <package>e.log

The script will also generate a setup-scripts subdirectory that contains set-up files for the various packages (i.e. ccp4.setup for the core suite and programs, and ccp4-others.setup for CCP4mg and Coot) for Bourne/Bash shell (in the sh subdirectory of setup-scripts) and C-shell (in the csh subdirectory). To start using the software you should only need to source the appropriate set-up files for your shell.

Note that the set-up files for other shells (e.g. Z-shell) can be found in the usual place i.e. in $CCP4/include. All the set-up files are specific to the directory where the packages were installed: if you later move the packages to a different location, you will need to manually edit the set-up files.

If you answer "no" to keeping the default parameters, then you will be asked a series of questions to customise the installation.

Manual installation can also be performed if you don't want to install.sh script, by manually unzipping and untarring the packages.tar.gz file (and tools.tar.gz if relevant). This will unpack the packages into subdirectories as described above, and you can then build and install each package as outlined in the the section Manually Building the Suite on Unix/Linux. Manual installation will be necessary if you want to build a package with options not covered by the automated installation.

The $CEXAM/unix/runnable/run-all script can be run to test the installation, see Testing the Suite.

Installing CCP4 on Microsoft Windows

A version of the CCP4 suite is available which will run under a number of different Microsoft Windows versions, including:

The installation procedure is different from the Unix/Linux platforms but is straightforward, and uses an installer wizard. Two types of installer are available from the website:

Once you have downloaded the appropriate installer you just need to run it in the usual fashion, and follow the dialogs the wizard provides for you. Both installers are designed to allow you to select which packages you wish to install, and will set up everything required to run the selected packages. For example, although Tcl/Tk and BLT are included in the installation, the wizard will check if it is already installed on your system. If not then the installer will allow you to install at this point.

It is recommended that you uninstall any previous versions of CCP4 before attempting to install a new version. There is currently no update mechanism and the results of installing over an existing version could have unpredictable results.

However it is possible to retrospectively modify the current installation after it is finished, by rerunning the installer that was originally used. The wizard will offer the options of modifying, repairing or uninstalling. Alternatively selecting CCP4-Packages in the Add or remove programs function (accessed from the Windows Control Panel) will give the choices of changing or removing the packages.

More detailed information is available from the CCP4 on Microsoft Windows webpage.

Installing CCP4 on Macintosh OS-X

For Mac OS-X there are several packages available as disk mountable image files (.dmg). These comprise of ccp4 (with cctbx, chooch and tcl/tk), CCP4MG and coot, along with seperate source code archives where applicable. The disk images contain installer scripts.

For ccp4/cctbx download and run the ccp4-6.3.0-osx-i386.dmg.gz file, and then follow the steps as per a standard Mac OS-X package installation. (Note that the i386 version will not run on ppc machines). The default installation location is /usr/local/ccp4-<version>

CCP4MG and coot are both installled in the Applications folder. Note that the ccp4mg .dmg is os version dependent.

Appendix A - Directory Structure

The directory structure of the CCP4 distribution under the top ccp4-<version-number> directory:
bin
default destination for installed program binaries
ccp4i
CCP4 graphical user interface
doc
plain text documentation files (generated from the html files)
etc
Unix shell scripts
examples
example scripts
data
miscellaneous data required by some of the runnable example scripts
toxd
example data and model coordinates for a small protein (alpha-dendrotoxin from green mamba venom: "toxd")
rnase
example data and model coordinates for Ribonuclease from Streptomyces aureofaciens: "rnase")
unix
non-runnable
scripts which can't be run for want of data
runnable
scripts which can be run with the data in toxd or rnase
tutorial
tutorial material for using the CCP4 suite for Data Processing, MIR and MAD phasing, Molecular Replacement and Refinement plus an introduction to the suite and to CCP4i
data
data files with which to run the tutorials
html
instructions on running the tutorials
results
results files from the tutorials
html
library and program documentation in HTML form
include
setup scripts and the .def files needed for program startup
lib
default destination for installation of the binary library files
ccif
C library for low level operations on mmCIF files
cctbx
CCTBX crystallographic libraries (if included)
clipper
Kevin Cowtan's Clipper package
data
machine-independent `library' files such as the symmetry operators
fftw
source code for the MIT `fftw' fast-Fourier transform library version 2.1.5
lapack
blas/src
source code for netlib reference BLAS maths routines
src
source code for netlib version of LAPACK maths routines
regex
regex taken from glibc-2.3 (in the unlikely event it is missing in the OS)
rxdispencer
a modified version of Henry Spencer's regular expression library
src
source code for the CCP4 library
mmdb
Eugene Krissinel's MMDB library
man
Unix manpages for the CCP4 suite
manual
LATEX source for the manual and corresponding PostScript file
src
program source
phaser
PHASER package (if included)
unsupported
source for the `unsupported' programs
x-windows
X-Windows (X11) programs (xloggraph etc.)

Appendix B: Alternative Compression Formats

We believe contemporary Unix/Linux systems all support gzip compression - if yours does not then:

a) Let us know or we are liable to stop distributing alternatives.
b) FTP the ccp4-all.tar.Z file and uncompress:

> zcat ccp4-all.tar.gz | tar xf -

Appendix C: Introduction to compilers, configure and make

CCP4 programs are written in either the C, C++ or Fortran programming languages (and the source files are called either foo.c, foo.cpp or foo.f respectively) and so need either the C compiler (usually called cc), the C++ compiler (which has various names depending on the platform, for example CC or cxx) or Fortran compiler (usually called f77) to compile and link them. Compilers are really just programs which process the code to make other programs. 'Compiling' is converting the program source code (human readable) into machine code (computer readable) in an object file called foo.o.

Large programs are written in several separate source files and so will be made up from several object files. Programs also require some standard 'libraries' of commonly used utilities - these are similar to object file and have file names like foo.a. There are some 'system' libraries which are a standard part of the operating system of your computer and some CCP4 libraries such as mtzlib and symlib which provide utilities to handle MTZ files and symmetry which are used by many programs. The CCP4 libraries also start out as source code in $CCP4/lib/src and need compiling too. The next stage after compiling is linking (this is still done by the 'compiler') which is pulling together all the object files and libraries needed for the program to work and creating the 'executable' file (usually just called foo).

There is an alternative approach to linking programs with the libraries and that is using 'shared libraries'. Linking programs to the default archive library foo.a involves including the necessary library routines in the program executable. Over the suite, therefore, the same code is included many times, and a lot of extra disk space is used. In contrast, when a program is linked to a shared library foo.so (created from foo.a by the link editor ld) only a reference to the library routine is included - it is not actually loaded until runtime. This means that the program executable takes up less disk space, but also that the program must be able to find the library routine at runtime. For this to work requires that the environment variable LD_LIBRARY_PATH is set correctly. This variable is a list of the directories that should be searched to find shared object libraries; it may include the names of several 'system' libraries or libraries for non-CCP4 programs but should also include $CCP4/lib.

The process of compiling and linking a complex system such as the CCP4 Suite is best done by a utility called make which interprets the instructions in a makefile. The great usefulness of make is that the system 'understands' the dependencies between the various levels of the programming system so that, for example, if you change one file which is used to build many different programs than the make procedure 'knows' to recompile and relink all the programs that are affected by the change. The make procedure will also control other necessary functions such as installing the software (which is basically moving files from where they are first built to where you want them to be used) or any configuring of data or documentation files. It will also clean up by deleting intermediate files.

It is usual in a complex program system to have several makefiles, one in each directory, always called Makefile. Within CCP4 there are separate Makefiles in several directories. There is a master Makefile in the $CCP4 directory which 'calls' all the other makefiles and so to run the make should only require typing the command make in the $CCP4 directory - this command will, by default, use the file called Makefile. The make utility can be got to perform some of its other functions by commands such as make install or make clean.

There is one more important part of the process, configure. The differences in operating systems and user requirements means it is impossible to write a makefile which would work for everybody. The way round this is to edit the makefile before it is run to customise it for your system. This process is done by a script called configure (which is in $CCP4 directory) which converts a standard initial file, Makefile.in, into a customised Makefile for your system. The configure process will do a lot of automatic checks on your system (do you have compilers? what system libraries are present?) and has a large number of options for you to set your own requirements.

So the minimal instructions you will actually need to type for a standard Unix installation are the trilogy:

> configure
> make
> make install

(but see Manually Building the Suite on Unix/Linux before trying it on CCP4!).

Appendix D: Configure Systems

When running configure it is necessary to specify the system that you are installing on. The following systems are supported:

linux Linux operating system with GNU compilers
linux_intel_compilers Linux operating system using Intel compilers
Darwin Mac OS-X operating system with GNU compilers
Darwin_intel_compilers Mac OS-X operating system using Intel compilers
ia64_linux_intel Itanium Linux using Intel compilers
freebsd FreeBSD operating system for x86 compatible, AMD64, DEC Alpha, IA-64, PC-98 and UltraSPARC® architectures

If your system does not appear on the list of supported systems for configure then use generic.

Appendix E: Configure Options

The main configuration parameters (apart from the system type) are controlled by command-line flags to configure of the form

        --<flag>=<value>

where <flag> may be abbreviated uniquely and = may be replaced by whitespace. The currently-implemented flags of this form are:

--srcdir
The CCP4 source directory (referred to as $CCP4)
--bindir
Where the program binaries get installed
--libdir
Where the library files get installed

Other common options are controlled by flags of the form --with-feature:

--with-shared-libs
Build shared versions of the libraries libccp4c, libccp4f, libmmdb and libccif, and link against these (this saves disk space, see Reducing Disk Space Requirements).
--with-netlib-lapack
Force building of the netlib `reference' LAPACK and BLAS libraries, over-riding use of any system versions.

Other options are:

--help
Display options and systems.
--disable-x
Disable installation of the X-Windows-based (X11) programs. By default these will be built and installed.
--disable-ccif
Disable installation of the libccif library (held in $CCP4/lib/ccif). This is purely a safety feature in case libccif fails on some systems (it shouldn't!). Disabling the ccif library will mean that data harvesting won't work for some programs. One possible cause of failure of the CCIF build is the lack of a local regular expression package, in which case check out the --with-rxdispencer option first.
--with-rxdispencer
Build the distributed version of Henry Spencer's regular expression library (held in $CCP4/lib/rxdispencer). Most unix systems should have their own regular expression library, and this option should only be used if this isn't the case. Currently, the regular expression library is only needed by the CCIF library.
--disable-cctbx
Disable installation of the cctbx library (held in $CCP4/lib/cctbx, if present on the system). By default if the cctbx source code is found then the cctbx libraries will be configured and built. Note that disabling cctbx automatically also disables installation of the PHASER and Aimless packages (even if the source code is available)
--disable-phaser
Disable installation of the PHASER package (held in $CCP4/src/phaser, if present on the system). By default if the PHASER source code is found (and the the cctbx libraries are also present and enabled) then PHASER will be configured and built.
--disable-lapack
Disable configuration of the LAPACK linear algebra package. By default configure will search for LAPACK libraries already installed in some form on the system, if none are found then otherwise it will build the netlib `reference' libraries which are distributed with the suite (in $CCP4/lib/lapack).
To force building of the reference LAPACK libraries regardless of those already on the system, use the --with-netlib-lapack option.
See the MODLIB documentation for more details about LAPACK.
--disable-mapslicer
Disable installation of the MapSlicer Tcl command library. Without this library the MapSlicer viewer will not work.
--disable-pdb_extract
Disable installation of the PDB_EXTRACT suite. Certain functionality in the Data Harvesting Manager will not be available.
--disable-ssm
Disable configuration and installation of the SSM library and the superpose program.
--disable-clipper
Disable configuration and installation of the Clipper package.
--disable-rapper
Disable configuration and installation of Rapper. This is enabled by default if the rapper package is in $CCP4/src.
--disable-pisa
Disable configuration and installation of Pisa. This is enabled by default if the pisa package is in $CCP4/src.
--disable-pointless
Disable configuration and installation of pointless and aimless. This is enabled by default if the pointless package is in $CCP4/src.
--disable-fftw
Disable configuration and installation of the FFTW package included with the CCP4 suite.
--with-fftw=dir
Use an existing FFTW library located in dir, rather than installing the version included with the CCP4 suite. Note that this must be version 2.x of FFTW.
--enable-prerelease
Enable the building of any packaged experimental software.
--with-warnings
Include extra compilation flags which output all compiler warnings (normally compilation is silent). This option may be useful for some developers.
--non_shared
Don't allow linking against shared libraries (including system libraries) when building the program executables. This results in significantly larger executables but does make them more portable. Not compatible with --with-shared-libs.
N.b. This option is only implemented for OSF1 and Linux systems, and can cause problems if programs are linked against system libraries which are only available as shared libraries. Use this option with caution.
--onlylibs
Build only the object libraries and not the CCP4 programs (See Installing only the Object Libraries).
--prefix=...
Install binary elements under prefix
--with-f2c
Use the free f2c compiler rather than a `native' one.
N.b. This option is strongly deprecated.
--tmpdir=
Explicitly set the value of TMPDIR used in configure.

configure currently assumes that the shareable (plain-text) library files remain in $CCP4/lib/data rather than being installed elsewhere.

If the build is not done in $CCP4, configure creates src and lib directories mirroring those in $CCP4 and does the compilations there.

Configure examines the environment variables necessary for running the suite as defined by ccp4.setup and tries to check them as far as reasonable, but these are not actually used in the build process. There are a number of configuration parameters that you don't usually want to change, but whose defaults can be over-ridden by the values of environment variables when configure is run:

CC
the C compiler (default is system's native compiler; another possibility might be gcc )
COPTIM
C compiler optimisation flags (default is the highest optimisation which is considered safe or unaggressive)
XCFLAGS
any extra flags you need to give to the C compiler, apart from those for optimisation (these are system dependent)
CXX
the C++ compiler (default is system's native compiler; another possibility might be g++ )
CXXOPTIM
C++ compiler optimisation flags (default is the highest optimisation which is considered safe or unaggressive)
XCXXFLAGS
any extra flags you need to give to the C++ compiler, apart from those for optimisation (these are system dependent)
FC
the Fortran compiler (default is system's native compiler); there isn't currently any support for other than native compilers
FOPTIM
Fortran compiler optimisation flags. The default varies, but will have no debugging extras. In some cases (notably IRIX), the default is used because there are so many problems encountered with the optimiser. You may want to try higher optimisation levels and see if there's a significant performance improvement and that the code still works
XFFLAGS
any extra flags necessary for the Fortran compiler, apart from those for optimisation; you might want to change this, for instance to make smaller binaries at the expense of debugging ability
XLDFLAGS
any extra flags needed for ld, typically extra libraries
RANLIB
dummy on SysV Unix, ranlib on BSD, depending on whether 'ar -r' builds a symbol table itself
LNS
indicates how to make symbolic links in the filesystem (usually 'ln -s'), else 'cp' to copy rather than link. 'ln' (hard links) could be used in the absence of symbolic links if you don't need to operate across file systems
M4
how to run the m4 macroprocessor and define a symbol to indicate the system type (see configure source)
MAKE
the name of the `make' program for use in recursive Makefiles if the system's make doesn't define the symbol MAKE. Usually null
SETFLAGS
set appropriately if individual programs need special Fortran compiler flags (most likely to suppress optimisation) - see the configure source
INSTALL_PROGRAM
a command to install executables. Uses 'install -c' if a BSD version is available (to avoid interfering with running programs), else 'cp'
INSTALL_DATA
a command to install non-executables. If BSD install is available, uses 'install -m 644', else 'cp'
TCL_CONFIG
The full path of the tclConfig.sh file which is used to acquire compiler flags and library paths when configuring the MapSlicer command library. If this is not set then configure searches for the tclConfig.sh file itself.
TCL_INCDIR
The full path of the directory containing the tcl.h header file which when building the MapSlicer command library. This should be determined automatically from tclConfig.sh.
TK_CONFIG
The full path of the tkConfig.sh file which is used to acquire compiler flags and library paths when configuring the MapSlicer command library. If this is not set then configure searches for the tkConfig.sh file itself.
TK_INCDIR
The full path of the directory containing the tk.h header file which when building the MapSlicer command library. This should be determined automatically from tkConfig.sh.
F_LIBS
Additional fortran compiler libraries and options required when a linkerother than the fortran compiler is used for objects generated using the fortran compiler.
C_LIBS
Additional C compiler libraries and options required when a linker other than the C compiler is used for objects generated using the C compiler.
CXX_LIBS
Additional C++ compiler libraries and options required when a linker other than the C++ compiler is used for objects generated using the C++ compiler.

There are corresponding Makefile variables that you can override, e.g.:

        make FOPTIM=-g

Suppose you support several types of system and share the CCP4 source and system-independent data files across a network with a distributed file system such as NFS, AFS or RFS. Then it may be convenient to have a subdirectory in $CCP4 for each system in which to build and, possibly, in which to keep the resulting binaries. Directories for binaries can be anywhere, though.

Appendix F: X-windows (X11) Graphics Programs

The most used X-Windows-based programs are Mosflm (integrate diffraction images), xplot84driver (used to view map sections and other xplot84 graphics files) and xloggraph (used to view graphs in log files but now superseded by loggraph which is part of ccp4i). The X-windows programs will run on SGIs, DECs and Linux.

The X-windows programs are built and installed by default and you must use the configure --disable-x option to suppress this.

If you are using the X-windows programs then you must ensure that the programs have access to the resources' relevant application defaults files. This is installation specific - there is an example for Unix users in ccp4.setup.

Appendix G: Multiple Installations

If you wish to do multiple installations for multiple machine types from a single source code tree, then you should run the script duptree first. This is in the $CCP4 directory and will produce a further script called duptree_ccp4 containing information on the source directory tree. This second script should then be run once for each machine type - in each case a duplicate source tree is created with soft links to the original source. The configure script should then be run within each duplicate source tree. The script duptree contains further details.

Appendix H: Reducing Disk Space Requirements for Executables

The program executables are big, and you may want to make them smaller. There are various possible ways to do this which are system-dependent:

Increasing the optimisation level of the compiler might make the executables bigger or smaller. However, under aggressive optimisation the programs may not work at all. To do this see either Configure Options for ways to change the configuration parameters or use parameters to make, (for example make FOPTIM=-O ).

Using shared libraries will definitely make executables smaller - use the configure argument --with-shared-libs to build a shared version of the CCP4 library to link against. You must make sure you have the correct setup, with LD_LIBRARY_PATH defined, to find the shared CCP4 library at runtime. Beware, using shared libraries has caused problems in the past when upgrading the operating system or moving executables.
All systems supporting the dlopen mechanism should be able to use shared libraries.

In extremis you can probably save some space by removing all debugging information from the executables using the strip program or installing using install -s (see Unix man pages for strip or install). This will prevent diagnosing some problems, though.

Appendix I: Testing the Suite

If you want to test the suite before installation then it is necessary to have the $CCP4/src directory, which contains the built executables, on the path, for example, for csh:

> set path=($CCP4/src:$path)

This is not necessary if testing after installation, when the executables will be in the correct bin directory. You will find two example datasets in $CCP4/examples/toxd and $CCP4/examples/rnase, and some runnable scripts in $CCP4/examples/unix/runnable. The script $CCP4/examples/unix/runnable/run-all can also be used to automatically run the example scripts in sequence and report any failures. The run-all script can be launched by "make runtest" at the top level.

In addition a set of htmlised CCP4i-based tutorials are available in $CCP4/examples/tutorial. These tutorials cover various aspects of the structure determination process using the CCP4 programs, including Data Processing, MIR and MAD phasing, Molecular Replacement and Refinement.

Appendix J: Problems Building the Suite

The amount of testing we can do is limited and there may be installation problems particularly on less widely-used systems. If you have a problem then:

If you find a solution please let CCP4 know; sending patches (made with diff -c) is very helpful. If you successfully use a configure option which reports itself as `untested' or `not properly tested' please let CCP4 know.

Appendix K: Scratch Directories

You will need at least 40Mbytes of scratch space to run many of the programs. The conventional Unix scratch area of /tmp or /usr/tmp is rarely setup with this much space. Many programs run more efficiently if the space is local to the machine that they are running on. But a potential problem is that if each machine in your laboratory has its own scratch directory then it is often confusing for users to find files on the scratch directory on a remote machine.

   setenv CCP4_SCR /scratch
   setenv BINSORT_SCR /scratch

Alternatively users can just use the subdirectory in their $HOME area so:

    setenv CCP4_SCR   $HOME/temporary
    setenv BINSORT_SCR   $HOME/temporary

Note that the user interface, CCP4i, also has the idea of a TEMPORARY directory which by default is the same as CCP4_SCR but this may need modifying for some setups.

Appendix L: Installing only the Object Libraries

To build the CCP4 library to support other software systems and avoid the make procedure attempting to access the src directory, use essentially the same procedure for Manually Building the Suite on Unix/Linux outlined above, with the following differences:
  1. Use the --onlylibs option when running configure, and
  2. Run make onlylib rather than make.

(Note that it is still necessary to set up and source the appropriate ccp4.setup file prior to running the configure and make steps.)

Appendix M: Installing Updates to Individual Programs

Bug-fix updates (more usually now referred to as patches) for known problems in the most recent releases are routinely made available via the CCP4 Problems Pages. Links are provided (where appropriate) to the CCP4 FTP server to enable download of bug-fix updates distributed as patch files, which can then be applied "manually" using the patch program to apply them. There is a short guide to applying patches on the CCP4 web site.

Alternatively, version 6.3.0 of the CCP4 suite includes a script in the $CETC directory called patch_ccp4.sh, which will attempt to download and apply all available source code patches automatically. To invoke the script, do e.g.:

> cd $CETC
> ./patch_ccp4.sh

Please note that:

  1. The script doesn't have to be run in any special location as it uses the CCP4 environment variables to correctly locate the files that are to be updated.
  2. If you have a binary distribution then only data files and scripts (such as CCP4i) will be updated.
  3. The script applies the fixes but does not recompile the programs, this must still be done by hand - see below.
  4. This will only work for UNIX-like systems.

To rebuild individual programs - put the new source code in $CCP4/src

> cd $CCP4/src
> make foo

Alternatively if you used make empty-targets when you did the original installation then a simple make will just rebuild the necessary. When you have installed the changes to your satisfaction remove the .orig files left behind by patch.

> make clean

Appendix N: Installing Updates to the Whole Suite

Patch releases of the whole suite may be made from time to time - these collect together all bug-fix updates made since the previous release, and do not introduce any new functionality to the programs.

Patch releases are made available in a number of ways:

  1. as complete new source code in a compressed tar file;
  2. as precompiled binaries;
  3. as a source code patch file.

If downloading the suite for the first time it is best to take the most recent available version. Otherwise, the source code patch file can be applied to a previously installed copy of the suite as follows:

> cp patchfile $CCP4 (Move the patch file to the top CCP4 directory)
> cd $CCP4
> patch -p1 -N -i patchfile >& patch.log (Apply the patch to the source code using the patch program, and save the output to a log file)

If you wish to rebuild any updated libraries and programs then you further need to:

> ./config.status (Rerun configure with the original settings to regenerate Makefiles)
> make [install] (Rebuild [and install] any updated programs)

Finally, patch generates files with the extension .orig (one .orig file for each patched file). These files can be removed by running make clean afterwards.

If using the precompiled binaries you can simply download the latest executables as outlined in the section on Installing Pre-Built CCP4. (Please note that it is still a good idea to apply the source code patch in this case, as this will address bugs in the uncompiled elements of the suite, such as CCP4i and the documentation.)

Finally, we recommend that you read the notes below before applying patches for the first time, and to report any problems with the procedure to CCP4 (ccp4@ccp4.ac.uk).

Notes on applying patches to the whole suite

  1. Patch files from CCP4 will have names of the form ccp4-oldversion-newversion.diff, which indicates that the patch will upgrade version old-version to version new-version. Applying a patch file to the wrong version will have unpredictable (and most likely undesirable) results.
  2. It is intended that patches should be incremental, so if you have e.g. CCP4 4.2 and wish to upgrade to 4.2.2 via the patches, then it will be necessary to first apply the upgrade from 4.2 to 4.2.1, and then that for 4.2.1 to 4.2.2. (It is not necessary to rebuild the suite in between applying the patches however.)
  3. The patch files cannot upgrade binary files (such as MTZ data files, images, and precompiled Java files) which are included in the distribution.
  4. It is sensible to keep a log file from the patching procedure (see instructions above), in case the patching is unsuccessful. In the event of a problem, please send the details along with the log file to ccp4@ccp4.ac.uk.
  5. To reverse the patch follow the instructions above, including an additional -R option on the patch command.

Appendix O: Uninstalling the Suite

Please note that the following information is relevant to Unix-based installations built from source code. Windows-based installations can be uninstalled by selecting the appropriate option in the CCP4 folder of the Windows Start menu.

If it is necessary or desirable to undo the installation and start again from scratch (for example you wish to run the configuration with different options) then the following prescription should restore the original distribution:
> cd $CCP4
> make uninstall         (Delete all the installed files, e.g. libraries in $CCP4_LIB and binaries in $CBIN)
> make distclean         (Delete all the files created in configuring and building the suite, including the Makefiles)

It will then be necessary to re-run configure etc as if starting from scratch. This procedure should also work for individual sections of the suite, for example if you want to uninstall and remake the libraries then cd $CLIBS instead before executing make uninstall and make distclean.

Appendix P: Contacting CCP4

Completed licence agreements and requests for commercial software licences should be mailed or faxed to:

The Secretary to CCP4
Daresbury Laboratory
Warrington WA4 4AD
UK

Telephone: (+44) 1925 603929 (direct line)
Facsimile: (+44) 1925 603825
e-mail: ccp4@ccp4.ac.uk

Technical queries including reports of problems with downloading, installing or running the software should in the first instance be sent to us by e-mail at ccp4@ccp4.ac.uk.