Installation of CCP4 Interface.
The CCP4 suite is divided up into a number of separate components (referred to as packages) in order to deal with user preferences, portability and installation issues. The packages are:
Building the Suite
A number of tools are required in order to be able to build the suite from the source code, and you should check that these are available before attempting to install.
Systems which supply a "developer's toolkit" or similar will normally
include the appropriate compilers, tools and libraries.
Note however that on some systems these toolkits are no longer supplied or installed by default.
Please check with your vendor - or alternatively if you are installing the operating system yourself (for example on a Linux box), then ensure that you install the appropriate packages so that you have access to these tools, as it is possible that some or all will be missing from a "default" or "standard" installation.
The CCP4 software is available as a set of discrete packages in a number of different forms (see Downloading the Software. The table below gives a representative set of approximate sizes for the downloaded packages:
|Component||Compressed archive (Mb)||Uncompressed archive (Mb)||Total|
|CCP4 core suite (source code)||325||1083||1407|
|CCP4 core suite (Linux binaries)||553||1732||2284|
|Phaser and CCTBX (source code)||34||172||205|
|Phaser binary only (for Linux)||93||346||439|
|CHOOCH (source code)||3||16||20|
|CCP4mg (source code)||62||152||214|
|CCP4mg (Linux executables)||292||1053||1345|
|Coot (Linux executables)||112||373||485|
The required disk space is obviously dependent on which packages are downloaded. For example, to uncompress a ``full'' distribution containing source code for CCP4, CCTBX, Phaser and Chooch plus binaries for CCP4mg and Coot would require ~2040 Mbytes (~770 Mbytes for the compressed archive files plus ~1270 Mbytes for the unpacked source code).
Note that the download pages report the compressed sizes for each package.
Disk requirements for the fully built and installed suite depends further on your system and particularly if you use shared libraries. In addition performing make realclean (see "Reducing Disk Space Requirements for Executables") can further reduce the size of the final installation (in the past reductions of ~30% have been observed).
Please note also that some programs use a lot of temporary scratch space to execute - you will need 10s of MBytes free in the directory pointed at by the environment variable $CCP4_SCR, see Scratch Directories.
The CCP4 graphical user interface CCP4i requires the Tcl/Tk and BLT packages to be installed on your system before it can be run. (See the separate CCP4i Installation page for more details).
Imosflm, the new mosflm interface requires various other tcl-tk extensions. These are
MrBUMP, XIA2 and the cctbx and CCP4 dispatchers require that the Python scripting language is installed on your system.
The CCP4 packages can be obtained directly from the CCP4 FTP site, or via the download pages. The pages are the recommended route for selecting the appropriate files based on the target system and the user's requirements. For Linux and Mac systems, one can alternatively use a package manager to download the binary packages (recommended for Linux binaries).
The pages have an associated FAQ page which is recommended for more detailed information.
The CCP4 FTP site is at ftp.ccp4.ac.uk and arranges the downloadable files (which are mostly in the form of tar archive files compressed with gzip) in a browsable hierarchy. At the top level are each of the available packages, followed by the package version, within which are a number of subdirectories:
---ccp4---6.3.1-----|--binaries--|--ccp4-6.3.1-linux-i386.tar.gz (32 bit) | |--ccp4-6.3.1-linux-x86_64.tar.gz (64 bit) | |--(plaform specific binaries) |--packed |--ccp4-6.3.1-core-src.tar.gz (source code) |--patches (patches to source code release) |--prerelease (prerelease software) ---cctbx_based ---|--binaries-------(platform specific binaries) |--packed -------(source code archive) ---ccp4mg-----2.6.0-----binaries-----(platform specific binaries) ---coot -----0.6.2-----binaries-----(platform specific binaries) ---chooch-----5.0.2--|--binaries----(platform specific binaries) |--packed ----(source code archive) ---balbesdb (balbes database)
If you are installing the binary packages on Linux, then first you need to decide where you want to put the software: for example this might be /usr/local (following usual Linux convention). (The chosen directory is later referred to by the environment variable $CCP4_MASTER.)
FTP the compressed tar files for each of the packages that you wish to install (for example ccp4-6.3.1-linux-x86_64.tar.gz, chooch.tar.gz etc) to this directory. See CCP4 Packages for information about each of the available packages.
This section describes the process for manually installing the suite from source code on Linux and Unix-based platforms.
The following sections outline the procedures for the following packages:
Manual installation of the core CCP4 and Phaser/CCTBX packages are described below:
Uncompress the gzipped files:
> gzip -dc ccp4-6.3.1-core-src.tar.gz | tar xvf -
> gzip -dc ccp4-6.3.1-phaser-cctbx.tar.gz | tar xvf -
This will create a new ccp4-6.3.1 directory, in turn containing a set of new directories (see Directory Structure for details).
Provided that the Phaser and CCTBX source code are present in the correct locations within the CCP4 directory tree (see Directory Structure) there are no other special actions to take - Phaser and CCTBX will be built as part of the standard installation. Conversely the configure options --disable-phaser and --disable-cctbx will suppress the installation of the two packages - see Configure Options.
> echo $SHELL
which will return something like /bin/tcsh or /bin/bash. The following assumes you are working in (t)csh, but applies to other shells.
If you have a previously installed CCP4 Suite it is worthwhile looking at the old $CCP4/include/ccp4.setup as it may save re-editing the new file (but do a comparison to check there are no new features).
The setup file is sourced by users (typically from their .cshrc file) to create the environment to run CCP4 programs. You will need to customise this file for your site.
|> cd ccp4-6.3.1/include|
|> cp ccp4.setup-csh.in ccp4.setup||(to make two copies of the file -
one is left unchanged for reference).
Edit ccp4.setup following the instructions in the comments - the minimum that you will need to do is:
Source this file - the setting of the environment variables is necessary for the rest of the installation:
> source ccp4.setup
This script also places CCP4 executables and scripts on the user's path, places any shared libraries in LD_LIBRARY_PATH, and sets some other paths. ccp4.setup has an option ccp4_first_in_path to determine the priority of CCP4 programs. Be aware that there may be name conflicts with non-CCP4 programs.
See Introduction to compilers, configure and make if you are a novice installer.
Make sure that you are in the top CCP4 directory and look at a listing of currently supported systems using the configure help option (see Configure Systems for more information - essential if you have multiple operating systems). Then start the configure process proper:
> cd $CCP4
> ./configure help
> ./configure system options
where the system is one of those listed by configure help, and must be supplied. There are options for setting non-default directories, for controlling whether or not certain parts of the Suite are built, and for overcoming possible problems. The options that you are most likely use are --with-shared-libs and --with-x. See Configure Options for details and a full list of options.
For academic/non-profit users of the software, you should return a completed and signed copy of the current CCP4 licence agreement, if this has not already been done. Copies of the academic licence agreement can be found in two formats in the $CCP4 directory:
It is possible to tweak the installation of some programs before starting the compilation:
MOSFLM: the Mosflm files include two shell scripts
to_small.csh and to_big.csh (both in the
$CCP4/x-windows/Mosflm/ directory). to_small.csh
can be run prior to performing the make step, and changes the
parameters to make a smaller Mosflm interface window suitable for
e.g. laptop displays.
to_big.csh restores the parameters for the standard window size.
REFMAC5: the Refmac5 files include two shell scripts
to_small_refmac.csh and to_big_refmac.csh
(both in the $CPROG/refmac5_4_/ directory).
to_small_refmac.csh reduces the array sizes used in
Refmac5 for machines running with smaller memory.
to_big_refmac.csh restores the array sizes for the standard Refmac5 installation.
When the configuration is complete be sure that you are in the $CCP4 directory and then run make (to build the executables) and make install to move executables from the src to the bin directory (unless an alternative target directory was specified with the configure --bindir option).
A reminder for a few people using the configure --bindir option: if you intend to reuse the same executables directory as for a previous CCP4 installation then remove the installed executables with make uninstall.
|> cd $CCP4|
|> make||(or make > make.log to keep a log file)|
|> make install||(or make install >> make.log to append to the log file)|
|> make runtest||(optional test of the most important programs)|
It is generally unnecessary but if you want to test the suite before installation look at Testing.
You can save disk space by removing many files which have been moved elsewhere:
> make realclean
or, rather than completely deleting files you can leave dummy copies of the executables so that if the sources get updated, make will only rebuild the relevant ones:
> make clean empty-targets
If you have problems see Problems.
The CCP4Dispatchers package provides wrappers for CCP4 programs using the Python scripting language. This package is automatically generated during installation of the binary distributions of CCP4. However, the manual step of editing the ccp4 setup file for source installations means that running the dispatcher generator must also be done manually.
In the directory $CCP4/libexec there are two python scripts, envExtractor.py and dispatcherGenerator.py. The first of these is used with your ccp4.setup file to create a CCP4 environment definition file:
> cd $CCP4/libexec
> python ./envExtractor.py /path/to/your/ccp4.setup
This creates the file 'ccp4-env.sh' containing all environment definitions from your ccp4.setup file in an easy-to-parse format. Once this file is created, the dispatcherGenerator can be run to generate dispatchers for the programs in $CCP4/bin. The commands below do so, creating the package under $CCP4/share/python. See the documentation for further explanation of use of the dispatcherGenerator and the CCP4Dispatchers package it creates.
> python ./dispatcherGenerator.py -p $CCP4/share/python -l $CCP4/share/python/CCP4Dispatchers/bin ./ccp4-env.sh $CCP4/bin
> mv ccp4-env.sh $CCP4/share/python/CCP4Dispatchers/bin/
If you have Tcl/Tk/BLT installed, and have set the environment variables in ccp4.setup correctly, then you should be ready to go. Just type ccp4i. Information on Tcl/Tk/BLT and on configuring the interface for local preferences can be found in Installation of CCP4 Interface.
Uncompress the gzipped file :
> tar zxf chooch.tar.gz
This will create a new chooch directory, in turn containing a set of new directories plus a README document:
CHOOCH does not have to be unpacked in any special place relative to the rest of the CCP4 suite.
Building CHOOCH requires linking against the GSL and Cgraph libraries, which may already be on your system or else easily installable from the installation disks or internet. Otherwise the versions supplied with the CHOOCH distribution should be built and installed first.
Hint: By default both GSL and Cgraph will attempt to install themselves in /usr/local. If you don't have write permission to this area then you can reset the install location using the --prefix=PATH option of the GSL and Cgraph configure scripts to install in a different place. Be aware that in this case you will need to tell the CHOOCH configure script where to find GSL using the --with-gsl-prefix=PATH, and where to find Cgraph using --with-cgraph-prefix=PATH.
Other options that are useful if not installing GSL and Cgraph in system-wide locations are --enable-static and --disable-shared, which will improve the portability of the resulting CHOOCH binary.
A recipe for installing CHOOCH from source code might look like:
# GSL build and install cd chooch/gsl-1.4/ ./configure --prefix=mylocal --enable-static --disable-shared make make install # Cgraph build and install cd ../../chooch/cgraph-2.04/ ./configure --prefix=mylocal --enable-static --disable-shared make make install # CHOOCH build and install cd ../../chooch/chooch-5.0.2/ ./configure --with-cgraph-prefix=mylocal --with-gsl-prefix=mylocal --prefix=$CCP4 make make install cd ../..
For manual build instructions for CCP4mg source code, please refer to the online instructions: http://www.ccp4.ac.uk/MG/ccp4mg_help2/build.html.
If you have downloaded the precompiled executables instead, the CCP4mg archive file contains everything that is required to run the software.
Once you have downloaded the CCP4mg gzipped archive file, uncompress it
using e.g. :
> gzip -dc ccp4mg-2.6.0-Linux-CentOS-5.8-i386.tar.gz | tar xvf -
This will create a new ccp4mg-2.6.0 directory in the current directory. Note that this does not have to be unpacked in any special location relative to the CCP4 suite. To start CCP4mg, execute the command e.g. :
CCP4mg does not require ccp4.setup to be sourced in order to operate. However adding the CCP4mg "bin" directory to your PATH in ccp4.setup will mean that you do not have to type the full pathname in order to invoke the program from the command line.
More information on CCP4mg can be found at the project website: http://www.ccp4.ac.uk/MG.
Large programs are written in several separate source files and so will be made up from several object files. Programs also require some standard 'libraries' of commonly used utilities - these are similar to object file and have file names like foo.a. There are some 'system' libraries which are a standard part of the operating system of your computer and some CCP4 libraries such as mtzlib and symlib which provide utilities to handle MTZ files and symmetry which are used by many programs. The CCP4 libraries also start out as source code in $CCP4/lib/src and need compiling too. The next stage after compiling is linking (this is still done by the 'compiler') which is pulling together all the object files and libraries needed for the program to work and creating the 'executable' file (usually just called foo).
There is an alternative approach to linking programs with the libraries and that is using 'shared libraries'. Linking programs to the default archive library foo.a involves including the necessary library routines in the program executable. Over the suite, therefore, the same code is included many times, and a lot of extra disk space is used. In contrast, when a program is linked to a shared library foo.so (created from foo.a by the link editor ld) only a reference to the library routine is included - it is not actually loaded until runtime. This means that the program executable takes up less disk space, but also that the program must be able to find the library routine at runtime. For this to work requires that the environment variable LD_LIBRARY_PATH is set correctly. This variable is a list of the directories that should be searched to find shared object libraries; it may include the names of several 'system' libraries or libraries for non-CCP4 programs but should also include $CCP4/lib.
The process of compiling and linking a complex system such as the CCP4 Suite is best done by a utility called make which interprets the instructions in a makefile. The great usefulness of make is that the system 'understands' the dependencies between the various levels of the programming system so that, for example, if you change one file which is used to build many different programs than the make procedure 'knows' to recompile and relink all the programs that are affected by the change. The make procedure will also control other necessary functions such as installing the software (which is basically moving files from where they are first built to where you want them to be used) or any configuring of data or documentation files. It will also clean up by deleting intermediate files.
It is usual in a complex program system to have several makefiles, one in each directory, always called Makefile. Within CCP4 there are separate Makefiles in several directories. There is a master Makefile in the $CCP4 directory which 'calls' all the other makefiles and so to run the make should only require typing the command make in the $CCP4 directory - this command will, by default, use the file called Makefile. The make utility can be got to perform some of its other functions by commands such as make install or make clean.
There is one more important part of the process, configure. The differences in operating systems and user requirements means it is impossible to write a makefile which would work for everybody. The way round this is to edit the makefile before it is run to customise it for your system. This process is done by a script called configure (which is in $CCP4 directory) which converts a standard initial file, Makefile.in, into a customised Makefile for your system. The configure process will do a lot of automatic checks on your system (do you have compilers? what system libraries are present?) and has a large number of options for you to set your own requirements.
So the minimal instructions you will actually need to type for a standard Unix installation are the trilogy:
> make install
(but see Manually Building the Suite on Unix/Linux before
trying it on CCP4!).
When running configure it is necessary to specify the system that you are installing on. The following systems are supported:
|linux||Linux operating system with GNU compilers|
|linux_intel_compilers||Linux operating system using Intel compilers|
|Darwin||Mac OS-X operating system with GNU compilers|
|Darwin_intel_compilers||Mac OS-X operating system using Intel compilers|
|ia64_linux_intel||Itanium Linux using Intel compilers|
|freebsd||FreeBSD operating system for x86 compatible, AMD64, DEC Alpha, IA-64, PC-98 and UltraSPARC® architectures|
If your system does not appear on the list of supported systems for configure then use generic.
where <flag> may be abbreviated uniquely and = may be replaced by whitespace. The currently-implemented flags of this form are:
Other common options are controlled by flags of the form --with-feature:
Other options are:
configure currently assumes that the shareable (plain-text) library files remain in $CCP4/lib/data rather than being installed elsewhere.
If the build is not done in $CCP4, configure creates src and lib directories mirroring those in $CCP4 and does the compilations there.
Configure examines the environment variables necessary for running the suite as defined by ccp4.setup and tries to check them as far as reasonable, but these are not actually used in the build process. There are a number of configuration parameters that you don't usually want to change, but whose defaults can be over-ridden by the values of environment variables when configure is run:
There are corresponding Makefile variables that you can override, e.g.:
Suppose you support several types of system and share the CCP4 source and system-independent data files across a network with a distributed file system such as NFS, AFS or RFS. Then it may be convenient to have a subdirectory in $CCP4 for each system in which to build and, possibly, in which to keep the resulting binaries. Directories for binaries can be anywhere, though.
The X-windows programs are built and installed by default and you must use the configure --disable-x option to suppress this.
If you are using the X-windows programs then you must ensure that the programs have access to the resources' relevant application defaults files. This is installation specific - there is an example for Unix users in ccp4.setup.
Increasing the optimisation level of the compiler might make the executables bigger or smaller. However, under aggressive optimisation the programs may not work at all. To do this see either Configure Options for ways to change the configuration parameters or use parameters to make, (for example make FOPTIM=-O ).
Using shared libraries will definitely make executables smaller
- use the configure argument --with-shared-libs to build a
shared version of the CCP4 library to link against. You must make sure
you have the correct setup, with LD_LIBRARY_PATH defined, to find the shared
CCP4 library at runtime. Beware, using shared libraries has caused
problems in the past when upgrading the operating system or moving executables.
All systems supporting the dlopen mechanism should be able to use shared libraries.
In extremis you can probably save some space by removing all debugging information from the executables using the strip program or installing using install -s (see Unix man pages for strip or install). This will prevent diagnosing some problems, though.
If you want to test the suite before installation then it is necessary to have the $CCP4/src directory, which contains the built executables, on the path, for example, for (t)csh:
> set path=($CCP4/src:$path)
This is not necessary if testing after installation, when the executables will be in the correct bin directory. You will find two example datasets in $CCP4/examples/toxd and $CCP4/examples/rnase, and some runnable scripts in $CCP4/examples/unix/runnable. The script $CCP4/examples/unix/runnable/run-all can also be used to automatically run the example scripts in sequence and report any failures. The run-all script can be launched by "make runtest" at the top level.
In addition a set of htmlised CCP4i-based tutorials are available in $CCP4/examples/tutorial. These tutorials cover various aspects of the structure determination process using the CCP4 programs, including Data Processing, MIR and MAD phasing, Molecular Replacement and Refinement.
The amount of testing we can do is limited and there may be installation problems particularly on less widely-used systems. If you have a problem then:
First check our problem page (http://www.ccp4.ac.uk/problems.php). As well as known problems with the most recent version of the suite, it is also worth checking the list of general problems (http://www.ccp4.ac.uk/problems/general-probs.php), which lists many platform and version independent issues.
If there is no reported fix then please try to solve the problem - if you suspect it relates to the library try building the library and running the test program for it with make testlib. If the problem is in building some of the programs (make fails on the srcdir target, having done the libdir target successfully) you can build as much as possible using:
|> cd $CCP4|
|> make -i|
|> cd $CCP4/src||(or cd $CCP4/unsupported/src)|
|> make instsome||(This will install the executables which are built and ignore what is missing)|
Finally if you can not solve the problem contact CCP4 (email@example.com).
If you find a solution please let CCP4 know; sending patches (made with
diff -c) is very helpful. If you successfully use a
configure option which reports itself as `untested' or `not properly tested'
please let CCP4 know.
setenv CCP4_SCR /scratch
Alternatively users can just use the subdirectory in their $HOME area so:
setenv CCP4_SCR $HOME/temporary
Note that the user interface, CCP4i, also has the idea of a TEMPORARY
directory which by default is the same as CCP4_SCR but this may need modifying
for some setups.
(Note that it is still necessary to set up and source the appropriate ccp4.setup file prior to running the configure and make steps.)
Please note that the following information is relevant to Unix-based installations built from source code. Windows-based installations can be uninstalled from Control Panel.
If it is necessary or desirable to undo the installation and start again from scratch (for example you wish to run the configuration with different options) then the following prescription should restore the original distribution:
|> cd $CCP4|
|> make uninstall||(Delete all the installed files, e.g. libraries in $CLIB and binaries in $CBIN)|
|> make distclean||(Delete all the files created in configuring and building the suite, including the Makefiles)|
It will then be necessary to re-run configure etc as if starting from scratch.
This procedure should also work for individual sections of the suite, for
example if you want to uninstall and remake the libraries then cd $CLIBS
instead before executing make uninstall and make distclean.
Completed licence agreements and requests for commercial software licences should be mailed or faxed to:
|Telephone: (+44) 1235 567726 (direct line)|
|Facsimile: (+44) 1235 567720|
|e-mail (preferred): firstname.lastname@example.org|
Technical queries including reports of problems with downloading, installing or running the software should in the first instance be sent to us by e-mail at email@example.com.