To use this you first need to compile your mpi binary in the usual way. Use mpisub to submit an mpi parallel job on the Myrinet nodes.
To invoke mpisub use the following:
mpisub nx1 ./myexec.exe ["arguments"] mpisub nx2 ./myexec.exe ["arguments"]This will submit an mpi parallel job using n compute nodes with either 1 or 2 cpus per node. For example:
mpisub 1x2 /home/nrcb/benchmarks/DLPOLY.parallel/execute/DLPOLY.X mpisub 2x2 /home/nrcb/benchmarks/DLPOLY.parallel/execute/DLPOLY.X mpisub 4x1 /home/nrcb/benchmarks/DLPOLY.parallel/execute/DLPOLY.X
would submit a 2,4 and a 4 process DLPOLY.X job running. The 2x2 job uses 2 compute nodes (2 processes per node) and the 4x1 job uses 4 compute nodes (4 processes per node). The job script that mpisub writes is placed in the current directory and named the same as the executable appended with .sh . If you don't specify a full path to the executable, mpisub assumes that it is in the current directory.
NOTE: Any arguments to the executable must be placed inside double quotes. e.g.
mpisub 2x1 ./PMB-MPI1 "-npmin 2 Pingpong Sendrecv"
You can set the QSUB_OPTIONS variable - this is a list of options to the qsub command. See man qsub for details. If you invoke mpisub without arguments it will print out a usage guide. Additionally you may set the SCORE_OPTIONS variable with a list of scrun options. See the html man pages for scrun in file:/opt/score/doc/html/en/man/index.html on magnum or at the pccluster.org web site for a list of scrun options. For instance you may want to turn on mpi_zerocopy, which you can do using:
setenv SCORE_OPTIONS mpi_zerocopy=onThen invoke mpisub.