IBM® Blue Gene® is a massively parallel computer system: The number of nodes in a Blue Gene system ranges from 1,024 to 65,536 or more. The nodes are grouped into partitions when running a job. Typical parallel Blue Gene applications are written using the Message Passing Interface (MPI) and submitted to the system using mpirun.

A new feature of the Blue Gene/P software for Version 1, Release 2, is support for applications running in high throughput computing (HTC) mode. In HTC mode, a different executable file can run on each node of the partition, and each runs independently of the others. These executables don't use MPI and are submitted to the system using the new submit command. For more information about HTC mode on Blue Gene, please see the High Throughput Computing on Blue Gene/P white paper.

Partitions and jobs running on the Blue Gene system are typically under the control of a scheduler such as LoadLeveler. The scheduler allocates partitions and calls mpirun to run an HPC job. LoadLeveler does not currently have direct support for submitting HTC jobs.

IBM Scheduler for High Throughput Computing on IBM Blue Gene/P is a lightweight scheduler that allows the user to reliably submit a large number of HTC jobs. It can be used with LoadLeveler to take advantage of LoadLeveler features such as partition management and accounting.

Included in this package are buildable scheduler source code in a TAR (tape archive) file, a source RPM, and a TAR file containing the binary RPMs built for PPC64 Linux®.

How does it work?

The scheduler server daemon keeps track of how many Blue Gene nodes are available, the state of all submitted jobs, and a queue of jobs to be run.

Client programs are provided in order to submit jobs to the scheduler server daemon, check the status of jobs, cancel jobs, and perform administrative tasks.

The startd daemons connect to the server daemon and wait for jobs to execute. The server daemon will send each startd daemon a job to run in round-robin order. When a startd daemon receives a job to run, it forks and executes the submit program. It uses the submit plug-in interface to monitor for errors. When the job is completed, the startd daemon sends the exit information to the server.

Brant Knudson is a staff software engineer for the Blue Gene Control System in Rochester, Minnesota. He joined IBM in 1999 and worked on the Tivoli® Directory Server (LDAP) before joining the Blue Gene project in 2003.