Worksheet 4

Worksheet 4
CS599 Parallel Programming
Introduction to OpenMP on Stampede

This worksheet is based upon the UNIX labs from the CS2351 course at Oklahoma State University, previously available at cs.okstate.edu/newuser/index.html. The purpose of this lesson is to learn to about command line debugging, pointers, threads, and processes in the C programming language on the Stampede Supercomputer.

1. Enter, compile, and run the following application:

#include <stdio.h>

int main(int argc, char ** argv)
{
int arr[3] = {1,2,3};
int x = 7;
int * intPointer = &x; //This is an integer pointer and contains the
//address of x.

printf("Welcome to C!\n");
printf("arr contains %d, %d, and %d\n", arr[0], arr[1], arr[2]);

printf("x is %d, the address of x is %u, the address of arr is %u\n", x, intPointer, arr);

//Notice that an integer pointer can also hold the address of an integer array.
intPointer = arr;
printf("The address of arr is %u\n", intPointer);

//The * operator, when applied to a pointer, dereferences a pointer or an address
intPointer = &x;

printf("Dereferencing the address of x gives its value, which is %d\n", *intPointer);

printf("Dereferencing arr provides arr[0] which is %d\n", *arr);
printf("Dereferencing arr+1 provides arr[1] which is %d\n", *(arr+1));
printf("Dereferencing arr+2 provides arr[2] which is %d\n", *(arr+2));

return 0;
}

You may type this file into the nano text editor, or copy and paste it.

PuTTY users will open PuTTY and connect to the LittleFe server as shown in class. Log on to LittleFe and create a new file called Welcome.c with the nano editor. Users wishing to copy and paste the text into nano should return to this webpage, highlight the text of the source code, right click, and click copy. Return to the nano editor, and right click inside the editor. The text of the source code should now be inside the editor. Press Ctrl-X to save and exit from nano.

Non-PuTTY users will log on to LittleFe as usual and create a new file called Welcome.c with the nano editor. Users may copy the code from this website. When inside the nano editor, right click and choose paste. The text of the source code should now be inside the editor. Press Ctrl-X to save and exit from nano.

Introduce the following errors, one at a time to the program Welcome.c. Compile the program and write down any error messages that the compiler produces. Fix the previous error message before you introduce a new error. If no errors were produced, explain why.

a. Change argc to argx.

b. Change Welcome to welcome.

c. Remove the first quote in the printf statement.

d. Remove the last quote in the printf statement.

e. Change main to mai.

f. Change printf to println.

g. Remove the semicolon at the end of the printf statement.

h. Remove the last curly bracket in the program.

i. Remove arr[0], arr[1], and arr[2] from the second printf statement.

j. Change arr[0] to arr[4].

k. Change arr[0] to arr[50000] (this should cause a segmentation fault).

2. The following program prints hello from ten OpenMP threads. Read through the following program and run it.

#include <stdio.h>
#include <stdlib.h>
#include <omp.h>

#define NUM_THREADS 10
//Add your static array here

//Define our thread function
void HelloThread()
{
long tid = omp_get_thread_num(); // This thread's identifier
//Modify the static array here

printf("Hello from thread %ld\n", tid);

}

int main(int argc, char ** argv)
{
long i; //Counter to keep track of the current thread

#pragma omp parallel num_threads(NUM_THREADS)
{
HelloThread();
}

//Print the static array here
return 0;
}

To compile the program above assuming it is named ws4Threads.c, use the following command on any Linux system:

gcc ws4Threads.c -fopenmp -o ws4Threads.exe

3. Add a static integer array of size NUM_THREADS immediately after the line containing the #define in the thread program from problem 2. The code for this array follows below.

static int arr[NUM_THREADS];

Set arr[(int) tid] to a value of your choice within each thread. Note that this value must be different for every thread. It could be the same as the thread identifier, if you wish.

Print the contents of this array immediately before you end the main thread.

4. Compile and run the both the program from problem 3 and the following program on Stampede (Host Name: stampede.tacc.utexas.edu ). You should be able to login to Stampede using PuTTY and your Stampede username. Note that you are not allowed to run your program (also called a job) on the login node. Instructions to submit, run, check, and, if necessary, to kill job follow below. Note that Stampede runs a variation of Linux called CentOS. Most of the commands you used on Littlefe such as ps, ls, cat, nano, etc., work on Stampede.

On Stampede, you must submit batch scripts to run your programs. This means your job waits in a virtual line before it may run. The batch queue scheduler on Stampede is called SLURM (like the drink on Futurama). Write a SLURM batch script for your program as follows:

#!/bin/bash
#SBATCH -A TG-SEE150001 #Account number
#SBATCH -n 1            #Number of tasks (processes)
#SBATCH -J helloWorldOMP    #Job name
#SBATCH -o hwOMP.o%j      #Output file name
#SBATCH -p normal        #Queue to use
#SBATCH -t 00:15:00    #Run (wall) time 15min
export OMP_NUM_THREADS=16
./hello_world_omp.exe

Save this batch script in a file called helloOMP.sbatch on Stampede.

Compile your program on the head node using the following command. Note that this is ok because the compilation uses very few resources.

icc hello_world_omp.c -openmp -O3 -o hello_world_omp.exe

Note that the executable file name matches exactly with what will run in the batch script.

Fix any errors in your program, and submit your program to the batch queue using the following command:

sbatch helloOMP.sbatch

Once your program finishes, the output will show up in a file called hwOMP.oJobNumber, where JobNumber is the job number assigned to your job by Stampede. You can find this file by using the ls command.

While your program is in the batch, run the following commands on Stampede. If necessary, submit your program to the batch queue again and quickly run the commands below. Note that the up and down arrow keys can be used to scroll through previously issued commands.

showq

What happened after running showq?

squeue -u Username

Where Username is your Stampede username. What happened after running the command above?
Next, try creating and running an MPI Program as shown below.

#include <stdio.h>
#include <stdlib.h>
#include "mpi.h"

int main (int argc, char** argv)
{
int number_of_processes;
int my_rank;
int mpi_error_code;

mpi_error_code = MPI_Init(&argc, &argv);

mpi_error_code = MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);

mpi_error_code = MPI_Comm_size(MPI_COMM_WORLD, &number_of_processes);

printf("%d o%d: Hello, world!\n", my_rank, number_of_processes);

mpi_error_code = MPI_Finalize();
}

Save this program in a file named hello_world.c on Stampede.

Create a SLURM batch script for your program as follows:

#!/bin/bash
#SBATCH -A TG-SEE150001 #Account number
#SBATCH -n 16            #Number of tasks (cores)
#SBATCH -J helloWorld    #Job name
#SBATCH -o hw.o%j      #Output file name
#SBATCH -p normal        #Queue to use
#SBATCH -t 00:15:00    #Run (wall) time 15min
ibrun hello_world.exe

Save this batch script in a file called hello.sbatch on Stampede.

Compile your program on the head node using the following command. Note that this is ok because the compilation uses very few resources.

mpicc hello_world.c -O3 -o hello_world.exe

Note that the executable file name matches exactly with what will run in the batch script using ibrun.

Fix any errors in your program, and submit your program to the batch queue using the following command:

sbatch hello.sbatch

Once your program finishes, the output will show up in a file called hw.oJobNumber, where JobNumber is the job number assigned to your job by Stampede. You can find this file by using the ls command.

showq

What happened after running showq?

squeue -u Username

Where Username is your Stampede username. What happened after running the command above?

Note that you may have a program that runs in an infinite loop or enters a condition called deadlock or livelock. To end such a program that is running on Stampede, you may first determine the job number using squeue as shown above and killing the program with the following command:

scancel JobNumber