View source for Application development (Fimm)

== Modules ==

Environment Modules allows you to dynamically modify your user environment by using information provided by "modulefiles". This make it easy to change between environments or settings, e.g. the Intel compiler environment and the PGI compiler environment. If you have problems during compiling, running the "module list" command could help you see if you have missing or wrong environment modules loaded.

When writing a PBS job script (see [[Job execution (Hexagon)|Job execution]] for more information), the wanted environment has to be set inside the script using the modules command. The reason for this is that the user environment is not inherited by the PBS script. The same applies for interactive jobs (i.e. qsub -I).

The "module" command have several subcommands, e.g. "module avail". 

The following list shows some of the subcommands used with "module".

{|
| '''Subcommand'''                  ||   '''Description'''
|-
| avail                                   ||   Lists all available modules
|-
| list                                     ||   Lists the modules you are using
|-
| load "module_name"             ||  Loads module "module_name"
|-
| unload "module_name"          ||  Unloads module "module_name"
|-
| show "module_name"            ||  Displays "module_name"'s configuration settings
|-
| swap "old_mod" "new_mod"  ||  Unloads the "old_mod" and loads the "new_mod"
|}

To load the netcdf module into your environment you type:
 module load netcdf
If you want a specific version of the module you instead specify:
 module load netcdf/3.6.2
Please avoid using version numbers unless strictly necessary since older versions of packages may be removed at a later time.

If you want to change from the PGI compiler (default) to the Intel compiler you type:
 module swap pgi intel
You should also use swap if you want to load a different version of the same module, this will e.g. replace your current pgi version with 8.0.6:
 module swap pgi pgi/8.0.6

A complete list of subcommands can be found in the module man page or [http://modules.sourceforge.net/man/module1.html here].

Please note, if the module command does not work inside your job scripts, add the line "export -f module" to your ~/.bashrc file. This should be automatically set for new users and is only valid if your shell is bash.
For other shells you may source the corresponding file in /opt/modules/default/init/ inside your qsub script before you use any "module" command.

== Compilers and programming languages ==
Four different compilers are available on Fimm:
* PGI (default)
* GNU
* Intel

To switch between compilers module command must be used:
 module switch pgi gnu
By default the latest available version will be loaded. You can switch to another compiler version with e.g.:
 module switch pgi pgi/8.0.6

=== How to invoke the compiler ===

Compiling an application for use on the compute node should be done by the wrappers specified below. Running the command "module list" will give you list of software which is already loaded properly to your environment. 

====Compiling programs  ====
When using the compiler wrappers,  the wrappers take care of MPI and all additional modules switches/settings automatically.

{|
! colspan="2" | Compute node compiler wrappers
|-
| Fortran 90/95 programs || ftn
|-
| Fortran 77 programs || f77
|-
| C programs || cc
|-
| C++ programs || CC
|}
'''NOTE:''' These wrappers also handles MPI and openMP, so you should not compile with mpicc, mpif90 or similar, nor should you need to add any reference to MPI libraries in CFLAGS or similar variables.

Compiling the C program test.c can be done by the command:
 cc -o test.out test.c
Where test.out is my selected name of the executable file.

=== Compiler version(s) ===
Currently installed Programming Environments for compilers:
{|
! Compiler || Module
|-
| PGI || pgi
|-
| GNU || gnu
|-
| Intel || intel
|}

=== Frequently used compiler options ===
'''Compiling OpenMP programs'''
To activate OpenMP directives, compile and link with

Fortran:
{| border="1"  cellspacing="0"
| -mp=nonuma || for the PGI compiler
|-
| -mp || for the Pathscale compiler
|}

C and C++:
{| border="1" cellspacing="0"
| -mp || for the PGI compiler
|-
| -mp || for the Pathscale compiler
|}

=== Recommended compiler options ===
Normally if you use compiler wrappers all recommended options will be included.

In some cases you may need to use "--enable-static" during configure for running on compute nodes.

'''Usefull optimization flags for quadcore'''

When using PGI the "-tp barcelona-64" flag will improve the performance of your code. For Pathscale the flag for optimizing for quadcore is "-march=barcelona". These options are automatically provided by the module ''xtpe-barcelona''.

=== Recommended environment variable settings ===
We recommend you to have the module ''xtpe-barcelona'' loaded. It will automatically add recommended optimization flags.

Additionally, the "xt-libsci" module contains optimized versions of common scientific/math libraries (e.g. LAPACK, BLAS).

== Debugging tools ==

strace - trace system calls and signals.

http://people.gnome.org/~newren/tutorials/developing-with-gnome/html/ch03s02.html

gdb - The GNU Debugger

http://www.gnu.org/software/gdb/

==== Local file systems ====
On all compute nodes there are local disk which user can use 

compute-0-x

compute-1-x

compute-2-x 

compute-7-x
  
 50 GB /state/partition1

compute-3-x

compute-6-x

  85 GB /state/partition1

== Recommended reading ==


[[Category:Fimm]]