Recapitulating CAM3.1 on Ranger

Not sure why this works, or whether getting to the bottom of it is useful.

=========

HOW TO BUILD CAM UNDER PGF90 on RANGER:

use the default Portland Group settings for MPI

cd to the root of the CAM tree, then issue the following

unsetenv USER_FC
module load netcdf
setenv INC_NETCDF /opt/apps/pgi7_2/netcdf/3.6.2/include
setenv LIB_NETCDF /opt/apps/pgi7_2/netcdf/3.6.2/lib/
setenv INC_MPI /opt/apps/pgi7_2/mvapich/1.0.1/include
setenv LIB_MPI /opt/apps/pgi7_2/mvapich/1.0.1/lib
mkdir buildpar
cd buildpar
../cam1/models/atm/cam/bld/configure -spmd

then edit the Makefile, line 190, replacing $(FC) with mpif90 .

then type

make

NOTE: Build takes about 7 minutes.

===

RUNNING ON RANGER NODES

You'd think they'd provide an example.

Before running CAM, I try to establish how to run something. I got a random MPI source off the net; stupidly, it has interactive I/) so my first run was inconclusive.

Anyway, after several bashes at it, I got this script

#$ -V
#$ -cwd
#$ -j y
#$ -A A-ig2 
#$ -l h_rt=00:10:00
#$ -q development
#$ -N test
#$ -o ./$JOB_NAME.out
#$ -pe 16way 16
ibrun ./a.out

which is passed to the queue submission command "qsub".

As far as I can figure

#$ -pe 16way 16

is the smallest possible allocation on ranger. And I'm only asking for ten minutes on the dev queue (also tried the normal queue). Yet it takes forever to get loaded.

qstat shows a job number but the queue is marked as empty. Should I worry about this?

SETTING UP THE RUN:

this namelist works for an initial run, based on a single CPU exepriment:

&camexp
 absems_data  = '/work/00671/tobis/inputdata/atm/cam/rad/abs_ems_factors_fastvx.c030508.nc'
 aeroptics  = '/work/00671/tobis/inputdata/atm/cam/rad/AerosolOptics_c040105.nc'
 bnd_topo  = '/work/00671/tobis/inputdata/atm/cam/topo/topo-from-cami_0000-09-01_64x128_L26_c030918.nc'
 bndtvaer  = '/work/00671/tobis/inputdata/atm/cam/rad/AerosolMass_V_64x128_clim_c031022.nc'
 bndtvo  = '/work/00671/tobis/inputdata/atm/cam/ozone/pcmdio3.r8.64x1_L60_clim_c970515.nc'
 bndtvs  = '/work/00671/tobis/inputdata/atm/cam/sst/sst_HadOIBl_bc_64x128_clim_c020411.nc'
 caseid  = 'camrun.bsi'
 iyear_ad  = 1950
 mss_irt  = 0
 nrevsn   = '/work/00671/tobis/camrun/restart/camrun.bsi.cam2.r.0021-01-01-00000'
 rest_pfile  = './cam2.camrun.bsi.rpointer'
 ncdata = '../inputdata/init/control_initial.cam2.i.0013-01-01-00000.nc'
 nestep  = 586943
 nsrest  = 0 
/
&clmexp
 nrevsn   = '/work/00671/tobis/camrun/restart/camrun.bsi.clm2.r.0021-01-01-00000'
 rpntpath  = './lnd.camrun.bsi.rpointer'
 fpftcon  = '/work/00671/tobis/inputdata/lnd/clm2/pftdata/pft-physiology'
 fsurdat  = '/work/00671/tobis/inputdata/lnd/clm2/srfdata/cam/clms_64x128_USGS_c030605.nc'
/

and using qsubscript

#$ -V
#$ -cwd
#$ -j y
#$ -A A-ig2 
#$ -l h_rt=03:10:00
#$ -q normal
#$ -N testCAM3
#$ -o ./$JOB_NAME.out
#$ -pe 16way 64
ibrun ./cam

issue

qsub < qsubscript

obviously the input data set is in $WORK/inputdata