I want to utilize the fact that I can run the correlation function on multiple nodes of riemann. Eric helped me learn to use qsub to run a script on multiple nodes. I've got it working as follows:
In the file on riemann 091118jess.script there is the following:
#PBS -l nodes=1
#PBS -N jessjob1
#PBS -j oe
#PBS -M kirkpatrick@berkeley.edu
#PBS -V
cd /home/jessica/repository/ccpzcalib/Jessica/pythonJess
./runcode.script
where runcode.script contains the following:
python runReconstructionMock.py
I make both runcode.script and 091118jess.script executables, and then type in the terminal:
qsub 091118jess.script
I then exit from riemann/insure (didn't close it using x button, not sure if this makes a difference).
When I log back in the code is still running. (check using qstat)
Now I need to integrate this into the python code so that qsubs are generated for each correlation function for each redshift slice. This should make things go a lot faster.
In the file on riemann 091118jess.script there is the following:
#PBS -l nodes=1
#PBS -N jessjob1
#PBS -j oe
#PBS -M kirkpatrick@berkeley.edu
#PBS -V
cd /home/jessica/repository/ccpzcalib/Jessica/pythonJess
./runcode.script
where runcode.script contains the following:
python runReconstructionMock.py
I make both runcode.script and 091118jess.script executables, and then type in the terminal:
qsub 091118jess.script
I then exit from riemann/insure (didn't close it using x button, not sure if this makes a difference).
When I log back in the code is still running. (check using qstat)
Now I need to integrate this into the python code so that qsubs are generated for each correlation function for each redshift slice. This should make things go a lot faster.
The email from Eric:
----------
From: Eric Huff
Date: Wed, Nov 18, 2009 at 3:06 PM
Subject: qsub
To: Jessica Kirkpatrick
Here's a script example:
#PBS -l nodes=1
#PBS -l walltime=24:00:00
#PBS -m bea
#PBS -M emhuff@berkeley.edu
#PBS -V
ulimit -s unlimited
export PHOTO_DATA=/clusterfs/riemann/raid006/bosswork/groups/boss/photo/data
export PHOTO_REDUX=/clusterfs/riemann/raid006/bosswork/groups/boss/photo/redux
cd /home/emhuff/Coadds/Code/Coadd_Data_Root/Scripts
echo `hostname`
/home/emhuff/Coadds/Code/Coadd_Data_Root/Scripts/r.out 380720 6 0 00
And here's some online documentation for qsub:
http://www.doesciencegrid.org/public/pbs/qsub.html
http://njal.physics.sunysb.edu/tutorial-1.html
I'm running the full reconstruction on the mock data, I might have a fully working code tomorrow. Woo hoo!
No comments:
Post a Comment