Funemployment

Hey everyone.  I've submitted my PhD thesis, and decided to take a little break between school and starting my next phase of life.  I will be traveling for the next few months and not posting here.  If you want to keep up with my travel adventures, please check out my travel blog: travelerjess.blogspot.com

Making Awesome Latex Tables

Yep, still disserating... thus all the posts are more about the process of writing than actual science research.  Here is a post about how to make awesome looking tables using latex and deluxetable.

I like using the deluxetable style in latex because the tables look nicer than normal latex tables, and you have more flexibility.  Below is an example of a cool table with come nice features that I made for my thesis:

Latex Code:

\begin{deluxetable}{cccccccc}

\tabletypesize{\footnotesize}

\tablecolumns{8} 

\tablewidth{0pt}

\tablecaption{ Cross-correlation Fit Details

\label{tab:lumresults1}}

\tablehead{

\colhead{QSO} \vspace{-0.2cm}& \colhead{$R$ Range} &  & & &  &\colhead{Separation} & \colhead{Result}\\ \vspace{-0.2cm}

& & \colhead{$\langle f \rangle$} & \colhead{$r_0$} &  \colhead{$\gamma$}& $W$ & & \\

\colhead{Division} & \colhead{(Mpc/h)} & \colhead{} & \colhead{} & \colhead{}&& \colhead{(\%)} & \colhead{Strength}} 

\startdata 

\vspace{-0.2cm} \nicefrac{1}{3} Bright &  & $4.24 \cdot 10^{-4}$ & 6.19 && 96.97 & &\\ \vspace{-0.2cm}

& [0.3,3] & & & 1.77 & &  96.7 & 1.9$\sigma$ \\ 

 \nicefrac{2}{3} Dim & & $4.26 \cdot 10^{-4}$ & 4.48 & & 52.77

\enddata

\vspace{-0.8cm}

\tablecomments{Luminosity dependent quasar clustering using a cross-correlation technique between CS82 galaxies ($M < 23.5$) and SDSS, BOSS, and 2SLAQ quasars \hbox{($0.5$)}}

\end{deluxetable}

Note that you need the file deluxetable.sty from here.

Fun with Python Axes

Sometimes you want to make a plot with two axes, and sometimes you also want the tickmarks on both those axes to look a certain way.  Here's how you do it:

fig = plt.figure(figsize = (11,10))

ax = fig.add_subplot(111)

for i in range(size(magcuts)): ax.plot(xdata[i],ydata[i], label=label[i], linewidth = 2.0)

leg = ax.legend(loc=3,shadow=False,prop=FontProperties(size=25),fancybox = True)

leg.get_frame().set_alpha(0) 

ax.set_xlabel("$\\chi \\mathrm{\\ (Mpc/h)}$",fontsize=30)

ax.set_ylabel("$f(\\chi)$",fontsize=30)

bx = twiny()

bx.plot(xdata2,ydata,alpha=0)

bx.set_xlabel("$\\mathrm{redshift\\ }(z)$",fontsize=30)

ticklabels = bx.get_xticklabels()

for label in ticklabels:

    label.set_fontsize(18)

    label.set_family('serif')   

ticklabels = ax.get_xticklabels()

for label in ticklabels:

    label.set_fontsize(18)

    label.set_family('serif')            

ticklabels = ax.get_yticklabels()

for label in ticklabels:

    label.set_fontsize(18)

    label.set_family('serif') 

    

    

filename = "plot2save"

savefig(filename + '.eps',format = 'eps', transparent=True)

savefig(filename + '.pdf',format = 'pdf', transparent=True)

savefig(filename + '.png',format = 'png', transparent=True)

TeXShop and .ps or .eps Files

I am frantically writing my thesis right now, so most of my "research" has to do with struggling with LaTex at the moment.

I am using TeXShop on Mac OS X to LaTeX my thesis.  I've encountered the problem that TeXShop doesn't seem to like .ps or .eps files.   So it requires some finagling to get TeXShop to handle these files.  Here are some tricks I've found:

1) Convert the .(e)ps files to PDF:

In a terminal window write the following command
ps2pdf filetoconvert.ps
eps2pdf filetoconvert.eps

This will produce the following file: filetoconvert.pdf

If you want to do a whole slew of files in a directory you can use the following script:


for x in *.ps; do
echo $x;
ps2pdf $x ${x/%ps/pdf};
done

2) Use Conversion Packages in TeXShop:


\usepackage{epstopdf}

\usepackage{ps2pdf}

\usepackage{pslatex}

Pretty Plots - 2D Histogram with 1D Histograms on Axes

Note: This post has been cross-posted to AstroBetter with a slightly more readable version of the code.  You might want to check it out there.

In a long list of plots that Martin wants me to add to the paper draft I sent out last week, the following:

Is it possible to show the histograms projected along each axis in addition to the 2D density? I know people do this in IDL frequently, I'm not sure how to do this in matplotlib. If it's possible we could play a similar game with the L-z figure. The 1D histograms contain lots of useful information, including how significant our clustering detection is in each bin.

Ask and you shall receive Martin. Below is the "money plot" (the temperature 2D histogram which shows amplitudes the bright versus dim correlation function) with histograms on the side of either axis.

I based this plot on code from here.

There are some cool features that I'll describe in the comments below. I think this plot rocks.

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.ticker import NullFormatter
def makeTempHistogramPlot(xdata,ydata,rexp,filename=None,xlims=-99, ylims =-99 , \
nxbins = 50,nybins=50, bw=0, nbins=100,contours=1,sigma=1,line=1):
#bw = 0 for color, = 1 for black and white
#line = 0 for no line, =1 for line
#sigma = 1 for display % below line, =0 for not
#contours = 1 for display 1,2,3 sigma contours, = 0 for not.

# Define the x and y data
x = xdata
y = ydata

# Set up default x and y limits
if (xlims == -99): xlims = [0,max(x)]
if (ylims == -99): ylims = [0,max(y)]

# Set up your x and y labels
xlabel = '$\mathrm{Your\\ X\\ Label}$'
ylabel = '$\mathrm{Your\\ X\\ Label}$'
mtitle = ''

# Define the locations for the axes
left, width = 0.12, 0.55
bottom, height = 0.12, 0.55
bottom_h = left_h = left+width+0.02

# Set up the geometry of the three plots
rect_temperature = [left, bottom, width, height] # dimensions of temp plot
rect_histx = [left, bottom_h, width, 0.25] # dimensions of x-histogram
rect_histy = [left_h, bottom, 0.25, height] # dimensions of y-histogram

# Set up the size of the figure
fig = plt.figure(1, figsize=(9.5,9))

# Make the three plots
axTemperature = plt.axes(rect_temperature) # temperature plot
axHistx = plt.axes(rect_histx) # x histogram
axHisty = plt.axes(rect_histy) # y histogram

# Remove the inner axes numbers of the histograms
nullfmt = NullFormatter()
axHistx.xaxis.set_major_formatter(nullfmt)
axHisty.yaxis.set_major_formatter(nullfmt)

# Find the min/max of the data
xmin = min(xlims)
xmax = max(xlims)
ymin = min(ylims)
ymax = max(y)

# Make the 'main' temperature plot
xbins = linspace(start = 0, stop = xmax, num = nxbins)
ybins = linspace(start = 0, stop = ymax, num = nybins)
xcenter = (xbins[0:-1]+xbins[1:])/2.0
ycenter = (ybins[0:-1]+ybins[1:])/2.0
aspectratio = 1.0*(xmax - 0)/(1.0*ymax - 0)
H, xedges,yedges = N.histogram2d(y,x,bins=(ybins,xbins))
X = xcenter
Y = ycenter
Z = H

# Plot the temperature data
if(bw): cax = axTemperature.imshow(H, extent=[xmin,xmax,ymin,ymax], \
interpolation='nearest', origin='lower',aspect=aspectratio, cmap=cm.gist_yarg)
else : cax = axTemperature.imshow(H, extent=[xmin,xmax,ymin,ymax], \
interpolation='nearest', origin='lower',aspect=aspectratio)

# Plot the temperature plot contours
if(bw): contourcolor = 'black'
else: contourcolor = 'white'

if (contours==0):
print ''
elif (contours==1):
xcenter = N.mean(x)
ycenter = N.mean(y)
ra = N.std(x)
rb = N.std(y)
ang = 0
X,Y=ellipse(ra,rb,ang,xcenter,ycenter)
axTemperature.plot(X,Y,"k:",ms=1,linewidth=2.0)
axTemperature.annotate('$1\\sigma$', xy=(X[15], Y[15]), xycoords='data',xytext=(10, 10), textcoords='offset points',horizontalalignment='right', verticalalignment='bottom',fontsize=25)
X,Y=ellipse(2*ra,2*rb,ang,xcenter,ycenter)
axTemperature.plot(X,Y,"k:",color = contourcolor,ms=1,linewidth=2.0)
axTemperature.annotate('$2\\sigma$', xy=(X[15], Y[15]), xycoords='data',xytext=(10, 10), textcoords='offset points',horizontalalignment='right', verticalalignment='bottom',fontsize=25, color = contourcolor)
X,Y=ellipse(3*ra,3*rb,ang,xcenter,ycenter)
axTemperature.plot(X,Y,"k:",color = contourcolor, ms=1,linewidth=2.0)
axTemperature.annotate('$3\\sigma$', xy=(X[15], Y[15]), xycoords='data',xytext=(10, 10), textcoords='offset points',horizontalalignment='right', verticalalignment='bottom',fontsize=25, color = contourcolor)
else:
xcenter = N.mean(x)
ycenter = N.mean(y)
ra = N.std(x)
rb = N.std(y)
ang = contours*N.pi/180.0
X,Y=ellipse(ra,rb,ang,xcenter,ycenter)
axTemperature.plot(X,Y,"k:",ms=1,linewidth=2.0)
axTemperature.annotate('$1\\sigma$', xy=(X[15], Y[15]), xycoords='data', xytext=(10, 10), textcoords='offset points',horizontalalignment='right', verticalalignment='bottom',fontsize=25)
X,Y=ellipse(2*ra,2*rb,ang,xcenter,ycenter)
axTemperature.plot(X,Y,"k:",ms=1,linewidth=2.0, color = contourcolor)
axTemperature.annotate('$2\\sigma$', xy=(X[15], Y[15]), xycoords='data', xytext=(10, 10), textcoords='offset points',horizontalalignment='right', verticalalignment='bottom',fontsize=25, color = contourcolor)
X,Y=ellipse(3*ra,3*rb,ang,xcenter,ycenter)
axTemperature.plot(X,Y,"k:",ms=1,linewidth=2.0, color = contourcolor)
axTemperature.annotate('$3\\sigma$', xy=(X[15], Y[15]), xycoords='data', xytext=(10, 10), textcoords='offset points',horizontalalignment='right', verticalalignment='bottom',fontsize=25, color = contourcolor)

#Plot the % below line
belowline = 1.0*size(where((x - y) > 0.0))/size(x)*1.0*100
if(sigma): axTemperature.annotate('$%.2f\%%\mathrm{\\ Below\\ Line}$'%(belowline), xy=(xmax-100, ymin+3),fontsize=20, color = contourcolor)

#Plot the axes labels
axTemperature.set_xlabel(xlabel,fontsize=25)
axTemperature.set_ylabel(ylabel,fontsize=25)

#Make the tickmarks pretty
ticklabels = axTemperature.get_xticklabels()
for label in ticklabels:
label.set_fontsize(18)
label.set_family('serif')

ticklabels = axTemperature.get_yticklabels()
for label in ticklabels:
label.set_fontsize(18)
label.set_family('serif')

#Plot the line on the temperature plot
if(line): axTemperature.plot([-1000,1000], [-1000,1000], 'k-', linewidth=2.0, color = contourcolor)

#Set up the plot limits
axTemperature.set_xlim(xlims)
axTemperature.set_ylim(ylims)

#Set up the histogram bins
xbins = N.arange(xmin, xmax, (xmax-xmin)/nbins)
ybins = N.arange(ymin, ymax, (ymax-ymin)/nbins)

#Plot the histograms
if (bw):
axHistx.hist(x, bins=xbins, color = 'silver')
axHisty.hist(y, bins=ybins, orientation='horizontal', color = 'dimgray')
else:
axHistx.hist(x, bins=xbins, color = 'blue')
axHisty.hist(y, bins=ybins, orientation='horizontal', color = 'red')

#Set up the histogram limits
axHistx.set_xlim( 0, max(x) )
axHisty.set_ylim( 0, max(y))

#Make the tickmarks pretty
ticklabels = axHistx.get_yticklabels()
for label in ticklabels:
label.set_fontsize(12)
label.set_family('serif')

#Make the tickmarks pretty
ticklabels = axHisty.get_xticklabels()
for label in ticklabels:
label.set_fontsize(12)
label.set_family('serif')

#Cool trick that changes the number of tickmarks for the histogram axes
axHisty.xaxis.set_major_locator(MaxNLocator(4))
axHistx.yaxis.set_major_locator(MaxNLocator(4))

if(filename):
savefig(filename + '.eps',format = 'eps', transparent=True)
savefig(filename + '.pdf',format = 'pdf', transparent=True)
savefig(filename + '.png',format = 'png', transparent=True)

return 0

no display name and $DISPLAY environment variable

I've been spending a lot of time organizing my code for the cross correlation project to run more efficiently. Part of this is setting up my code so that it can process/run a lot of different runs simultaneously. It's been a lot of fun learning all the cool tricks I can do with python to help with this. I should blog about some of these tricks.... but not right now....

This blog is about an error I was getting when I tried to run my newly optimized code on riemann. I use qsub/PBS to submit jobs, and I was getting the following errors in my qsub.out file:

File "/clusterfs/riemann/software/matplotlib/1.0.0/lib64/python2.7/site-packages/matplotlib/pyplot.py", line 270, in figure **kwargs)
File "/clusterfs/riemann/software/matplotlib/1.0.0/lib64/python2.7/site-packages/matplotlib/backends/backend_tkagg.py", line 83, in new_figure_manager
window = Tk.Tk() File "/clusterfs/riemann/software/Python/2.7.1/lib/python2.7/lib-tk/Tkinter.py", line 1685, in __init__
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable

I know that when I run jobs on other riemann nodes using qsub, that the X11 forwarding doesn't work properly. Or in other words, I can't view any of the plots I make when I am running an interactive PBS session. However, for this code, I simply wanted to save plots to .pdf files. However, for some reason there isn't a simple command (that I can find) to plot something in python directly to a file. If anyone out there knows how to do this, please help me!

But I did come across the following from matplotlib's faq page:
~~~~~~~~~~~~~~~~~
How To Generate Python/Matplotlib Images Without Having A Window Appear

The easiest way to do this is use a non-interactive backend such as Agg (for PNGs), PDF, SVG or PS. In your figure-generating script, just call the matplotlib.use() directive before importing pylab or pyplot:

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
plt.plot([1,2,3])
plt.savefig('myfig')
~~~~~~~~~~~~~~~~~
You need to make sure to call matplotlib.use() before pyplot or you will get the following error

UserWarning: This call to matplotlib.use() has no effect because the the backend has already been chosen;
matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.
if warn: warnings.warn(_use_error_msg)

I actually had an alias set up so that I call pyplot when I open up python, so I was getting this error until I changed that.

So now, I don't the the "$DISPLAY environment variable" error, because python knows to use the non-interactive backend Agg instead of trying to display to the screen. Yippee!

Update

I've been pretty bad at blogging lately because I've been job-hunting and spending most of my time traveling around and giving job talks at various universities. However, I am back in Berkeley now and ready to finish my thesis! Apologies to anyone who reads this (probably just my parents) and was disappointed at my lack of posting for the past several months. I hope to get back to posting (almost) daily moving forward.

I've also started another blog for non-research related stuff that I am doing. Happy reading!