Friday, March 30, 2012

no display name and $DISPLAY environment variable

I've been spending a lot of time organizing my code for the cross correlation project to run more efficiently. Part of this is setting up my code so that it can process/run a lot of different runs simultaneously. It's been a lot of fun learning all the cool tricks I can do with python to help with this. I should blog about some of these tricks.... but not right now....

This blog is about an error I was getting when I tried to run my newly optimized code on riemann. I use qsub/PBS to submit jobs, and I was getting the following errors in my qsub.out file:

File "/clusterfs/riemann/software/matplotlib/1.0.0/lib64/python2.7/site-packages/matplotlib/pyplot.py", line 270, in figure **kwargs)
File "/clusterfs/riemann/software/matplotlib/1.0.0/lib64/python2.7/site-packages/matplotlib/backends/backend_tkagg.py", line 83, in new_figure_manager
window = Tk.Tk() File "/clusterfs/riemann/software/Python/2.7.1/lib/python2.7/lib-tk/Tkinter.py", line 1685, in __init__
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable

I know that when I run jobs on other riemann nodes using qsub, that the X11 forwarding doesn't work properly. Or in other words, I can't view any of the plots I make when I am running an interactive PBS session. However, for this code, I simply wanted to save plots to .pdf files. However, for some reason there isn't a simple command (that I can find) to plot something in python directly to a file. If anyone out there knows how to do this, please help me!

But I did come across the following from matplotlib's faq page:
~~~~~~~~~~~~~~~~~
How To Generate Python/Matplotlib Images Without Having A Window Appear

The easiest way to do this is use a non-interactive backend such as Agg (for PNGs), PDF, SVG or PS. In your figure-generating script, just call the matplotlib.use() directive before importing pylab or pyplot:

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
plt.plot([1,2,3])
plt.savefig('myfig')
~~~~~~~~~~~~~~~~~
You need to make sure to call matplotlib.use() before pyplot or you will get the following error

UserWarning: This call to matplotlib.use() has no effect because the the backend has already been chosen;
matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.
if warn: warnings.warn(_use_error_msg)

I actually had an alias set up so that I call pyplot when I open up python, so I was getting this error until I changed that.

So now, I don't the the "$DISPLAY environment variable" error, because python knows to use the non-interactive backend Agg instead of trying to display to the screen. Yippee!

18 comments:

  1. Ah, THANK you for this. I spent forever trying to figure out how to save my images on a server when I didn't have the correct X11 authentication. I appreciate it mightily!

    ReplyDelete
  2. Also, I just looked at your blog information after posting that previous comment, and get this -- the program I needed this for I am writing for David Hogg! Small world...

    ReplyDelete
  3. Thanks But I want to see the plot on my SSH terminal, or how do I see the plot from the saved file?

    ReplyDelete
  4. Thanks..it helped..

    ReplyDelete
  5. Thank you! Really helpful, solved my problem straight away.

    ReplyDelete
  6. Thank you! Saved the day!!

    ReplyDelete
  7. Anyone working on this topic might want to look at:

    import os
    import tempfile
    os.environ['MPLCONFIGDIR'] = tempfile.mkdtemp()
    import matplotlib

    Setting the MPLCONFIGDIR var first. :)

    Have a good one ppl.

    ReplyDelete
  8. THANKS!!! This helped me a lot

    ReplyDelete
  9. Thanks a bunch..trying to figure this out for soo long!

    ReplyDelete
  10. Thank you! Very helpful when trying to plot over SSH connection. Many thanks!

    ReplyDelete
  11. This comment has been removed by a blog administrator.

    ReplyDelete
  12. Thanks a lot for help!
    This was really useful!

    ReplyDelete
  13. Still keeping on, two and a half years later. Thanks a lot.

    ReplyDelete
  14. april 2015 checking in... still fighting the good fight ! -uh, err I ended up dockerizing/vnc-ing , but this amazingly is still relevant !

    ReplyDelete
  15. Thank you Jessica for this great post. These tiny bugs are often the most frustrating! Thanks to you I only wasted 2 hours trying to figure out the issue before coming here.

    ReplyDelete
  16. Thank You very very much for your solution

    ReplyDelete