Notes from the Beach

I have spent the last two weeks at two conferences. First the American Astronomical Society (AAS) 215th Meeting in Washington DC, second Essential Cosmology for the Next Generation: Cosmology on the Beach in Playa del Carmen, Mexico. Both were really helpful for both learning new astrophysics, as well as getting lots of new ideas for my work.

I thought I would summarize here the highlights (with links to talks, and a list of things I would like to look into myself).

Max Tegmark coined the phrase "Goldilocks Galaxies" when talking about the Luminous Red Galaxies (LRGs) in SDSS. This is because "Quasars are too sparse, common galaxies are too dense, but LRGs are just right."

Mark also talked about 21 cm tomography comparing it to the CMB in it's potential importance to cosmology. His description unfortunately was pretty general and I don't think I understand how it works. Need to read a review paper on this. Looks like this might be a good one.

Nicholas Suntzeff had an interesting slide at the end of his talk where he gave advice to young cosmologists about how to be competitive for jobs. I've repeated it below:

• Don’t keep on doing your thesis over and over again
• Establish prominent collaborators and mentors, but appear independent
• Publish, publish, publish. Include useful tables of summary and colorful figures that can be easily captured.
• Apply for external funding
• Luck verus hard work
• Become the leader in your field
• Think carefully about joining large projects with time scales of > 5 years.
• Spergel’s law
• Don’t be afraid to go out on a limb and say something weird (people will remember you).
• The Aaronson effect in obsevations.
• When you apply for jobs, make sure you know all about the department – and brown nose a bit. Write your application as if there is no other job out there. Know your audience.
  

Berian James is a new post-doc at the BCCP starting this spring. He had some ideas about Newman project and had talked to others who are working on similar things (he named John Peacock and Hannah Parkinson at Edinburgh). He also had an interesting idea that it would be great if we could positions from photometry and a redshift distribution and generate a correlation function from these two things. He was wondering if anyone has done this. I should talk to him more about this when he gets to Berkeley.

Dovi Poznanski suggested I look at quasar variability using overlapping Sloan plates. Apparently 20% of the data has multiple epochs. Josh Bloom also told us that we could get good variability information using Palomar data. Nic Ross and I should set up a meeting with him when we get back.

Daniel Matthews is Jeff Newman's graduate student. He presented a poster at the AAS (see Jan 2nd email, subject: poster, he sent me a copy). In this poster he talked about doing redshift distribution reconstructions of "DEEP-like" data. He does something different than what Newman does in his paper. He claimed that he couldn't get Newman's method to work -- this is discouraging. Maybe it's time to try to talk to Newman directly about the project to clarify this?

The Joe Wolf Effect.

Marilena LoVerde discussed lensing of quasars from hydrogen in the ly-alpha forest. I talked to her afterwards and it doesn't seem like this effect his that large, but might be relevant for BOSS/BIG BOSS. The talk isn't posted on the Cosmology on the Beach web page as of right now.

Eyal Kazin has a web page with LRG catalogs. He said he is interested in the Newman Project and is very familiar with SDSS data, and to contact him if we want help. These LRGs might be useful data to use instead of the ones I downloaded from CAS, and don't really trust.

I had some thoughts about if I should request authorship on future DRIFT analysis papers, considering they didn't allow me to publish my work. I don't know what the protocol for this type of thing is, or if it even matters at this point in my career.

I had a great time at these conferences and learned a lot. Time to get back to real work now! Science to follow....

Single Epoch Likelihood

I've been playing with the single epoch likelihoods cross checked with the truth table. The rest of the targeting will be on single epoch measurements (i.e. not stripe 82) so these are a more accurate measurement of how well the likelihood is doing. I've been playing with adjusting various parameters like how we define "variability" in terms of the chi^2 cut, if we should add variable objects from the L_everything catalog to L_QSO.

Below are the raw numbers of quasars targeted (normalized to targeting of ~40/degree^2) as well as the accuracy of each method:

No variability Method
; number quasars
1029
;percent accuracy
0.545986
; total targeted
1881


Variable objects with chi^2 > 1.5 subtracted from L_everything
; number quasars
1047
;percent accuracy
0.556619
; total targeted
1881


Variable objects with chi^2 > 1.5 subtracted from L_everything
Variable objects with chi^2 > 1.5 added to L_QSO
; number quasars
1033
;percent accuracy
0.549176
;total targeted
1881


Variable objects with chi^2 > 1.1 subtracted from L_everything
; number quasars
1053
;percent accuracy
0.559809
; total targeted
1881



Variable objects with chi^2 > 1.2 subtracted from L_everything
; number quasars
1055
;percent accuracy
0.560872
; total targeted
1881


Variable objects with chi^2 > 1.3 subtracted from L_everything
; number quasars
1054
;percent accuracy
0.560340
; total targeted
1881


Variable objects with chi^2 > 1.4 subtracted from L_everything
; number quasars
1051
;percent accuracy
0.558745
; total targeted
1881


Variable objects with chi^2 > 1.2 subtracted from L_everything
Variable objects with chi^2 > 1.6 added to L_QSO
; number quasars
1036
;percent accuracy
0.550771
;total targeted
1881

Variable objects with chi^2 > 1.2 subtracted from L_everything
Variable objects with chi^2 > 1.5 added to L_QSO
; number quasars
1035
;percent accuracy
0.550239
;total targeted
1881

Variable objects with chi^2 > 1.2 subtracted from L_everything
Variable objects with chi^2 > 1.4 added to L_QSO
; number quasars
1034
;percent accuracy
0.549708
;total targeted
1881


Variable objects with chi^2 > 1.2 subtracted from L_everything
Variable objects with chi^2 > 1.3 added to L_QSO
; number quasars
1029
;percent accuracy
0.547049
;total targeted
1881

It seems that the method with the best accuracy (56%) is variable objects with chi^2 > 1.2 subtracted from L_everything.

Here is a plot of the L_everything (non variable) vs L_QSO:

Magenta objects are targeted QSOs.
Cyan objects are targeted stars.
Green objects are not targeted stars.
Red objects are missed qsos.

It looks like we might be able to make the L_ratio a function of L_QSO to perhaps pick up a few more of the red objects at the top right corner.

Another interested plot is looking at likelihood-ratio vs g-band flux:

The color scheme is the same as first plot

We might be able to get more missed QSOs (red) if we target faint (g-flux less than 5) objects with a L_ratio close to the cut. I've also plotted likelihood-ratio vs g-r magnitude:

The color scheme is the same as first plot

Again, we might be able to use the fact that the missed QSOs (red) are clustered.

-------------

Cool Tip of The Day:
To download file directly from web using the command line:
wget http://www.downloadingurl.com/filetodownload.fits

If it is from a password protected page (like the wiki) then do the following:
wget --http-user=myusername --http-password=mypassword http://www.downloadingurl.com/filetodownload.fits

Likelihood Optimatization

I've been playing with various parameters in the likelihood method trying to find the most efficient cuts. The three things I have been playing with are:

added errors (what to use as the input adderr into likelihood_compute)
everything - variability (what happens when we take away variable objects from the L_everything file)
QSO + variability (what happens when we add the variable objects to the L_QSO likelihoods)

Here are my findings (these are on the co-added fluxes, next step is to re-run with single epoch):

Normal Errors (adderr = [0.014, 0.01, 0.01, 0.01, 0.014])
Percent of those targeted are quasars (based on 40/degree^2 targeting)
No variability: 0.649914
Variability everything: 0.656196
Variability everything + QSO = 0.651057

5X Errors (adderr = 5*[0.014, 0.01, 0.01, 0.01, 0.014])
Percent of those targeted are quasars (based on 40/degree^2 targeting)
No variability: 0.641348
Variability everything: 0.645346
Variability everything + QSO = 0.603655

7X Errors (adderr = 7*[0.014, 0.01, 0.01, 0.01, 0.014])
Percent of those targeted are quasars (based on 40/degree^2 targeting)
No variability: 0.627641
Variability everything: 0.624786
Variability everything + QSO = 0.572244

No Errors (adderr = 0.0*[0.014, 0.01, 0.01, 0.01, 0.014])
Percent of those targeted are quasars (based on 40/degree^2 targeting)
No variability: 0.644203
Variability everything: 0.627070
Variability everything + QSO = 0.619075

It looks like the errors we were running have the best numbers, and using the variability everything, but not the variability QSO.

I am going to play more with the definitions of variable everything and variable qso to see if I can get these to work better. I also want to play with not cutting on a L_ratio = 0.01, but perhaps changing L_ratio as a function of L_QSO (it seems we might be able to get a few more objects if we have L_ratio cut decrease as L_QSO gets large.

Variable Likelihood

David Schelgel and I added variability information into the "everything" likelihood training file. This allows us to separate objects that vary (QSOs are included in this list) from objects that don't vary. This should improve our training by removing quasars from the "everything" file.

The training file now has chi^2 information of the changes in fluxes over repeat observations. The likelihood computation splits up L_EVERYTHING into chi^2 bins between the following :

L_EVERYTHING_ARRAY[0] : objs1.flux_clip_rchi2[2] LT 1
L_EVERYTHING_ARRAY[1] : objs1.flux_clip_rchi2[2] GE 1.1 AND
objs1.flux_clip_rchi2[2] LT 1.2
L_EVERYTHING_ARRAY[3] : objs1.flux_clip_rchi2[2] GE 1.2 AND
objs1.flux_clip_rchi2[2] LT 1.3
L_EVERYTHING_ARRAY[4] : objs1.flux_clip_rchi2[2] GE 1.3 AND
objs1.flux_clip_rchi2[2] LT 1.4
L_EVERYTHING_ARRAY[5] : objs1.flux_clip_rchi2[2] GE 1.4 AND
objs1.flux_clip_rchi2[2] LT 1.5
L_EVERYTHING_ARRAY[6] : objs1.flux_clip_rchi2[2] GE 1.5 AND
objs1.flux_clip_rchi2[2] LT 1.6
L_EVERYTHING_ARRAY[7] : objs1.flux_clip_rchi2[2] GE 1.6

Changing L_Ratio to be L_QSO / total(L_EVERYTHING_ARRAY[0:5] (which removes the L_EVERYTHING of the objects with variablility (chi^2 less than 1.5)), we get more QSOs. From the truth table data, using different chi^2 cuts:

L_ELSE = total(likechi.L_EVERYTHING_ARRAY[0:1],1) ; chi^2 LT 1.2

;new quasars selected
QNEWSELECT LONG = Array[77]
;new stars selected (not quasars)
SNEWSELECT LONG = Array[106]
; #new_quasars/#new_stars
0.726415

L_ELSE = total(likechi.L_EVERYTHING_ARRAY[0:2],1) ; chi^2 LT 1.3
;new quasars selected
QNEWSELECT LONG = Array[69]
;new stars selected (not quasars)
SNEWSELECT LONG = Array[92]
; #new_quasars/#new_stars
0.750000

L_ELSE = total(likechi.L_EVERYTHING_ARRAY[0:3],1) ; chi^2 LT 1.4
;new quasars selected
QNEWSELECT LONG = Array[67]
;new stars selected (not quasars)
STILLMISSING LONG = Array[159]
; #new_quasars/#new_stars
0.797619

L_ELSE = total(likechi.L_EVERYTHING_ARRAY[0:4],1) ; chi^2 LT 1.5
;new quasars selected
QNEWSELECT LONG = Array[65]
;new stars selected (not quasars)
SNEWSELECT LONG = Array[79]
; #new_quasars/#new_stars
0.822785

L_ELSE = total(likechi.L_EVERYTHING_ARRAY[0:5],1) ; chi^2 LT 1.6
;new quasars selected
QNEWSELECT LONG = Array[62]
;new stars selected (not quasars)
SNEWSELECT LONG = Array[72]
; #new_quasars/#new_stars
0.861111

This is L_EVERYTHING (chi2 less than 1.4) vs L_QSO
Green: Not quasars
Magenta: Quasars that the old likelihood method (no variability) targeted
Cyan: Quasars that the likelihood + variability method targets
Red: Quasars we are still missing (not targeting).

On another note, possible Thesis Title: "Putting Galaxies in their Place." Thoughts?

Back to the Likelihoods

It's been a while since I have worked on the likelihood QSO selection method. With the next deadline for target selection is coming up, it's time to go "Back to the Likelihoods." Note to self: It more time efficient to keep working on something continuously than to not work on it for weeks and then waste a day trying to remember what I was doing.

Where we left off...
Below is a color-color (ug - gr) plot of the final likelihood selection objects for the commissioning data.

The white data points are a random sampling of 20,000 possible objects to target. The red data points are objects who's likelihood ratio is greater than 0.1, where likelihood ratio is defined as:

where L_QSO and L_everything, as described in "A Likely Result" are defined as:

The green data points are objects are objects who's likelihood ratio is greater than 0.1 and L_everything is greater than 10^-6. This is eliminate classification of "fringe" objects that are not close to any objects (and therefore have a small everything likelihood.

The likelihood was then run on the co-added Stripe 82 data and all of the above green objects were submitted as targets for the commissioning data.

What we need to work out...

• Why are the likelihoods so small/large? The likelihoods should be a probability, but we have likelihood's spanning from 0-11.
  
• Why is our completeness and efficiency on the MMT data so poor?
• How does the likeliness compare to QSOs based on variability?
• How well does this method work on single epoch Stripe 82 data versus the co-added images?