Phase 3 data reduction and keyword work to do

Reference : any referral to the SPD is the "ESO Science Data Product Standard" January 2013 release: http://www.eso.org/sci/observing/phase3/p3sdpstd.pdf

EFOSC2 spectroscopy files - to do

1. SPEC_RES= 359.5606862048376 / [nm] Reference spectral resolution (FWHM) DONE

the comment should be "Spectral resolving power", the number is correct - just the comment needs updated.

SPECSYS = 'HELIOCEN' / Observed frame DONE

change to

SPECSYS = 'TOPOCENT' / Observed frame

Sorry, I was wrong previously. Should be topocentric (i.e. as observed from La Silla,

no correction for earth or solar motion).

2. Can we delete these : DONE

HIERARCH ESO DPR CATG = 'SCIENCE ' / Observation category

HIERARCH ESO DPR TECH = 'SPECTRUM' / Observation technique

HIERARCH ESO DPR TYPE = 'OBJECT ' / Observation type

As the SDP manual now says : "However, the hierarchical products from the DPR category, like HIERARCH ESO DPR CATG, are an exception. Actually, science data products shall not contain keywords from the DPR category [AD3]. The value of the keyword DPR TECH is normally propagated to the keyword OBSTECH in the product tile."

3. The unit, or number here is wrong. The unit is required to be in degrees ! DONE

i.e. should be 2.778e-4 in degrees, for 1 arcsec slit

APERTURE= 1.0 / [deg] Aperture diameter

4. Unit here is wrong. You have it defined correctly, i.e. DONE

TELAPSE=MJD-END - MJD-OBS

But this is in days of course.

TELAPSE = 0.001620376155187842 / Total elapsed time [s]

You need to multiply by 24*3600 = 86400 to get it into seconds.

5. CCD parameters :

Summary of measurements here : http://wiki.pessto.org/efosc2-ccd40-quality-control-and-measurements

GAIN :

there is a discrepancy between what is written in the header for CCD40 and what is on the EFSOC2 webpages. The header value of 0.91 e-/ADU is wrong. Morgan has measured the gain and readnoise from April 2013 and October 2012 and finds GAIN = 1.18 e- per ADU. Hence I think we should change

HIERARCH ESO DET OUT1 GAIN = 0.91 / Conversion from electrons to ADU DONE

to

HIERARCH ESO DET OUT1 GAIN = 1.18 / Conversion from electrons to ADU

and also we need to add a header keyword :

GAIN = 1.18 / Number of electrons per data unit DONE

DETRON :

"Detector Readout noise in electrons"

We need to add this (Section 2.5 in the SDP manual). It is the detector read noise. Again there is a difference in the ESO header. From the same runs in April 2013 and October 2012, Morgan finds DETRON = 11.00 e- (October 2012) and 12.20 e- for April 2013. Hence I think we should change

HIERARCH ESO DET OUT1 RON = 7.80 / Readout noise per output (e-) DONE

to

HIERARCH ESO DET OUT1 RON = 11.6 / Readout noise per output (e-)

and also we need to add a header keyword :

DETRON = 11.6/ Detector readout noise in electrons DONE

EFFRON :

"Effective readout noise in electrons" SET as DETRON (at the moment)

set this equal to DETRON, as we have single science frames (not

combined) and typically we have a large number of bias and flat frames

which will reduce their contribution to EFFRON. SJS to CHECK THIS DURING REDUCTIONS.

6. Add keyword for LACOSMIC : DONE

vwe need to add a header keyword to note if cosmic ray cleaning has been applied

this should be boolean (T/F). We are encouraged to add our own keywords, as long as

they are standard, are useful and are described in the data release paper

I suggest :

LACOSMIC = T /TRUE if Laplacian cosmic ray rejection has been applied to the image

7. The 2D wavelength calibrated file

There's a revised way of submitting this, and making the fitsheaders compatible. It's a little

different to what we have, and needs some minor changes

The 2D file is an "associated ancillary data" product, not a product in itself. It needs

specifically linked from the 1D spectum file (i.e. in our *_1_e.fits file)

In this 1D spectrum file (*_1_e.fits), we must point to this, so we

need these headers in the original

*_1_e.fits file :

ASSON1 ='NAME of 2D file goes here....' / Name of associated file DONE

ASSOC1 ='ANCILLARY.2DSPECTRUM' / Category of associated file

Important - *delete* this header keyword from the 2D files (and only the 2D files): DONE

PRODCATG= 'SCIENCE.IMAGE' / Data product category

Delete it completely - as ESO don't want it. They just want the 1D spectrum file to reference the 2D "ancillary file.

EFOSC2 image files (i.e. photometry, not 2D spectral images)

SV: ALL THIS WERE NOT REAL PROBLEM, SINCE TEST WAS DONE ON NOT PROPER REDUCED IMAGES

1. Keyword value "ORIGIN" is set wrong in the image files. It should be "ESO"

FILE ORIGIN

LSQ12dyw_20120808_R642_56432_1.fits NOAO-IRAF FITS Image Kernel July 2003

SN2012ca_20120824_r784_56432_1.fits NOAO-IRAF FITS Image Kernel July 2003

SN2012ec_20120818_r784_56432_1.fits NOAO-IRAF FITS Image Kernel July 2003

tCHASEJ20032484-5557192_20120807_Gr13_Free_slit1.0_56432_1.fits ESO

tCHASEJ20032484-5557192_20120807_Gr13_Free_slit1.0_56432_1_2df.fits ESO

tCHASEJ20032484-5557192_20120807_Gr13_Free_slit1.0_56432_1_e.fits ESO

tCHASEJ20032484-5557192_20120807_Gr13_Free_slit1.0_56432_1_ex.fits ESO

tCHASEJ20032484-5557192_20120807_Gr13_Free_slit1.0_56432_1_f.fits ESO

tCHASEJ21253403-4016432_20120807_Gr13_Free_slit1.5_56432_1.fits ESO

2. Keyword TEXPTIME is missing in the imaging (but it is there in the spectral files). For EFOSC2 this is just set to EXPTIME, as we are not submitting merged products.

3. Following keywords are missing in the imaging (but are there OK in the spectral files)

MJD-END

PROG_ID

OB_ID

NCOMBINE

OBSTECH (should be set to "IMAGE")

PROCSOFT

REFERENC

PRODCATG (should be set to SCIENCE.IMAGE)

BUNIT (should be set to "ADU")

These also need added (as above for spectral files)

GAIN

DETRON

EFFRON

4. FLUXCAL is completely missing in the file LSQ12dyw_20120808_R642_56432_1.fits but it is set correctly in the others

/Users/sjs/data/pessto/Phase3/May2013-test1>dfits *.fits | fitsort FLUXCAL

FILE FLUXCAL

LSQ12dyw_20120808_R642_56432_1.fits

SN2012ca_20120824_r784_56432_1.fits UNCALIBRATED

SN2012ec_20120818_r784_56432_1.fits UNCALIBRATED

tCHASEJ20032484-5557192_20120807_Gr13_Free_slit1.0_56432_1.fits ABSOLUTE

tCHASEJ20032484-5557192_20120807_Gr13_Free_slit1.0_56432_1_2df.fits ABSOLUTE

tCHASEJ20032484-5557192_20120807_Gr13_Free_slit1.0_56432_1_e.fits ABSOLUTE

tCHASEJ20032484-5557192_20120807_Gr13_Free_slit1.0_56432_1_ex.fits ABSOLUTE

tCHASEJ20032484-5557192_20120807_Gr13_Free_slit1.0_56432_1_f.fits ABSOLUTE

tCHASEJ21253403-4016432_20120807_Gr13_Free_slit1.5_56432_1.fits ABSOLUTE

6. Similarly, the following are completely missing in the file LSQ12dyw_20120808_R642_56432_1.fits

but set correctly in the others

CUNIT1

CSYER1

CRDER1

PHOTZP

ABMAGLIM

ABMAGSAT

PSF_FWHM

ELLIPTIC

7. DATAMIN and DATAMAX are both set to 0 in the image files. Should be "the minimum/maximum valid physical value across the pixel array in units defined by BUNIT." I guess this should be 0 and 65536

Submission of photometric imaging files

See the new ESO Phase3 faq, for some new information http://www.eso.org/sci/observing/phase3/faq.html

i.e. see the specific answers to the following two questions at the bottom :

Q: How to submit image data without accurate astrometric/photometric calibration via Phase 3?

Hence we need to decide if we will submit our images as "calibrated" or "uncalibrated" images. We can do either, but the submission process and file linkage is different depending on which route we choose.

Here is the proposed method :

1. Acquisition images :

Should all be run through PESSTOEFOSCPHOT. Always submit them as an associated file of a main SCIENCE product (i.e. our fully reduced SCIENCE.SPECTRUM fits binary table spectrum). This means that the header in the spectrum file should point to

ASSONi = 'name_of_acquistion_image__should-contain-object-name.fits'.

ASSOC1 = 'ANCILLARY.IMAGE'

And this is important - we must delete the header item PRODCATG from the acquisition image. As this file is not a SCIENCE product, it is just an association.

This part of the header keyword work will be done at QUB, during the creation of the binary table file. Dave will do the association between the fits binary table and the images (Morgan/Cosimo don't need to do anything, Stefano does not need to adjust pipeline for this).

As this is an associated file, it is not necessary that we have astrometrically and photometrically calibrated images. But as PESSTOEFOSCPHOT will usually do both (to some level of accuracy), we will always run them through this process and submit with the pipeline calibration of astrometry and photometry in the header items (i.e.according to Sect 2.6 WCS keywords and Sect. 2.9 Image photometric zeropoint in the SPD).

To keep things homogeneous, I suggest we set the relevant keywords exactly the same as below for the "Multi-colour follow-up imaging"

2. Multi-colour follow-up imaging :

Should all be run through PESSTOEFOSCPHOT. These will all be submitted as SCIENCE products. Hence they must have

PRODCATG=SCIENCE.IMAGE

They must have WCS and photometric information header keywords, or else they will not qualify as standard ESO products. But these header keywords can be set in an appropriate way if we do not have reliable astrometry or photometry (see the FAQ above).

2.1 If pipeline determines a ZP from reference stars in the field, then :

PHOTZP - set to a measured ZP if the pipeline finds suitable reference stars in the field.

PHOTZPER - This is "an optional keyword to indicate the (1 sigma) uncertainty of the photometric zeropoint." Stefano, does the pipeline set this sensibly ?

SV: The PHOTZPER in not set at the moment since it is optional

PHOTSYS - set to AB, Vega etc.

and set

FLUXCAL = ‘ABSOLUTE’

2.2 If there is no way to set a ZP from reference field stars

can be done looking the zeropoint of a full run at the end of the reduction at the marshall level ?

PHOTZP - set to the instrumental zeropint, plus an extinction correction based on the filter, airmass and standard La Silla extinction curve. See

here http://www.eso.org/sci/facilities/lasilla/instruments/efosc/inst/zp/.html

SV: At the moment is set to 9999

PHOTZPER -set this to 2.00

PHOTZP - set to AB, Vega etc.

and set

FLUXCAL = ‘ABSOLUTE’ SV: At the moment it is set to UNCALIBRATED

In the data release paper, we will discuss the cases where PHOTZPER is set to 2.00 and explain that this is a nominal zerpoint from instrumental characteristics.

In addition, the following keywords must be set if FLUXCAL = 'ABSOLUTE'

ABMAGLIM,MAGLIM,ABMAGSAT,PSF_FWHM

I think these are all handled fine in the pipeline, as long as there is a value for ZPT. Correct, Stefano ??

As far as the astrometry is correct PSF_FWHM,ELLIPTIC,CRDER1,CRDER2,CUNIT1,CUNIT2,CSYER1,CSYER2 are set

as far as the zeropoint is calculated PHOTZP,PHOTSYS,ABMAGSAT,ABMAGLIM are set

if we are using a nominal zeropoint PHOTZP,PHOTSYS,ABMAGSAT,ABMAGLIM can be set using this nominal zeropoint (as far as the astrometry works)

3. WCS Astrometric calibration

Handled within the pipeline - we always set to calibrated and Stefano has the keywords set appropriately. If astrometry completely fails, we handle on case by case basis

e.g. do not submit the data file, or make it an association.

Q: How to submit an image without photometric and astrometric calibration?