.help mscguide Jan98 mscred .sp 3 .ce \fBGuide to the NOAO Mosaic Data Handling Software\fR .ce Francisco Valdes .ce February 1998 .ce MSCRED Version 1.1 .sh Introduction This document discusses handling and reducing CCD mosaic data, particularly data from the NOAO CCD Mosaic Camera, using IRAF and the \fBmscred\fR package. It is not a beginner's guide and assumes some previous experience with IRAF and CCD reductions. The first section discusses the mosaic data format and how to use it in IRAF. This format is more complex than single CCD images because of the multiple CCDs. To keep the data from each exposure self-contained the multiple CCD images are stored in a single file. This multiple image per file has many advantages but it does mean that some commands for dealing with images behave differently. The second section describes the tools used to examine the mosaic data. These tools are used during observing as well as during data reductions. The last section describes the reduction of mosaic data. This includes basic CCD instrumental calibration and combining mosaic exposures into single images. .sh 1. Multiextension FITS Files The data format used by the NOAO Mosaic Data Handling Software is a multiextension FITS (MEF) file. This format is produced by the the Data Capture Agent (DCA) when observing with the NOAO CCD Mosaic Camera. The MEF file for the NOAO Mosaic currently consists of nine FITS header and data units (HDU). The first HDU, called the primary or global header unit, contains only header information which is common to all the CCD images. The remaining eight HDUs, called extensions, contain the images from the eight CCDs. The fact that the image data is stored as a FITS file is not significant. Starting with IRAF V2.11, FITS files consisting of just a single primary image may be used in the same way as any other IRAF image format. The significant feature of the mosaic format is it's multi-image structure. With multiextension FITS files you must either use tasks which are specifically designed to operate on these files as a unit or explicitly specify the image within the file that is to be operated upon by general IRAF image processing tasks. The tasks in the \fBmscred\fR package are designed to operate on the mosaic MEF files and so you only need to specify the filename. For image tasks outside the \fBmscred\fR package you must specify the image in the MEF file using the syntax .nf filename[extension] .fi where "filename" is the name of the MEF file. The ".fits" filename extension does not need to be used in the filename. The image "extension" is specified either using an extension name or the position of the extension in the file (where the first extension is 1). The extension names in the NOAO Mosaic data are "im1" through "im8" for the eight CCDs. Two of the most common tasks that require specifying an image extension are \fBdisplay\fR to display a single CCD image (the task \fBmscdisplay\fR is used to display all the images at once) and \fBimheader\fR to list the header of a particular CCD. So, for example, the following commands might be used. .nf ms> display obj012[im2] 1 ms> imhead obj012[3] l+ .fi Other tasks you may use this way are \fBimexam\fR and \fBimplot\fR. A common question is how to specify a list of extensions. Modification of the syntax to allow wildcard templates in the extension specification is under study. Currently you must specify each extension explicitly, though the filename itself may be a wildcard; for example the first image in a set of files can be collectively specified with .nf obj*[im1] .fi There are two methods for specifying some or all extensions in tasks that operate upon lists of images. One is to make @files. This can be done explicitly with an editor. However the \fBproto\fR task \fBimextensions\fR can expand MEF files into an @file as in the following example. .nf ms> imexten obj012,obj13 > list ms> imhead @list .fi Read the help page for further information, additional parameters, and examples. Another method is to use the special \fBmscred\fR task \fBmsccmd\fR. This task can be used on the command line or as a simple interactive command interpreter. The idea is that you use the special designations "$input" and "$output" for task parameters which allow lists of images. Then lists of MEF filenames are specified for the input and output which are expanded and substituted into the task parameters when it is executed. For example, .nf ms> msccmd "imhead $input l+" input=obj012,obj013 .fi For additional information and examples consult the help page for that task. Note that the tasks \fBimstat\fR and \fBimarith\fR are so useful and common that there are specific \fBmscred\fR tasks \fBmscstat\fR and \fBmscarith\fR that operate on all or a subset of image extensions. So these tasks need not be used with \fBmsccmd\fR or with @files. We conclude with a discussion of the special operations of copying, renaming, deleting, and reading and writing FITS tapes as they apply to the mosaic MEF files. To copy a mosaic file as a unit use \fBcopy\fR, making sure to explicitly specify the "fits" extension. If you use \fBimcopy\fR it will expect you to specify a particular extension and will copy only that extension. While \fBimcopy\fR is not the way to copy an complete MEF file the tasks \fBimrename\fR and \fBimdelete\fR are the commands for renaming and deleting these files; though \fBrename\fR and \fBdelete\fR will also work provided you are explicit with the extension. Finally the mosaic data should be kept as a MEF file and so the the special mosaic tasks \fBmscwfits\fR and \fBmscrfits\fR should be used. The current \fBwfits\fR and \fBrfits\fR are not intended for this type of data. .sh 2. Examining Mosaic Data During observing a small set of IRAF commands are commonly used to examine the data. This section describes these commands. While this section is oriented to examining the data at the telescope during the course of observing, the tools described here are also used when reducing data at a later time. .sh 2.1 Displaying the Data: \fBdisplay\fR and \fBmscdisplay\fR The two commands \fBdisplay\fR and \fBmscdisplay\fR are used to display the data in a display server window. The display server is a separate process which must be running before displaying the images. The observing environment at the telescope will generally have the XIMTOOL display server already running with a window in a separate monitor. If it is not running for some reason it can be started with a menu selection. Away from the telescope you would start XIMTOOL or SAOIMAGE as you do normally. The display server must be told what size "frame buffer" to allocate for holding the display pixels. This determines how many pixels may be loaded at one time. Note that the display window may be smaller than this size and the display server allows you to move the portion viewed and zoom/unzoom any region. If the image size is larger than the frame buffer you can display a portion of the image at full resolution or the full image at a lower resolution. The frame buffer size is queried and set with the commands: .nf ms> show stdimage imt2048 ms> set stdimage=imt1024 .fi There are trade-offs in the frame buffer selection. A large frame buffer allows you to have higher resolution for the large mosaic images but it uses more memory and takes longer to load. The \fBdisplay\fR task is used to display individual images in the display server. This task is a standard IRAF task about which you are assumed to have some basic knowledge. There are many display options which are discussed in the help page. The only special factor in using this task with the mosaic data is that you must specify which CCD image to display using the image extension syntax discussed previously. As an example, to display the central portion of extension im3 in the first frame and the whole image in the second frame: .nf ms> display obj123[im3] 1 fill- ms> display obj123[im3] 2 fill+ .fi The \fBmscdisplay\fR task is based on \fBdisplay\fR with a number of specialized enhancements for displaying mosaic data. It displays the entire mosaic observation in a single frame by "filling" each image in a tiled region of the frame buffer. The default filling (defined by the order parameter) subsamples the image by uniform integer steps to fit the tile and then replicates pixels to scale to the full tile size. The resolution is set by the frame buffer size. As mentioned before, trying to increase the resolution with a larger buffer size has the penalty of longer display times. An example display command is: .nf ms> mscdisplay obj123 1 .fi Many of the parameters in \fBmscdisplay\fR are the same as \fBdisplay\fR and there are also a few that are specific to the task of displaying a mosaic of CCD images. The mapping of the pixel values to gray levels includes the same automatic or range scaling algorithms as in \fBdisplay\fR. This is done for each image in the mosaic separately. The new parameter "zcombine" then selects whether to display each image with it's own display range ("none") or to combine the display ranges into a single display range based on the minimum and maximum values ("minmax"), the average of the minimum and maximum values ("average"), or the median ("median") of the minimum and maximum values. The independent scaling is most appropriate for raw data while the "minmax" scaling is recommend for processed data which are gain calibrated. During data acquisition the \fBmscdisplay\fR task is used to display mosaic data as it is being written to disk by the DCA. It begins execution shortly after the readout begins and displays the portion of the recorded image which has been written to disk. It then alternately waits and displays new data which has been written by the DCA. It continues to sleep and display until all the data has been displayed. The automatic gray level scaling is computed on the initial amount of data. .sh 2.2 Examining the Data: \fBmscexamine\fR The task \fBmscexamine\fR allows interactive examination of mosaic images. It is essentially the same as the standard \fBimexamine\fR task except that it translates the cursor position in a tiled mosaic display into the image coordinates of the appropriate extension image. Line and column plots also piece together the extensions at the particular line or column of the mosaic display. To enter the task after displaying an image the command is: .nf ms> mscexam .fi As with \fBimexamine\fR, one may specify the mosaic MEF filename to be examined and if it is not currently displayed it will be displayed using the current parameters of \fBmscdisplay\fR. .sh 2.3 Examining the Headers: \fBimheader\fR, \fBhselect\fR, \fBccdlist\fR There was discussion earlier concerning the use of generic image tasks with the NOAO Mosaic data. The tasks \fBimheader\fR and \fBhselect\fR fall into this category. The two important points to keep in mind are that you must specify either an extension name or the extension position and that the headers of an extension are the combination of the global header and the extension headers. Often one does not need to list all the headers for all the extensions. The image title and many keywords of interest are common to all the extensions. Thus one of the following commands will be sufficient to get header information about an exposure or set of exposures: .nf ms> imhead obj*[1] l- # Title listing ms> imhead obj123[1] l+ | page # Paged long listing ms> hselect obj*[1] $I,filter,exptime,obstime yes .fi If you need to list header information from all the extensions then you need to take the additional step of creating an @file or using \fBmsccmd\fR. For example to get the default read noise and gain values for each CCD: .nf ms> imextensions obj123 > list123 ms> hselect @list123 $I,rdnoise,gain yes or ms> msccmd "hselect $input $I,rdnoise,gain yes" input=obj123 .fi The \fBccdlist\fR task in the \fBmscred\fR package is specialized for the mosaic data. It provides a compact description of the name, title, pixel type, filter, amplifier, and processing flags. The "extname" parameter may be used to select a particular extension, a set of extensions, or all extensions. Because all extensions should generally be at the same state of reduction it may be desirable to list only the first extension. Like most of the CCD reduction tasks you can also select only a certain type of exposure for listing. Examples of the two modes are: .nf # Summary for all exposures ms> ccdlist *.fits extname=im1 # Summary for all object exposures ms> ccdlist *.fits extname=im1 ccdtype=object # List of all extensions. ms> ccdlist obj123 extname="" .fi .sh 2.4 Determining Best Focus: \fBmscfocus\fR Focus sequence frames can be evaluated for the best focus using the task \fBmscfocus\fR. This displays a focus exposure with \fBmscdisplay\fR (if needed) and then lets you select one or more bright stars to measure. This task is customized so that all you need do is mark the top image in any CCD. For NOAO Mosaic data header information tells the task how many exposures, the spacings between the exposures, and the focus values. After the measurements are made they are displayed and analyzed graphically and written to the terminal and logfile. This task is the mosaic analog of the \fBkpnofocus\fR and \fBstarfocus\fR tasks for single CCD data. .sh 3. Data Reductions The reduction of CCD mosaic camera data can be divided into two stages. The first is the basic calibration of the individual CCDs. This stage is very similar to reducing data from single CCD exposures except that the calibration operations are repeated for all the CCDs in the mosaic. The only significant difference is that any scaling of an exposure, such as in normalizing the flat field calibration, must be done uniformly over all the CCDs. The details of repeating the calibrations for all CCDs and the scaling of the calibration data are taken care of by the software and the data format so that these operations appear the same as with single CCD data. There is one step of the basic CCD calibration stage which has generally been ignored or forgotten with smaller single CCD formats. The large field of view provided by a mosaic and the optics required to provide it can lead to a significant variation in the pixel scale. This effect is important with the NOAO 4meter telescopes and is also present in the NOAO 36inch data to a smaller degree. It is likely to be present in other telescopes as well. When the pixel scale varies significantly the standard flat field calibration operation will cause the photometric zero point to vary. A simple calibration step can be performed to remove this effect. However, if you intend to produce single images from the mosaic of CCDs this step is not necessary since the resampling operation naturally accounts for this effect. The second stage of data reductions is unique to mosaic data. This stage is the combining of the multiple CCD images and multiple exposures into a single image. Since creating a single image from a single mosaic exposure is of marginal value, the thrust of this stage of the reductions is the combining of multiple exposures which have been spatially offset or "dithered" to cover both the gaps between the individual CCDs and any defects (which are quite extensive in the "engineering grade" CCDs of the first NOAO Mosaic Camera). The steps required to produce a single deep integration from dithered exposures consists of accurately registering the images, mosaicking the exposures into single images with the same spatial sampling, measuring changes in the intensity scale due to variations in transparency and sky brightness, and combining the individual images into a single deep image with the gaps and bad pixels removed. .sh 3.1 Some Preliminaries The command \fBsetinstrument\fR is used to set default parameters for the tasks in the \fBmscred\fR package appropriate to a particular instrument. For users of the NOAO Mosaic Camera it is recommended you run this command the first time you reduce data. Subsequently you should not do this since it will reset parameters you later changed. To set the parameters for reducing the NOAO Mosaic data type the command .nf ms> setinstrument 4meter review- .fi Substitute "36inch" for "4meter" if the data is from the Kitt Peak 0.9 meter telescope. For some of the operations it is useful to specify lists of exposures corresponding to a dither set. The examples in this guide show using using @files for dither sets. An @file is simply a list of filenames. These can be created in several ways including using a text editor. One way is with the \fBfiles\fR command to expand a file template. For example, .nf ms> files obj021,obj022,obj023,obj024,obj025 > field1 ms> dir @field1 obj021 obj002 obj003 obj004 obj005 .fi .sh 3.2 Basic CCD Calibration Basic CCD instrumental calibrations consist of correcting each CCD for electronic bias levels, zero exposure patterns, dark counts, and pixel sensitivities. A cosmetic replacement of bad pixels may also be included. An additional calibration is required to correct for the variable pixel scale across the field of view if you intend to do photometry on the individual CCD images. .sh 3.2.1 Calibration Data to Obtain At the Telescope Good data reductions begin with obtaining good calibration data at the telescope. This section discusses the NOAO CCD Mosaic Camera but the general principles will apply to other detectors, though the relative importance of the different calibration will depend on the quality of the CCDs. The first generation (engineering grade) NOAO Mosaic CCDs operate at warmer temperatures than usual, so good dark count exposures (darks) are required in addition to the usual zero level exposures (zeros). Further, we recommend that you obtain darks with exposures similar to your science exposures. While many of the CCD features will scale with time, there are regions of charge overflow that may not be completely linear. Dome flat fields (dome flats) provide a fair basic flattening of the data to 2% or so, but dark sky flat fields (sky flats) or illumination corrections will be required to produce dithered data that can be combined without introducing obvious artifacts. Dark sky flats can flatten the data to 0.1%. We currently lack experience with twilight flats so we cannot comment on them. The default NOAO Mosaic dither pattern is a sequence of 5 exposures designed to insure at least 80% coverage for all portions of an astronomical image, given the gaps between the CCDS \fIand\fR many of the large defects, which in a number of cases are even wider than the gaps. Other things to be aware of include the small wells and nonlinear response of CCD 4. This chip saturates at only ~4X10^4 e^-, and becomes nonlinear by more than 1% above 1.5X10^4 e^-. At this time we are investigating the nonlinearity, which is unexpected, but the full well still sets a hard limit to the exposure level. In contrast, the other 7 CCDs appear to be highly linear up to this exposure level, and have much larger wells. Lastly, good astrometry is required to register and stack the Mosaic images. We have excellent solutions for the R and V band filters (which are automatically included in the data taken at the telescope), but the scale varies slightly with color, so you may want to image an astrometric field if you are using filters not bracketed by these colors. However, simple scaling of the default solutions can be done either to register data between filters or to register to a few stars with known coordinates to achieve quite good astrometry. So astrometric calibrations are not critical. Note that this guide and other documentation does not yet discuss how to create the coordinate system solutions and the Mosaic support team will eventually supply solutions for all of the filters. .sh 3.2.2 Preparing Calibration Data This section describes how to prepare the basic calibration data. The steps are virtually the same as with the \fBccdred\fR package and, in fact, the command names and parameters are the same. The basic calibration data of zero level, dark count, and dome or projector flat fields are generally taken as a sequence of identical exposures which are combined to minimize the noise. A later section discusses preparing a sky flat field calibration using the object exposures or twilight exposures. The calibration exposures are individually reduced by \fBccdproc\fR and then combined. Thus, it is necessary to first set the \fBccdproc\fR parameters. Because this task knows which operations are appropriate for particular types of calibration exposures you can set all the parameters. Below is a typical set of parameters. The main optional setting is whether or not to replace bad pixels by interpolation, which is purely a cosmetic correction. .nf I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = ccdproc images = List of Mosaic CCD images to process (ccdtype= object) CCD image type to process (noproc = no) List processing steps only? (oversca= yes) Apply overscan strip correction? (trim = yes) Trim the image? (fixpix = no) Apply bad pixel mask correction? (zerocor= yes) Apply zero level correction? (darkcor= yes) Apply dark count correction? (flatcor= yes) Apply flat field correction? (sflatco= no) Apply sky flat field correction? (biassec= !biassec) Overscan strip image section (trimsec= !trimsec) Trim data section (fixfile= ) List of bad pixel masks (zero = Zero) List of zero level calibration images (dark = Dark) List of dark count calibration images (flat = Flat*) List of flat field images (sflat = Sflat*) List of secondary flat field images (minrepl= 1.) Minimum flat field value (interac= no) Fit overscan interactively? (functio= legendre) Fitting function (order = 1) Number of polynomial terms or spline pieces (sample = *) Sample points to fit (naverag= 1) Number of sample points to combine (niterat= 1) Number of rejection iterations (low_rej= 3.) Low sigma rejection factor (high_re= 3.) High sigma rejection factor (grow = 0.) Rejection growing radius .fi The first step is generally to process and combine sequences of zero, dark, and dome flat exposures. This is done using the tasks \fBzerocombine\fR, \fBdarkcombine\fR, and \fBflatcombine\fR. The combining must be done in the following order since the processing of later calibration data requires the preceding calibration data. .nf ms> zerocombine *.fits ms> darkcombine *.fits ms> flatcombine *.fits .fi Each of these tasks search for all exposures of the particular type so it is fine to specify all files. Also \fBflatcombine\fR has the feature that it will combine the data separately for each filter. However, you can use explicit file lists, templates, or @files to limit the input files. The output combined names have some standard default values which the above settings for \fBccdproc\fR use. It is a good idea to first check that the different calibration types and filters are correctly identified by the software. This is done using the \fBccdlist\fR command .nf ms> ccdlist *.fits extname=im1 .fi Unless you change the parameters \fImscred.backup\fR and \fImscred.bkuproot\fR the original raw files will be saved in the subdirectory "Raw/". If you want to start over, delete the processed files and copy the raw files back to the working directory. If disk space is a concern and you are satisfied with the combined calibration files you can delete the individual processed calibration files. There is a parameter in the combining tasks that will delete the individual files automatically after processing and combining. .sh 3.2.3 Object Exposure Reductions At this point you will have some subset of combined zero level, dark count, and flat field calibration data. The calibration data is applied to the object exposures, either in bulk or as observations are completed, using the task \fBccdproc\fR. The command is simply .nf ms> ccdproc .fi .sh 3.2.4 Dark Sky or Twilight Sky Flat Fields You will notice that there are two flat field corrections which can be performed by \fBccdproc\fR. The first one is for an initial flat field such as the dome flat obtained at the beginning of the night, a standard flat field from a previous night or run, or a final combined dome flat and sky flat from some other night or run. The second is for a dark sky or twilight sky flat field prepared from the object exposures after they have been calibrated with the first flat field. Sky flat fields are created by combining object exposures with objects removed by using data in each pixel that is only sky. In principle one could use exposures of the twilight sky. We have no experience with this type of sky flat. You are welcome to take some exposures and try using them. We have found that dark sky flat fields derived from the object exposures do work quite well. Mosaic observations already typically dither a field. One will do even better by combining observations from other fields. The more data used the better the resulting sky flat will be. The main criterion for including data is to avoid observations contaminated by varying background light from the moon or scattered light from bright stars off the field. Of course, another factor that has to be considered is whether a field has a very large extended object which appears in many of the observations. These will not be useful. The sky flat field is created using the task \fBsflatcombine\fR with parameters selected to reject objects appearing above a median. We don't have much experience with creating sky flats currently so some experimentation with parameters may be required. Below is one possibly set of parameters. .nf I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = sflatcombine input = List of images to combine (output = Sflat) Output sky flat field root name (combine= average) Type of combine operation (reject = avsigclip) Type of rejection (ccdtype= object) CCD image type to combine (subsets= yes) Combine images by subset parameter? (scale = mode) Image scaling (statsec= ) Image section for computing statistics (nkeep = 1) Minimum to keep (pos) or maximum to reject (neg) (nlow = 1) minmax: Number of low pixels to reject (nhigh = 1) minmax: Number of high pixels to reject (mclip = yes) Use median in sigma clipping algorithms? (lsigma = 6.) Lower sigma clipping factor (hsigma = 3.) Upper sigma clipping factor (rdnoise= rdnoise) ccdclip: CCD readout noise (electrons) (gain = gain) ccdclip: CCD gain (electrons/DN) (snoise = 0.) ccdclip: Sensitivity noise (fraction) (pclip = -0.5) pclip: Percentile clipping parameter (blank = 1.) Value if there are no pixels (grow = 3.) Radius (pixels) for neighbor rejection .fi This task is a combination of \fBccdproc\fR to first process the images, if they have not previously been processed, and \fBcombine\fB to combine the offset images with rejection of object pixels. A new feature of this package is the \fIgrow\fR parameter which now provides a two dimensional circular rejection of pixels around pixels rejected by the rejection algorithm selected. Whatever rejection algorithm is used it is likely that the best results will be when the clipping sigmas are non-symmetric as shown above. Note that a very low rejection threshold or very large grow radius will make the task quite slow. After producing a good sky flat that has no evidence of objects it may be applied directly to the data by using it as the second flat field correction. .nf ms> ccdproc sflatcor=yes sflat=Sflat* .fi Note that the object exposures used in creating the sky flat will already have been processed except for the application of the sky flat so \fBccdproc\fR will only apply the sky flat field calibration. The sky flat field includes corrections at all scales from pixel-to-pixel sensitivity variations to large scale illumination differences. If the signal-to-noise is poorer than the dome flat field you might wish to apply a filtering/smoothing operation to the sky flat data thus relying on the dome flat field for the pixel-to-pixel sensitivity calibration and the sky flat field for larger scale illumination corrections. There are a number of filtering tasks in IRAF. A median is a good filter and there is the choice of a ring median or box median. To apply one of these general filtering tasks you would use \fBmsccmd\fR to run it on all the CCDs .nf ms> msccmd msccmd: median $input $output 10 10 Input files: SflatV Output files: SflatMedV msccmd: q .fi Because the object exposures are first processed with the dome flat (or other flat field) you would normally run \fBccdproc\fR again on the data using the sky flat and any observations that have not been processed at all will use both the dome flat and the sky flat. However, if you want to make a single flat field to apply to raw data, say if starting over or using it for a second night, you can combine the two flat field corrections into a single flat field to be used as the only flat field correction. This is done by multiplying the two flat fields using \fBmscarith\fR .nf ms> mscarith FlatV * SflatV FinalflatV .fi .sh 3.2.4 The Variable Pixel Scale and Zero Point Uniformity A key assumption in the traditional reduction of CCD images is that the pixel scale is uniform and that a properly reduced blank sky image will have a uniform and flat appearance. Unfortunately, this is not correct when the pixel scale varies over the field. In the case of the NOAO Mosaic at the Mayall 4meter, the pixel scale decreases approximately quadratically from the field center, with the pixels in the field corners being 6% smaller in the radial direction, and 8% smaller in area. Pixels in field corners thus would properly detect only 92% of the sky level seen in the field center, even with uniform sensitivity. At the same time the same number of \fItotal\fR photons would be detected from a star regardless of how many pixels the PSF would be distributed over. Forcing the sky to be uniform over the image has the deleterious effect of causing the photometric zeropoint to vary from center to field corners by 8%. Note that this effect is different from vignetting where the flux actually delivered to the image margins is less than that at the center, an effect that \fIis\fR corrected by the flat field. In practice, the photometric effect of the variable pixel scale can be ignored provided that the reduced images will be part of a dither-sequence to be stacked later on. As discussed below, prior to stacking the images, they first must be re-gridded, which produces pixels of essentially constant angular scale. This is done with the \fBmscimage\fR task, which re-grids the pixels and has a "flux conservation" option that can scale the pixels photometrically by the associated area change. If this function is disabled, then "improperly" flattened images will have a uniform zero point restored. In short, the flat field adjusted (if inappropriately) for the different pixel sizes, so \fBmscimage\fR would then do no further adjustment. Stars would be too bright in the corners of the flattened images, but after re-gridding, their total fluxes would be seen to be scaled down to the appropriate values. If the Mosaic images are to be analyzed individually, however, as might be done for standard star fields, then after the flat field reductions are complete, the differential scale effects must be restored. At present we are in the process of developing a routine in the \fBmscred\fR package to do this, without actually re-gridding the image with \fBmscimage\fR (which can also be done with images not part of a dither-set). The correction process is simple; the scale at any point in the Mosaic field is already known from the astrometry, so one just calculates and multiplies by the correction. The final image would appear to have a variable sky level, but would be photometrically uniform. .sh 3.3 Coordinate Calibration For some projects the basic flux calibrated CCD exposures may be all that is required. However, if you want to obtain coordinate information or combine multiple exposures which are dithered on the sky or taken with different filters then you must calibrate the celestial world coordinate system (WCS) of the data. This may be done in an absolute or relative sense; an absolute calibration ties the data coordinates to catalog coordinates while a relative calibration ties multiple exposures to the same coordinates. Determining the WCS from scratch is a complicated business and requires special observations of astrometry fields. However, for NOAO Mosaic data a standard coordinate calibration determined earlier is automatically inserted into your data by the data capture agent. The calibration consists of an approximate "linear" solution in each header and a reference to a complete "plate" solution database that is part of the \fBmscred\fR package. [In the future the plate solution will be recorded in the header to provide the full coordinate system accuracy and to eliminate the tie to an external file.] The default coordinate system is sufficiently accurate for most purposes and just requires some small adjustments as described below. To piece a single exposure into a single image that does not require registration to any other data you may use the default WCS and skip the WCS calibration steps. The WCS is a mapping from pixels in the mosaic data to celestial coordinates relative to a reference point on the sky. The reference point, or zero point, is set using the telescope pointing coordinate. The telescope pointing is generally off by a small amount, though it could be completely wrong in some hardware/software error situations. In addition, differential atmospheric refraction introduces small axis scale changes and rotations, which are significant due to the large field of view of the mosaic even during the course of single set of dithered exposured. Putting observations from different filters onto the same coordinate system also requires mapping small scale changes since currently there is only a single standard WCS solution derived through one filter. [In the future filter dependent solutions will be made available.] The WCS calibration operations consist of adjusting the standard coordinate system calibration to a desired the zero point and applying small axis scale changes and rotations. This is done using objects (usually stars) in the exposures. Unlike a full WCS calibration, which requires a high density of stars with accurate catalog coordinates, the adjustments to the default WCS calibration require only a few objects; only one objects is needed to provide a zero point correction. The WCS adjustments are determined by specifying coordinates for one or more objects in the data. The coordinates can be obtained from a reference catalog or, more commonly, by measuring coordinates from one reference exposure to which other exposures are to be "registered". A combination of using a catalog coordinate for one object in the field to set the zero point in a reference exposure and then measuring the positions of other stars in the reference image based on that zero point calibration may also be done. The two tasks you will use are \fBmsczero\fR and \fBmsccmatch\fR. \fBMsczero\fR is used to interactively set the zero point of the coordinates, register multiple exposures closely, and generate a list of coordinates in a reference exposure to which other exposures in a dither set are registered. \fBMsccmatch\fR finds objects at the positions specified by a list of coordinates and determines corrections for the zero point, axis scale change, and axis rotation. .sh 3.3.1 Setting Coordinate Zero Points and Measuring Coordinates \fBMsczero\fR is an interactive display task for mosaic exposures that allows measuring coordinates and adjusting the WCS zero point. The task parameters are shown below. The last set of parameters are for the task to query and maintain lists. .nf I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = msczero images = List of mosaic exposures (nframes= 2) Number of frames to use (logfile= default) Log file for measurements ra = RA (hours) dec = DEC (degrees) update = yes Update WCS zero point? (fd1 = ) (fd2 = ) .fi The task displays each exposure in the list, in turn, and responds to cursor key commands. You can go forward and backward through the input list or quit at any point. The exposures are displayed by cycling through the number of frames starting with the first frame. As an aid to efficiency, if the exposure is already loaded in the appropriate frame then the display step is skipped. This task has several uses (type '?' to get the list of command options). .nf 1. Set the WCS zero point by specifying the coordinate of a star. 2. Create a list of coordinates for use with \fBmsccmatch\fR and \fBmscimatch\fR. 3. Report coordinates at the cursor position. .fi It may be that the WCS zero points, based on the telescope pointing coordinates, are accurate enough that you can use this task on only a reference exposure to generate a list of coordinates for use with \fBmsccmatch\fR and \fBmscimatch\fR. However, because it is fairly quick to explicitly check and set the zero point of all the exposures in a dither set to the same coordinate for a common reference star it is recommended you do this first. To check and set the zero points for a set of dithered exposures run \fBmsczero\fR with a list of the exposures .nf ms> msczero @field1 .fi After the first exposure is displayed either find a reasonably bright unsaturated star which will be in all the exposures or find a star whose coordinate is known from a catalog such as the HST Guide Star Catalog. Move the cursor to the star and type 'z' (zero) to invoke a centering algorithm. Note that even though the exposure may be displayed at lower resolution the centering is done with the full resolution data. The task will then tell you what it thinks the coordinate is and ask you for a new coordinate. The first time 'z' is typed it will prompt with the measured coordinate and thereafter it will prompt with the last entered value. If you are referencing all the exposures to the first exposure in the list accept the measured coordinate (and write the value down in case you need it later) otherwise enter the desired coordinate. Note that all further measurements of the image will automatically apply the measured zero point correction but the exposure WCS is not actually updated until you type 'n' (next) or 'q' (quit). If you want to print coordinates without changing the zero point correction use the space bar or 'c' (center) to center on an object and print the centered coordinate. If you changed the WCS zero point you will shown the zero point offsets and given the option to update the WCS in the data file when you type 'n'. Then the next exposure in the list will be displayed. Find the same star and type 'z' again. Since it will retain the last entered coordinate you should only need to accept the prompted coordinates. When you have done this for all the exposures their coordinate systems will be registered at least at that point. The WCS in the dither set may still not be registered all over the field due to refraction effects. Also the intensity scales of the dithered exposures may not be the same due to changes in transparency and sky brightness. These effects are calibrated by matching objects throughout the field in position and brightness. This requires a list of coordinates tied to one of the dithered exposures as a reference. Usually the first exposure in the set is used as the reference. \fBMsczero\fR is used to create a list from objects in the reference exposure. .nf ms> msczero obj021 .fi Select objects, usually stars, throughout the field and type 'x' for each one. This will center on the object and and record the coordinate in a logfile. The default logfile name "default" creates a log file beginning with "Coords." and followed by the name of the exposure. In the example this will be "Coords.obj021". To be useful for coordinate matching this list should have a good number of stars, say three or four from each CCD, with emphasis on the field edges but allowing for the dithering. For the intensity matching you want to have stars with a range of brightness (though not saturated or extremely faint) and which are mostly isolated so that a region around them may be used for sky. The lists for the coordinate and intensity matching do not have to be the same but it is reasonable to just create one list. .sh 3.3.2 Matching Coordinate Systems The task \fBmsccmatch\fR determines and applies a linear correction to the WCS to match objects, generally stars, in an exposure to a set of reference celestial coordinates. This correction maintains the detector geometry and optical distortions while adjusting for changes in apparent sky position such as produced by atmospheric refraction and telescope pointing errors. The linear correction consists of a zero point shift, scale changes in the right ascension and declination axes, and rotations of the axes. To use this task you need a list of reference celestial coordinates, right ascension in hours and declination in degrees, and the mosaic exposure coordinate system must be relatively close to the reference coordinate system. The default WCS plus telescope pointing may be close enough, but if not you would use \fBmsczero\fR to register the zero points at some point in the exposures. Since it is relatively simple to register a set of dithered exposures to a common star with \fBmsczero\fR this is recommended procedure before using \fBmsccmatch\fR. The reference coordinates should cover all of the mosaic field of view to be sensitive to the small rotation and scale effects. The coordinate list might be obtained from a catalog or measured from one of the exposures to which other overlapping exposures will be matched. For the purposes of making a well aligned stacked image from a set of dithered exposures one generally uses one of the exposures as the source of the reference coordinates. \fBMsccmatch\fR operates on a set of input mosaic exposure in turn. For an exposure it converts each input celestial coordinate to a pixel coordinate in one of the extensions using the current WCS. If the the coordinate does not fall in any extension the coordinate is not used. The pixel coordinate is used as a starting point for the \fBapphot.center\fR task. If the centering fails for some reason, such as the object being too near the edge or the final position being too far from the initial position, the coordinate is not used. For those objects successfully found a fit is made between the original celestial coordinates and the measured coordinates expressed as arc seconds from the exposure tanget point. The fit is constrained to yield some combination of shift, scale change, and rotation for each of the celestial coordinate axes. These parameters are then used to update the exposure WCS so that the adjusted measured coordinates best agrees with the reference coordinates. The task parameters are shown below. .nf I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = msccmatch input = List of input mosaic exposures coords = Coordinate file (ra/dec) (nfit = 4) Min for fit (>0) or max not found (<=0) (rms = 2.) Maximum fit RMS to accept (arcsec) (maxshif= 5.) Maximum centering shift (arcsec) (fitgeom= rxyscale) Fitting geometry (update = yes) Update coordinate systems? (interac= yes) Interactive? (fit = yes) Interactive fitting? (verbose= yes) Verbose? accept = yes Accept solution? .fi The input are a list of mosaic exposures and a file of reference celestial coordinates. The exposures should all include a significant number of objects from the list of coordinates. The task can be run interactively or non-interactively based on the \fIinteractive\fR parameter. In interactive mode you can graphically interact with the fitting (selected with the \fIfit\fR parameter) and accept or reject a fit based on the printed fit parameters. The fitting is done using the task \fBgeomap\fR and the interactive mode allows you to view the distribution of coordinates, residuals verses the input coordinates, delete bad values, and possibly change the fitting constraints (see the help for \fBgeomap\fR for more information). The linear transformation may be constrained by the \fIfitgeometry\fR parameter as described in the help for \fBgeomap\fR. This may be desirable if there are only a few coordinates or if you want to impose some physical assumption. Note that the effects of atmospheric refraction actually do cause independent scale changes and rotations in the two axes so the default "rxyscale" should be used. There are a some conditions which are placed on the task. The \fImaxscale\fR parameter limits how far the objects may be found from the initial coordinates. This constraint protects against incorrect identifications and tells the centering routine how much of the image to look at. This parameter should be as small as possible consistent with the errors in the WCS. If you first zero the coordinates then the objects should be found quite close to the initial coordinates. When the \fIverbose\fR parameter is set the results of the centering will be printed consisting of the image extension name, the final pixel coordinates, the shift in pixel coordinates from the initial value, and the formal uncertainties in the pixel coordinates. If an error occurs one of the error codes from \fBapphot.center\fR will be reported such as "BigShift" for objects with too big a shift from the initial position and "EdgeImage" for objects to near the edge of the image. The \fInfit\fR parameter requires a certain number of coordinates to be included in the fit. If specified as a negative number the parameter is interpreted as a maximum number that may be lost from the input list due to being off the exposure or failing to be centered. The \fIrms\fR parameter requires that the final RMS of the residuals about the fit for each axis be less than a certain value. .sh 3.4 Putting the Pieces Together This section tells you how to make single images from each exposure and how to combine sets of dithered images into a final deep image free from gaps and artifacts. Obtaining good results depends on having well-flattened data, a uniform sky, a dither pattern that samples the gaps and bad regions of the detectors, and accurately registered world coordinates. Most difficulties are caused by variable sky conditions or scattered light within a dither sequence or the data used to create a sky flat. .sh 3.4.1 Constructing Single Images Making a single image from a mosaic exposure is done by mapping the pixels from each extension to a single uniform grid on the sky. The WCS calibrations described in previous sections provide this. For making a single image from a single exposure the WCS calibration is not critical and the default WCS is sufficient. For combining multiple dithered exposures all the exposures must be registered to a common coordinate system, either relative to one reference exposure or to a set of catalog stars, and each exposure must be resampled to the same final coordinate system. The task that makes single images from mosaic exposures is \fBmscimage\fR. It's parameters are shown below. .nf I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = mscimage input = List of input Mosaic exposures output = List of output images (referen= ) Reference image (pixmask= no) Create pixel mask? (verbose= )_.verbose) Verbose output? # Resampling parmeters (blank = 0.) Blank value (interpo= linear) Interpolant (nearest,linear,poly3,poly5,spline3) (fluxcon= no) Preserve flux per unit area? (ntrim = 7) Edge trim in each extension (nxblock= 2048) X dimension of working block size in pixels (nyblock= 1024) Y dimension of working block size in pixels # Geometric mapping parameters (interac= no) Fit mapping interactively? (wcssol = yes) Use WCS plate solution? (nx = 10) Number of x grid points (ny = 20) Number of y grid points (fitgeom= general) Fitting geometry (functio= chebyshev) Surface type (xxorder= 4) Order of x fit in x (xyorder= 4) Order of x fit in y (xxterms= half) X fit cross terms type (yxorder= 4) Order of y fit in x (yyorder= 4) Order of y fit in y (yxterms= half) Y fit cross terms type .fi An output image is created for each input mosaic exposure. The output image is created with a coordinate system defined by the specified \fIreference\fR image. If no reference image is specified then the first input mosaic exposure is used to define a simple tangent plane coordinate system with optical distortions removed and that coordinate system is used for all the input mosaic exposures. The important point is that for a set of dithered exposures all the output images must be created with the same coordinate system grid so that they may be combined by simple integer shifts along the image axes. The normal usage is to specify all the mosaic exposures in a dither set as the input, give a matching list of output images, and leave the reference image unspecified. If all the exposures in a dither set are not done at the same time then you must specify one of the earlier output images as the reference image to continue to create the output images on the same coordinate grid. The output images are created with a size that just covers the input data and initially filled with the specified \fIblank\fR value. This is the value that the mosaic gaps will have in the final output image. Then each extension is resampled into the approprate part of the output image. The coordinate mapping is generated by \fBgeomap\fR using the geometric mapping parameters which you don't need to change. The resampling is done with the specified interpolation function. Linear interpolation is the fastest and most straightforward. Other interpolation functions are available and new ones, such as sinc, are under development. The small rotations in the CCDs produce edge effects in the interpolated output pieces so a small trim is required to eliminate these. It is useful for the later combining step to make bad pixel masks that reflect the interpolation and resampling from the input data. These may be created by setting the \fIpixmask\fR parameter. If this parameter is set and the input mosaic data have bad pixel masks defined through the header BPM keywords (default bad pixel masks are provided in the NOAO Mosaic data) then the masks will be interpolated in exactly the same way as the data. The interpolated masks will appear in the working directory with names related to the output image names and with the output images containing the BPM keyword pointing to these masks. The input bad pixel masks are assumed to have zero for good data and one for bad data and the output masks have zero for good data and values between zero and one thousand for bad data. The value is the result of interpolation and reflects the relative contribution of good and bad data. The \fIfluxconserve\fR parameter applies a pixel area correction if selected. As discussed earlier, standard flat fielding distorts the flux per unit area in pixels of different projected size by making them have the same flux per pixel. In effect this applies half of the flux conservation operation by adjusting the pixel values without adjusting the pixel sizes. \fBMscimage\fR does the second half by adjusting the pixel sizes. So for standard flat fielded data, the usual route to making a combined dithered image, the flux conservation parameter should not be used to arrive at a proper final flux per unit area in the resampled data. Flux conservation would only be used if the input mosaic data has previously been corrected back to proper flux per unit area through adjustment of the flat field or data for the variable pixel size inherent in the mosaic coordinate system. Below are two examples; one using prepared @files and one illustrating advanced usage of filename templates. .nf ms> mscimage @dither1 @outdither1 pixmask+ ms> mscimage obj02![2-5]* %obj%mos%02![2-5]* pixmask+ .fi In the second example the input template expands to obj022.fits to obj025.fits and the output template matches the input template using the first part of the %% substitution field and then replaces the "obj" with "mos" to give output images mos022.fits to mos025.fits. .sh 3.4.2 Matching Intensity Scales When stacking dithered exposures (the single images created in the previous step) to fill in the mosaic gaps and remove bad pixels and cosmic ray events it is critical that the intensity scales of the images match. Otherwise you will see artifacts from the gaps, places with bad data, and around objects as the combined intensity level jumps when data from an exposure is missing or rejected. Also the rejection algorithms require that the image intensities match both at the sky level and in objects. There are two parameters that must be determined to match the intensity scales. One is a additive offset caused by sky brightness variations. The second is a multiplicative scale change caused by transparency and exposure time variations. Matching the intensity scales for a set of dithered exposures consists of determining values for these two scaling parameters relative to a reference exposured and setting them in the image headers. The actual adjustment of the pixels values occurs when stacking the exposures. The intensity matching values are determined by the task \fBmscimatch\fR. The task parameters are shown below. .nf I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = mscimatch input = List of input images coords = Coordinates (referen= ) Reference image (box1 = 21) Box size for statistics (box2 = 51) Box size for statistics (lower = 1.) Lower limit for good data (upper = INDEF) Upper limit for good data (update = yes) Update images? (interac= yes) Interactive? (fit = yes) Fit interactively? (verbose= yes) Verbose? accept = yes Accept scaling? .fi The input consists of a list of images to be matched and a file of celestial coordinates (RA in hours and DEC in degrees) to use in computing the matching parameters. The input images are the single images constructed from the mosaic exposures for a set of dithered observations. The reference image to use may be specified explicitly otherwise the first image in the input list is used. Of course, all the images in the dither set must be matched relative to the same reference image. The scaling parameters are determined by measuring the mean flux in a set of matching regions between each input image and the reference image. The centers of the regions are specified by their celestial coordinates. The list of coordinates should consist of the positions of objects in the field. These objects should span a range of brightness. As noted previously, you would normally use the same coordinate list as used with \fBmsccmatch\fR, which is generally obtained using \fBmsczero\fR. However, you can use any IRAF task that produces a list of celestial coordinates from images with a WCS. One possibility is to use \fBrimcursor\fR on one of the displayed single images with the \fIwcs\fR parameter set to "world" and the \fIwxformat\fR set to "%.2H" to produce right ascension values in hours instead of degrees. The now accurately aligned coordinate systems in the images are used to identify the matching pixel coordinate center in each image. The regions to be measured consist of square boxes of the specified sizes about the pixel coordinate center. There are two boxes, an inner box and an outer box which excludes the inner box. The box sizes are intended to define photometry apertures for the objects and nearby background. It is not critical that they exactly fit the objects or that the objects necessarily be stars but this is usually how they will be set. Because of possible PSF variations the inner box should be large enough include all the light from stars over the whole data set. If the inner box is not fully contained in the input or reference image that box is not used for that pair. Similarly the outer box must be fully contained in the images but if only the outer box is outside one or both images the measurement for the inner box may still be used. In order to exclude regions that include the gaps or bad data in one or both of the pair of images all pixels in a box must have values between the specified good data limits. Those regions with values outside the limits are eliminated from the intensity matching. The mean fluxes in each region are used to fit the relation .nf mean(reference) - mean(input) = A + (B - 1) * mean(input) .fi This formulation is chosen so that one may constrain the fit to just finding A (the zero point offset) or both the zero point and scale. The fitting is done using the task \fBcurfit\fR. If both the \fIinteractive\fR and \fIfit\fR parameters are set then this task is excuted interactively and you can examine the fit and delete bad points and refit. Remember that what is plotted is the reference minius input flux verses input flux. You may change the fitting order to 1 to just fit the zero point offset or to 2 to fit the zero point and intensity scale. Any higher order is not allowed. You may delete points and turn on iterative rejection but these do not carry across to the next execution. If the interactive fitting is not selected but the task is run interactively or with the \fIverbose\fR parameter set a plot is produced of the fit. In verbose or interactive mode the result of the fit is printed with the offset adjusted to give the offset at the indicated reference intensity from the minimum reference image mean flux used (i.e. indicative of the offset at the sky level). In interactive mode you may then accept or reject the result. If the scaling parameters are accepted then the keywords \fBmsczero\fR and \fBmscscale\fR are recorded in the input image header when the \fIupdate\fR parameter is set. Note that the reference image is assigned values of 0 and 1 for these header keywords. .sh 3.4.2 Making the Final Stack Image After \fBmscimage\fR produces single images of each of the dithered mosaic exposures with a common coordinate system grid a final image is created with the task \fBmscstack\fR. The task \fBmscimatch\fR is generally used to match the intensity scales of the images before this step as described in the previous section. However, for quick reductions or for other reasons the images may be stacked either with no intensity matching or using the \fIscale\fR and \fIzero\fR options of \fBmscstack\fR. The task parameters are shown below. .nf I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = mscstack input = List of images to combine output = Output image (plfile = ) List of output pixel list files (optional) (combine= median) Type of combine operation (median|average) (reject = none) Type of rejection (masktyp= none) Mask type (maskval= 0.) Mask value (blank = 0.) Value if there are no pixels (scale = !mscscale) Image scaling (zero = !msczero) Image zero point offset (weight = none) Image weights (statsec= ) Image section for computing statistics (lthresh= 1.) Lower threshold (hthresh= INDEF) Upper threshold (nlow = 1) minmax: Number of low pixels to reject (nhigh = 1) minmax: Number of high pixels to reject (nkeep = 1) Minimum to keep (pos) or maximum to reject (neg) (mclip = yes) Use median in sigma clipping algorithms? (lsigma = 3.) Lower sigma clipping factor (hsigma = 3.) Upper sigma clipping factor (rdnoise= 0.) ccdclip: CCD readout noise (electrons) (gain = 1.) ccdclip: CCD gain (electrons/DN) (snoise = 0.) ccdclip: Sensitivity noise (fraction) (sigscal= 0.1) Tolerance for sigma clipping scaling corrections (pclip = -0.5) pclip: Percentile clipping parameter (grow = 0) Radius (pixels) for 1D neighbor rejection .fi This task is a simple variant of \fBcombine\fR that registers the images using the coordinate systems and has the default threshold parameters set to ignore values below one DN based on the default \fIblank\fR value in \fBmscimage\fR for the gaps. If you have also generated bad pixel masks for the resampled images you can exclude them as well by setting \fImasktype\fR to "goodvalue". The real art in using this task is deciding how to scale and reject bad data not covered by the bad pixel masks. A \fIcombine\fR of "median" is the simplest but it does not optimize the signal-to-noise for the number of images. If you "average" the data you will probably want to apply a rejection algorithm such as "avsigclip". Careful flat fielding will make each separate image have the same sky level across the different CCDs. However, the sky levels and transparency may still be varying from exposure to exposure. If you simply combine such data you will see imprints of the gaps. So it is generally a good idea to scale the images. This is done using the \fIscale\fR and \fIzero\fR parameters which can be set to header keyword values, files containing the values, or special values to compute image statistics in a particular region of the data. The recommended method for scaling is to use the intensity matching task \fBmscimatch\fR described in the previous section and use the image header keywords \fBmscscale\fR and \fBmsczero\fR produced by that task. An example of using this task to create a final image is given below. .nf ms> mscstack @field1 Field1 combine=average rej=avsigclip .fi .endhelp