[News] TERAPIX quality control tool

[Emmanuel Bertin]

As requested during the last Tele"k"on, please find below the specs of the
quality control-tool (here after the "Derfotron") developed at IAP. Two examples
of "bad" outputs from MEGACAM images can be found at
http://pix1.iap.fr:8080/astrowise/692019o/ldac/
http://pix1.iap.fr:8080/astrowise/691741o/ldac/
You may click the names of the CCDs in the bottom table to see their individual
thumbnails.
Note that these are images from technical nights, not for scientific use, prior
to instrument finalization; hence the bad flat-fielding, halos (incomplete
baffling) and unstable focusing. But of course this is very useful for testing
quality control. So please do not distribute.
Now the specs.
The Derfotron has been developed by Frederic Magnard, in C and Perl. It is a
fully-automatic, "memoryless" process (its behaviour does not depend on past
history): at TERAPIX it is directly part of the data reception stage.  It is
also capable of generating weight-maps and flag-maps; a flat-field and a
bad-pixel mask can be used as a basic gain map. Minimal WCS info (approximate
pixel scale and image location and orientation), approximate zero-point and
effective wavelength are required for working (predefined values can be used).
The goal of the Derfotron is not to take decisions about the quality of images,
this is done at a later stage in the pipeline, through the pipeline "selection
cart". For instance, a defocused image may be perfectly suitable for photometric
calibration, but not for science use. An image with elongated PSFs may be OK for
detecting X-band drop-outs but not weak-lensing analyses. What the Derfotron
does is only generate weight-maps and flag maps (quality control at the pixel
level), and produce stats about sensitive image parameters (quality control at
the exposure level).
What has been completed:
- It is a fully-automatic, "memoryless" process (its behaviour does not depend
on past history): at TERAPIX it is directly part of the data reception stage.
- Generates weight-maps and flag-maps on the fly (including CR footprints and
saturated pixels).
- Diagnostic summary stats published in FITS (binary table) and XML formats
- Plots and Thumbnails published in both PNG and Postcript formats
- Support both basic FITS files and MEFs, with arbitrary positioning of CCDs
(thanks to image warping)
- Provide binned image and weight-map thumbnails in PNG (lossless) format.
- Provide thumbnails of low-surface brightness maps of the background.
- Generate a tabulated model of the PSF for each CCD.
- Track background level, background noise, source density and PSF variations
across the field of view.
- Provide rough star and galaxy counts together with model comparison.
- Current processing speed is 220kpix/s end-to-end on a 2GHz AMD system (to be
improved by parallelizing further, although some stages are already
I/O-limited).
- The final product will be autoconfigurable (not a difficult task).
What remains to be done:
 (quality control side):
- Provide more advanced background stats: residual fringing level, ghost counts,
circular/linear gradients.
- Plot of the local sky (USNO, GSC2) with the camera footprint (a prototype is
already working).
- Ellipticity, skewness and kurtosis stats for the PSF model.
- Adaptive histogram binning.
- Improved star-galaxy separation
- Add star count model.
- Add North-West arrows on thumbnails.
- Add a link to the header content.
 (weight-map side):
- Identify and flag automatically trails and spikes.
- Identify and flag automatically large optical ghosts.

Because Fred has to leave to Hawaii on Sunday, and will be back on the 28th of
May, the release has been pushed to mid-June.
Cheers,
					Emmanuel.