[News] Some issues
Dr. Mark Neeser
neeser at usm.uni-muenchen.de
Wed Nov 12 16:29:32 CET 2003
Hi Roberto,
here some answers to your questions:
On Tue, 11 Nov 2003, Roberto Silvotti wrote:
> Dear Mark,
>
> As we are planning to compare the AW pipeline results on the
> Capodimonte Deep Field WFI data obtained by Philippe Héraudeau
> with the results that we obtained here with IRAF,
> I would like to ask you a few more details on your tests on AW
> pipeline vs. Bonn pipeline vs. IRAF, in order to better understand
> what you did in Munich and use at best all these tests during our
> future discussions next week in Groningen.
>
>
> > Hello All,
> >
> > As a test of data reduction routines (astro-wise vs. Bonn pipeline vs.
> > IRAF) we have immersed ourselves (primarily Jan Snigula and
> > Yuliana Goranova) in the astro-wise pipeline using our WFI data.
> >
> > Here some results/comments:
> >
> > - Dataset used for testing: WiFI- images taken over 6 nights
> > (approx. 50GB).
> >
> > Time for testreduction: approx. 1 week (most of the delays were caused
> > by unannounced pipeline changes occurring during the reduction.
> >
> > Details:
> >
> > - COSMICFILTER:
> > We tried to implement our cosmic filtering routine (cosmicfits from
> > Claus Goessl) in the pipeline. This attempt failed, due to the
> > current handling of bad columns/pixels in the images and the
> > existance of negative pixel values. The latter problem can be
> > circumvented in the program call, but the bad columns/pixels would
> > need to be replaced (PRIOR to executing our cosmic filter routine) with
> > a well defined NaN value, e.g. 0. Essentially, our cosmic filter
> > routine treats the bad columns as cosmic rays and spends an inordinate
> > amount of time trying to detect/correct them. It would be easy to
> > mask these as NaN's prior to running "cosmicfits".
> >
> > - BAD COLUMNS:
> > Bad colums in general must be taken care of at an earlier stage of
> > the pipeline than currently implemented (see example ps
> > file). This was already discussed by Mark and Roeland.
> >
> > - SWARP:
> > The pipeline currently uses a beta version of Swarp (swarp 2.0b),
> > that fails to create usable weight images. Reasonable weight
> > images are essential for obtaining a fast run-time with cosmicfits,
> > and are crucial when the frames on which cosmic rays are detected
> > have strong intensity slopes.
> >
> > - UPDATES:
> > Unannounced pipeline changes that change object definitions, require
> > changes to the database, that can be time consuming, given the fact,
> > that we have to figure out the changes from the code, and guess
> > the needed Database changes. Usually Danny sends out an e-mail
> > explaining the required changes (usually after talking with Roeland
> > about the changes in the pipeline), but these e-mail come about 2
> > days after the changes, causing severe interruptions.
> >
> > Suggestion: Create a stable branch of the pipeline in the CVS with
> > weekly? updates from a development branch, and the changes in these
> > updates should be discussed with the DBAs before, so that updates to
> > the pipeline cause as few interruptions as possible.
> >
> > - TESTS:
> >
> > - Bias:
> >
> > Subtracting resulting masterbias from its raw input frames
> >
> > rawbias file name median stddev average stddev
> > seWFI.2003-01-05T20C54C42.283_4.fits -0.199997 18.876 -0.244914 18.8759
> > seWFI.2003-01-05T20C55C44.983_4.fits -0.100006 18.9255 -0.161555 18.9254
> > seWFI.2003-01-05T20C56C42.354_4.fits 0.100006 18.89 0.00905352 18.8897
> > seWFI.2003-01-05T20C57C39.851_4.fits 0.100006 18.9164 -0.00907611 18.916
> > seWFI.2003-01-05T20C58C34.232_4.fits 0 18.8818 -0.0418041 18.8818
> > seWFI.2003-01-05T20C59C33.157_4.fits 0 18.881 -0.0651421 18.8809
> > seWFI.2003-01-05T21C00C32.066_4.fits 0 18.8516 -0.0559038 18.8515
> > seWFI.2003-01-05T21C01C33.362_4.fits 0 18.888 -0.0407198 18.888
> > seWFI.2003-01-05T21C02C30.338_4.fits 0.100006 18.8894 0.000104276 18.8891
> > seWFI.2003-01-05T21C03C29.919_4.fits 0.100006 18.9252 0.0489991 18.9252
> >
> > (Is there a numerical round-off occurring in the median computation?).
>
> I suppose that the master was calculated with the AW pipeline.
> But it is not clear to me what these numbers tell us: just that each raw
> frame
> has a median or average very similar to the master ?
Yes, the master was calculated with the AW pipeline. We were not looking
for anything deeply philosophical in these numbers, but just wanted to
test the pipeline computations. We wanted to be sure that the final
master bias resulted in a median that was comparable to the input frames.
The number confirm this.
>
> > - DomeFlat:
> >
> > Flatfielding biassubtracted raw domeflatframes using the processed
> > domeflat. ( SEM = std. error)
> >
> > flatfielded raw flatfield filename median stddev average stddev
> > dseWFI.2003-01-06T21C59C26.278_4.fits 17778.8 90.3731 17778.1 90.3702
> > dseWFI.2003-01-06T22C00C53.969_4.fits 19056.3 94.5017 19055.6 94.4991
> > dseWFI.2003-01-06T22C02C15.050_4.fits 19074.4 94.4472 19073.4 94.4418
> >
> > - TwilightFlats:
> >
> > Flatfielding biassubtracted raw twilightflatframes using the processed
> > twilightflat. ( SEM = std. error)
> >
> > flatfielded raw flatfield filename median stddev average stddev
> > dseWFI.2003-01-04T00C17C43.159_4.fits 17885.3 122.664 17886 122.662
> > dseWFI.2003-01-04T00C19C14.085_4.fits 17161.4 134.724 17161.9 134.723
> > dseWFI.2003-01-04T00C21C01.056_4.fits 16581.6 151.299 16582.4 151.297
>
> Here again I do not see what these numbers tell us.
> The stddev are relatively small but, for example, we are not able
> to say anything about flatness.
Again, we just wanted to see that the numbers are reasonable. We were not
looking for flatness.
>
> > - Astrometry:
> > From visual inspection of stars from the USNO catalog overplotted
> > on the image, the astrometric solution determined by the pipeline
> > seems to be very accurate even out to the edges.
> >
> > Hard numbers:
> > mean position differences in arcsec:
> > RA DEC
> > -0.0002 0.0001
> >
> > mean sigma of the position differences in arcsec:
> > RA DEC
> > 0.4199 0.4021
>
> For what concerns astrometry, the mean differences very close to zero
> tell us
> that there are no offsets in the RA and DEC.
Exactly. Since the frames were dithered, these numbers are good. We
just wanted to confirm that the astrometric solution produced by LDAC
was good.
> It is less clear the meaning of the mean sigmas:
> 0.4 arcsec is an accuracy of the same order of the USNO catalogue
> (~0.3).
> So this values are normal if they represent absolute accuracy.
> But they would be not good enough for our purposes if they represent the
> relative
> accuracy, for example the difference between one dithering and another.
> So it would be interesting to known exactely what are these numbers.
Clearly, the relative accuracy of our frames (the FIRST numbers) is
extremely good, at levels of about 0.001 pixels. . . more than adequate
for anything we need.
The SECOND numbers are the absolute errors. Since the pmm 1 catalogue
has rms errors of order 0.3 to 0.4 arcsec, this will be our limit in
absolute terms.
Cheers,
Mark
More information about the News
mailing list