[Issues] why the illumination correction is underestimated in aw
E. M. Helmich
helmich at astro.rug.nl
Tue Nov 24 12:25:41 CET 2009
Oops, plots...
E. M. Helmich wrote:
> Hi Philippe, Fedor,
>
> I agree the procedure we use while fitting the 2-D polynomial is wrong.
> It seems we indeed fit to this jig-saw puzzle your plots show Philippe,
> and this is clearly cannot work. As to how to fix this; I'm looking into
> it, but I don't have a good solution yet; I'm not sure I see all the
> issues involved. One factor that influences the relative (CCD-to-CCD)
> ADU -> magnitude scale is the gain. By replacing this line:
>
> sigclipped_rawzeros_list.append(sigclipped_rawzeros_dict[star][0] - median)
>
> by this line:
>
> sigclipped_rawzeros_list.append(sigclipped_rawzeros_dict[star][0] +
> 2.5*math.log10(gain.gain))
>
> (query for the gain object for the appropriate CCD first) in the "get
> inputs_for_fit_routine" method, I think I'm getting a better result, but
> I don't think this is the whole story (and there still seem to be
> systematics in the distribution of offsets, especially along X-direction
> of the CCD mosaic, see attached plots). Note that the plots are for an
> exposure by MEGACAM of SA 107, where I made a fake photometric reference
> catalog from SDSS DR5 sources for this field.
>
> I tried to use your method Fedor, but I get errors trying to get it to
> work; you must have added more code than just this method to get it to
> run. Also it seems to me if the moon is up during the exposure, using
> the skybackground will give wrong results. Any further insight would be
> appreciated!
>
> Regards,
> Ewout
>
> Fedor I. Getman wrote:
>
>> Dear Philippe,
>>
>> the idea of my modification was to determinate IlluminationCorrection
>> as best as possible, but with minimal modification of AW dataflow.
>>
>> PhotSrcCatalog object has skybackground already as magnitude value:
>> skybackground = -2.50 * math.log10(frame.image.median/frame.EXPTIME)
>>
>> If you recall, in AW calibration flow the IlluminationCorrection
>> produced from not many exposures, but only one. It doesn't allowed to
>> store into DB more then 1 source exposure, used to produce
>> IlluminationCorrection.
>> The idea to use several exposures must be evaluated taking into
>> account possible variation and stability of illumination over FOV.
>>
>> With best regars,
>> Fedor Getman
>>
>> heraud at astro.uni-bonn.de wrote:
>>
>>> Dear Fedor,
>>>
>>> Thus you also noticed the problem! That's good!
>>> Thanks for the code. I will test it when I find some time.
>>> But I am wondering if you should not have a -2.5log10 somewhere since I
>>> think the "background" derived by Sextractor is in ADU/Frame, which you
>>> use to correct magnitude?
>>> And does it work if we combine exposures with different exposure times (I
>>> do not see here that the exposure time is used)?
>>>
>>> Best regards,
>>> Philippe
>>>
>>>
>>>
>>>
>>>> Dear Philippe and Gijs,
>>>>
>>>> first of all, my apologise for not commited to cvs my modification of
>>>> illumination correction done several mounth before.
>>>> Here the modified method "get_inputs_for_fit_routine()" wich must be
>>>> invocked instead original one in "
>>>>
>>>>
>>> ()" of IlluminationCorrection
>>>
>>>
>>>> class.
>>>> The idea is to adjust before fitting the residuals into the same
>>>> surface, using common value over FOV - sky background (as determined by
>>>> sExtractor during source detection).
>>>>
>>>> In our tests with 3c273 mosaic, that modified method to derive
>>>> IlluminationCorrection give reasonable results in comparision to SDSS
>>>> mags.
>>>>
>>>> -------------------------------------------------------------------------------------------------
>>>> def get_inputs_for_fit_routine_new(self):
>>>> '''This routine accumulates source information from the input
>>>> PhotSrcCatalog objects in preparation for the overall fit.
>>>> '''
>>>> sigclipped_rawzeros_list = []
>>>> pixelpos_list = []
>>>> error_list = []
>>>>
>>>> gridpositions_dict = {}
>>>> sigclipped_rawzeros_dict = {}
>>>>
>>>> for photcat in self.photcats:
>>>> for gridposition in self.gridpositions[photcat.chip.name]:
>>>>
>>>> gridpositions_dict[photcat.chip.name+"_"+str(gridposition.index)] =
>>>> gridposition
>>>>
>>>> for src in photcat.photsourcelist:
>>>> corrected_dmag = src.mag - src.instmag -
>>>> (self.photcats[0].skybackground-photcat.Background)
>>>> dmag_error = math.sqrt(src.mag_err**2 +
>>>> src.instmag_err**2)
>>>>
>>>> sigclipped_rawzeros_dict[photcat.chip.name+"_"+str(src.index)] =
>>>> (corrected_dmag, dmag_error)
>>>>
>>>> # do sigma-clipping
>>>> mean, stdev =
>>>> get_mean_and_stdev([sigclipped_rawzeros_dict[key][0] for key in
>>>> sigclipped_rawzeros_dict.keys()])
>>>> median = do_median([sigclipped_rawzeros_dict[key][0] for
>>>> key in sigclipped_rawzeros_dict.keys()])
>>>> sigma_clip = abs(self.process_params.SIGCLIP_LEVEL * stdev)
>>>>
>>>> for key in sigclipped_rawzeros_dict.keys():
>>>> if abs( sigclipped_rawzeros_dict[key][0] - median ) >
>>>> sigma_clip:
>>>> sigclipped_rawzeros_dict.pop(key)
>>>>
>>>> # Add the gridpositions and the corrected raw zeropoints to the
>>>> lists defined
>>>> # above
>>>> for star in sigclipped_rawzeros_dict.keys():
>>>>
>>>> sigclipped_rawzeros_list.append(sigclipped_rawzeros_dict[star][0] -
>>>> median)
>>>> pixelpos_list.append((gridpositions_dict[star].x,
>>>> gridpositions_dict[star].y))
>>>> error_list.append(1.)
>>>>
>>>> return pixelpos_list, sigclipped_rawzeros_list, error_list
>>>> -------------------------------------------------------------------------------------------------
>>>>
>>>>
>>>> Philippe Heraudeau wrote:
>>>>
>>>>
>>>>> Dear Gijs,
>>>>>
>>>>> As we discussed yesterday the illumination correction is underestimated
>>>>> in awe because of the rescaling of each individual chip to the median of
>>>>> the zeropoints (see ./astro/main/IlluminationCorrection.py)
>>>>> I simulated this in the 2 ps files you can download with:
>>>>>
>>>>> wget http://www.astro.uni-bonn.de/~heraud/illum_4_4.ps.gz
>>>>> wget http://www.astro.uni-bonn.de/~heraud/illum_8_8.ps.gz
>>>>>
>>>>> which illustrate this effect for 4x4 chip and 8*8 chip cameras.
>>>>> The higher number of chips the more "smoothing" and that's why you
>>>>> notice more the underestimation with CFHTLS data than with WFI.
>>>>>
>>>>> The figures show:
>>>>> a) the input illumination correction which is about 25% maximum from
>>>>> peak to valley.
>>>>> b) the 2D 2nd order polynomial fit of the input
>>>>> c) the input illumination after dividing the field by the number of
>>>>> chips and rescaling each chip to the median to simulate what is done in
>>>>> aw for 4x4 chips in illum_4_4.ps and 8x8 chips in illum_8_8.ps
>>>>> d) the 2D 2nd order polynomial fit of c) which clearly underestimate the
>>>>> input: only a few percent (6-7%) are seen instead of 25%
>>>>> Note: The minimum of the plotted surfaces is always set to 0 and the
>>>>> median of c is 0.
>>>>>
>>>>> We should solve this issue to get the proper illumination correction...
>>>>>
>>>>> Cheers,
>>>>> Philippe
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Issues mailing list
>> Issues at astro-wise.org
>> http://listman.astro-wise.org/mailman/listinfo/issues
>>
>>
>
>
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Drs. Ewout Helmich
Kapteyn Astronomical Institute,
Astro-WISE/OmegaCEN
Landleven 12
P.O.Box 800
9700 AV Groningen
The Netherlands
email: helmich at astro.rug.nl
tel: +31(0)503634548
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ill_cor_with_gain_X.png
Type: image/png
Size: 1156870 bytes
Desc: not available
Url : http://listman.astro-wise.org/pipermail/issues/attachments/20091124/f738d76d/attachment-0002.png
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ill_cor_with_gain_Y.png
Type: image/png
Size: 1171046 bytes
Desc: not available
Url : http://listman.astro-wise.org/pipermail/issues/attachments/20091124/f738d76d/attachment-0003.png
More information about the Issues
mailing list