[Issues] why the illumination correction is underestimated in aw
Fedor I. Getman
tig at na.astro.it
Wed Nov 25 15:17:37 CET 2009
Hi Ewout,
As Philippe said, gain already taken into account by flat-fielding
(which done on single-chip level, not on FOV level).
I commit my changes into cvs (import of get_mean_and_stdev() and
median() was missed), so you can do check.
Fedor.
E. M. Helmich wrote:
> Hi Philippe, Fedor,
>
> I agree the procedure we use while fitting the 2-D polynomial is
> wrong. It seems we indeed fit to this jig-saw puzzle your plots show
> Philippe, and this is clearly cannot work. As to how to fix this; I'm
> looking into it, but I don't have a good solution yet; I'm not sure I
> see all the issues involved. One factor that influences the relative
> (CCD-to-CCD) ADU -> magnitude scale is the gain. By replacing this line:
>
> sigclipped_rawzeros_list.append(sigclipped_rawzeros_dict[star][0] -
> median)
>
> by this line:
>
> sigclipped_rawzeros_list.append(sigclipped_rawzeros_dict[star][0] +
> 2.5*math.log10(gain.gain))
>
> (query for the gain object for the appropriate CCD first) in the "get
> inputs_for_fit_routine" method, I think I'm getting a better result,
> but I don't think this is the whole story (and there still seem to be
> systematics in the distribution of offsets, especially along
> X-direction of the CCD mosaic, see attached plots). Note that the
> plots are for an exposure by MEGACAM of SA 107, where I made a fake
> photometric reference catalog from SDSS DR5 sources for this field.
>
> I tried to use your method Fedor, but I get errors trying to get it to
> work; you must have added more code than just this method to get it to
> run. Also it seems to me if the moon is up during the exposure, using
> the skybackground will give wrong results. Any further insight would
> be appreciated!
>
> Regards,
> Ewout
>
> Fedor I. Getman wrote:
>> Dear Philippe,
>>
>> the idea of my modification was to determinate IlluminationCorrection
>> as best as possible, but with minimal modification of AW dataflow.
>>
>> PhotSrcCatalog object has skybackground already as magnitude value:
>> skybackground = -2.50 * math.log10(frame.image.median/frame.EXPTIME)
>>
>> If you recall, in AW calibration flow the IlluminationCorrection
>> produced from not many exposures, but only one. It doesn't allowed to
>> store into DB more then 1 source exposure, used to produce
>> IlluminationCorrection.
>> The idea to use several exposures must be evaluated taking into
>> account possible variation and stability of illumination over FOV.
>>
>> With best regars,
>> Fedor Getman
>>
>> heraud at astro.uni-bonn.de wrote:
>>> Dear Fedor,
>>>
>>> Thus you also noticed the problem! That's good!
>>> Thanks for the code. I will test it when I find some time.
>>> But I am wondering if you should not have a -2.5log10 somewhere since I
>>> think the "background" derived by Sextractor is in ADU/Frame, which you
>>> use to correct magnitude?
>>> And does it work if we combine exposures with different exposure
>>> times (I
>>> do not see here that the exposure time is used)?
>>>
>>> Best regards,
>>> Philippe
>>>
>>>
>>>
>>>> Dear Philippe and Gijs,
>>>>
>>>> first of all, my apologise for not commited to cvs my modification of
>>>> illumination correction done several mounth before.
>>>> Here the modified method "get_inputs_for_fit_routine()" wich must be
>>>> invocked instead original one in "
>>>>
>>>
>>> ()" of IlluminationCorrection
>>>
>>>> class.
>>>> The idea is to adjust before fitting the residuals into the same
>>>> surface, using common value over FOV - sky background (as
>>>> determined by
>>>> sExtractor during source detection).
>>>>
>>>> In our tests with 3c273 mosaic, that modified method to derive
>>>> IlluminationCorrection give reasonable results in comparision to SDSS
>>>> mags.
>>>>
>>>> -------------------------------------------------------------------------------------------------
>>>>
>>>> def get_inputs_for_fit_routine_new(self):
>>>> '''This routine accumulates source information from the input
>>>> PhotSrcCatalog objects in preparation for the overall fit.
>>>> '''
>>>> sigclipped_rawzeros_list = []
>>>> pixelpos_list = []
>>>> error_list = []
>>>>
>>>> gridpositions_dict = {}
>>>> sigclipped_rawzeros_dict = {}
>>>>
>>>> for photcat in self.photcats:
>>>> for gridposition in self.gridpositions[photcat.chip.name]:
>>>>
>>>> gridpositions_dict[photcat.chip.name+"_"+str(gridposition.index)] =
>>>> gridposition
>>>>
>>>> for src in photcat.photsourcelist:
>>>> corrected_dmag = src.mag - src.instmag -
>>>> (self.photcats[0].skybackground-photcat.Background)
>>>> dmag_error = math.sqrt(src.mag_err**2 +
>>>> src.instmag_err**2)
>>>>
>>>> sigclipped_rawzeros_dict[photcat.chip.name+"_"+str(src.index)] =
>>>> (corrected_dmag, dmag_error)
>>>>
>>>> # do sigma-clipping
>>>> mean, stdev =
>>>> get_mean_and_stdev([sigclipped_rawzeros_dict[key][0] for key in
>>>> sigclipped_rawzeros_dict.keys()])
>>>> median = do_median([sigclipped_rawzeros_dict[key][0]
>>>> for
>>>> key in sigclipped_rawzeros_dict.keys()])
>>>> sigma_clip = abs(self.process_params.SIGCLIP_LEVEL * stdev)
>>>>
>>>> for key in sigclipped_rawzeros_dict.keys():
>>>> if abs( sigclipped_rawzeros_dict[key][0] - median ) >
>>>> sigma_clip:
>>>> sigclipped_rawzeros_dict.pop(key)
>>>>
>>>> # Add the gridpositions and the corrected raw zeropoints to
>>>> the
>>>> lists defined
>>>> # above
>>>> for star in sigclipped_rawzeros_dict.keys():
>>>>
>>>> sigclipped_rawzeros_list.append(sigclipped_rawzeros_dict[star][0] -
>>>> median)
>>>> pixelpos_list.append((gridpositions_dict[star].x,
>>>> gridpositions_dict[star].y))
>>>> error_list.append(1.)
>>>>
>>>> return pixelpos_list, sigclipped_rawzeros_list, error_list
>>>> -------------------------------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>> Philippe Heraudeau wrote:
>>>>
>>>>> Dear Gijs,
>>>>>
>>>>> As we discussed yesterday the illumination correction is
>>>>> underestimated
>>>>> in awe because of the rescaling of each individual chip to the
>>>>> median of
>>>>> the zeropoints (see ./astro/main/IlluminationCorrection.py)
>>>>> I simulated this in the 2 ps files you can download with:
>>>>>
>>>>> wget http://www.astro.uni-bonn.de/~heraud/illum_4_4.ps.gz
>>>>> wget http://www.astro.uni-bonn.de/~heraud/illum_8_8.ps.gz
>>>>>
>>>>> which illustrate this effect for 4x4 chip and 8*8 chip cameras.
>>>>> The higher number of chips the more "smoothing" and that's why you
>>>>> notice more the underestimation with CFHTLS data than with WFI.
>>>>>
>>>>> The figures show:
>>>>> a) the input illumination correction which is about 25% maximum from
>>>>> peak to valley.
>>>>> b) the 2D 2nd order polynomial fit of the input
>>>>> c) the input illumination after dividing the field by the number of
>>>>> chips and rescaling each chip to the median to simulate what is
>>>>> done in
>>>>> aw for 4x4 chips in illum_4_4.ps and 8x8 chips in illum_8_8.ps
>>>>> d) the 2D 2nd order polynomial fit of c) which clearly
>>>>> underestimate the
>>>>> input: only a few percent (6-7%) are seen instead of 25%
>>>>> Note: The minimum of the plotted surfaces is always set to 0 and the
>>>>> median of c is 0.
>>>>>
>>>>> We should solve this issue to get the proper illumination
>>>>> correction...
>>>>>
>>>>> Cheers,
>>>>> Philippe
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Issues mailing list
>> Issues at astro-wise.org
>> http://listman.astro-wise.org/mailman/listinfo/issues
>>
>
More information about the Issues
mailing list