[Issues] why the illumination correction is underestimated in awyioplkmb
Edwin A. Valentijn
valentyn at astro.rug.nl
Mon Nov 30 19:41:13 CET 2009
sent from my HTC
----- Oorspronkelijk bericht -----
Van: E. M. Helmich <helmich at astro.rug.nl>
Verzonden: maandag 30 november 2009 17:37
Aan: Fedor I. Getman <tig at na.astro.it>
CC: Philippe Heraudeau <heraud at astro.uni-bonn.de>; issues at astro-wise.org
Onderwerp: Re: [Issues] why the illumination correction is underestimated in aw
Hi,
Looking at the results for your method it does seem to work better Fedor
(better also than incorporating the gain as I did); I would still prefer
to arrive at such a solution for instance by deriving scale factors from
the relative scale factors for the flat-fields. I'm trying that now.
Ewout
Fedor I. Getman wrote:
> Hi Ewout,
>
> As Philippe said, gain already taken into account by flat-fielding
> (which done on single-chip level, not on FOV level).
> I commit my changes into cvs (import of get_mean_and_stdev() and
> median() was missed), so you can do check.
>
> Fedor.
>
>
> E. M. Helmich wrote:
>> Hi Philippe, Fedor,
>>
>> I agree the procedure we use while fitting the 2-D polynomial is
>> wrong. It seems we indeed fit to this jig-saw puzzle your plots show
>> Philippe, and this is clearly cannot work. As to how to fix this; I'm
>> looking into it, but I don't have a good solution yet; I'm not sure I
>> see all the issues involved. One factor that influences the relative
>> (CCD-to-CCD) ADU -> magnitude scale is the gain. By replacing this line:
>>
>> sigclipped_rawzeros_list.append(sigclipped_rawzeros_dict[star][0] -
>> median)
>>
>> by this line:
>>
>> sigclipped_rawzeros_list.append(sigclipped_rawzeros_dict[star][0] +
>> 2.5*math.log10(gain.gain))
>>
>> (query for the gain object for the appropriate CCD first) in the "get
>> inputs_for_fit_routine" method, I think I'm getting a better result,
>> but I don't think this is the whole story (and there still seem to be
>> systematics in the distribution of offsets, especially along
>> X-direction of the CCD mosaic, see attached plots). Note that the
>> plots are for an exposure by MEGACAM of SA 107, where I made a fake
>> photometric reference catalog from SDSS DR5 sources for this field.
>>
>> I tried to use your method Fedor, but I get errors trying to get it
>> to work; you must have added more code than just this method to get
>> it to run. Also it seems to me if the moon is up during the exposure,
>> using the skybackground will give wrong results. Any further insight
>> would be appreciated!
>>
>> Regards,
>> Ewout
>>
>> Fedor I. Getman wrote:
>>> Dear Philippe,
>>>
>>> the idea of my modification was to determinate
>>> IlluminationCorrection as best as possible, but with minimal
>>> modification of AW dataflow.
>>>
>>> PhotSrcCatalog object has skybackground already as magnitude value:
>>> skybackground = -2.50 * math.log10(frame.image.median/frame.EXPTIME)
>>>
>>> If you recall, in AW calibration flow the IlluminationCorrection
>>> produced from not many exposures, but only one. It doesn't allowed
>>> to store into DB more then 1 source exposure, used to produce
>>> IlluminationCorrection.
>>> The idea to use several exposures must be evaluated taking into
>>> account possible variation and stability of illumination over FOV.
>>>
>>> With best regars,
>>> Fedor Getman
>>>
>>> heraud at astro.uni-bonn.de wrote:
>>>> Dear Fedor,
>>>>
>>>> Thus you also noticed the problem! That's good!
>>>> Thanks for the code. I will test it when I find some time.
>>>> But I am wondering if you should not have a -2.5log10 somewhere
>>>> since I
>>>> think the "background" derived by Sextractor is in ADU/Frame, which
>>>> you
>>>> use to correct magnitude?
>>>> And does it work if we combine exposures with different exposure
>>>> times (I
>>>> do not see here that the exposure time is used)?
>>>>
>>>> Best regards,
>>>> Philippe
>>>>
>>>>
>>>>
>>>>> Dear Philippe and Gijs,
>>>>>
>>>>> first of all, my apologise for not commited to cvs my modification of
>>>>> illumination correction done several mounth before.
>>>>> Here the modified method "get_inputs_for_fit_routine()" wich must be
>>>>> invocked instead original one in "
>>>>>
>>>>
>>>> ()" of IlluminationCorrection
>>>>
>>>>> class.
>>>>> The idea is to adjust before fitting the residuals into the same
>>>>> surface, using common value over FOV - sky background (as
>>>>> determined by
>>>>> sExtractor during source detection).
>>>>>
>>>>> In our tests with 3c273 mosaic, that modified method to derive
>>>>> IlluminationCorrection give reasonable results in comparision to SDSS
>>>>> mags.
>>>>>
>>>>> -------------------------------------------------------------------------------------------------
>>>>>
>>>>> def get_inputs_for_fit_routine_new(self):
>>>>> '''This routine accumulates source information from the input
>>>>> PhotSrcCatalog objects in preparation for the overall fit.
>>>>> '''
>>>>> sigclipped_rawzeros_list = []
>>>>> pixelpos_list = []
>>>>> error_list = []
>>>>>
>>>>> gridpositions_dict = {}
>>>>> sigclipped_rawzeros_dict = {}
>>>>>
>>>>> for photcat in self.photcats:
>>>>> for gridposition in
>>>>> self.gridpositions[photcat.chip.name]:
>>>>>
>>>>> gridpositions_dict[photcat.chip.name+"_"+str(gridposition.index)] =
>>>>> gridposition
>>>>>
>>>>> for src in photcat.photsourcelist:
>>>>> corrected_dmag = src.mag - src.instmag -
>>>>> (self.photcats[0].skybackground-photcat.Background)
>>>>> dmag_error = math.sqrt(src.mag_err**2 +
>>>>> src.instmag_err**2)
>>>>>
>>>>> sigclipped_rawzeros_dict[photcat.chip.name+"_"+str(src.index)] =
>>>>> (corrected_dmag, dmag_error)
>>>>>
>>>>> # do sigma-clipping
>>>>> mean, stdev =
>>>>> get_mean_and_stdev([sigclipped_rawzeros_dict[key][0] for key in
>>>>> sigclipped_rawzeros_dict.keys()])
>>>>> median =
>>>>> do_median([sigclipped_rawzeros_dict[key][0] for
>>>>> key in sigclipped_rawzeros_dict.keys()])
>>>>> sigma_clip = abs(self.process_params.SIGCLIP_LEVEL *
>>>>> stdev)
>>>>>
>>>>> for key in sigclipped_rawzeros_dict.keys():
>>>>> if abs( sigclipped_rawzeros_dict[key][0] - median ) >
>>>>> sigma_clip:
>>>>> sigclipped_rawzeros_dict.pop(key)
>>>>>
>>>>> # Add the gridpositions and the corrected raw zeropoints
>>>>> to the
>>>>> lists defined
>>>>> # above
>>>>> for star in sigclipped_rawzeros_dict.keys():
>>>>>
>>>>> sigclipped_rawzeros_list.append(sigclipped_rawzeros_dict[star][0] -
>>>>> median)
>>>>> pixelpos_list.append((gridpositions_dict[star].x,
>>>>> gridpositions_dict[star].y))
>>>>> error_list.append(1.)
>>>>>
>>>>> return pixelpos_list, sigclipped_rawzeros_list, error_list
>>>>> -------------------------------------------------------------------------------------------------
>>>>>
>>>>>
>>>>>
>>>>> Philippe Heraudeau wrote:
>>>>>
>>>>>> Dear Gijs,
>>>>>>
>>>>>> As we discussed yesterday the illumination correction is
>>>>>> underestimated
>>>>>> in awe because of the rescaling of each individual chip to the
>>>>>> median of
>>>>>> the zeropoints (see ./astro/main/IlluminationCorrection.py)
>>>>>> I simulated this in the 2 ps files you can download with:
>>>>>>
>>>>>> wget http://www.astro.uni-bonn.de/~heraud/illum_4_4.ps.gz
>>>>>> wget http://www.astro.uni-bonn.de/~heraud/illum_8_8.ps.gz
>>>>>>
>>>>>> which illustrate this effect for 4x4 chip and 8*8 chip cameras.
>>>>>> The higher number of chips the more "smoothing" and that's why you
>>>>>> notice more the underestimation with CFHTLS data than with WFI.
>>>>>>
>>>>>> The figures show:
>>>>>> a) the input illumination correction which is about 25% maximum from
>>>>>> peak to valley.
>>>>>> b) the 2D 2nd order polynomial fit of the input
>>>>>> c) the input illumination after dividing the field by the number of
>>>>>> chips and rescaling each chip to the median to simulate what is
>>>>>> done in
>>>>>> aw for 4x4 chips in illum_4_4.ps and 8x8 chips in illum_8_8.ps
>>>>>> d) the 2D 2nd order polynomial fit of c) which clearly
>>>>>> underestimate the
>>>>>> input: only a few percent (6-7%) are seen instead of 25%
>>>>>> Note: The minimum of the plotted surfaces is always set to 0 and the
>>>>>> median of c is 0.
>>>>>>
>>>>>> We should solve this issue to get the proper illumination
>>>>>> correction...
>>>>>>
>>>>>> Cheers,
>>>>>> Philippe
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>> _______________________________________________
>>> Issues mailing list
>>> Issues at astro-wise.org
>>> http://listman.astro-wise.org/mailman/listinfo/issues
>>>
>>
>
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Drs. Ewout Helmich
Kapteyn Astronomical Institute,
Astro-WISE/OmegaCEN
Landleven 12
P.O.Box 800
9700 AV Groningen
The Netherlands
email: helmich at astro.rug.nl
tel: +31(0)503634548
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
_______________________________________________
Issues mailing list
Issues at astro-wise.org
http://listman.astro-wise.org/mailman/listinfo/issues
More information about the Issues
mailing list