<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix"><br>
Hi Gijs and all,<br>
<br>
thanks for summing up things, here below some feedback which I
prepared after having heard from Francesco who seems didn't
receive this message (please add him to this mailing list). Let me
announce that the 52 catalogs are going to be produced tomorrow
(we got a window on the beowulf to run KIDSCAT in parallel).
Francesco has indeed adapted KIDSCAT to make use of the 60 beowulf
CPUs and we are in the condition that we can produce 400 catalogs
in 1h. So we are ready for massive extraction runs.<br>
<br>
<br>
On 11/15/12 3:33 PM, Gijs Verdoes Kleijn wrote:<br>
</div>
<blockquote cite="mid:50A4FD2B.6040902@astro.rug.nl" type="cite">
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
Dear CATALOG-QC-TEAM,<br>
<br>
The QC for KiDS catalogs falls in 2 parts: single-band and 4-band
QC.<br>
<br>
* SINGLE-BAND SOURCELISTS<br>
<br>
We have the following QC checks in place in KiDS-CAT (thanks!), is
my understanding:<br>
<ul>
<li class="level1">
<div class="li"> FWHM vs SNR. DONE in KiDS-CAT. NN+HB can
send algorithm/code to HB so Hugo can check implementation
in AWE.</div>
</li>
<li class="level1">
<div class="li"> star/galaxy separation. DONE in KiDS-CAT.
NN+HB can send algorithm/code to HB so Hugo can check
implementation in AWE. </div>
</li>
</ul>
</blockquote>
<br>
here I guess you mean NRN+FLB. Algorithm/code can be send straight
away, if this is needed to start implementing the codes in AWE and
possibly solve all the system compatibility problems. Be aware that
the S/G separation also is not yet implemented in KIDSCAT but still
as matlab code. You can either wait both tasks (S/G and limiting mag
belo) are translated in some pgplot script or you can implement in
AWE as they are. <br>
I don't know whether this might be really high priority as we don't
think that the current version of the catalog produced will be
already ingested as integrated sourcelist rather than external
sourcelist (see below).<br>
<br>
<br>
<blockquote cite="mid:50A4FD2B.6040902@astro.rug.nl" type="cite">
<ul>
<li class="level1"> <br>
</li>
<li class="level1">
<div class="li"> limiting magnitude. FLB can send a matlab
(actually octave script) performing the analysis of the S/N
vs. 2''mag diagram to HB Hugo Buddelmeijer who adds
algorithm to <a moz-do-not-send="true"
href="http://www.astro-wise.org/viewvc/awe/astro/experimental/kids/qcSourceLists_INTDR2.py?view=markup&sortby=date"
class="urlextern"
title="http://www.astro-wise.org/viewvc/awe/astro/experimental/kids/qcSourceLists_INTDR2.py?view=markup&sortby=date"
rel="nofollow"> qcSourceLists_INTDR2.py</a> </div>
</li>
</ul>
</blockquote>
<br>
as below, matlab codes will be sent<br>
<br>
<blockquote cite="mid:50A4FD2B.6040902@astro.rug.nl" type="cite">
<ul>
<li class="level1"> <br>
</li>
</ul>
The NN+HB / FLB+HB indicate that I think it will be useful to
implement those in Astro-WISE acting upon the ingested KiDS-CAT
sourcelists. Nicola and Francesco can send the relevant info/code
to Hugo Buddelmeijer who coordinates the implementation in
Astro-WISE.<br>
</blockquote>
<br>
Currently catalogs contain 136 columns if the VIGNET ones are thrown
away. We don't know yet how many of these columns will be kept and
how many can be possibly added. I've been told that AWE can handle
100 columns as integrated sourcelist. How about using the current
version of the catalogs as external sourcelist? Linked to the
original Coadd together with the other plots (FWHM vs SNR, S/G,
limiting, completeness, etc.) in the DBviewer as below?<br>
<br>
<br>
<blockquote cite="mid:50A4FD2B.6040902@astro.rug.nl" type="cite"> <br>
The following QC checks are planned for implementation:<br>
<ul>
<li class="level1">
<div class="li"> HB/DONE?: The qualityWISE link for KiDS-CAT
sourcelists in Astro-WISE (<a moz-do-not-send="true"
href="http://dbview.test.astro-wise.org/DbView?project=KIDS&set_cookie=&QSort=creation_date&QDesc=descending&2ndQSort=creation_date&2ndQDesc=descending&numrows=100&fcontext_project_only=on&show_expanded=no&Exportselect=NoDownload&class_str=SourceList&mode=table_res&depth=1&python_dict=%2528dp0%250AS%2527query%2527%250Ap1%250A%2528dp2%250AsS%2527expanded%2527%250Ap3%250A%2528dp4%250AS%2527SourceList%2527%250Ap5%250A%2528lp6%250AS%2527open%2527%250Ap7%250AaS%2527root%2527%250Ap8%250Aag5%250AasS%2527SourceList.chip%2527%250Ap9%250A%2528lp10%250AS%2527prefilled%2527%250Ap11%250AaS%2527link_property%2527%250Ap12%250AaS%2527Chip%2527%250Ap13%250AasS%2527SourceList.filter%2527%250Ap14%250A%2528lp15%250Ag11%250AaS%2527link_property%2527%250Ap16%250AaS%2527Filter%2527%250Ap17%250AasS%2527SourceList.instrument%2527%250Ap18%250A%2528lp19%250Ag11%250AaS%2527link_property%2527%250Ap20%250AaS%2527Instrument%2527%250Ap21%250AassS%2527deriv
%0A%20%0A
%20ed%2527%2%0A50Ap22%250A%2528dp23%250As.&expanded_leaf=&expanded_leaf_type=&expanded_leaf_class=&derived_leaf=&node_state=&SourceList.expanded=open&SourceList.chip.expanded=prefilled&SourceList.filter.expanded=prefilled&SourceList.instrument.expanded=prefilled&SourceList.SLID.op=%3D&SourceList.SLID=5967351&SourceList.detection_frame.base=BaseFrame&SourceList.frame.base=CoaddedRegriddedFrame&SourceList.instrument.name=OMEGACAM&SourceList.instrument.name.op=%3D"
class="urlextern"
title="http://dbview.test.astro-wise.org/DbView?project=KIDS&set_cookie=&QSort=creation_date&QDesc=descending&2ndQSort=creation_date&2ndQDesc=descending&numrows=100&fcontext_project_only=on&show_expanded=no&Exportselect=NoDownload&class_str=SourceList&mode=table_res&depth=1&python_dict=%2528dp0%250AS%2527query%2527%250Ap1%250A%2528dp2%250AsS%2527expanded%2527%250Ap3%250A%2528dp4%250AS%2527SourceList%2527%250Ap5%250A%2528lp6%250AS%2527open%2527%250Ap7%250AaS%2527root%2527%250Ap8%250Aag5%250AasS%2527SourceList.chip%2527%250Ap9%250A%2528lp10%250AS%2527prefilled%2527%250Ap11%250AaS%2527link_property%2527%250Ap12%250AaS%2527Chip%2527%250Ap13%250AasS%2527SourceList.filter%2527%250Ap14%250A%2528lp15%250Ag11%250AaS%2527link_property%2527%250Ap16%250AaS%2527Filter%2527%250Ap17%250AasS%2527SourceList.instrument%2527%250Ap18%250A%2528lp19%250Ag11%250AaS%2527link_property%2527%250Ap20%250AaS%2527Instrument%2527%250Ap21%250
AassS%252
7deri
ved%2527%
250Ap22%250A%2528dp23%250As.&expanded_leaf=&expanded_leaf_type=&expanded_leaf_class=&derived_leaf=&node_state=&SourceList.expanded=open&SourceList.chip.expanded=prefilled&SourceList.filter.expanded=prefilled&SourceList.instrument.expanded=prefilled&SourceList.SLID.op=%3D&SourceList.SLID=5967351&SourceList.detection_frame.base=BaseFrame&SourceList.frame.base=CoaddedRegriddedFrame&SourceList.instrument.name=OMEGACAM&SourceList.instrument.name.op=%3D"
rel="nofollow"> example link in DBVIEWER</a>) directs to
the QualityWISE pages of their coadds. These show catalog
based information: see <a moz-do-not-send="true"
href="http://quality.test.astro-wise.org/QualityWISE?object_str=SourceList&object_id=ce10420f2fbdeb45e0407d81e60e7648&project=KIDS"
class="urlextern"
title="http://quality.test.astro-wise.org/QualityWISE?object_str=SourceList&object_id=ce10420f2fbdeb45e0407d81e60e7648&project=KIDS"
rel="nofollow"> example</a>. These plots are NOT based on
the KiDS-INT-DR2 catalogs. They use a different sextractor
run. Should we do something about this? </div>
</li>
</ul>
</blockquote>
<br>
yes Francesco and I agree that this should be very straightforward
and we can provide all plots as above that you need to link to a
page like this.<br>
<br>
<blockquote cite="mid:50A4FD2B.6040902@astro.rug.nl" type="cite">
<ul>
<li class="level1"> </li>
<li class="level1">
<div class="li"> FLB: completeness. FLB: you probably have
best insight now how to do this.</div>
</li>
</ul>
</blockquote>
<br>
<div>this is work in progress. Francesco will send an email next
week about completeness and its implementation in KIDSCAT.</div>
<div>If you like, you can contact him directly and discuss this
issue.</div>
<div><br>
</div>
<br>
<blockquote cite="mid:50A4FD2B.6040902@astro.rug.nl" type="cite">
<ul>
<li class="level1"> <br>
</li>
<li class="level1">
<div class="li"> GVK: spatial distribution of sources and flag
values. GVK's proposal: a KiDS images with source/flag
information overlayed as step1?</div>
</li>
</ul>
<br>
</blockquote>
<br>
that would be very useful, although this might depend on the way we
ingest the catalogs, right?<br>
<br>
<br>
<blockquote cite="mid:50A4FD2B.6040902@astro.rug.nl" type="cite">
Summing these up: the CATALOG-QC-TEAM should verify we are not
missing crucial basic ones:<br>
<ul>
<li class="level1">
<div class="li"> CATALOG-QC-TEAM: CFHTLS at Terapix had its
final release (T0007). One product is <a
moz-do-not-send="true"
href="http://terapix.iap.fr/cplt/T0007/table_syn_T0007.html"
class="urlextern"
title="http://terapix.iap.fr/cplt/T0007/table_syn_T0007.html"
rel="nofollow"> synoptic table of CFHTLS</a>, giving
access to QC plots/numbers. Which ones do you think are
important at this stage for KiDS QC? </div>
</li>
</ul>
</blockquote>
<br>
From what I can see we are can provide all the plot/numbers as in
that page (either already in progress or to be derived). <br>
In particular <br>
<br>
Stacking method: we are still stacked on only one right?<br>
u g r i : done <br>
chi2 : ? <br>
cats-{u-g-r-i} : done<br>
Mask- {u-g-r-i} : done<br>
Mask [binary] : done <br>
cats-{ugriz} +e(b-v): TBD<br>
Galaxy Counts : can be done<br>
Star: Color-Color tracks: can be done<br>
Seeing (FWHM in arc-second): done <br>
Completeness limit point-like[80%/50%]: in progress<br>
ompleteness limit extended (disk) [80%/50%]: in progress <br>
unmasked MegaCam field (deg2): done<br>
Median stellar Mag-Diff [KiDS-SDSS]: TBD (FLB, MR) <br>
<br>
<br>
<blockquote cite="mid:50A4FD2B.6040902@astro.rug.nl" type="cite">
<ul>
<li class="level1"> <br>
</li>
</ul>
What does CATALOG-QC-TEAM deliver for KiDS-INT-DR2 single-band
sourcelists?: <br>
My proposal:<br>
1.the QualityWISE page: those we have in hand, they come for free
with Astro-WISE coadd production. Input on improving the plots
presented there is welcome.<br>
</blockquote>
<br>
OK<br>
<br>
<blockquote cite="mid:50A4FD2B.6040902@astro.rug.nl" type="cite"> 2.
A set of additional plots and/or numbers per catalog that
complements the QualityWISE page with KiDS-INT-DR2 specific stuff.
This can be either presented as one page per coadd or gallery of a
certain plot for the 52 single-band sourcelists. Danny is looking
into how we can implement such a presentation. I see this set as
an experiment from which we can choose how to develop QualityWISE
for single-band sourcelists further after KiDS team meeting in
December. <br>
</blockquote>
<br>
see previous comment on the DBviewer.<br>
<br>
Hope this can help.<br>
<br>
Cheers,<br>
<br>
Nicola<br>
<br>
<blockquote cite="mid:50A4FD2B.6040902@astro.rug.nl" type="cite"> <br>
Discussion has opened: all input welcomed (keep emailing <a
moz-do-not-send="true" class="moz-txt-link-abbreviated"
href="mailto:kids@astro-wise.org">kids@astro-wise.org</a>!)<br>
<br>
Above information and more is contained at our wiki spot for
CATALOG-QC-KIDS-INT-DR2 for single-bands:<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.astro-wise.org/projects:kids:data_deliveries:internaldelivery2.0_scratch#qc_based_on_single-band_catalogs">http://wiki.astro-wise.org/projects:kids:data_deliveries:internaldelivery2.0_scratch#qc_based_on_single-band_catalogs</a><br>
<br>
Regarding 4-BAND CATALOGS: I propose to leave that effort as a
separate task. Details will follow in another email. <br>
<br>
Best wishes, Gijs<br>
<br>
<pre class="moz-signature" cols="72">--
--------------------------------------------------------------------
|dr Gijs Verdoes Kleijn | astronomer |
|e-mail: <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:verdoes@astro.rug.nl">verdoes@astro.rug.nl</a> | OmegaCEN / Kapteyn Institute / |
|www: <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="http://www.astro.rug.nl/%7Everdoes">www.astro.rug.nl/~verdoes</a> | Target |
|tel: +31-50-3638326 | University of Groningen |
|mobile: +31-654658050 | postal address: |
| | Kapteyn Astronomical Institute |
| | Postbus 800, 9700 AV, Groningen|
| | The Netherlands |
--------------------------------------------------------------------</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
KiDS mailing list
<a class="moz-txt-link-abbreviated" href="mailto:KiDS@astro-wise.org">KiDS@astro-wise.org</a>
<a class="moz-txt-link-freetext" href="http://listman.astro-wise.org/mailman/listinfo/kids">http://listman.astro-wise.org/mailman/listinfo/kids</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Nicola R. Napolitano
Research Astronomer
INAF - Observatory of Capodimonte
Salita Moiariello, 16, 80131, Napoli (Italy)
phone: +39 081 5575509; fax: +39 081 456710
email: <a class="moz-txt-link-abbreviated" href="mailto:napolita@na.astro.it">napolita@na.astro.it</a>
web page: <a class="moz-txt-link-freetext" href="http://www.na.astro.it/~napolita">http://www.na.astro.it/~napolita</a>
</pre>
</body>
</html>