[News] Minutes Nov05
Gijs Verdoes Kleijn
g.verdoes at astro.rug.nl
Tue Dec 6 15:43:58 CET 2005
Dear astro-wisers,
Please find attached the minutes of the PI meeting, the DBA meeting and
Action Items of the Astro-wise workshop 14-18 November at the Lorentz
Centre, Leiden, The Netherlands.
Best wishes, Edwin & Gijs
-------------- next part --------------
===============================================================================
Minutes of the AstroWise PI meeting 18 Nov 2005 11:30 - 13:30
Lorentz center Leiden
===============================================================================
Present: T. Erben (Bonn), Y. Mellier (Terapix), M. Neeser (USM), R. Silvotti (OAC), G. Verdoes-Kleijn (OmegaCEN), E.A. Valentijn (OmegaCEN)
===============================================================================
PIs, local coordinators
=======================
M.Neeser just moved to ESO-DMD. He will stay involved in OmegaCAM and AstroWise in his new position.
Dave Wilman will replace Mark Neeser as the local coordinator at USM/MPE (mail from Saglia after meeting)
T.Erben represents Bonn- AstroWise satellite (O. Cordes attends DBA splinter).
G.Verdoes is new AstroWise Postdoc at OmegaCEN and will have liaison function within the consortium.
ADMINISTRATIVE
==============
EU: The zero-cost 5th year extension has been signed by all partners and EU.
Cost statements of year 3 have been approved without any modification.
Third instalment has been received and will be distributed soon over partners.
Before doing this two issues will have to be resolved:
1) ESO is the only partner with a major positive balance after year 3 and ESO has been approached for transferring the surplus amount back to NOVA (action EV).
2) the legal situation of OAC/INAF should be clarified; also regarding the signature for the forthcoming cost statements (action RS,MC).
(Note 6 dec 2005- all issues resolved- paymant inititated - ev)
Partners have to carefully overlook the spending profile in year 5, since this is the last year we operate under the present contract. Cost made beyond 1 Dec 2006 can not be declared to this contract.
EVALUATION (current) WORKSHOP/COURSE
====================================
All partners were very positive about the workshop held last week and the progress made. It was good to see how completely new people could find their way into the system.
It was agreed that next year an AstroWise workshop should be held again at the Lorentz center. PI booked the Lorentz center for week 47 (20 - 24 November in 2006). This period falls just within the AstroWise 5th year. The expectation is that at that time OmegaCAM data and analysis can be presented and discussed.
NODE REPORTS / NEXT STEPS
=========================
The AstroWise Portal www.astro-wise.org/portal is now publicly announced. It contains extensive documentation, how_tos and various web and Grid services. Public WFI at 2.2m data can be accessed.
DBA
---
Now with the AstroWise system being well in place the highest priority for the coming quarter is to make sure that the system is federated (partly mirrored) at the AstroWise nodes. Parallel to the current PI meeting a DBA splinter meeting
is held to inventories the necessary steps. The minutes of this DBA meeting are attached here. In the near future the federation will be upgraded at USM/MPE, OAC and Bonn. Terapix will install AstroWise in the coming month (action F. Magnard). For the coming half year databases will be replicated using Oracle "data pumping" mechanism. Several research and development lines are on-going to advance this to more intelligent mechanisms.
Science operations
------------------
The system is now well enough developed to run science projects on it. At OmegaCEN 3 PhDs and few PostDoc make use of the system for their research. Now, with the official release of the AstroWise portal, which includes lots of documentation, it becomes feasible to involve researchers at other nodes in the system. This will provide the necessary feed-back but will also accelerate local expertise.
A large set of 150.000 public WFI at 2.2m CCD readouts are ingested into the system ready for scientific research. The system is also ready for INT (La Palma), MDM and OmegaCAM data. PI will contact the director of CFHT to explore the possibility
to ingest MegaCAM data into the system - Terapix is also very interested in this (action ev).
USM
---
R.Saglia wrote before the meeting that USM/MPE is planning WFI at 2.2m research projects. Further, Johannes Koppenhoefer (PhD) will support OmegaTrans development as a new AstroWise person. Jan Snigula will continue as local DBA, AstroWise system support and will support the photo-z tool.
OAC
---
OAC plans to give a local AstroWise workshop in the near future and to involve researchers in the system. F.Getman will continue as local DBA and AstroWise support. AstroWise persons Radovich and Alcala can assist in familiarizing local staff with the system. OAC will further fine-tune the 'variability tool" VODIA. OAC will continue testing and fine-tuning the QC tools.
Terapix
-------
Anis Rojbi started working at Terapix on 4 Nov 2005. He will implement the Panorapix large_image visualisation tool. OpenGL has developed substantially last year making this task much simpler. A working prototype is expected within the coming 6 month. Magnard will start implementing AstroWise before the end of the year and in Januari and Februari 2006 a feed-back period is planned.
E. Bertin expects to build the PSF homogenization code coming year. Y.Mellier will deliver "automask" to AstroWise (action YM -> E.Helmich)
Bonn
----
O.Cordes is local DBA and AstroWise system supporter. The system will be upgraded in the near future and then the (EIS) Deep Public Survey will be re-processed with AstroWise, for comparison, but now including the illumination correction.
PI will assist in acquiring Oracle licenses (action EV).
OmegaCEN
--------
Staff at OmegaCEN will support the distribution of the system to partners.
Most of the major development work is behind us, but some substantial work has to be done on smart database federations, implementing more complex context privileges (without making the system more complex ...) as was requested by project managers at the workshop and to advance the push-buttom GUI for target processing. List of AIs include settling of secondary photometric standards, provision of thumbnails views and many small items originating from users feedback.
It is planned to include a surface photometry package in the system, together with OAC, Peletier and perhaps Bochum. OmegaCEN list of Ais is posted weekly on their website. The coming period the system will be qualified and stress tested with WFI at 2.2m data.
ASTRO-WISE > year 5
===================
OmegaCEN has secured some funding for AstroWise support beyond 1 dec 2006.
Partners agreed to undertake additional steps coming year for asking additional national and EU funds for AstroWise operational support beyond the present EU contract (action all PIs).
First AstroWise based research paper is submitted (Sikkema et al) and an AstroWise project descriptive paper is in preparation.
==================================================================================
end minutes PI meeting
===================================================================
Astro-Wise DBA splinter - 18 November 2005, Lorentz Center, Leiden.
===================================================================
Present: K. Begeman, O. Cordes, J. Dietrich, J. McFarland, F. Getman, A. Grado, J. Snigula, D. Boxhoorn.
Conclusions from the meeting.
-----------------------------
Prime focus of the DBA activities in the coming quarter will be the complete replication/mirroring/synchronisation of the AstroWise system over the different nodes. Staff at OmegaCEN will support and document this, local DBAs have the responsibility for the local installations.
In order to reach the following actions have been defined.
A full Database Federation is now urgent, amongst other things, because of the
following ongoing partnerships:
Amata Mercurio, Chris Haines, et al. (Napoli) & Gert Sikkema (Groningen)
VESUVIO - Particular science data and corresponding calfiles.
Jan (M�nchen) & Kor (Groningen)
Photometric Redshift with SourceLists.
Oliver (Bonn) & Danny (Groningen)
Reduce science data in Bonn with calfiles from OmegaCEN.
The following points were flagged as crucial for reaching our goals:
. Frequent and intensive communication is necessary, even more so with
the upcoming Federation effort in mind. Thursday is the preferred day for DBA
activities and all DBA's try to keep this day free in their agenda.
This becomes important as type evolution will be more synchronized when the
data are federated. Every Thursday a DBA network meeting will be organized,
chaired by DRB( we propose to do this with Skype- please buy nice headset)
(action DRB, all DBAs).
. Installation documentation of all different AWE components has to be updated
to reflect the current situation (action KGB, DRB).
. Database user authentication via LDAP needs to be in place to support
single sign-on for all users. The Oracle authentication server needs to be
installed at OmegaCEN (action DRB, KGB) and the databases at the Astro-Wise
sites need to use the authentication server (action all DBAs).
. The Astro-Wise environment, database (Oracle 10g Release 2), dataserver
and parallel processing (dpu) software need to be installed as explained
in the updated documentation (action all DBAs).
Install new database hardware where this arrives in the coming quarter
(action FG, and maybe OC).
After the Astro-Wise system has been setup at an Astro-Wise site, verify the
installation, by local export/import of database objects via Oracle DataPump
and store/retrieve of images on the local dataserver (action all DBAs).
To connect the dataservers between all Astro-Wise sites and storing/retrieving images their hostname and port need to be known. The federated dataservers
will be tested for store/retrieve/cache and delete operations between
all participating Astro-Wise sites (action all DBAs).
With the local Astro-Wise system in place, distribute the DDL (metadata for
TYPE/TABLE/VIEW) from OmegaCEN (action DRB) to be imported at the partner
sites (action all DBAs).
Finally, automatically export and import data that is created at each site
with a fixed frequency (initially on a daily basis) using Oracle DataPump
(action all DBA).
. The procedure for type evolution at the different Astro-Wise sites has
to be settled. All type changes should be put in CVS instead of just
communicating them via the wp3 mailing list (action DRB).
. Oracle Automatic Storage Management (ASM) is the preferred way to store
databases. It seems better to have Oracle manage the (raw) disks, instead of
operating RAID5. This is true both for performance and maintenance reasons.
In Groningen ASM is in use, in Napoli its use is planned and it was concluded
that for Bonn and M�nchen ASM is recommended.
. Give functional description of CONTEXT, incorporating feedback from the
workshop, i.e.
- Project specific Process Parameters.
- Project specific Source code.
- Project dependent selection of specific calfiles, as a sort of
"superflag per Project".
Write a proposal for implementation of the required functionality.
(action DRB).
==================================================================================
DESIRABLE DEVELOPMENT ITEMS Noted at the workshop (EV and KK)
==================================================================================
- Trendanalysis: zoom in on outlyers - find data item
- all raw FITSHEADERS in db
- timestamps settings and flag bad data from Awe promp
- Thumbnail views on images in db view
- cutouts same RA, DEC images
- cutouts on all
- date/time
- wild cards
- make it more vtransparant
- help(dpurun)
- evaluate handling of edges in Associate lists
- Associate Source list from different projects
- list of functionalities CONTEXT
- implement "project favourite flag" allowing projects to lock on their favourite CalFiles
- transparency on version control of the software
- include echoes of the commands entered in logs
- importing coordinate lists: how are epochs (precession of the sky) handled?
- associated lists: can the stats of the distances between matches be kept (eg min, max, median, rms offset distance)
- sextractor 2-image mode: is this supported? Will be vital for any matched-aperture photometry (equivalently: can astrowise do list-driven photometry on image files)
- filter properties (throughput curves as function of field position, perhaps location of ghosts, etc etc) be made an attribute of the filter class?
- add interstellar extinction
- handling segmented filters - Does ingestion turn one segmented filter into different filter names for the 4 quadrants?
(eg Halpha --> Halpha_0, Halpha_3000, Halpha_6000, Halpha_9000)
- AstroWise on laptop (Oracle client)
==================================================================================
More information about the News
mailing list