[News] WFI-data ingest

[Edwin A. Valentijn]

Dear Astro-Wiser,

This mail is about initiating WFI at 2.2m archive studies using the 
AstroWise system.

The development of the  Astro-Wise system is now  fully  engaging Phase II.
Following the conclusions of the workshop in November last year a major 
iteration on the Class definitions is currently  implemented.  The  
expectation is, that by the 1 March these implementations will be 
completed both on pipeline level and on database level.
At the same time we are working on a major improvement of the 
documentation of the system, which is also expected to be completed 
around the 1 March.
 >From then on, our system will  be ready to host first scientific 
projects, clearly still in an experimental phase, but well enough 
documented and stabilized to allow pioneer PhD students and others to 
run "small" projects.

At the workshop we discussed how to process WFI at 2.2m data, particularly 
the photometric calibration.
We will use the sky brightness to do the chip-tp-chip calibration, while 
the zeropoint will be derived for a single chip. Code for this will be 
available, but it will be still  laborious to reduce WFI data due to 
variations in data acquisition scenarios. Nevertheless, we are now in a 
position to  ingest  and process big quantities of publicly available 
WFI data, and for the coming year this will be  important for the 
development and qualification  of the Astro-Wise system.

ESO is developing a new data distribution system, using swappable hard 
disks, which will be mailed to the users. This new system will be 
announced in one of the coming issues of the Messenger. Benoit Pirenne 
of ESO's archive group has kindly offered to test this new system with 
the OmegaCAM consortium in the coming months. We will buy a couple of 
normal IDE hard disks, with special box in which the hard disk slides 
(not expensive) which will be used for this data transfer. Typically, a 
harddisk will contain 200-300 Gbyte of data, i.e 1600-2400 images with 
raw data. We can test this procedure by issuing WFI data requests. 
In Groningen, but also at other Astro-Wise sites we can then ingest 
these data into the Awe system,  and further process it. OmegaCEN will 
reserve sufficient amount of RAID disk towers (typically 2 Tbyte) to 
ingest these data and store processed data.

An involvement of the consortium partners in  this activity would be 
very much welcomed, and I 'll try to co-ordinate this in the near future 
together with Philippe Heraudeau.
Kapteyn and OAC already indicated that they want to request datasets for 
their supercluster projects.

If you want to participate I would appreciate receiving a "statement of 
intent" before 1 February,
whether you want to embark on this an what kind of targets your are 
considering.
Next, we will organize some co-ordination for  the preparations for a 
data request.
You can check yourself the publicly available  part of the archive at

http://archive.eso.org/wdb/wdb/eso/observations/form

It will take some extra effort to analyse /query the archive and thus it 
is a good idea to first mail me what you want  and we might further 
coordinate the querying/analysing work.

Note, that there are several options to further distribute the data over 
the consortium:
-  harddisks can be passed along the consortium
-  remote processing is an option
-  Oracle STREAMS and/or FTP might provide us electronic data distribution
In any case, all derived calibration data will be shared over the sites.

So, please if you are interested respond to this message to Philippe and 
myself before 1 February.

looking forward

Edwin

-- 
----------------------------------------------------------------------
Edwin A. Valentijn                  OmegaCEN www.astro.rug.nl/omegacen
Kapteyn Astronomical Institute      tel   : +31 (0)50 3634011
P.O.Box 800                         mobile: +31 (0)6 48276416
NL-9700 AV Groningen                e-mail: valentyn at astro.rug.nl
The Netherlands                     www   : www.astro.rug.nl/~valentyn
---------------------------------------------------------------------- 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://listman.astro-wise.org/pipermail/news/attachments/20040121/37e4c99f/attachment.htm