UKIDSS Survey Heads meeting, Durham 27 July 05

Present: Omar Almaini, Alastair Edge, Richard Jameson, Nige Hambly, Phil Lucas, Mike Irwin, Simon Dye, Steve Warren

Actions labeled e.g. **MJI

Status of pipeline

The sequence of data reduction is:

1. linearise, 2. dark subtract, 3. flat field, 4. decurtain, 5. sky subtract

Step (1) appears to be unnecessary and is not implemented. In step (3) sky flats are used, created by combining 5s flats taken with counts that are substantially higher than in the data. In step (4) in arrays #2 and #4 single channels are occasionally visibly low, but this is not viewed as an important concern. In (5) the main worry is blobs produced by crud on the field lens, which can vary in intensity quite rapidly in time, and can shift slightly spatially. In ZYJH it is believed to be due to scattered light (mostly from moon) while in K thermal emission also contributes. Currently under study, and not yet implemented are removal of cross-talk and persistence ghosts. Cross talk produces ghost images in nearby channels that (strangely) are the derivative along (? or across) the channel of the image of bright stars. Cross-talk ghosts of heavily saturated stars look like washers, while ghosts of unsaturated stars are negative-positive images. Although undesirable, cross-talk images are unlikely to be classified as real. Persistence images are more pernicious as they could be classified as real.

The catalogues do not currently include all the relevant information. Missing items include photometric errors, and Petrosian and Kron magnitudes.

The entire May-June observing block has been pipelined. It is planned to redo the catalogue generation at some stage in the near future.

The WCS system is ZPN which IRAF does not understand. Jim Lewis has written an IRAF patch.

Catalogues are generated for a product e.g. an interlaced pawprint, or an image stack. Quality control parameters are put into the catalogue header, but it was unclear if these appeared anywhere in the archive at present. The DXS wanted to have QC information for every frame taken if possible.

QC plots presented show rather large drifts in ellipticity over relative short periods. It was suggested that this could be due to drift of the focus.

**MJI to clarify on Science Verification page the consequences for IRAF users of the ZPN WCS, and to ensure the patch is made available.

**MJI check whether dark flushing is implemented after filter changes, and if this should be included in future MSBs

**MJI requested to have current pipeline procedures, issues, and items missing from catalogues summarised on the Science Verification TWiki page.

**NCH to check that QC parameters from catalogue headers are propagated.

**SJW to follow up with JAC the drift in ellipticity seen in images on timescales of about one hour.

**MJI,AE continue discussion on QC parameters for every DXS (and presumably UDS) frame

Archive

We discussed the propagation of fluxes for stars and galaxies from the detection tables (i.e. single band) to the source tables (i.e. lists of matched objects, therefore including colours). Colours need to be useful for both stars and galaxies, but the user should have the flexibility of defining the exact star/galaxy boundary. This implies propagating both a suitable stellar flux (FWHM preferred until psf implemented), and a galaxy flux (2" aperture magnitude in the absence of Petrosian magnitudes) for all objects in all bands. The issue becomes moot once the list-driven photometric catalogue appears. Users are invited to attempt to justify addition of any further flux measure.

Overheads, microstepping, integration times

Email of 26/7 from Andy Adamson was discussed. With expected software improvements, using the basic 5s integration results in 59% efficiency. This is unsatisfactorily slow. All present found it curious that the data taken was substantially slowed down by 'synchronisation loop and data handling' activities. If these could somehow be executed in parallel very significant efficiency gains would be made. The total time saved over the lifetime of the surveys could easily justify substantial cost to achieve this.

The DXS and UDS will now adopt 10s as standard, which should be 74% efficient. The LAS, GPS, GCS then discussed whether they should drop microstepping in order to improve efficiency. There was a general feeling that microstepping doesn't gain you much in photometric accuracy, but could be important for astrometry. A compromise solution might be to microstep in the repeat bands only (K for GPS and GCS, and J for LAS). Then if improved sampling were important for other science e.g. galaxy surface-brightness profiles, this would be achieved in the astrometry band.

**Survey Heads to decide on future microstepping strategy in consultation with working groups.

SDT and MSB design

Noone was keen on the JAC suggestions that MSBs should be limited to 15min in length. Arguments against i) it wouldn't suit the pipeline, which works better with long runs in a single filter, ii) the number of MSBs would be 4 times greater, presumably demanding more frequent uploads to keep the number of MSBs in the database to a manageable size, iii) the problem of getting contiguous coverage will be made worse.

There is a problem that the SDT sometimes selects galaxies as guide stars. The understanding is that this is because it uses USNO-A (useable online). What we really want is USNO-B. SDSS has been mentioned, but it would be better to have an all-sky catalogue so that the same procedure is used for all surveys.

**SD to liaise with Martin Folger on how to get USNO-B used, with galaxies excluded.

**SD to liaise with Martin Folger on how to get a numbering scheme into the naming of MSBs in an automated way

**SD to liaise with Martin Folger to check whether the revised measurement of the exact location of the guide field, realtive to the arrays, has been propagated to the ST

Observing

There does not seem to be a detailed protocol for observing standards, other than it should be done every hour. This means that there is no guarantee that the extinction is measureable from a night of observations. This will get taken up at a planned meeting of the calibration working group.

We need a protocol for dealing with interrupted MSBs in the situation where the MSB will be fine if just part of it is redone (e.g. say just the last 10mins). Some observers have successfully redone the last part, but insufficient information was available to CASU to process the data in the way it should have been.

**SJW to raise this with JAC.

Charting progress

The best would be a zoomable sky map in each band (as in the WSA, but more detailed). Every MSB would be shown, and numbered, with 3-colour coding (unobserved, observed, in archive). So we need to devise a numbering scheme so that every MSB is unique, and on the map.

Selection of MSBs to observe, prioritisation, etc.

The two survey heads who have observed expressed misgivings about the QT. Having divided a survey into blocks, if a particular block is selected then we would like the MSBs to proceed sequentially by ranking. For example for stable conditions, in the LAS if the HK MSB has been done, we would like the next MSB executed to be the YJ MSB on the same field (and they are ranked to achieve this). With the current algorithm this does not necessarily occur, probably because of the weighting by airmass. In any case if there are over 100 MSBs in a project inevitably some MSBs get the same ranking. Can a decimal point be added to the ranking? Survey heads are unwilling to break down their projects into smaller blocks. The LAS has 16 blocks, and it would be difficult to keep track of a greater number. Nevertheless any one block could have 500 MSBs, which JAC find uncomfortably large.

Here, using the LAS as an example, is an outline of how we would like it to work. UKIDSS produce all MSBs for a block (as many as 500). Every MSB has a number or label that is unique. If the block is selected, the MSBs are executed sequentially (the airmass range is not large across a block).

Omar suggested that the surveys produce an observing guide, to ease choices for observers at the summit. This should certainly be considered, but what we want will depend on our discussions with JAC on the above.

**Survey heads: produce useful naming/numbering scheme for all future MSBs

**SJW: liaise with JAC over these issues, and develop discussion between JAC and survey heads

OMP

The OMP is less obviously useful than for PATT projects. Nevertheless with a better naming/numbering scheme it could be, particularly if the blocks proceed sequentially.

-- SteveWarren - 29 Jul 2005