NDWFS Image Cutout Access

Home Data-Access Data-Processing NDWFS-Publications Related-Surveys Links Additional-Documentation

NDWFS Image Cutout Data Service

Access to the bulk NDWFS data products is available via the NOAO archive FTP server. For many types of analysis however, all that is really needed is small science-grade image "cutouts" around objects of interest, e.g., for computing morphological parameters of objects of interest or for comparision with cutouts of the same regions from other surveys to perform multiwavelength studies.

The image cutout service described herein provides this capability. The image cutout Web service allows users to access the archive server and extract and download cutout images on-the-fly, returning a FITS file to the user. Using the cutout facility it is possible to download thousands of small image cutouts in less time than it would take to download a single one of the 300-400 MB bulk NDWFS data product images.

The NDWFS web-based image cutout services are illustrated in the figure above. A variety of Web clients may be used to download cutout images. These range from a simple browser-based fill-out-the-form facility, to Web client programs that you run locally. You can also write your own client-side software to access the URL-based image cutout web service directly.

The image cutout Web service is accessed via a simple URL which calls a PHP script which in turn calls the IRAF cutout task to extract cutouts from the NDWFS survey image database. The IRAF cutout task extracts a subregion centered at the specified position from the survey image database, and returns a FITS image which may be saved to local disk by the browser or other client program. Unique file names based on position and filter are automatically generated for each cutout region.

Client Software Options

Downloading Cutouts with a Web Browser

The browser interface to the cutout service provides two forms for accessing image cutout data. In the simplest case the user supplies an RA and DEC position for the desired region and optionally a filter or image size. When submitted, a FITS image is returned which will create a dialog box allowing the user to save the file locally. By default the form will return a 200 pixel square subregion of the field.

For multiple cutouts, a second form allows one to upload a file to specify the regions for which cutouts are to be generated. Each line of the file specifies a single cutout region, giving a position and optional parameters such as size, filter, and coordinate system (see the cutout task help page for details). A FITS image for each region is created and a gzip-compressed tar file of the resulting images is returned as the file ndwfs_images.tgz (the .tgz extension is commonly used to indicate this type of file) which again may be saved locally using the dialog box.

The browser interface to the cutout service is available at

http://archive.noao.edu/ndwfs/cutout-form.html

Using WGET and similar UNIX Clients

Web client programs which can download a URL and save it to a file can also be used to generate and download image cutouts, for example the wget utility. This approach is convenient for downloading multiple cutouts using simple techniques such as a host shell script, and has the advantage that Web clients such as wget are widely available and indeed may already be installed on a given host. The disadvantage with this approach is that the unique file name created on the server is normally not preserved (see ndwfsget below for a way to get around this limitation).

A number of such client programs are available, for example: wget, snarf, curl, and lynx All of these provide a command-line interface to URLs and can be used from a variety of scripting languages to process the cutout requests automatically. For example, a wget request might look like

    % wget "http://archive.noao.edu/ndwfs/cutout.php?ra=14:28&dec=34:55" -O pix.fits

The result of the query shown would be a FITS file "pix.fits" of the requested cutout region.

There are several caveats to this approach when using any of the above clients:

The URL itself must either be enclosed in quotes or otherwise escaped (depending on the shell or scripting language used) to prevent characters such as '?' or '&' from being interpreted as metacharachers instead of URL delimiters.
The name of the file returned is generally the name of the requested URL, in this case the full cutout.php command line including all parameters. Users will need to rename the file themselves (e.g. using the -O option to wget) to something more meaningful locally. (Consult the cutout task help page for a description of the naming convention used by the task on the server if you wish to reproduce the names within the script).

Despite these limitations, such clients still provide a powerful method for scripting a solution to retrieve multiple regions relatively easily. All tasks mentioned are freely available open-source software and will run on all major platforms. Wget is readily available as a standard component on many Unix-based PC platforms.

Using the IRAF NDWFSGET task for downloads

The ndwfsget program is an IRAF-based Web client for extracting FITS cutout images from the NDWFS survey. This task is installed in the xdwred.xastcat external package for use at NOAO. For sites outside NOAO, this task is also available as a standalone application (see cutout.tar.Z for a tar file of the sources, and cutout.readme for installation instructions).

The ndwfsget task can be called either from within IRAF or at the host level. Unlike generic Unix programs such as wget, this task preserves the file names and automatically writes all images to the local disk. Users may supply a regions file or filter list to download any number of cutout images with a single command.

The default parameter set is similar to the following:

so> lpar ndwfsget
       output = "default"       The output image root name
           ra = "14:28:07.00"   The cutout center right ascension in hours
          dec = "+34:55:00.0"   The cutout center declination in degrees
     (rawidth = "1.0")          The cutout right ascension width in minutes of 
    (decwidth = "1.0")          The cutout declination width in minutes of arc
    (fcsystem = "ICRS")         The cutout center coordinate system
     (filters = "Bw,R,I")       List of filters to extract
     (regions = "none")         The input cutout centers file
       (blank = 0.)             Pixel value for undefined regions
     (verbose = yes)            Print status messages ?
        (mode = "ql")

Running ndwfsget with the parameters shown above would produce Bw, R, and I band 1.0 arcminute square cutouts around the position ra = 14:28:07.0 dec = +34:55:00.0. Cutout regions can also be defined in a file in which case the ra, dec, rawidth, and decwidth parameters are ignored. X and Y widths in pixels, rather than rawidth and decwidth in arcminutes, can be defined by appending a 'p' character to the rawidth and decwidth parameters, e.g. rawidth='250p' decwidth='250p'. A detailed desciption of the task can be found in the ndwfsget task help page.

The Image Cutout Web Service Interface

Cutout images are generated using a URL of the form:

    http://archive.noao.edu/ndwfs/cutout.php?ra=14:28:07&dec=34:55:40

where cutout.php is the name of the script, and various task parameters are given as param=value pairs on the command line. These are delimited from the script name using a '?' and from each other using a '&' character, no spaces or parameter abbreviations are allowed in the URL. The only required parameters are the ra and dec position for the cutout. The default cutout image will be 100x100 pixels in the Bw filter unless the rawidth/decwidth and/or filter parameter are also set to specify a different size or filter name.

Any application which can build such a URL and connect to the NDWFS server can use the cutout web service directly to download image cutouts. Indeed, the ndwfsget task above does just this, constructing a URL for each image and writing the results to a local file. Similar software can be written to integrate the cutout service into any program.

Options supported by the image cutout Web service include the following:

       output = "default"        The output image root name
           ra = "00:00:00.00"    The cutout center RA in hours
          dec = "+00:00:00.0"    The cutout center DEC in degrees
     (rawidth = "100p")          The cutout RA width in minutes of arc
    (decwidth = "100p")          The cutout DEC width in minutes of arc
    (fcsystem = "ICRS")          The cutout center coordinate system
      (filter = "Bw")            Filter to extract
       (blank = 0.)              Pixel value for undefined regions
      (imroot = "NDWFS")         The default root image name

See the cutout task help page for a complete description of the task parameters and their values.

The IRAF CUTOUT Task

CUTOUT is an IRAF task for extracting cutout images from a survey dataset using world coordinate information. Cutout is also the server-side task being used to create the NDWFS cutout images, and while it will not normally be called directly by the user it is worth looking at to understand the details of how image cutouts are generated. The task is sufficiently general that it may, but is not guaranteed to, work with other image survey data.

Given a set of images comprising the survey, the task can be used in scan mode to create a database file of images to be used for later extractions. In cutout mode (i.e. the mode used by the cutout service) the task will extract a subregion from the survey at a position and size specified by the task parameters. The interface to the task is via the cutout.php URL mentioned earlier, therefore any application which can construct the URL and connect to the server can access the task and retrieve the data directly.

The task has been installed in the xdwred$xastcat external package for use at NOAO. For sites outside NOAO, this task is also available as a standalone application (see cutout.tar.Z for a tar file of the sources, and cutout.readme for installation instructions).

See the cutout task help page for a complete description of the task parameters and their values.

NOAO Deep Wide-Field Survey -- Last modified: September 06 2007 11:01:48. -- Feedback: jannuzi@noao.edu