|Downloading data directly from the NOAO Archive using cURL|
Alternatively, you can fetch your data using cURL, a command line tool that allows batch downloads of data directly from the archive without going through the FTP staging area. In certain circumstances, this may be faster or more robust than standard FTP. Here we describe the procedure for using cURL to retrieve data from the NOAO Archive.
Scripts and certificates that you will need in order to retrieve data this way are available in the NOAO Archive contributed software repository.
1. Obtain a copy of cURLcURL is a command line tool for transferring files using URL syntax. It is provided by most Linux systems but supports other systems as well.
2. Read the secure_get instructionssecure_get.sh is a script that uses cURL and an SSL certificate (which is necessary if the data are proprietary) to retrieve a list of archive files specified by their URIs (Uniform Resource Identifiers). The procedure for creating a list of URIs for archive data is described in step 5, below.
3. Download the secure_get script
4. To retrieve proprietary data, get an SSL certificateThe SSL certificate encodes your username and login information so that the secure_get script can retrieve your proprietary data from the NOAO Archive.
If you wish to retrieve only public (non-proprietary) data, you can skip this step.
5. Query the NOAO Archive the data you wish to retrieveSearch the NOAO Archive for the data you wish to retrieve following standard procedures described in the tutorial on how to Search and Retrieve NOAO Data:
- Fill out the Query Form and click Search
- On the Results page, select the data you wish to retrieve by clicking the checkboxes for individual data sets, or by using the Selection pulldown menu to select all results.
- Click Download selected rows as VOTable.
6. Retrieve your data using curl, your SSL certificate, and your query result VOTableFrom your local computer, execute the following commands:
% chmod +x secure_get.sh % ./secure_get.sh rows_as_votable_1303942326_4420.vot.xml --cert copernicusn.certSubstitute the actual name of your downloaded VOTable and SSL certificate.
If you are retrieving only non-proprietary data, you can leave out the --cert option.