ESO Archive Frequently Asked Questions

WDB Query Forms (http://archive.eso.org/wdb/wdb/....)

• Hints on how to improve your search strategy
  • Do a general search first, look at the result and use values from the result table to refine your query; gradually narrow down your search.
  • On some query forms one can select the columns that appear in the tabular output. Mark all fields which are not displayed by default prior to the search and after looking at the result decide yourself which columns are crucial in the given context.
  • Browse also the programme (proposal) information and search for keywords or known authors to get references to particular observations that cannot otherwise be matched with some search criteria on a given form.
  • When querying for targets make sure the radius of the box around the target coordinates is large enough to cover the field of view of the instruments in which you are interested.
  • Consult the online help for a given field on a form if available.
• How does SIMBAD resolve the target name?
  SIMBAD, via its SESAME service is a database of basic data, cross-identifications and bibliography for astronomical objects outside the solar system. The SIMBAD client routines take your target name and return the center coordinates for the object, provided the object has been mentioned in the literature or is listed on a published catalog. We then query the archive for all observations whose pointing falls within the radius given in the form.
  NED is an extragalactic database, hence galactic targets should be resolved by SIMBAD.
  The client routines are provided by courtesy of the Centre de Données astronomiques de Strasbourg.
• How is the DIMM seeing measured? Do you store the FWHM measured on the image itself?
  The DIMM seeing is defined as the Full Width Half Maximum of a stellar image observed with a perfect large telescope, at 500nm wavelength and at zenith. It is measured over periods of a few minutes all night long. See the ESO Astroclimatology Overview web pages for more information.
  ESO does not routinely extract and store a FWHM measured directly on image pixels.
• What is a DP ID and what is Orig Filename?
  A DP ID is a unique identifier assigned to every data product created by ESO. This identifier is composed of the instrument acronym followed by the date and time of start of the exposure (e.g. FORS1.2005-03-29T09:37:42.829).
  The Orig Filename is not unique and is used only at the instrument control workstation at the telescope. These names are displayed at the quick-look console and therefore may be used by visiting astronomers.
• What is the coding scheme for ESO Programme IDs?
  Read the documentation on the ESO programme identification code.
• What is the meaning of the check-box located left of most query fields?
  The check-box allows you to decide which fields will be included in the results table. Checked fields are listed as columns.
  More information on controlling the navigation options is given in the WDB help page.
• What is the relation between instrument IDs and prefixes of archival dataset identifiers for raw data?
  An archival dataset identifier (also known as DP ID) for raw data is composed of a prefix, followed by a timestamp up to the millisecond.
  The prefix usually corresponds to the instrument ID (ins_id parameter in the main archive query form).
  Example: instrument ID = UVES, archive filename = UVES.2009-03-20T09:22:28.489
  
  However, there are a few exceptions which are listed below:
  
  

  instrument ID	DP ID prefix(es)
  CRIRES	CRIRE
  EMMI	ONTT
  ESPRESSO	ESPRE
  FORS	FORS1, FORS, or OFORS
  GIRAFFE	GIRAF
  GRAVITY	GRAVI
  ISAAC	ISAAC or OISAAC
  MATISSE	MATIS
  NAOS+CONICA	NACO
  PIONIER	PIONI
  SINFONI	SINFO
  SOFI	ONTT
  SPHERE	SPHER
  SUSI, SUSI2	ONTT
  VIRCAM	VCAM
  SHOOT XSHOOTER	SHOOT XSHOO
  OmegaCAM	OMEGA

  
  
  The query forms take care of the equivalence between instruments and their different DP ID prefixes and instrument IDs, but the user should take into account that for some instruments it is not possible in all cases to tell, just from the DP ID or file name, which instrument was used to generate a particular FITS file.
• When I use the wild-card on some queries (e.g. Filter: R*), the results take much longer to show up. Why?
  The speed of completion for a given database query may vary     depending on the amount of data to be searched and whether or not     the database server uses the indexes associated to it.  Under     some conditions (specially when using wild-cards), these indexes     are ignored and the server scans the complete database. Since     the ESO databases tend to be large, scan times of up to a minute     can be expected.
• When browsing raw data, should I use the main ESO Archive interface or the instrument specific ones?
  The main ESO Archive interface is intended to aid general queries, especially to astronomers with no previous experience with ESO telescopes and instruments.
  The instrument specific pages will help in finding observations which are either specific to the observing strategy, like stacks in the infrared, or to the detector or instrument setups. These pages should be used when you are searching for specific data (e.g. infrared spectra) for a given object or when you are searching individual observation programs.
• How do I export search results?
  In order to export search results from large datasets, users can follow one of the two options below:

  Export links

  
  There is an export option into three different ASCII data formats, with values separated by tabs (\t), commas, or pipe simbols (|). This can be combined with the "Return a maximum of n rows" setting, in order to retrieve a really large number of rows.
  This can be accomplished by performing the following actions on the ESO Archive main query form:
  • Check the Use tabular output radio button
  • Check the Exports results to file checkbox, selecting additionally whether a Tab-separated-values, Comma-separated-values, or Pipe-separated-values is to be generated.
  • Optionally, set the "Return a maximum of" to the number of expected rows (ie, 20000).
  • When more than 10000 rows are expected, it is advised to uncheck the "Number of Frames / Instrument" and "Cumulative Exptime / Filter" options, as they might cause a timeout precluding the form from completion, and might end up not showing the export link.

After the query is completed (it might take as long as 15 minutes for a 20000 results query), the download link provided will be available for XXX time [please check] for reuse.

Command-line downloads, and VOTables

For users who prefer to download query results using command-line tools, they will have to build a query URL that can be used with downloading tools such as curl or wget.

Building a query URL

Query parameters have to be identified on the ESO Main archive query form (where they are identified as INPUT HTML tags). A selection of them follows:

1. max_rows_returned: controls the maximum numbers of rows to be returned; used by the "Return a maximum of" option.
2. Parameters governing extra fields to be returned:
   • tab_object: target information
   • tab_dp_cat: category information
   • tab_target_coord: actual target coordinates
   • tab_night: extra night information
   • tab_obs_name: actual name of the observing block
   • tab_origfile: adds the orinal file name as provided to the archive system (i.e., as saved in the PI pack)
3. Common query parameters:
   • target: target name; it must be used jointly with resolver, so that the target is converted to coordinates.
   • resolver: one of simbad or ned; indicates which name resolver service will be used to convert the target name into coordinates.
   • ra: right ascension; only if no target name is provided; can be a blank separated sexagesimal value, or a decimal one. See deg_or_hour.
   • dec: declination; only if no target name is provided; can be a blank separated sexagesimal value, or a decimal one.
   • deg_or_hour: can be one of degrees, or hours, and affects only the interpretation of the ra field when given in sexagesimal format.
   • format: affects how coordinates are going to be shown in the results. SexaHour means sexagesimal format, with RA in HMS format; DecimHour means decimal format, with RA in hours, and DecimDeg means decimal format, with both RA and Dec in degrees. Requires tab_target_coord set to true.
4. Format parameters:
   • votable_out_mode: when set to true, returns an on-the-fly generated VOTable. The VOTable data format is a metadata-rich output format, that can be manipulated by GUI tools such as TOPCAT or command-line tools such as http://archive.eso.org/wdb/wdb/eso/eso_archive_main/query?tab_object=true&target=vega&resolver=simbad&tab_dp_cat=true&category=SCIENCE&category=CALIB&max_rows_returned=2000&deg_or_hour=degrees&tab_target_coord=true&format=DecimDeg&votable_out_mode=true

Which if used would cause a browser, or command line tool, to download up to 2000 rows (max_rows_returned=2000) of calibration data (category=CALIB), corresponding to observations around an object known as Vega (target=vega), as resolved by Simbad (resolver=simbad) in VOTable format (votable_out_mod=true), with target coordinates written in decimal degrees (tab_target_coord=true&format=DecimDeg).
In order to download such VOTable, we would have to issue a suitable curl or wget command (see examples below), or used in a web browser, or users' favourite download manager.Curl command

curl -o output.votable.xml   "http://archive.eso.org/wdb/wdb/eso/eso_archive_main/query?tab_object=true&target=vega&resolver=simbad&tab_dp_cat=true&category=SCIENCE&category=CALIB&max_rows_returned=2000&deg_or_hour=degrees&tab_target_coord=true&format=DecimDeg&votable_out_mode=true"Wget command

wget -O output.votable.xml "http://archive.eso.org/wdb/wdb/eso/eso_archive_main/query?tab_object=true&target=vega&resolver=simbad&tab_dp_cat=true&category=SCIENCE&category=CALIB&max_rows_returned=2000&deg_or_hour=degrees&tab_target_coord=true&format=DecimDeg&votable_out_mode=true"
In order to manipulate the VOTable from the command line, a package such as STILTS can be used. A few examples follow using the URL above, but more information can be found on the STILTS package documentation.STILTS commands

The output.votable.xml file can later be transformed into ASCII, CSV, TSV, or even a FITS binary table using the STILTS package, or interactively with the TOPCAT table editor (based on the same libraries as STILTS).

As the STILTS toolset allows for the concept of "remote files", the URL can be used as input parameter:

stilts tcopy ofmt=ascii in="http://archive.eso.org/wdb/wdb/eso/eso_archive_main/query?tab_object=true&target=vega&resolver=simbad&tab_dp_cat=true&category=SCIENCE&category=CALIB&max_rows_returned=2000&deg_or_hour=degrees&tab_target_coord=true&format=DecimDeg&votable_out_mode=true" out=-
 
By using the tpipe option, some additional filtering can be performed:

stilts tpipe ofmt=text in="http://archive.eso.org/wdb/wdb/eso/eso_archive_main/query?tab_object=true&target=vega&resolver=simbad&tab_dp_cat=true&category=SCIENCE&category=CALIB&max_rows_returned=2000&deg_or_hour=degrees&tab_target_coord=true&format=DecimDeg&votable_out_mode=true" out=- cmd='fixcolnames' cmd="select 'equals(data_products_dp_cat,\"SCIENCE\")'" cmd="select 'equals(data_products_dp_type,\"OBJECT\")'"
 
The fixcolnames command substitutes characters problematic for STILTS in column names by underscores (in the case of the ESO archive, the dot between data_products and dp_cat, for instance.