HEASARC support for unencrypted FTP access will be ending on September 20, 2019. Please see this notice for details.

Search:

[Advanced Search]



Browse Browse Help Help
Latest Version
New/Updated Tables

Using the Batch Interface to the HEASARC Databases

The HEASARC Batch interface allows users to interrogate the HEASARC databases non-interactively. The Batch interface can be used from within a script to query the database and retrieve results. This document discusses how to use this interface.

The HEASARC Database Batch Interface

Topics:

The batch interface allows a user to interrogate the HEASARC databases as if they were local to the users machine. The user downloads a pair of small Perl scripts which act as a local client to query the database. The client is configured to be easily called from programs or from within command scripts -- though it may be used interactively.

Getting Started and Software Installation

Note that only Object Name/Coordinates Searches can be executed using the the batch interface. Also, in order to use the batch facility, you will need access to a Unix workstation a version of Perl greater that 4.0 compiled and installed on it.

There are two versions of the batch program. The older version requires two scripts to be downloaded from the HEASARC anonymous FTP server:

  • browse_extract.pl/webquery.pl    These scripts do not meet the HTTPS-Only Standard and are no longer available. Use browse_extract_wget.pl instead.

browse_extract_wget.pl uses wget instead of webquery to do the data download. Only one script is needed for this version however it is necessary to have wget installed. Many systems have wget pre-installed and you can easily download a version from the web if not.

Once you've downloaded these files, make sure they have executable permissions and place them in your executable path. These scripts assume your system has the Perl command installed in /usr1/local/bin. If Perl has been installed elsewhere on your machine, you should edit the first line of each script to change:
#!/usr1/local/bin/perl5
to the correct location.

Note: These scripts are in the public domain. Please feel free to copy and modify them to use however you wish. However, we can only support the versions of the scripts that we have made available.

Usage

To use the Batch Interface, simply use browse_extract_wget.pl command at the Unix shell prompt. Many options are available, but you only need to specify the table to be searched and the astronomical position(s) of interest.

The syntax of the command is:

browse_extract_wget.pl table=table name

optional arguments:
[position=name_or_position]
[coordinates=csys]
[equinox=year]
[radius=arcmin]
[fields=STANDARD/ALL/list]
[name_resolver=NED/SIMBAD]
[infile=input_list]
[outfile=output_file]
[format=batch|FITS|VOTable|Excel|HTML]
[sortvar=column_name]
[param=name,value /or/ name=value]...

or

browse_extract_wget.pl table=xxx
to just get a list of available tables. Only VizieR tables directly linked within the HEASARC will be noted, but any VizieR table can be queried using browse_extract_wget.

All arguments are case insensitive except Vizier table parameters.

Explanation of command line arguments:

table
This is the abbreviated or short table name as used in the HEASARC, e.g., ABELL, XTEOBS. This is given as the short name in the Browse Web interfaces.
position
is either the name of an object a set of coordinates around which the search is to take place. If a name is given it will be resolved using the service given in the name_resolver keyword or SIMBAD by default. The syntax for coordinates is that supported in the Browse Web services. If the coordinates string contains embedded spaces, then this argument should be enclosed in quotes. Multiple positions may be separated by semicolons but these will then be processed together giving a combined count for all specified targes.

The Position argument can be specified as "none" or "null" if the user does not wish to specify and positions.

coordinates
This argument should be either "Equatorial" or "Galactic". The default is Equatorial. It is used to determine the input coordinates if used and the display coordinate system for the primary coordinates in the table.
equinox
is the equinox year for input and output coordinates. It defaults to 2000 (and is ignored for Galactic coordinates).
radius
Radius gives the radius in arcminutes out to which the search is to take place. This defaults to 60.

Note that this is different from the interactive Browse system where the default differs from table to table.

fields
This argument indicates which parameters are to be retrieved from the table. The default, "Standard", indicates that only a limited set of parameters will be retrieved. "All" will retrieve all parameters from the table. A list of specific fields separated by commas may also be specified.
name_resolver
may be used to select the system used to convert names into coordinates. The currently supported services are NED and SIMBAD. The default is SIMBAD.
infile
specifies a file containing objects to be searched. Each line in the file will be used as the Position. If no Infile or Position argument is give then the positions will be read from the standard input.
outfile
specifies a file to contain the table of returned results. If not specified the results will be printed on standard output
format
specifies the desired output format. When anything other than the batch format is requested, all positions will be searched at once. In batch queries each line of the input is specified as a distinct query. Current valid formats include:
Batch - The default format.
HTML - The Text Table format in the Web version of Browse
FITS - A FITS ASCII table (The results are in the first extension).
EXCEL - An Excel compatible output format.
VOTable - The Virtual Observatory Table format.
Text - The Pure Text format in the Web version of Browse
sortvar
may be used to specify the field on which the results will be sorted. This variable need not be displayed.
param
param=name,value argument is used to do parameter searches. The syntax of the value parameter is the same as used in the Browse Web interface, e.g., 3000, >5000, 4..10, 3C*273 are possible values which search for data with a value equal to 3000, greater than 5000, between 4 and 10 or matching the strings '3C273', '3CXXXXX273', etc., respectively. If the name of the parameter does not conflict with one of the other arguments to browse_extract_wget, then the simpler syntax
name=value
may be used.

In some environments characters in the value may need to be escaped, e.g., exposure='>2000' or exposure=\>2000

Users may specify the target positions using the position argument, using a predefined file specified with infile, or from the standard input. In the latter two cases each line until an EOF will be used as a position.

Examples

  • Are there any public ROSAT observations of 3C273?

    % browse_extract_wget.pl table=rosmaster position=3c273 name_resolver=ned

    should print to standard output a table like the following: 

    
seq_id     |instrument|exposure|ra(2000)  |dec(2000)  |name              |public_date(ISO)|
+-----------+----------+--------+----------+--------+--------------------+-------+
|RH701971N00|HRI       |    1720|12 29 06.5|02 03 07|3C273               | 0.0622|
|RH701952N00|HRI       |    1189|12 29 07.2|02 03 00|3C273               | 0.1902|
|RH701953N00|HRI       |    1533|12 29 07.2|02 03 00|3C273               | 0.1902|
|RH701954N00|HRI       |    1373|12 29 07.2|02 03 00|3C273               | 0.1902|
|RH701955N00|HRI       |    1586|12 29 07.2|02 03 00|3C273               | 0.1902|
|RH701956N00|HRI       |    1704|12 29 07.2|02 03 00|3C273               | 0.1902|
|RH701957N00|HRI       |    1831|12 29 07.2|02 03 00|3C273               | 0.1902|
|RH701958N00|HRI       |    1659|12 29 07.2|02 03 00|3C273               | 0.1902|
...
Search of table rosmaster around 3c273 with a radius 60' returns 63 rows

  • An example using an input and output file

    I might first do a query of all WGACAT sources within 80' of the galactic center using:

    % browse_extract_wget.pl table=wgacat radius=80 coordinates=galactic position='0.,0.' outfile=wgacat_gc.list
    The results of that query can be edited (manually or by a simple script) to produce at file like: 
    359.386118, 1.149945
359.510470, 1.223261
359.274779, 0.933392
359.279818, 0.934399
359.383583, 0.977861
359.389096, 0.979161
359.392070, 0.972419
359.292038, 0.907242
359.389873, 0.967603
359.390891, 0.967811
359.393223, 0.969269
...
    plus 340 more lines.

    If this result is stored as wgacat_galcen.dat, we can find nearby HST Guide Star Catalog positions with: 

    browse_extract_wget.pl table=gsc coordinates=galactic infile=wgacat_galcen.dat outfile=wgacat_galcen_guidestars.dat
    It will take a while to process 350 targets...

  • A parameter query example: 
    % browse_extract_wget.pl table=rassfsc hardness_ratio_1-hardness_ratio_2='>1' position=null
  • A Vizier table example: 
    % browse_extract_wget.pl table=I/284 position="0.0,0.0" coordinates=equatorial equinox=2000.0 radius=12.0 B1mag=\>19.9
    Note that Vizier table parameters are case sensitive. Check table information pages for correct parameter spelling.
If you have questions concerning the installation or usage of these scripts please contact the HEASARC.

HEASARC Acknowledgment

Browse is provided by the Astrophysics Science Division at NASA/Goddard Space Flight Center. If using this service made a significant contribution to a research project, please make the following acknowledgment in any resulting publication:

"This research has made use of data obtained through the High Energy Astrophysics Science Archive Research Center Online Service, provided by the NASA/Goddard Space Flight Center."

Please send a preprint or reprint of the paper to:

	The HEASARC
	Code 660.2
	NASA/Goddard Space Flight Center
	Greenbelt, Maryland, 20771, USA
Page Author: Browse Software Development Team
Last Modified: Wednesday, 10-Jan-2018 09:27:06 EST