|
Using the Batch Interface to the HEASARC Databases
The HEASARC Batch interface allows users to interrogate
the HEASARC databases non-interactively. The Batch interface can
be used from within a script to query the database and retrieve results.
This document discusses how to use this interface.
The HEASARC Database Batch Interface
Topics:
The batch interface allows a user to interrogate the HEASARC databases as if
they were local to the users machine. The user downloads a pair of small
Perl scripts which act as a local client to query the database. The client
is configured to be easily called from programs or from within command
scripts -- though it may be used interactively.
Note that only Object Name/Coordinates Searches can be
executed using the the batch interface. Also, in order to use the batch
facility, you will need access to a Unix workstation a version of Perl
greater that 4.0 compiled and installed on it.
There are two versions of the batch program. The older version requires two scripts to be downloaded from the HEASARC anonymous FTP server:
- browse_extract.pl updated September 20 , 2010
- webquery.pl (Perl 5.x and higher) OR webquery.pl (Perl 4.x)
The newer version of the batch script was created in response to reports of problems running the webquery scripts from behind firewalls and proxies. This version uses wget instead of webquery to do the data download. Only one script is needed for this version however it is necessary to have wget installed. Many systems have wget pre-installed and you can easily download a version from the web if not. We recommend wget version 1.18 or later. (Some earlier versions have caused problems with Browse downloads.) After making sure wget is accessible on your system just substitute browse_extract_wget.pl for browse_extract.pl. The command syntax is the same.
- browse_extract_wget.pl updated September 20, 2010
Once you've downloaded these files,
make sure they have executable permissions and place them in your executable
path. These scripts assume your system has the Perl command installed in
/usr1/local/bin. If Perl has been installed elsewhere on your
machine, you should edit the first line of each script to change:
#!/usr1/local/bin/perl5
to the correct location.
Note: These scripts are in the public domain. Please feel free to copy
and modify them to use however you wish. However, we can only support the
versions of the scripts that we have made available.
To use the Batch Interface, simply use browse_extract.pl command
at the
Unix shell prompt. Many options are available, but you only
need to specify the table to be searched and the astronomical position(s) of
interest.
The syntax of the command is:
browse_extract.pl table=table name
optional arguments:
[position=name_or_position]
[coordinates=csys]
[equinox=year]
[radius=arcmin]
[fields=STANDARD/ALL/list]
[name_resolver=NED/SIMBAD]
[infile=input_list]
[outfile=output_file]
[format=batch|FITS|VOTable|Excel|HTML]
[sortvar=column_name]
[param=name,value /or/ name=value]...
or
browse_extract.pl table=xxx
to just get a list of available tables. Only VizieR tables
directly linked within the HEASARC will be noted, but any
VizieR table can be queried using browse_extract.
All arguments are case insensitive except Vizier table parameters.
Explanation of command line arguments:
- table
- This is the abbreviated or short table name as used in the HEASARC,
e.g., ABELL, XTEOBS, ROSPUBLIC. This is given as the short name in
the Browse Web interfaces.
- position
- is either the name of an object a set of coordinates around
which the search is to take place. If a name is given it will
be resolved using the service given in the name_resolver keyword
or SIMBAD by default. The syntax for coordinates is that supported
in the Browse Web services. If the coordinates string contains
embedded spaces, then this argument should be enclosed in quotes.
Multiple positions may be separated by semicolons but these will then be
processed together giving a combined count for all specified targes.
The Position argument can be specified as "none" or "null" if
the user does not wish to specify and positions.
- coordinates
- This argument should be either "Equatorial" or "Galactic".
The default is Equatorial. It is used to determine the input coordinates
if used and the display coordinate system for the primary coordinates
in the table.
- equinox
- is the equinox year for input and output coordinates. It
defaults to 2000 (and is ignored for Galactic coordinates).
- radius
- Radius gives the radius in arcminutes out to which the search
is to take place. This defaults to 60.
Note that this is different from the interactive Browse system where
the default differs from table to table.
- fields
- This argument indicates which parameters are to be retrieved
from the table. The default, "Standard", indicates that only
a limited set of parameters will be retrieved. "All" will retrieve
all parameters from the table. A list of specific fields separated
by commas may also be specified.
- name_resolver
- may be used to select the system used to convert names into
coordinates. The currently supported services are NED and SIMBAD.
The default is SIMBAD.
- infile
- specifies a file containing objects to be
searched. Each line in the file will be used as the
Position. If no Infile or Position argument is give then
the positions will be read from the standard input.
- outfile
- specifies a file to contain the table of returned results.
If not specified the results will be printed on standard output
- format
- specifies the desired output format. When anything other than
the batch format is requested, all positions will be searched at
once. In batch queries each line of the input is specified as
a distinct query. Current valid formats include:
Batch - The default format.
HTML - The Text Table format in the Web version of Browse
FITS - A FITS ASCII table (The results are in the first extension).
EXCEL - An Excel compatible output format.
VOTable - The Virtual Observatory Table format.
Text - The Pure Text format in the Web version of Browse
- sortvar
- may be used to specify the field on which the
results will be sorted. This variable need not be displayed.
- param
- param=name,value argument is used to do parameter searches.
The syntax of the value parameter is the same as used in the Browse
Web interface, e.g., 3000, >5000, 4..10, 3C*273 are possible values
which search for data with a value equal to 3000, greater than 5000,
between 4 and 10 or matching the strings '3C273', '3CXXXXX273',
etc., respectively. If the
name of the parameter does not conflict with one of the other arguments
to browse_extract, then the simpler syntax
name=value
may be used.
In some environments characters in the value may need to be escaped, e.g.,
exposure='>2000' or exposure=\>2000
Users may specify the target positions using the position argument, using a
predefined file specified with infile, or from the standard input. In the
latter two cases each line until an EOF will be used as a position.
- Are there any ROSAT observations of 3C273?
% browse_extract.pl table=rosmaster position=3c273 name_resolver=ned
should print to standard output a table like the following:
seq_id |instrument|exposure|ra(2000) |dec(2000) |name |public_date(ISO)|
RP600242 |PSPC | 3078|12 27 43.2|+01 36 00.0|GIOVANELLI-HAYNES | 1994-03-22|
RP600242A01|PSPC | 24830|12 27 43.2|+01 36 00.0|GIOVANELLI-HAYNES | 1994-03-22|
RH120001 |HRI | 0|12 29 04.8|+02 03 00.0|XRT/HRI NORTH DUMM| 1995-08-01|
WP141509N00|PSPC | 3332|12 29 04.8|+02 03 00.0|3C273 | 1994-09-28|
RP120000N00|PSPC | 916|12 29 04.8|+02 03 00.0|XRT/PSPC NORTH DUM| 1995-01-31|
WF700191 |PSPC | 3291|12 29 04.8|+02 03 00.0|3C273 | 1996-02-07|
WP700191 |PSPC | 6243|12 29 04.8|+02 03 00.0|3C273 | 1996-02-07|
RP141520N00|PSPC | 485|12 29 04.8|+02 03 00.0|3C273 | 1995-09-27|
WH700234 |HRI | 17174|12 29 07.2|+02 03 00.0|3C 273 | 1993-07-20|
...
Search of table ROSPUBLIC around '3c273' with a radius 60' returns 25 rows
- An example using an input and output file
I might first do a query of all WGACAT sources within 80' of the galactic
center using:
% browse_extract.pl table=wgacat radius=80 coordinates=galactic position='0.,0.' outfile=wgacat_gc.list
The results of that query can be edited (manually or by a simple
script) to produce at file like:
359.386118, 1.149945
359.510470, 1.223261
359.274779, 0.933392
359.279818, 0.934399
359.383583, 0.977861
359.389096, 0.979161
359.392070, 0.972419
359.292038, 0.907242
359.389873, 0.967603
359.390891, 0.967811
359.393223, 0.969269
...
plus 340 more lines.
If this result is stored as wgacat_galcen.dat, we can find nearby HST Guide
Star Catalog positions with:
browse_extract.pl table=gsc coordinates=galactic infile=wgacat_galcen.dat outfile=wgacat_galcen_guidestars.dat
It will take a while to process 350 targets...
-
A parameter query example:
% browse_extract.pl table=rassfsc hardness_ratio_1-hardness_ratio_2='>1' position=null
-
A Vizier table example:
% browse_extract.pl table=I/284 position="0.0,0.0" coordinates=equatorial equinox=2000.0 radius=12.0 B1mag=\>19.9
Note that Vizier table parameters are case sensitive. Check table information pages for correct parameter spelling.
If you have questions concerning the installation or usage of
these scripts please contact the HEASARC.
Browse is provided by the Laboratory for High Energy
Astrophysics at NASA/Goddard
Space Flight Center. If using this service made a significant
contribution to a research project, please make the following acknowledgment
in any resulting publication:
"This research has made use of data obtained through the High Energy
Astrophysics Science Archive Research Center Online Service, provided
by the NASA/Goddard Space Flight Center."
Please send a preprint or reprint of the paper to:
The HEASARC
Code 660.2
NASA/Goddard Space Flight Center
Greenbelt, Maryland, 20771, USA
|
