Up to March 1st 2012, the SSC was responsible for systematically processing all XMM-Newton observations to produce data products for the three scientific instruments. Since then, processing responsibilities have been transferred to the XMM-Newton SOC. This processing is done with a pipeline that uses the same SAS software packages that are available for users to interactively analyse XMM-Newton data. However, the pipeline infrastructure used to carry out the processing has been developed by, and has been specific to, the SSC. The SSC was responsible for the pipeline software used until July 2013 when the SOC took over full control, following the transfer of the pipeline used to generate the 3XMM-DR4 catalogue from the SSC to the SOC.
Each dataset processed is subject to a screening procedure by the SSC and SOC teams which aims to verify the quality of the processing and identify any processing problems.
The aim of the pipeline processing is to provide a set of data products which are of immediate value for the XMM-Newton observer as well as for the XMM-Newton Science Archive where they are also stored for eventual public release. The pipeline products, together with the ODF (see § 6.1.1), are distributed to observers by the XMM-Newton SOC via the XMM-Newton Science Archive.
The XMM-Newton data products include calibrated, ``cleaned'' event lists for all X-ray cameras, which are intended to provide the starting point for most interactive analysis of the data. A number of higher-level products are also provided, such as sky images, source lists, cross-correlations with archival catalogues (performed by the Astronomical Catalogues Database Subsystem (ACDS) at the Observatoire Astronomique de Strasbourg), and spectra and time series of sufficiently bright individual sources. These provide a useful overview of the observation for the XMM-Newton observer as well as constituting key archival resources.
Since late 2010 XMM-Newton slew data have also been pipeline processed to create images, event lists and exposure maps along the slew path.
An outline of the pipeline processing system, information about the pipeline
configurations and access to the pipeline software modules, are available at
The data products produced by the pipeline are detailed in documents available on the SOC Manuals web page
Up until early 2013 the SSC had the responsibility to perform bulk reprocessings of all XMM-Newton data from time to time. Bulk reprocessing enables the most up-to-date processing and calibrations to be applied to data and re-imposes uniformity on the archive. The latest bulk reprocessing of all XMM-Newton data took place in December 2012, using a version of the SAS very similar to the public SAS v12.0 and the latest instrument calibrations at that time. This configuration was used for the release of the 3XMM-DR4 catalogue on the 23 July 2013. From this date, both pipeline processing and SAS codes have been updated regularly with no implications on the analyis results. Hence, the XMM-Newton EPIC Serendipity Sources catalogue has been updated from 3XMM-DR4 up to 3XMM-DR8 without the need of a bulk reprocessing.
During 2019 the whole content of the XMM-Newton Science Archive is being reprocessed with a new pipeline processing code, using the most recent versions of the SAS software and calibration files. As a result of this re-processing a new version of the EPIC catalogue (4XMM-DR9) will be published by the last quarter of 2019 when new products will be made available in the XMM-Newton Science Archive .