j2ms2 and tConvert
The normal workflow of processing correlator data at JIVE is to take the correlator output (SFXC, UniBoard) and:
- use
j2ms2
to convert to CASA MeasurementSet format - inspect the data, do flagging, other stuff …
- convert MeasurementSet data to FITS-IDI format using
tConvert
for archiving and distribution to the PI
These tools are currently undergoing a big overhaul; modernizing the build system, changing the project layout from a CASA/AIPS++ subproject into a standalone one, migrating revision control system from CVS to git and, most importantly, addressing built up changes to the JIVE SFXC correlator capabilities.
Those tools are installed under the /eee/
directory on the EEE machine at JIVE.
There exist production and development versions. They are dynamically linked executables and as such the LD_LIBRARY_PATH
needs to be set correctly for the applications to find the shared libraries they were linked with.
Production
$> export LD_LIBRARY_PATH=/eee/lib:${LD_LIBRARY_PATH}
then find the executables:
/eee/bin/j2ms2
and /eee/bin/tConvert
Features (“bugs” in this context):
- somewhat broken
tConvert
; may lose up to a second of data between IDI-chunks for normal experiments, can be a lot worse for mixed-mode experiments where a significant number of stations drop a number of subbands - leaks metadata from multiple-experiments-per-single-vex-file into individual experiments' MeasurementSet (and consequently in the FITS-IDI files).
j2ms2
: if for some reason a baseline is to be written in reverse order then the spectrum would not be flipped
Development
$> export LD_LIBRARY_PATH=/eee/src/jive-casa/code/jive/lib/x86_64:${LD_LIBRARY_PATH}
after which the binaries can be found here:
/eee/src/jive-casa/code/jive/bin/x86_64-64/{j2ms2, tConvert}
Features:
- do not leak meta data; only meta data for antennas, frequencies and sources that have data in the MeasurementSet will be written. The filtering of the antenna table is off by default (ie it will leak all antenna data into all measurement sets) but can be enabled by adding
fo:squash_antenna_table
on thej2ms2
command line. - baseline data is now written correctly by
j2ms2
even when baselines are written in reverse order in the MS