j2ms2 and tConvert

The normal workflow of processing correlator data at JIVE is to take the correlator output (SFXC, UniBoard) and:

• use j2ms2 to convert to CASA MeasurementSet format
• inspect the data, do flagging, other stuff …
• convert MeasurementSet data to FITS-IDI format using tConvert for archiving and distribution to the PI

These tools are currently undergoing a big overhaul; modernizing the build system, changing the project layout from a CASA/AIPS++ subproject into a standalone one, migrating revision control system from CVS to git and, most importantly, addressing built up changes to the JIVE SFXC correlator capabilities.

Those tools are installed under the /eee/ directory on the EEE machine at JIVE. There exist production and development versions. They are dynamically linked executables and as such the LD_LIBRARY_PATH needs to be set correctly for the applications to find the shared libraries they were linked with.

$> export LD_LIBRARY_PATH=/eee/lib:${LD_LIBRARY_PATH}

then find the executables:

/eee/bin/j2ms2 and /eee/bin/tConvert

Features (“bugs” in this context):

• somewhat broken tConvert; may lose up to a second of data between IDI-chunks for normal experiments, can be a lot worse for mixed-mode experiments where a significant number of stations drop a number of subbands
• leaks metadata from multiple-experiments-per-single-vex-file into individual experiments' MeasurementSet (and consequently in the FITS-IDI files).
• j2ms2: if for some reason a baseline is to be written in reverse order then the spectrum would not be flipped

$> export LD_LIBRARY_PATH=/eee/src/jive-casa/code/jive/lib/x86_64:${LD_LIBRARY_PATH}

after which the binaries can be found here: /eee/src/jive-casa/code/jive/bin/x86_64-64/{j2ms2, tConvert}

Features:

• do not leak meta data; only meta data for antennas, frequencies and sources that have data in the MeasurementSet will be written. The filtering of the antenna table is off by default (ie it will leak all antenna data into all measurement sets) but can be enabled by adding fo:squash_antenna_table on the j2ms2 command line.
• baseline data is now written correctly by j2ms2 even when baselines are written in reverse order in the MS