Welcome to the jplotter online manual

On this page you'll find a quickstartguide and some indepth info with respect to the use of the jplotter. The jplotter is a collection of glish-scripts that enables visual inspection of data in an aips++ MeasurementSet. The jplotter is the successor of jivegui. The jivegui is meant to be used with MeasurementSet version 1, wheres jplotter was designed to be MeasurementSet version 2 based. The two don't mingle, since the underlying datastructure (i.e. the MeasurementSet) has changed considerably, going from version 1 to 2.

Before we get to the goodies, a word or two about font-styles used in this document

Style	Meaning
this	indicates a directory
this	indicates an application
this	indicates an application plus a link to further documentation about the application
this	indicates a command, as it could/should (depends on the context...) be typed at a command prompt
this	indicates a link to a command description
this	indicates a hint or useful, operational information

Table of contents

1. Getting started
2. Inside the Command Line Interface
3. The commands
   Description of the command syntax
   
   • System commands (independant of data)
   • Selection commands
   • Inspection commands
   • Plot commands

Getting started

Before we start, check the following:

• Currently, the libs/binaries are only available for SPARC/Solaris based systems (there is also Solaris for the Intel-x86 architecture...). This means that either you rlogin to a SPARC/Solaris based machine or you'll have to have one on your desktop.
• On that machine, the following directories should be available:
  
  1. /aips++, on dus0
  2. /aips++4, on dus0
  If they are not available, contact sysadmin to mount them for you.
• Make sure you have the following directory in your PATH environment: /aips++/verkout/release/aips++/sun4sol_egcs/bin; all the binaries are located here
• Make sure you have the following path in your LD_LIBRARY_PATH: /local/gcc-2.95.3/lib; the C++ code must be able to find its libraries.

Having checked this, we can proceed to starting either aips++ or glish. Starting glish is faster and usually sufficient for our needs.

Note: if you just start glish, you presumably will get all the log-messages in your terminal window. In order to get around that annoyance, start glish as
glish -l logger.g

Now it is time to include the glish-script containing the code for the jplotter:
at the glish prompt, type
include 'jplotter.g'
Now you will see a lot of messages on your terminal; it is the system telling you which version of which file it is loading. After some time the glish-prompt returns. Your screen will look something like this.

As the system mentions, you can enter the command-line interface by typing
jcli()

Inside the Command Line Interface

Envisaged modus operandi is something like this. The idea is to

• first open a dataset
• optionally list some of its properties/auxiliary information
• be able to select a subset of the dataset, if so desired
• different plots with different properties can be generated

All steps but the first may taken in random order. Obviously, selecting/plotting makes no sense before a dataset is opened. A dataset may be opened at anytime. Once a dataset is open you may select subsets out of the data. Once you're satisfied with the selection you can think of what propertie(s) of the selection you'd wish to be plotted. That done, you can set the system off into plotting. It will produce plots for all unique combinations in the subselection. The plots may be arranged in the way you'd like them to; as an end user you have control of how multiple datasets are grouped into separate plots.

Suppose you have made a subselection with three baselines and two polarization combinations, you have the choice of overplotting equal polarizations for all baselines in one plot, resulting in two plots, one for every polarization combination. On the other hand, you might want to compare the polarizations for every baseline by itself resulting in three plots, one for every baseline.

In order to support this, the command-line interface contains a number of commands, which can be put in a couple of categories.

• System commands (independant of data)
• Selection commands
• Inspection commands
• Plot commands

The commandlineinterface should feature commandline history/editing. However, sometimes this feature seems to be broken within glish. In principle, the history is so persistent that if you exit the commandline interface (but do not exit glish) and later on re-enter the command-line interface again, the commands from the previous run should still be available to you.

Description of the command syntax

As you will see in the command descriptions, use will be made of colors/styles to provide visual information as to which arguments are optional and which are required (note: it is possible for a required argument to appear in an optional argument...).

Normal command text (i.e. not the argument) will be displayed like
command

Optional arguments will be indicated as follows
command [<arg1> <arg2>...]

Required arguments will be indicated like this
command <arg1> <arg2>

 
 

The system commands

The system commands are a set of commands that do not act upon data and usually have no effect on the current state of the program, unless stated otherwise. The commands in this category are

Command	Description
list	List all commands
help	Get help on commands
exit	(...)
macro	Define/inspect macros
del	Delete macro

list

• 0 arguments: the system lists all currently defined macros and their definition
• 1 argument: the system will try to list the definition of macro <macroname>, if any
• 2 arguments: the system will define the macro <macroname> as <definition>.
  

  Note: when creating macros involving multiple commands (i.e. separated by semicolons), enclose the macrodefinition in single quotes to prevent immediate interpretation/execution of the statements. E.g. note the difference between;

     macro multiple ch mid;t start>end;
  	

  and

     macro multiple 'ch mid;t start>end;'
  	

  the first will not work (as intended!) whereas the second will

Macros can be persistent. If you wish some macros to be loaded every time you start the command-line interface the only thing you need to do is create a variable of a certain type in your ~/.glishrc. Every time a command-line interface object starts, it tests if a global variable named command is present, and if it is of type record. If these conditions are satisfied, the system checks if the record command contains a field named macros and if it also is of type record. If these conditions also hold, the system loops over all items in the variable command.macros and checks if they contain the fields name AND value. If the entry does, then a corresponding macro will be defined.

Example (excerpt from ~/.glishrc):

<..other glish stuff..>
command.macros := [
	# select midchannel+whole experiment
	een=[name='multiple', value='ch mid; t start>end;'],
	# select 80% of the band
	twee=[name='80pct',    value='ch 0.1*last:0.9*last;']
];

This piece of glish will make available two macros, multiple and 80pct.

Persistent macros can be very useful, e.g. for defining shorthands for your datadirectories! Eg:

command.macros := [
	#dataarea 0: disk on juw22
	een=[name='d0', value='/juw22_1/verkout/Experiments'],
	#dataarea 1: disk on juw16
	twee=[name='d1', value='/juw16_2/data',
	# C00C3 specific dir
	drie=[name='c00c3', value='/juw12_1/jops/data/C00C3']
];

I can now use commands like

	ms d0/N01X2/N01X2.ms

or

	ms c00c3/pass4lag.ms

to open a MeasurementSet.

Selection commands

The selection commands are a set of commands that enable the user to widen/narrow a subset of the currently opened dataset. Selection can be done arbitrarily along any of the interferometric axes (e.g. polarization, frequencychannel, subband, freqgroup, baseline, source, time).

At this point I think I should elaborate a little more on the MeasurementSet and how certain things work within jplotter. With the arrival of MS version 2, the system supports multiple frequency setups (freqgroups). Each of these freqgroups may consist of one or more subbands. These subbands may contain a number of frequency channels and a number of polarization combinations that have been calculated by the correlator.

The problem now is, that freqgroup/subband labeling in the MS is not predefined. The link between the frequency label you will find attached to the data in the main table, does not (directly) tell you anyting about the actual frequency; it is a pure logical index. Therefore, jplotter cannot make any assumptions based on this index. The boys from the aips++ project let you deal with this index; if you want to select a specific frequencyband, you'll have to work out yourself which index corresponds to what you really want.

Another complicating factor is the fact that the same spectral window (=subband) may appear in the main table with different labels, namely if the same frequency band was correlated with different polarizations or with a different number of channels.... each of these combinations yields a different label that is attached to the data in the main table, making it impossible to deduce directly from the label in the main table what data you're dealing with.

I decided to work around that and do things a little differently. jplotter inspects all the frequency information and does the following:

• groups together all subbands for a freqgroup
• within each freqgroup, the subbands are re-ordered to be in ASCENDING frequency

As a result, every freqgroup is identifieable by a unique number and within a freqgroup, the subbands are numbered 0..<numsubband-1>. This provides a generic mechanism for the end user who now only has to deal with freqgroups and subbands. Hence, frequency selection in jplotter is always done based on the combination of freqgroupid and subbandnr.

The selection commands (in no particular order) are

Command	Description
ms	Open a dataset
ch	Select frequency channels
bl	Select baselines
time	Select time(ranges)
src	Select sources
fq	Select freqgroup/subbands/polarizations

ms

<first>:<last>[:<increment>]

For your pleasure, the following symbols have been defined [first|mid|last|all] which will evaluate to the corresponding channel(s) in the current MS.

A useful feature in this command is that arithmetic is supported. You can use numerical expressions for the first,last,increment or individual channel. However, the fun really starts when you use the previously mentioned aliases, which you can also use in arithmetic expressions: e.g. 0.1*last:0.9*last would select the inner 80% of the channels... independant of how many channels actually are present in your current dataset.

bl

By executing this command (with argument(s)) you will overwrite the previous selection, lest the following exception occurs: if your selection comes out as being invalid (i.e. the specified baseline(s) are not in the dataset), the previous selection is left untouched.

Now for the format of <blcode>. Wildcards are allowed (* or ?), as well as RegularExpression-type formats! The baseline selection is always evaluated from left-to-right (this bears importance on the negated baselinecodes), adding (or removing!) baselines to the selection as the next argument is parsed. If a baselinecode is prefixed with an exclamationmark, the baselines matching the code will be removed from the selection so far. Of course the selection mechanism is case-insensitive.

It is advisory to put braces around one of the baselinecodes so the system knows where the separation between the individual codes is (it is dangerous to assume exact two letters per code). Also, it enables the system to be independant of in which order the baseline is in the dataset. As a matter of fact, the system always automatically searches for the baseline with the reversed code! This means you do not have to pay attention to the order in which you specify the baselincode; if you want EfJb, you'll get it!

For your pleasure, the following special names have been defined: [auto|cross|all] which will select the appropriate baselines out of the current MS...

Suppose we have the following set of baselines

EfJb EfNt WbJb JbJb WbWb NtWb NtCm NtJb EfOn

Selection command	Resulting selection	Comment
nt(wb) nt(ef)	NtWb EfNt	Select individual baselines
nt*	NtEf NtWb NtCm NtJb	All baselines to Nt
nt* !cm	NtEf NtWb NtJb	All baselines to Nt, except NtCm
jb(ef|wb)	EfJb WbJb	RegularExpression like
auto	JbJb WbWb	All autobaselines
cross !nt	EfJb WbJb EfOn	All crossbaselines, except those to Nt

If an argument of the first form is given, it indicates a time range, ranging from the first time (before the >) to the second time. If only a single time is give, the system selects the nearest integration within half an integration time of the indicated time, if such a time can be found.

The format for the time arguments is quite elaborate. In the following list, the letters hh, mm, ss.sss depict decimal numbers representing the hour, minute and second specification, DD, MM YY(YY) indicate day, month and year in decimal notation, MM can also be the name of the month. Other characters are literal unless specified otherwise.

Supported formats include (but are not necessarily limited to):

• hh:mm:ss.sss or hhHmmMss.sss. These formats indicate times on day 0 of the experiment.
• In order to be able to adress times on other days of the experiment, you can prefix the previous formats with the following: <reldaynr>/ to indicate a time on day <reldaynr> of the experiment. Note: <reldaynr> is zero-based!
• A full timespecification can be given, if desired. Prefix the first format with a date: DD-MM-YYY/[time], or YYYY/MM/DD/[time]. You cannot mix this format with the previous, relative daynr, format.

For your pleasure, the special time values [start|end] are defined which will evaluate to the experiment's start- and endtime

A property of the system to take care of/make use of is the following:

If a timerange is selected AND the second time is less than the first, the system will automagically place the second time on the day following the first time!

Suppose you entered the following timerange

	2/23:50:00>00:15:00

this will be changed by the system to

	2/23:50:00>3/00:15:00

A lot of the comments about the syntax that hold for the bl command, also hold for this command. <source> may be a full sourcename but it may as well contain wildcards, regular expression like matching and negating. Indeed, as with the baselineselection, this command is also case-insensitive. This command too, evaluates the source-selection-arguments left-to-right, adding or removing sources as each argument is parsed.

Some examples

Suppose you have the following sources in your experiment

DA193 DA195 J1245+11 J1245+13 J1245+15 NGC132

Selection string	Selected sources	Comment
da193 da195	DA193 DA195	Single source selection
da*	DA193 DA195	Wildcard
j1245+(11|13)	J1245+11 J1245+13	Regular expression like
* !J*	DA193 DA195 NGC132	All sources, except those starting with J
* !J* j*15	DA193 DA195 NGC132 J1245+15	All sources, except those starting with J, plus J1245+15

As discussed in the intro of this commandsection, jplotter groups and sorts the frequency info before presenting it to the user. You can use the lf command to see which freqgroups and subbands jplotter has found. Use this command to select your favourite subselection out of this.

As said before, the syntax of this command is quite complex. It has the advantage that there's almost no limit to what you want to select, subject to some minor constraints.

Lets start by looking at the full selection command, all other cases are subsets of this. A fully defined selection looks like <freqsel>/<subbandsel>/<polsel>, where

<freqsel>	The freqgroupselection
<subbandsel>	The subband selection
<polsel>	The polarization selection

In these definitions, <freqsel> and <subbandsel> are numerical ranges. Range in this context also includes single integers as well a set of discrete values. Technically speaking, the definition of a range is something like this (for all of you high on RE's :)... further on there will be examples wich will hopefully say more than a dozen RE's.

(i|(i:j))([,(i|(i:j))]*)

([xylr]\*)|(\*[xylr])|([xylr]{2})

Ok. Working from this, we can start interpreting the full command. The system attempts to select all the indicated subbands from all the given freqgroups. I deliberately left out the polarization since that one is treated a little different. If not all indicated subbands appear in all the given freqgroups the selection is not executed (it is 'invalid').

Do not despair since if you want to select different subbands from different freqgroups you can just pass multiple arguments to the fq command!

How the polarization selection is treated. As was mentioned before (in the intro), the same freqgroup/subband combination may be present more than once, sometimes even unbeknownst to the user. If you request a polarization selection from the system, the system will select data from the first combination of freqgroup/subband plus polarizationid that contains all of the requested polarizations. If no polarization combination is given, the system selects all polarizations found in the first polarizationsetup for that freqgroup.

If you have multiple arguments to the fq it is legal (according to the syntax) to select different polarizations in every argument. At the moment, jplotter can not deal with that. It is as yet unclear if it ever will. However, the system detects that you're trying to do such a thing, warns you and will stick to the polarization selection resulting from parsing the first argument!

DEFAULTS As you can see from the command-syntax, if you pass an argument to this command, only the subband(s) are a required parameter. This is not completely true since you may only omit the freqid(s) if and only if there is only one freqgroup present in the experiment. You can find this out using the lf command. As for the polarization(s), they may be defaulted. The selection defaults to all polarizationcombinations found in the first polarizationsetup for the indicated freqgroup.

I think it is time for some examples

Selection	Result
1/3	Will select all polarizations for subband 3 (0-based) from freqgroup 1
1/0:3	Will do almost the same as the previous selection, only now selecting the first 4 subbands
1/1,3	same, will only select subbands 1 and 3
1/1,3:5,8	... well.. you can figure this one out
1/1,3:5,8/r*	nearly same as previous, now only polarizations RR and RL are selected
1,3/0/rr	will select polarization RR for subband 0 from freqgroups 1 and 3
1:3/0,3/lr,rl	selects polarization LR and RL from subband 0 and 3 from freqgroups 1 through 3
1/0,2/rr 3/1:3/ll	Syntactically correct, however, the system will warn you and just selects the RR polarization from subbands 0 and 2 of freqgroup 1 AND from subbands 1 through 3 of freqgroup 3

Inspection commands

The inspection commands let you list/inspect properties of the dataset or the plot. They can be used to find out which sources, antennas, baselines and frequencygroups/subbands were found in the MS you opened. Also, current settings or the current selection can be viewed. Very little more can be said about these commands.

The commands in this section are

Command	Description
r	List all 'selectables' from the current MS
sl	Inspect the current selection
pp	Inspect the plot properties
la	List the antennas

r

1. All polarization setups found
2. All freqgroups plus subbands found therein (after reordering, see the intro)
3. All baselines in the dataset
   note: the system will list all possible baselines. The actual baselines present may be a function of time and hence cannot be determined from inspecting the antennatable alone
4. All sources found in the dataset
5. The start/end time found in the experiment

1. Your selected freqgroups/subbands/polarizations
2. Your selected channels
3. Your selected baselines
4. Your selected sources
5. Your selected timerange(s)/integration(s)

The system lists the following properties

1. plotType: indication which quantity will be plotted against which quantity. To get a list of available plottypes you can use the lp command. Use the pt command to set the current plottype
2. averageTime: whether or not you wish to average the selected data in time. If averaging is to be done it can be done either scalar or vector. Use the avt command to set/unset it
3. averageChannel: whether or not you wish to average the selected channels in your dataset. Channels can also be scalar- or vectoraveraged. Use the avc command to set/unset it
4. weightThreshold: which weightthreshold to apply. Use the wt command to set the threshold of your choice.
   Note: if data is to be averaged and not all datapoints have a weight>=threshold, the whole resulting datapoint is discarded!

Plot commands

The plot commands are implemented to control what you want to plot, if you desire any pre-processing (averaging) and how you would like the plots organized.

Let me first explain a bit about the philosphy behind jplotter. The idea is that once you've selected some (or all) data and have decided what you want to plot, the system can be set of. What happens is that the system 'iterates' over all unique combinations found in your selection. For each of the unique combinations one or more datasets are generated. E.g. of you want to plot amplitude and phase, two datasets per unique combination are generated, one containing the amplitudes, the other, surprisingly enough, the phases...:).

Having gathered this data, jplotter offers you the possibility of ordering these datasets into plots. Each plot may contain a number of datasets. As a user, you have control over how the datasets are merged into plots.

You could decide that you wish that for every unique baseline a plot be generated. This has the effect that all datasets for that baseline will be overplotted in one plot. If you selected 2 polarizations and subbands, there will be 6 datasets in such a plot. There will be <numbaselines in your selection> plots, under the assumption that data is available and has a weight>=threshold.

On the other hand you might decide that you would like to compare the polarizations for each subband for all baselines. In this case you could tell the system to order the datasets differently: put all datasets with equal polarization and subband labels into a plot. Each unique combination of polarization/subband label will now generate a plot. So there will be 6 plots (2 polarizations x 3 subbands), each plot containing <numbaselines in your selection> datasets.

For nearly all the commands in this section the following holds: changes won't be reflected in the plots until the plots have been re-created.

The commands in this section are

Command	Description
lp	List the available plottypes
pt	Control the current plottype
pp	Display the current plotproperties
pl	Create the plots
avt	Control timeaveraging
avc	Control channelaveraging
wt	Control weightthreshold
new	Control how datasets are merged into plots
(nxt|prev)	Navigate through pages with plots
n[x|y]	Control number of plots per page

lp

Note: timeaveraging means averaging over all integrations in your selection, even if they belong to different scans.

avc

There is a slight difference how the system deals with the weightthreshold when dealing with averaging

• channelaveraging: if one or more channels have a weight<weightthreshold, the whole resulting datapoint is dropped
• timeaveraging: if a datapoint has a weight<threshold, that datapoint is skipped.

<type> is one of

• p for polarization: a new plot is started for every polarization found
• ch for channel: a new plot is started for every channel found.
  Take care of this one, especially if you've got a lot of channels in your selection and you did not average them...
• sb for subband: start a new plot for every subband found
• fq for freqgroup: start a new plot for every freqgroup found
• bl for baseline: start a new plot for every baseline found
• src for source: start a new plot for every source
• time for time: start a new plot for every integration.
  Same as with the channels stuff: if you select the whole experiment and forget to timeaverage and this is set to true, you might end up with a lot of plots...

Some examples:
Suppose you have selected 3 baselines, 4 subbands and 2 polarizations. For the moment I won't discuss src/fq/time/ch since the principle is the same for all of them. The command

	new bl T p F ch F sb F fq F src F time F

would reset all values in one go. As a result, when you next plot something, everything for a baseline will be put in one plot since only the value for bl is true. You will end up with 3 plots (one for each baseline) and containing 8 datasets (4 subbands x 2 polarizations).

Now we enter the following command

	new bl F sb T

Only the values for bl and sb are modified. After re-plotting, you will have 4 plots now. One for each subband, since only the value for sb is true. Each plot will contain 6 datasets (3 baselines x 2 polarizations).

Let's issue the following command

	new p T

Now the values for p and sb are true. After re-plotting you will have 8 plots: 2 polarizations x 4 subbands, each combination giving a new plot. In each plot, there will be 3 datasets; one for each baseline.

As said before, in this way you are in complete control how you wish to overlay your datasets, depending on which relation you want to compare.

(nxt|prev)