2.1 Production Correlation

2.1.1 Sessions and Their Experiments

Session 1/2009 had a total of 20 user experiments correlated at JIVE. There were 10 spectral line experiments (nine 6.7GHz methanol maser and one 22GHz water maser), nine of which required multiple corrlator passes. Seven experiments used multiple MERLIN outstations in the EVN correlation. The new Yebes 40m telescope participated for the first time at 5cm, as did the Yamaguchi 32m telescope in Japan (data recorded as K5, then translated to Mark5B prior to shipping). With more than eight stations now available at 5cm, recirculation was used for the first time for user experiments to maintain the spectral resolution desired for methanol masers (typically 1024 frequency points over a 2 MHz subband, providing about 88 m/s velocity spacing).

Session 2/2009 had a total of 25 user experiments correlated at JIVE. There were 6 spectral line experiments (5 methanol, 1 OH). Recirculation avoided the need for multiple passes in four of those. Four other (continuum) experiments required multiple passes for separate correlation phase centers. Four experiments used multiple MERLIN out-stations. Yebes 40m participated for the first time at 6cm. The Chinese stations at Kunming and Miyun participated for the first time ever in EVN observations (S/X-band NME for Kunming; an X-band user experiment for both).

Session 3/2009 had a total of 17 user experiments correlated at JIVE. We discovered that recirculation and oversampling can't be used together (see section 2.1.3 Astronomical Features for more details). There were two wide-field mapping experiments that required multiple passes by pairs of subbands to keep the bandwidth smearing to the required level (combined correlator output about 880 GB, for these two).

In 2009, there were 29 e-EVN user experiments, including six ToOs between April and July. These ToOs inlcluded a 3-epoch experiment in which Kashima and three Australian stations participated (Kashima translated their K5-format data to Mark5B prior to transmission; this was successful by the third epoch). In the 24-25 March e-EVN day, we had our first truly Gbps real-time correlation in user experiments; after this time Gbps became the standard mode for continuum e-VLBI observations. An e-VLBI demo for the International Year of Astronomy saw the largest number of stations correlating at once in real-time e-VLBI at 12 (figure [##0]).

[figure 0 iyaANN2.png goes here] Figure [##0] Network throughput plot for the International Year of Astronomy e-VLBI demonstrations.

Session 1/2010 had a total of 23 user experiments correlated at JIVE. The KVAZAR stations Zelenchukskaya and Badary participated in ftp fringe tests for the first time, and Shanghai participated for the first time at 5cm, providing long baselines for 6.7 GHz methanol maser observations entirely within the EVN.

Session 2/2010 saw the first e-EVN experiments to be scheduled during a disk session. There were a total of 16 disk-based user experiments correlated at JIVE (6 e-EVN experiments scheduled during the session are included in the 2010 e-VLBI summary paragraph below). Zelenchukskaya and Badary participated in many user experiments (after joining only the NMEs in session 1/2010). In the course of this session, we made the first use of native Mark5B playback during correlation (as opposed to playing a Mark5B recording via the Mark5A+ firmware on a Mark5A unit); by this time, Westerbork, Yebes, Urumqi, and Badary were providing Mark5B recordings. Session 2/2010 also saw the first operational use of the SFXC correlator for a user experiment – one requiring pulsar gating.

Session 3/2010 continued the practice of placing e-EVN observations in the midst of the disk session, in order to accommodate a ToO. There were a total of 17 disk-based user experiments correlated at JIVE, plus that e-EVN ToO. The KVAZAR station Svetloe participated for the first time in NMEs and user experiments. Hartebeesthoek returned to the array following it successful polar-mount bearing repair. We had fringes for the first time to VERA_Mizusawa, at 5cm; these recordings had to be transferred from the original VERA tapes to a disk-pack in Mark5B format. Two more user pulsar-gating experiments were correlated on SFXC.

In 2010, there were 40 e-EVN user experiments, including 15 ToOs (there were also two disk-based ToOs correlated at JIVE in 2010). Seven e-EVN user experiments took place within regularly scheduled disk sessions. e-VLBI continued to book large gains in 2009-10, both absolutely and as a fraction of total EVN observing. Figure [##1] shows the evolution of annual EVN network hours since 2004, with the contribution of e-EVN representend by the shaded area (not all of the disk-based experiments were correlated at JIVE). Figure [##2] focuses on the e-EVN experiments, showing a division of annual observing hours into different categories: ToOs, triggered observations (see the Astronomical Features section), short (⇐ 2hr) exploratory observations not requiring a formal proposal, experiments proposed for disk recording, but conducted in e-VLBI (after consultation with the PI), and the standard e-EVN observations in regularly scheduled e-VLBI (or disk) sessions. By their nature, all e-EVN observations correlate at JIVE, and occupy a single correlator pass.

[figure 1 ntwkhr2010.png goes here] Figure [##1]: Annual EVN network hours, with the contribution by e-EVN observations shown by the shaded area.

[figure 2 etypes2010.png goes here] Figure [##2]: Division of annual e-EVN network hours into categories.

Tables [##1 and ##2] summarize projects observed, correlated, distributed, and released in 2009 and 2010. They list the number of experiments as well as the network hours and correlator hours for both user and test/NME experiments. Here, correlator hours are the network hours multiplied by any multiple correlation passes required (e.g., because of continuum/line, separate correlation by subband/pol to maximize spectral resolution, etc.).

                    User Experiments          Test & Network Monitoring
                  N   Ntwk_hr  Corr_hr          N   Ntwk_hr   Corr_hr

Observed 91 859 1148 35 176 181 Correlated 83 772 983 32 162 172 Distributed 88 824 1065 32 161 166 Released 91 854 1085 34 170 175 e-EVN experiments 29 224 224 e-EVN ToOs 6 68 68

Table [##1]: Summary of projects observed, correlated, distributed, and released in 2009.

                    User Experiments          Test & Network Monitoring
                  N   Ntwk_hr  Corr_hr          N   Ntwk_hr   Corr_hr

Observed 98 778 1041 16 51 51 Correlated 109 917 1236 19 67 67 Distributed 105 883 1183 20 73 73 Released 98 833 1155 16 61 61 e-EVN experiments 40 296 296 e-EVN ToOs 15 138 138

Table [##2]: Summary of projects observed, correlated, distributed, and released in 2010.

Figure [##3] presents the work division among various correlator tasks (production, clock-searching, network/correlator tests) as a number of hours per week, over the past three years (2009-10 highlighted). A six-week running average is shown. Troughs in the production hours correspond to periods when the correlator queue ran out of possible experiments prior to the start of the next EVN session, during which time we can focus more on correlator testing (e-VLBI, recirculation, fine-tuning the disk-servo'ing algorithm, etc). Bursts of time spent on network tests (cyan) in and immediately following sessions are also apparent.

[figure 3 ar0910_f3.png goes here] Figure [##3]: Work division among various correlator tasks, in hours per week.

Figure [##4] presents various measures of correlator efficiency. The red line plots the completed correlator hours per during time actively devoted to production correlation (the red line in figure [##3]). The green line shows completed correlator hours over the total operating time of the correlator – the red and green lines diverge more in periods when production takes up a smaller fraction of the total time available. The blue line shows completed network hours over total operating time – the green and blue lines diverge because some experiments require multiple passes. A twelve-week running average is shown to smooth out spurious peaks caused by periods with no remaining production correlation. The dip visible in network-hours per operating-hour (blue) at the very beginning of 2008 is the residue of the 17-pass GC029.

[figure 4 ar0910_f4.png goes here] Figure [##4]: Various measures of correlator efficiency.

Figure [##5] presents the size of the correlator queue at different stages in the processing cycle, showing a snapshot of the status at the end of each week. The red line plots the number of correlator hours that remain to be correlated. The blue line plots the number of correlator hours whose data remain to be distributed to the PI. The green line plots the number of correlator hours associated with recording media that have yet to be released back to the pool (in practice, release occurs prior to the following session, leading to a blocky pattern for the green line).

[figure 5 ar0910_f5.png goes here] Figure [##5]: Size of various correlator queues, measured in correlator hours.

Figure [##6] shows the number of user experiments and the number network and correlator hours correlated since 2003, with the hours for user experiments (diamonds) and the combination of user experiments and NME/test observations (squares). The bump in correlator hours in 2008 stemmed largely from the 17-pass GC029. The number of network hours correlated at JIVE, especially when considering both user experiments and NME/test observations, shows a steadily increasing trend. The growth in the number of user experiments has accelerated in recent years thanks to the new e-VLBI observations (more than tripled from 2005 to 2010). Both the increasing correlator hours and surging number of experiments place distinct pressures on the Science Operations and Support group, with PI interaction scaling more closely with numbers of experiments rather than their duration.

[figure 6 exps.png goes here] Figure [##6]: Amount of correlator and network hours plus the number of user experiments correlated in each year.

2.1.2 Logistics and Infrastructure

The disk-shipping requirements are derived from the recording capacity needed by a session (from the EVN scheduler) and the supply on-hand at the stations (from the TOG chairman). The EVN and VLBA stations follow different sets of guidelines:

a) the EVN policy that stations should buy two sessions' worth of disks,
   hence the disk flux should balance over the same 2-session interval.
   Following distribution to the stations for session 3/2010, we
   had "overdistributed" a net cumulative 47.2 TB of disk-pack capacity.

b) the VLBA's need for sub-session turn-around, which essentially requires
   pre-positioning the difference between what NRAO stations will
   observe in globals to be correlated at JIVE and what EVN stations
   will observe in globals to be correlated in Socorro.  Following the
   shipments in both directions for session 3/2010, we had "overdistributed"
   a net cumulative 213.12 TB of disk-pack capacity.  This has been built
   up entirely within 2010 (at the end of 2009, it was 6.23 TB) thanks to
   a series of long globals, now finished, that was correlated in Socorro.
   Prior to the distribution of packs for session 1/2011, the VLBA did
   return 52.13 TB, and since the preponderance of global proposals in 
   the pipeline to observe correlate at JIVE the bulk of the imbalance 
   should work itself away naturally.

Especially in 2010 sessions, experiments have been going to three different correlators. This added some complexity to the pre-session disk-distribution planning. The principal goal is to avoid individual packs containing data for more than one correlator. Should this occur, then some copying of raw data over the web would be required from correlator to correlator – this has occurred in a few instances between JIVE and Bonn, but ties up Mark5 units and usually renders the byte information in the field-system log unusable for a direct log-driven correlation preparation. Thus in the disk-distribution plan, the load for each target correlator for each station is computed separately. Packs on-hand at a station are applied to one of these individual-correlator loads prior to calculating what replenishment is required from JIVE. These loads are computed by

• ) assuming a 100% recording duty-cycle for the duration of each experiment
• ) subtracting 0.09TB from the capacity of each pack (to provide a buffer to compensate for unused space at the ends of packs du to the last + 1) Mark5 scan not having fit onto the pack).

in order to provide some insurance against loss of capacity due to bad packs, (but would likely be insufficient if a very large pack fails). The disk-distribution plan tabulates the specific set of packs to use per target correlator in terms of how many of which capacity. We try to use preferentially the largest-available packs for the farthest stations (to cut down on shipping), but consistent with minimizing unused capacity (e.g., 4.4TB = 3.2TB + 1.48TB instead of a 6TB or 8TB). Additionally, as was learned in session 1/2010, experiments destined for Socorro but well separated in time in a sesssion should be covered by sets of packs that can be shipped individually, without holding partially-recorded packs from the first to use in the second.

The current play-back line-up is 14 Mark5As, 2 Mark5Bs, 1 Mark5B+, and 7 Mark5Cs. The MarkIV correlator is still limited to a maximum of 16 stations. Among the standard EVN stations, Westerbork, Yebes, Urumqi, Badary, Zelechukskaya, and Svetloe currently provide Mark5B recordings (as do typically the Japanese stations and Kunming among the non-EVN stations correlated in this biennial period). We can play back via a Mark5B unit (bypassing the station units) or via a Mark5A unit using the capabilities of the 5A+ firmware. Because Mark5B units can not play back Mark5A recordings, we do need to retain enough 5A units to handle the maximum expected number of Mark5A stations (through 2010: ten EVN stations, plus VLBAs and Robledo) to minimize the conversions between Mark5 “flavors” of the individual units. Playback through the Mark5B units does result in lower statistical noise in the correlated phases, so there is emphasis on moving away from Mark5A as soon as the stations upgrade. Maintaining the proper mix of playback units in light of the phasing of these changes is reminiscent of that required in the initial tape-to-disk transformation. The SFXC could in principle accept a 24-station array, not limited by the number of station units, even for the Mark5A recordings.

We continue to encounter the occasional individual bad disk (or two) in an incoming pack. We maintain a small bench stock of disks of various sizes so that we could replace a bad disk locally if that is the most appropriate course of action (in light of warranty status, urgency of recycling, etc.), and then we would get a new disk from the pack's “owner” to replenish our bench stock. All but the highest data-rate recordings generally play back well with a bad individual disk disconnected. There was an unusually high amount of difficulty with Gbps recordings in session 1/2010, coincident with the introduction of some new 8TB SATA packs, that drove the processing factor (actual correlation time required over the number of correlator hours) up from a typical 2-2.4 to over 4. This did not recur subsequently, so was probably limited to a specific batch of disks.

2.1.3 Astronomical Features

The impact of e-VLBI on expanding the kinds of astronomy that can be successfully pursued with the EVN was truly felt in 2009-10. Figures in section 2.1.1 illustrate the rapid growth in e-EVN observating, both absolutely and as a fraction of total EVN time. Target-of-opportunity (ToO) experiments form the greatest advance. There have been 19 ToO proposals submitted in this two-year period (including one global and 2 disk-based ones), considerably more than in the previous decade – illustrating the change in mind-set with which astronomers view the EVN as an instrument that can respond to outbursting sources rapidly. These projects have addressed novae, outbursts in masers, X-ray transients and microquasars, newly discovered supernovae, and also provide the means to coordinate the EVN with obersvations at other wavelengths – typically high-energy satellite observations looking to study correlations of changes in radio brightness/structure with high-energy brightness variations. In 2010, a full 38% of the observed EVN network hours correlated at JIVE were e-VLBI observations, and 47% of these (18% of the total) were e-EVN ToOs. Triggered observations provide another means to approach transient behavior of a pre-selected set of sources without the stress of preparing a ToO proposal. Based on a proposal submitted at a regular deadline, the class of triggered observation enables e-EVN observations one of the listed source when its behavior prior to an e-EVN session shows that it has entered an interesting state, recognizable by a set of criteria. Once such a triggered proposal has been accepted, the proposing group only needs to submit a short trigger request, showing that the triggering criteria are met, up to 24\,hr before the start of an e-EVN session to be considered for observation in that session. To date, such proposals have focused on X-ray binaries, and there have been three “triggers” executed in this period. Short observations have always been possible with the EVN, requiring only a request letter to the PC chair in lieu of a full proposal. For e-EVN, short observations are considered to be up to two hours, and need to be requested up to three week before the desired e-EVN observing day (compared with 4 hr and 6 weeks before the beginning of a regular EVN session for disk). There have been 11 short e-EVN observations in 209-10, typically checking for calibrators or confirming compactness of a target prior to a full proposal for a more extensive study.

Recirculation is a means of time-sharing correlator resources for experiments that don't use the maximum sampling rate (32Mb/s; 16MHz Nyquist-sampled subbands), in order to increase the apparent spectral capacity in experiments that would otherwise have had spectral resolution limited by the number of stations or polarizations. The recirculation factor, R, is 32 over the sampling rate, up to a maximum of 8 (thus experiments with 16MHz subbands can't use recirculation), and the spectral capacity scales as R, subject to the maximum number of frequency points per baseline/subband/polarization that remains at 2048. The downside of recirculation is that the minimum integration time also scales with R, which may affect narrow-band spectral-line observations wanting a wide field of view (recirculation may not be available to them, which may in turn require limiting the array size). We also learned that the combination of recirculation on oversampled data does not work together. Oversampling is used almost exclusively to get to 0.5 MHz sub-bands (sampled at the current minimum 4 Msamp/s sampling rate, thus 4 times Nyquist). For 2 MHz sub-bands, recirculation can provide an 8x boost in spectral resolution, greater than the corresponding gain from oversamping – but the maximum of 2048 frequency points per baseline/sub-band/polarization would remain. So users wanting 1024 or 2048 frequency points across 0.5 MHz sub-bands would need to choose oversampling instead of recirculation, with the possible array-size limit of 8 stations. Recirculation was first used operationally for user experiments in session 1/2009, and has been on by default starting from session 2/2009 (we turn it off explicitly for short integration-time or oversampled experiments). Of course, no improved capability remains sufficient for long… some experiments now require both recirculation and multiple passes to attain their desired spectral resolution (usually 8MHz-subband spectral-line observations).

The possibilities for including additional MERLIN out-stations in the EVN correlation continue to expand. We had already incorporated one additional dual-pol and three additional single-pol stations into the Cambridge recording in user experiments (one station/pol per each of the VLBA recorder's four IFs). The extra out-station data can be placed into “unused” subbands – ones in the observing set-up not needed for the 128 Mbps data per out-station transmitted over the micro-wave link; lower data-rate experiments may require a separate, higher-rate schedule explicitly for Cm to create enough “unused” subbands to hold these additional out-stations. We can now use multi-casting techniques developed for e-VLBI to access all stations' data from the single Cm disk-pack directly. The benefit is avoiding having to copy the Cm pack; the cost is use of an additional two Mark5 units to simulate the e-VLBI transfer into the switch/router. In recent e-VLBI testing, more than one dual-pol out-station was able to be placed into the Cambridge data, by putting the 16 MHz signals from two station/pols into a single 500MHz IF by up-converting one of them when mixing into the IF and down-converting again when mixing into the BBCs.

High-sensivitivy wide-field mapping observations usually can't use recirculation; they either have 16MHz subbands and/or want the shortest possible integration times. Such observations also may need to be correlated in multiple passes by subsets of sub-bands in order to cut the bandwidth smearing. We now can combine these passes at the Measurement Set stage into a single Measurement Set containing all sub-bands. Included are extra steps to check for “orphan” sub-bands at any time, and to make sure the data stay in TB order. The PI avoids the need to VBGLU separate sets of FITS files back together, and the pypeline processing can provide a single ANTAB file applicable to the whole experiment. Two experiments from session 3/2009 went through this process: GV020A with 348 GB of raw correlator output and 230 GB of FITS files, and GF015A with 532 GB and 328 GB, respectively.

We discovered a problem playing back 512 Mbps VLBA-format data using 16 8MHz channels at fan-out=2 (the sched default for 512 Mbps would use 8 16MHz channels, which doesn't show this effect). The result is that in the higher subbands, weights of 0 ensue in autocorrelations (but not necessarily in the associated baselines, although those products are often suspect). For “native” VLBA recordings, this is intermittent, and can affect either polarization in the upper two subbands in a non-repeatable non-repeatable way. Mark5B data played back via the 5A+ firmware will also be in VLBA format; here the problem seems localized to the 7th subband. But for such Mark5B/5A+ situations, we have the flexibility to define how the Mark5B bit-streams will be converted into Mark5A tracks; use of fan-out=1 in this conversion avoids the problem. For native VLBA recordings, avoidance seems the only solution at present, and this is now a topic for the pre-session schedule-checking carried out by JIVE support scientists.

SFXC has provided the capability to provide pulsar-gated correlation at JIVE; three such experiments have been observed in the last two sessions of 2010. A number of independent bins can be placed within a single gate that has a start/stop phase with respect to the pulsar period. Each bin would produce a separate FITS file. Traditional gating in the MarkIII sense corresponds to 1 bin. Figure [##7] shows an example of a correlation with 100 bins spread over a gate of a tenth of the period for PSR 0329+54 at 1.4 GHz. Figure [##8] shows the Effelsberg single-dish pulse profile, illustrating that the SFXC pulse profile built up from the indenependent bins within the gate reproduces the pulse profile well. Other types of experiments that would immediately benefit from SFXC would include those with more than 16 stations and spectral-line experiments wanting more than 2048 frequency points per subband/pol.

[figure 7 profile-onpulse.png goes here] Figure [##7] Pulse profile for PSR 0329+54 built up from 100 separate bins within a gate of one-tenth of the pulsar's period, from a test observation at 1.4 GHz.

[figure 8 sieber2.png goes here; side-by-side with fig##7, or preferably on top of each other, with a horizontal scaling to make the pulses appear the same width (i.e., two side-pulses line up with each other in the two plots)] Figure [##8] Effelsberg single-dish pulse profile from Sieber et al. (1975, A&A, 38, 169).

2.2 EVN Support

Automatic-ftp fringe tests are included in all network monitoring experiments (NMEs) at the beginning of each new frequency sub-session within EVN sessions, or as a separate fringe-test observation when the NME does not appear first in the schedule or falls well outside working hours. Under the control of sched and the field system, a specified portion of a scan is sent directly to the software correlator computer at JIVE. In a three hour NME, there would typically be ftp transfers from three scans, providing the opportunity to confirm the resolution of any problems identified in the first transfer. Use of ftp transfer and near-real-time correlation permits stations that don't have a full e-VLBI fibre connection to participate. A skype chat session during the ftp fringe-test observations provides even more immediate feedback between the station friends and the JIVE support scientists. Correlation of the ftp fringe tests takes place on the SFXC correlator, and correlation results go to a web page available to all the stations, showing baseline amplitude and phase across the band as well as autocorrelations, and each plot is accessible by moving the cursor over color-coded baseline/subband/polarization cells. The web-based results from the first and probably the second ftp transfer would be available to the stations before the end of the NME. These ftp fringe tests continue to be very successful in identifying telescope problems and helping to safeguard user experiments by allowing the station friends to take care of any such problems before the actual astronomical observations begin.

The EVN pipeline now runs under ParselTongue (a Python interface to classic AIPS). The new pipeline is considerably easier to use, more robust and has much greater scope for future development due to the improved coding environment. The pipeline scripts are available from the ParselTongue wiki (RadioNet) and should provide a good basis for other (semi-)automated VLBI reduction efforts. We continue to process all experiments, including NMEs, via the pipeline, with results being posted to the EVN web pages. The pipeline provides stations with feedback on their general performance and in particular on their gain corrections, and identifies stations/frequency bands with particular problems. Timely delivery of ANTAB amplitude calibration results from the telescopes seems to be improving, but remains an issue in e-VLBI experiments due to the shorter time-scales involved.

We helped Onsala investigate the performance of their new optical-fiber system for getting the RF down from the 25m antenna. Jun Yang determined improved SEFDs for Jodrell Bank Mark2 and Cambridge from a series of NMEs from 2009. Torun appeared to be missing fringes in only some 5cm (methanol maser) experiments in some sessions in 2009; it was discovered to be a LO change between EVN and single-dish observations that isn't under field-system control. Once this was realized, we were able to go back and correlate with the appropriate LO offset (1 MHz) and see fringes in the NME from session 3/2009. However, a 1 MHz LO offset still would irreparably damage the typical methanol-maser observation that uses 2 MHz subbands with the emission centered in the band. The station took measures to avoid the problem subsequently. Session 1/2010 saw the first time Mark5B EVN data went to Socorro for correlation; we passed along the appropriate bit-stream/channel layouts for these stations in the various experiments.

There have been quite a few new stations participating in astronomical observations. The three KVAZAR stations have joined the EVN as members, and strengthen the array's capability to provide long baselines. We conferred with the station friends at IAA to understand how to provide set-up information in the user schedule files to work best with their data acquisition systems (see “PI Support” section). Stations in Japan (Yamaguchi 32m, VERA_Mizusawa) have participated in some methanol-maser astrometric observations. These are not under field-system control, and provide Mark5B-format disk-packs they generate by translating from their natural K5 format and/or VERA recording tapes. There were a couple of iterations with people in Mitaka about this process, resulting in recordings we could correlate reliably. Without a field-system log to control the antennas or associate bytes on the pack with scan start times, they record continuously, moving the antennas under local control to match the schedule and we dead reckon the byte/scan associations from knowing the begin/end times of the recordings (per constituent original VERA tape, if appropriate). Two new stations in China participated for the first time in EVN observations late in 2009: Kunming in in the Phoenix mountains in southwest China and Miyun, about 140 km northeast of Beijing (see figure [##9]). Jun Yang help organize the shipment of Westerbork's Mark4 data acquistion system to Kunming (superfluous at Westerbork since the establishment of their new fully digital TADUmax back-end, cf. section 2.2 of the JIVE biennial report 2007-8 for TADUmax details). This Mark4 equipment would enable Kunming to record at a full Gbps, up from their previous 128 Mbps limit.

[figure 9 MyKm2.png goes here] Figure [##9] The Miyun (My) and Kunming (Km) antennas, along with phase versus time for fringe-finder scans in the experiment EY008A (August 2009) on baselines formed by sub-array Shanghai, Urumqi, Miyun, and Kunming.

[this is just a cropping of a viewgraph shown at the EVN symposium; I think the constituent parts exist separately]

2.3 PI Support

The EVN Archive at JIVE provides web access to the station feedback, standard plots, pipeline results, and FITS files for experiments correlated at JIVE. Public access to the FITS files themselves and derived source-specific pipeline results is governed by the EVN Archive Policy – the complete raw FITS files and pipeline results for sources identified by the PI as “private” have a one-year proprietary period, starting from distribution of the last experiment resulting from a proposal. PIs can access proprietary data via a password they arrange with JIVE. PIs receive a one-month warning prior to the expiration of their proprietary period. The archive machine has 12.8 TB of dedicated disk space, with a buffer of another 1.8 TB that also houses the pipeline work area. The total size of the FITS files in the archive at the end of 2010 was 9.43 TB (a 3.75 TB gain in the two-year period); figure [##10] shows the growth of the FITS-file size in the EVN archive size over time.

[figure 10 archvgro2010.png goes here] Figure [##10]: Growth in the size of FITS files in the EVN archive. Experiments archived in this biennial period are plotted in red.

We have begun to post two new types of FITS data on the archive: (i) FITS files from the Westerbork-array data obtained during during the course of EVN observations, and (ii) the pipeline-calibrated UV-FITS files for individual sources. The Wb FITS files aren't directly available to users as such from the station; we take care of the data transformation from the Wb-archive Measurement Sets, and place the resulting FITS files on the archive along with the FITS files from the EVN correlation. We are working on expanding the pipeline to calibrate these Wb FITS data, to help with amplitude and polarization calibration of the full EVN data-set. In some cases (e.g., small-field continuum mapping), the Wb FITS data can be several times the size of the EVN FITS data, so this extra processing is currently driven by PI request. The pipeline-calibrated UV-FITS data for individual sources contain the cumulative effects of all steps of calibration within the EVN pipeline. These are now available via the main pipeline page for each experiment in the EVN archive. These pipeline-calibrated FITS files associated with “private” sources are protected by the same one-year proprietary period as are the plots/images of these sources and the full set of raw IDI FITS files.

The science operations and support group continues to contact all PIs once the block schedule is made public to ensure they know how to obtain help with their scheduling. There were 14 first-time EVN PIs in 2009 and another 13 in 2010. We also checked schedules PIs posted to VLBEER prior to stations downloading them, with safeguards in place to minimize the chance that different stations use different versions of an experiment's schedule. In session 3/2010, after the PIs had desposited their schedules we learned that the Lovell telescope would not be able to observe. Behind the scenes, we restored Jodrell Bank Mark2 to schedules that had used the Lovell, including reinserting scans it may have missed in fast cycle-time phase-referencing observations. There continues to be the very occasional instance (only three times this period) of a PI trying to work a non-authorized target into the schedule; we address these to the EVN PC chairman. The pre-observation communication also provides the opportunity to inform eligible PIs about the benefits of the RadioNet trans-national access programme, as well as the extra reporting they would eventually need to provide.

The preferred patchings for the individual KVAZAR stations run afoul of the various existing checking rules in sched, so beginning with session 2/2010, we provided PIs of experiments including them with plug-ins for their sched-input files that will pass the existing checks and are also relatively close to the stations' preferred patching. Once the PIs deposit such schedules, we then hand-edited them to conform to the stations' preferences as part of the normal JIVE-review period prior to releasing all the schedules to the stations for DRUDGing. Once we had a reasonable understanding of what to do, this process was fairly straightforward. We have identified the routines within sched to modify to allow the KVAZAR patching to be accepted properly (these reside in the parts for which JIVE has historically contributed as part of EVN support to sched). Similar extension of set-up checking is anticipated for the advent of the DBBCs at EVN stations early in 2011.

JIVE hosted 8 data-reduction visits in both 2009 and 2010. In addition, there were six post-graduate students who were co-supervised by members of JIVE staff during all or part of this period, and who visited frequently. The visitors room has five dual-processor PCs running linux, a MAC-mini, and a windows-based PC. Three of the linux PCs have been replaced with more powerful machines this period, and these are also dual-boot linux/windows.