1 UVCROSS UVCROSS calculates gain correction factors for each station based on the ratios of amplitudes at or near the uv-crossing points. Several data weighting options, based on amplitude errors, are available. Solutions are calculated in clusters of dependent stations. 2 Brief_Guide Have the VLBI package "initialised", then type uvcross In the simplest case, just specify the INPUT file, decide on the UVRADIUS (see below), and terminate input with a / This generates some output to the terminal, and an overview of the run in a file UVCROSS.LIS. That filename can be changed with LISTFILE, and the amount of output with MSGLEVEL (0 to 3, default is 1). If there are many stations, there can be hundreds of crossing points; in this case, one may wish to limit those printed, using NCPRINT. Preparation of a commandfile for VLBEDIT is often desirable; this is specified through EDITFILE. One may wish to exclude data taken at large zenith angles, using ZALIM, or to restrict the area of the uv-plane used, with UVWINDOW. There is no recipe for determining UVRADIUS, which specifies how close two baselines must come before they are taken to "cross". However, a reasonable choice might be somewhat larger than a pixel in the (u,v)-plane. So, for a greatest (u,v)-distance of 150 Mlamda, and 256 pixels in the final map, UVRADIUS=0.75 could be reasonable. You are encouraged to experiment with different values, but remember that the cpu-time required goes up steeply with UVRADIUS. The Example in the helpfile shows the parameter input discussed above. Further parameters are available for more sophisticated data selection and weighting; see the Parameters section in the helpfile. 2 Example $ uvcross INPUT = 'MYSOURCE.MRG' ! The Merge-file to be analysed. UVRADIUS = 0.75,0.75 ! "Crossing" when within 0.75 Mlamda MSGLEVEL = 2 ! Record more extensive information, LISTFILE = 'MYSOURCE.LIS' ! in a listing file MYSOURCE.LIS, NCPRINT = 50 ! but print only 50 ratios. EDITFILE = 'MYSOURCE.EDT' ! Prepare a commandfile for VLBEDIT. ZALIM = 70 ! Reject data when either telescope ! is at a zenith angle > 70 degrees. UVWINDOW = 100,100 ! Use only the inner 100 Mlambda. / This input is described in the Brief_Guide. 2 Limitations - The program accepts input files containing: (a) 20 or fewer stations, (b) 65536 or fewer (non-deleted and non-rejected) u,v-points. - The clustering algorithm is incomplete (see METHOD), but a warning is given when appropriate. - The least-squares problem is non-linear. However, while the solutions are approximate, the errors are usually dominated by noise in VLBI data. 2 Parameters All parameters are read through KEYIN (documented separately). They can be abbreviated as long as they are unique, except NOSOLVE (NOSolve). Parameters INPUT and UVRADIUS are mandatory, the others are optional, and the defaults are probably reasonable. 3 Input Input filename (max. 64 char.) Must be in Merge-1 or Merge-2 format. A warning is given if the amplitudes are not calibrated. 3 Editfile If specified (max. 64 char.), this file will contain a commandfile for running VLBEDIT to scale the visibility amplitudes and their errors. Check this file before running it. In particular, you may wish to change the output filename: for VMS, this is by default the next higher file version, while for UNIX, it is the old filename with ".UVCROSSED" appended. No commandfile will be made when EDITFILE=' ' (the default). 3 Listfile Specify this (max. 64 char.) to override the default UVCROSS.LIS. This file may contain a (very) extensive overview of the program run and the solutions found. The amount of information depends on MSGLEVEL and NCPRINT. If MSGLEVEL=0 the file is not made at all. By default MSGLEVEL=1. 3 Msglevel MSGLEVEL specifies the detail of output in the LISTFILE. Default: 1, basic listing. Note that at each MSGLEVEL, the number of ratios printed can be limited using NCPRINT. 0) : No LISTFILE. 1) : Summary of input parameters. Summary of data read. Gain solutions. Effect these solutions would have on the averaged ratios. 2): Same as 1), plus: Number of points found on each baseline. Extra information about the average for each ratio. 3): Same as 2), plus: Details for each individual ratio. 3 NCprint Default=0; meaning print ALL ratios. NCPRINT limits the number of ratios printed in the LISTFILE. This is useful when the (u,v)-plane is well filled, such that there are many hundreds of crossing points. Only the first NCPRINT that are found will be listed, at the level of detail appropriate for the MSGLEVEL selected; NCPRINT is ignored for MSGLEVEL=0. 3 UVRadius UVRadius = u ,v in Megalamda Cutoff distances in U and V beyond which baselines no longer "cross". By default, UVRADIUS defines the half-axes of an ellipse, but if DORTANGL is specified, it defines half the length and width of a rectangular box. The U value must be specified, but if the V value is left at 0, then it will be set equal to the U value, giving a circle or square. By default, UVRADIUS is constant throughout the (u,v)-plane, but this can be modified using UVPOWER (see the help on UVPOWER for details). 3 UVPower UVPOWER allows the effective UVRADIUS and UVTAPER to vary with (u,v)-distance. Leave this disabled (UVPOWER=0; the default) unless you fully understand the following: The radii can be calculated separately for each "reference point" by scaling with its (u,v)-coordinates U and V as: UVRADIUS_actual = UVRADIUS_given * (U^2 + V^2)^(UVPOWER/2) If UVPOWER is not 0, and, in particular, if UVPOWER = 1, then UVRADIUS_given is somewhat akin to a fraction of the (u,v)-distance, rather than being itself a distance. All numbers are in megalamda. This could be useful in case there is a wide range of baseline lengths in the dataset. A single radius could then lead either to "crowding" in the inner plane, or to a lack of crossings in the outer regions. UVTAPER, if specified, is scaled along: UVTAPER_actual = UVTAPER_given * (U^2 + V^2)^(UVPOWER/2) It is suggested to restrict UVPOWER to [0,1]. Also note that, with typical global 6 cm VLBI datasets, UVRADIUS would be of order 1 if constant, but of order 0.01 if used in conjunction with UVPOWER=1. If UVPOWER = 0 (the default), the above calculations are not performed repeatedly, leading to a significant gain in speed. 3 DORtangl If not left at -1 (the default), this indicates that the UVRADIUS should be taken as a rectangle, not an ellipse. This makes for slightly faster program runs. Specify DORTANGL without a value to select the option, and DORTANGL = -1 to reset to the default. 3 DOAbserr If not left at -1 (the default), this indicates that the weighting scheme should be based on the ABSOLUTE VALUES of the amplitude errors (the sigmas), rather than on the RELATIVE amplitude errors (signal-to-noise ratios). The latter is the default. Note that this option has no effect if WTPOWER = 0. Specify DOAbserr without a value to select the option, and DOAbserr = -1 to reset to the default. 3 Wtpower Weights are determined by the relative (default) or absolute (if DOABSERR specified) amplitude errors. These numbers are raised to WTPOWER. You may wish to stay with the default, -1, or experiment with -2 (variance weighting) or -0.5. Positive values of WTPOWER are obviously ridiculous, as they would give more weight to noisy data, but WTPOWER = 0 could be useful: all data points will then be given equal weight. Note that WTPOWER is applied before any UVTAPER. 3 UVTaper UVTaper = u ,v in Megalamda The impact of a pair of points on the average ratio can be (down)weighted by their separation (DU,DV), with the following exponential: EXP( -(DU/UTAPER)**2 -(DV/VTAPER)**2 ). If both values of UVTAPER are 0, nothing is done. If only 1 value is given, the other is assumed to be the same. By default, the values are constant throughout the (u,v)-plane, but this can be modified using UVPOWER (see the help on UVPOWER for details). 3 TDiff TDiff = t (minutes) If TDIFF is not 0, it prevents the use of a crossing u,v region when the data from the two baselines are further apart than TDIFF minutes. This option may be useful when combining data from different epochs, or if the observed source varies during the experiment. 3 Zalim Zalim = d in degrees If ZALIM < 90 degrees, all data points are rejected for which the zenith angle of either telescope is greater than ZALIM. Set ZALIM to 90 degrees (or more) to disable the calculation and test of zenith angles. Requesting the ZALIM test will lead to an error if orbiting telescopes are in the dataset. 3 TRadius TRadius = t (minutes) If TRADIUS is not 0, it restricts the used length of the timespan along each baseline separately. Counting from the point closest to the other baseline, other points must lie within +/- t minutes to enter into the calculated ratio. Useful in case of rapid calibration variations, or perhaps to restrict contamination by noisy data or source structure, combined with a judicious choice of UVRADIUS and UVPOWER. 3 UVWindow UVWindow = u ,v in Megalamda Restricts the location of used crossing points if not left at 0,0. Care is taken to allow full use of all data in case the crossing point is within the window, but part of the UVRADIUS around it is not. Specify positive values to select the inner portion of the uv-plane. Specify negative values to select the outer portion of the uv-plane. The u and v values apply separately, i.e., they define a rectangle. If only 1 value is given, the other is assumed to be identical. 3 NOSolve Specify the name of any station for which no gain correction should be calculated. Note that this prevents all ratios involving this station from being used (i.e. the gain is not simply taken to be 1.0). 2 Method - Amplitude ratios are found for all baseline pairs which "cross". Provisions are made to take advantage of long almost parallel tracks. - Clusters are determined, each containing a group of stations which are linked to eachother, but not to other stations, by the available ratios. Some stations may not be constrained at all. - For each cluster, the amplitude gains are determined through a least-squares solution. Since UVCROSS does not deal with models, the overall gain is not allowed to float: in each cluster, the product of the amplitude gains is forced to be 1. 3 Crossing points UVCROSS considers a baseline to "cross" another one when it has at least one (non-deleted) point which is within a user-specified U,V-distance of at least one point on the other baseline. For each crossing pair, an AVERAGE amplitude ratio is constructed. It is typically constructed from many "INDIVIDUAL" ratios, as will be explained below. There is no restriction to an ellipse around a theoretical crossing; all sorts of near misses are allowed. The procedure ensures that as many U,V points as possible can enter into the calculation, without the need to average amplitudes over a wide area of the U,V plane. This is useful, for example, for a baseline pair which runs almost parallel. For each pair, the two baselines are treated in turn as the "reference" and the "neighbour" baseline. The individual ratios are obtained for, and belong to, individual points on the reference baseline. All reference points are considered in turn. Points on the neighbour baseline are sought within a given U,V ellipse of the reference point. If there are any such close points, an individual ratio can be determined. A neighbour amplitude is derived as the weighted average of ALL neighbour-baseline amplitudes within the ellipse. Likewise, a reference amplitude is obtained as the weighted average of all points on the reference baseline which lie within the U,V ellipse of the reference point (which, by definition, therefore contributes to the reference amplitude). The weighting scheme involves ABSERR, WTPOWER, and UVTAPER (see the help for those keywords under the topic Parameters). The error-bar of the neighbour and reference amplitudes is given by the square root of the weighted average (as above) of the square of the error-bars of the data points. The ratio of the reference to the neighbour amplitudes determines the individual ratio for that specific point on the reference baseline. Its relative error is given by the square root of the sum of the squares of the relative errors of the two amplitudes. After considering each baseline as reference in turn, all individual ratios are averaged (weighted by their error-bar) to form the average ratio applicable to the pair of baselines as a whole. Thus, averaging in amplitude is only done within a (small) U,V ellipse, but averaging of the individual ratios then allows a much broader region of the U,V plane to be used, if the baselines stay close over a long distance. However, the error estimate for the final ratio on the baseline pair is not determined from the errors of the contributing individual ratios, since these are typically formed using many overlapping points: they are far from independent. Instead, the square root is taken of the weighted average of the squares of the relative errors of all the the points actually used as reference (i.e. those which have close neighbours on the other baseline). For this weighting, ABSERR and WTPOWER are taken into account, but UVTAPER is meaningless (each reference point is always at the centre of its own ellipse). Clearly, the error obtained by this procedure is not exactly correct, as not all points will enter equally frequently into the calculation of the average ratio, or with equal "tapering". However, while not proper in absolute value, the errors obtained for different baseline pairs should still be reasonably in proportion, such that they can provide relative weights in the least squares gain solutions (which is all they're used for). 3 Clustering The crossing ratios found specify relations between the gains of stations. Before solving for the gains, UVCROSS has to determine which stations are tied together by the relations. UVCROSS has to apply the restriction that the overall gain may not float to each cluster of stations separately in order to have enough constraints for the least-squares solutions (see the help on least-squares). The present clustering algorithm is INCOMPLETE, because the whole affair is very tedious. However, the "open ends" are known, and their encounter will be indicated on the screen and in the listfile (if requested at all). If non-optimal clustering is indicated, this COULD, but not necessarily DOES mean that the algorithm has left too many separate clusters uncombined. The gain solutions MAY therefore not represent the optimal use of the complete set of ratios, and they MAY be derived using the non-floating overall gain constraint too often. The trouble arises because there are two kinds of ratios: - Forks, of the form A-B / A-C - 4-ways, of the form A-B / C-D Forks are fully dealt with by the algorithm. They specify a direct relation between two stations (B,C in the above case). The clustering algorithm always ties those stations together in the same cluster, whether it be a new one, or a coalescense of the cluster(s) of which the stations may already have been part. One 4-Way ratio, unfortunately, gives insufficient information to constrain the gain of all 4 stations it involves. The current algorithm FIRST deals with all the forks to establish clusters. Then, the 4-ways are considered in an iterative loop, which ends when a pass has not led to any mutations in clustering. A 4-way ratio is ONLY ACCEPTABLE to the algorithm if it is IN ITSELF sufficient to: - create a new cluster, or - add a new member to a cluster, or - tie 2 clusters together. If more complicated 4-way ratios are left at the end of clustering process, a warning message is printed. For full details, see the source code. Of course, it is possible that the gain of some stations is not constrained at all by the data. These are left out of the clusters. 3 Least-squares The least-squares solver operates on each cluster in turn. The following function is minimised w.r.t. all gains (Gi): sum 1 Ga Gb 2 over --------- * [ log ( ----- * RATIO ) ] ratios SIGMA*SIGMA Gc Gd where Ga to Gd are the gain factors for the 4 stations involved in the ratio. This function is based on the following considerations (written up here in part because they differ from those in versions 1.* of UVCROSS): The gain corrections are made by MULTIPLICATION, so one should test by WHAT FACTOR the ratios differ from 1. In other words, a ratio of 0.1 should be considered equally bad as a ratio of 10, not as a ratio of 1.9 . The product of all corrected ratios should ideally be 1. Thus, the sum of the logs should be as close as possible to 0. To enforce least SQUARES, minimise the sum of the squared logs. Furthermore, each term in the sum of a least squares function should be weighted by a variance. The least-squares solver is iterative: it adjusts the gain of each station in turn such that chisq. reaches a minimum (grid search). The iterations are terminated when the relative change in chisq. is less than 0.001. After each iteration, the product of the gains of the stations is forced back to 1.0 . This rigid boundary condition is necessary because the overall gain error can never be determined from the data: the absolute flux scale of the map is unknown, and the ratios determine only the relative flux scale between the stations (and hence the baselines, and hence the features in the map). Take the simplest possible case as an example. There is only 1 ratio ( R = A-B / A-C), which determines the gain of station B relative to C: Gb = R*Gc. We clearly need an additional constraint to determine Gb and Gc; Gb*Gc = 1 seems a very logical choice. 2 History Version 1.0: 1984 Nov 12 - initial program written. M.W. Hodges Version 1.1: 1986 Jul 23 - general clean up; add keyin of parameters Version 1.2: 1986 Jul 31 - include SNR's in the solution Version 1.3: 1986 Aug 21 - fix MAXV bug; add UT's for crossings 1986 Aug 22 - declare MAXV as real (SCU) Version 1.4: 1987 Jan 26 - fix GETFLUX to look for multiple days Version 1.5: 1989 Feb 23 - fix it with maximum crossing points upto 1000. From: ST%"zhang%buasta@buenga.BU.EDU" 23-FEB-1989 15:52 Version 1.6: 1989 Aug 30 - correct bug in computing number of stations (TJP). Version 2.0: 1990 Apr 02 Overhaul: Program substantially rewritten. - Main program; search for close points: - Emphasis now on UV constraints, not time limits. No theoretical crossings calculated "near misses" generalised so "true crossings" no longer special case. - Data weighting options more elaborate. - SOLVE subroutine (see comments there too): - Closer to true unbiased least-squares. - Solve in clusters of dependent stations. - NOSOLVE options added. Rene C. Vermeulen Version 2.1 1990 May 04 - Proper handling of baselines/stations without data. - Add EDITFILE, MSGLEVEL. - Minor bugfixes and cleanup of output. Rene C. Vermeulen Version 2.2 1990 Nov 30 - CPU-time saved by V sorting, bracketing. - Remove infinite loop on file-opening errors, by introduction of error-count NERR. - ZALIM and UVPOWER parameters added. - Annotation expanded, NCPRINT introduced. - Write # records rather than points in editfile. Rene C. Vermeulen Version 2.3 1991 Apr 23 - Bug-fix in errorbar weighting scheme. - UVTAPER now depends on UVPOWER. - Fix inconsistency in handling of "4-way" ratios - Expand and clarify handling of 4-way ratios. - Add TRadius and UVWindow options. - Change default MSGLEVEL from 0 to 1. - Print minimum crossing distances in listfile. Rene C. Vermeulen Version 2.4 1992 Sep 14 - Add "errmult" lines to EDITFILE. Rene C. Vermeulen