File Inspection (Experimental)¶

by Josh Dillon, Aaron Parsons, and Tyler Cox, last updated September 25, 2022

This notebook is designed to infer as much information about the array from a single file, including pushing the calibration and RFI mitigation as far as possible

Here's a set of links to skip to particular figures and tables:

  • Figure 1: Plot of autocorrelations with classifications
  • Figure 2: Summary of antenna classifications prior to calibration
  • Figure 3: Redundant calibration of a single baseline group
  • Figure 4: chi^2 per antenna across the array
  • Figure 5: Summary of antenna classifications after redundant calibration
  • Table 1: Complete summary of per antenna classifications
In [1]:
import time
tstart = time.time()
In [2]:
import os
os.environ['HDF5_USE_FILE_LOCKING'] = 'FALSE'
import numpy as np
from scipy import constants
import copy
import matplotlib
import matplotlib.pyplot as plt
import pandas as pd
pd.set_option('display.max_rows', 1000)
from uvtools.plot import plot_antpos, plot_antclass
from hera_qm import ant_metrics, ant_class, xrfi
from hera_cal import io, utils, redcal, apply_cal, datacontainer, abscal
from IPython.display import display, HTML
import linsolve
display(HTML("<style>.container { width:100% !important; }</style>"))
_ = np.seterr(all='ignore')  # get rid of red warnings
%config InlineBackend.figure_format = 'retina'

Parse inputs¶

To use this notebook interactively, you will have to provide a sum filename path if none exists as an environment variable. All other parameters have reasonable default values.

In [3]:
# get file names
SUM_FILE = os.environ.get("SUM_FILE", None)
# SUM_FILE = '/mnt/sn1/zen.2459797.30001.sum.uvh5'  # If sum_file is not defined in the environment variables, define it here.
DIFF_FILE = SUM_FILE.replace('sum', 'diff')
PLOT = os.environ.get("PLOT", "TRUE").upper() == "TRUE"
if PLOT:
    %matplotlib inline
print(f"SUM_FILE = '{SUM_FILE}'")

SUM_FILE = '/mnt/sn1/2459847/zen.2459847.27893.sum.uvh5'

Parse settings¶

Load settings relating to the operation of the notebook, then print what was loaded (or default).

In [4]:
# parse plotting settings
PLOT = os.environ.get("PLOT", True)
if PLOT:
    %matplotlib inline

# parse omnical settings
OC_MAX_DIMS = int(os.environ.get("OC_MAX_DIMS", 4))
OC_MIN_DIM_SIZE = int(os.environ.get("OC_MIN_DIM_SIZE", 8))
OC_SKIP_OUTRIGGERS = os.environ.get("OC_SKIP_OUTRIGGERS", "TRUE").upper() == "TRUE"
OC_MAXITER = int(os.environ.get("OC_MAXITER", 50))
OC_MAX_RERUN = int(os.environ.get("OC_MAX_RERUN", 4))

# print settings
for setting in ['PLOT', 'OC_MAX_DIMS', 'OC_MIN_DIM_SIZE', 'OC_SKIP_OUTRIGGERS', 'OC_MAXITER', 'OC_MAX_RERUN']:
    print(f'{setting} = {eval(setting)}')

PLOT = True
OC_MAX_DIMS = 4
OC_MIN_DIM_SIZE = 8
OC_SKIP_OUTRIGGERS = True
OC_MAXITER = 50
OC_MAX_RERUN = 4

Parse bounds¶

Load settings related to classifying antennas as good, suspect, or bad, then print what was loaded (or default).

In [5]:
# ant_metrics bounds for low correlation / dead antennas
am_corr_bad = (0, float(os.environ.get("AM_CORR_BAD", 0.3)))
am_corr_suspect = (float(os.environ.get("AM_CORR_BAD", 0.3)), float(os.environ.get("AM_CORR_SUSPECT", 0.5)))

# ant_metrics bounds for cross-polarized antennas
am_xpol_bad = (-1, float(os.environ.get("AM_XPOL_BAD", -0.1)))
am_xpol_suspect = (float(os.environ.get("AM_XPOL_BAD", -0.1)), float(os.environ.get("AM_XPOL_SUSPECT", 0)))

# bounds on zeros in spectra
good_zeros_per_eo_spectrum = (0, int(os.environ.get("MAX_ZEROS_PER_EO_SPEC_GOOD", 2)))
suspect_zeros_per_eo_spectrum = (0, int(os.environ.get("MAX_ZEROS_PER_EO_SPEC_SUSPECT", 8)))

# bounds on autocorrelation power
auto_power_good = (float(os.environ.get("AUTO_POWER_GOOD_LOW", 5)), float(os.environ.get("AUTO_POWER_GOOD_HIGH", 30)))
auto_power_suspect = (float(os.environ.get("AUTO_POWER_SUSPECT_LOW", 1)), float(os.environ.get("AUTO_POWER_SUSPECT_HIGH", 80)))

# bounds on autocorrelation slope
auto_slope_good = (float(os.environ.get("AUTO_SLOPE_GOOD_LOW", -0.2)), float(os.environ.get("AUTO_SLOPE_GOOD_HIGH", 0.2)))
auto_slope_suspect = (float(os.environ.get("AUTO_SLOPE_SUSPECT_LOW", -0.4)), float(os.environ.get("AUTO_SLOPE_SUSPECT_HIGH", 0.4)))

# bounds on autocorrelation RFI
auto_rfi_good = (0, float(os.environ.get("AUTO_RFI_GOOD", 0.075)))
auto_rfi_suspect = (0, float(os.environ.get("AUTO_RFI_SUSPECT", 0.15)))

# bounds on chi^2 per antenna in omnical
oc_cspa_good = (0, float(os.environ.get("OC_CSPA_GOOD", 3)))
oc_cspa_suspect = (float(os.environ.get("OC_CSPA_GOOD", 3)), float(os.environ.get("OC_CSPA_SUSPECT", 4)))

# print bounds
for bound in ['am_corr_bad', 'am_corr_suspect', 'am_xpol_bad', 'am_xpol_suspect', 
              'good_zeros_per_eo_spectrum', 'suspect_zeros_per_eo_spectrum',
              'auto_power_good', 'auto_power_suspect', 'auto_slope_good', 'auto_slope_suspect',
              'auto_rfi_good', 'auto_rfi_suspect', 'oc_cspa_good', 'oc_cspa_suspect']:
    print(f'{bound} = {eval(bound)}')

am_corr_bad = (0, 0.2)
am_corr_suspect = (0.2, 0.4)
am_xpol_bad = (-1, -0.1)
am_xpol_suspect = (-0.1, 0.0)
good_zeros_per_eo_spectrum = (0, 2)
suspect_zeros_per_eo_spectrum = (0, 8)
auto_power_good = (5.0, 30.0)
auto_power_suspect = (1.0, 80.0)
auto_slope_good = (-0.2, 0.2)
auto_slope_suspect = (-0.4, 0.4)
auto_rfi_good = (0, 0.075)
auto_rfi_suspect = (0, 0.15)
oc_cspa_good = (0, 3.0)
oc_cspa_suspect = (3.0, 4.0)

Load sum and diff data¶

In [6]:
hd = io.HERADataFastReader(SUM_FILE)
data, _, _ = hd.read(read_flags=False, read_nsamples=False)
hd_diff = io.HERADataFastReader(DIFF_FILE)
diff_data, _, _ = hd_diff.read(read_flags=False, read_nsamples=False)
In [7]:
ants = sorted(set([ant for bl in hd.bls for ant in utils.split_bl(bl)]))
auto_bls = [bl for bl in data if (bl[0] == bl[1]) and (utils.split_pol(bl[2])[0] == utils.split_pol(bl[2])[1])]
antpols = sorted(set([ant[1] for ant in ants]))
In [8]:
# print basic information about the file
print(f'File: {SUM_FILE}')
print(f'JDs: {hd.times} ({np.median(np.diff(hd.times)) * 24 * 3600:.5f} s integrations)')
print(f'LSTS: {hd.lsts * 12 / np.pi } hours')
print(f'Frequencies: {len(hd.freqs)} {np.median(np.diff(hd.freqs)) / 1e6:.5f} MHz channels from {hd.freqs[0] / 1e6:.5f} to {hd.freqs[-1] / 1e6:.5f} MHz')
print(f'Antennas: {len(hd.data_ants)}')
print(f'Polarizations: {hd.pols}')

File: /mnt/sn1/2459847/zen.2459847.27893.sum.uvh5
JDs: [2459847.27887263 2459847.27898448] (9.66368 s integrations)
LSTS: [20.35994958 20.36264129] hours
Frequencies: 1536 0.12207 MHz channels from 46.92078 to 234.29871 MHz
Antennas: 180
Polarizations: ['nn', 'ee', 'ne', 'en']

Classify good, suspect, and bad antpols¶

Run ant_metrics¶

This classifies antennas as cross-polarized, low-correlation, or dead. Such antennas are excluded from any calibration.

In [9]:
am = ant_metrics.AntennaMetrics(SUM_FILE, DIFF_FILE, sum_data=data, diff_data=diff_data)
am.iterative_antenna_metrics_and_flagging(crossCut=am_xpol_bad[1], deadCut=am_corr_bad[1])
In [10]:
# Turn ant metrics into classifications
totally_dead_ants = [ant for ant, i in am.removal_iteration.items() if i == -1]
am_totally_dead = ant_class.AntennaClassification(good=[ant for ant in ants if ant not in totally_dead_ants], bad=totally_dead_ants)
am_corr = ant_class.antenna_bounds_checker(am.final_metrics['corr'], bad=[am_corr_bad], suspect=[am_corr_suspect], good=[(0, 1)])
am_xpol = ant_class.antenna_bounds_checker(am.final_metrics['corrXPol'], bad=[am_xpol_bad], suspect=[am_xpol_suspect], good=[(-1, 1)])
ant_metrics_class = am_totally_dead + am_corr + am_xpol

Classify antennas responsible for 0s in visibilities as bad:¶

This classifier looks for X-engine failure or packet loss specific to an antenna which causes either the even visibilities (or the odd ones, or both) to be 0s.

In [11]:
zeros_class = ant_class.even_odd_zeros_checker(data, diff_data, good=good_zeros_per_eo_spectrum, suspect=suspect_zeros_per_eo_spectrum)

Examine and classify autocorrelation power, slope, and RFI occpancy¶

These classifiers look for antennas with too high or low power, to steep a slope, or too much excess RFI.

In [12]:
auto_power_class = ant_class.auto_power_checker(data, good=auto_power_good, suspect=auto_power_suspect)
auto_slope_class = ant_class.auto_slope_checker(data, good=auto_slope_good, suspect=auto_slope_suspect, edge_cut=100, filt_size=17)
auto_rfi_class = ant_class.auto_rfi_checker(data, good=auto_rfi_good, suspect=auto_rfi_suspect)
auto_class = auto_power_class + auto_slope_class + auto_rfi_class

/home/obs/mambaforge/envs/RTP/lib/python3.10/site-packages/hera_filters/dspec.py:2000: ComplexWarning: Casting complex values to real discards the imaginary part
  model[i], _, info_t = _fit_basis_1d(x=x[1], y=_y, w=_w, filter_centers=filter_centers[1],
In [13]:
def autocorr_plot():    
    fig, axes = plt.subplots(1, 2, figsize=(14, 5), dpi=100, sharey=True, gridspec_kw={'wspace': 0})
    labels = []
    colors = ['darkgreen', 'goldenrod', 'maroon']
    for ax, pol in zip(axes, antpols):
        for ant in auto_class.ants:
            if ant[1] == pol:
                color = colors[auto_class.quality_classes.index(auto_class[ant])]
                ax.semilogy(np.mean(data[utils.join_bl(ant, ant)], axis=0), color=color, lw=.5)
        ax.set_xlabel('Channel', fontsize=12)
        ax.set_title(f'{utils.join_pol(pol, pol)}-Polarized Autos')

    axes[0].set_ylabel('Raw Autocorrelation', fontsize=12)
    axes[1].legend([matplotlib.lines.Line2D([0], [0], color=color) for color in colors], 
                   [cls.capitalize() for cls in auto_class.quality_classes], ncol=1, fontsize=12, loc='upper right', framealpha=1)
    plt.tight_layout()

Figure 1: Plot of autocorrelations with classifications¶

This figure shows a plot of all autocorrelations in the array, split by polarization. Antennas are classified based on their autocorrelations into good, suspect, and bad, by examining power, slope, and RFI-occupancy.

In [14]:
if PLOT: autocorr_plot()

Summarize antenna classification prior to redundant-baseline calibration¶

In [15]:
final_class = ant_metrics_class + zeros_class + auto_class
In [16]:
def array_class_plot():
    fig, axes = plt.subplots(1, 2, figsize=(14, 6), dpi=100, gridspec_kw={'width_ratios': [2, 1]})
    plot_antclass(hd.antpos, final_class, ax=axes[0], ants=[ant for ant in hd.data_ants if ant < 320], legend=False, title='HERA Core')
    plot_antclass(hd.antpos, final_class, ax=axes[1], ants=[ant for ant in hd.data_ants if ant >= 320], radius=50, title='Outriggers')

Figure 2: Summary of antenna classifications prior to calibration¶

This figure shows the location and classification of all antennas prior to calibration. Antennas are split along the diagonal, with ee-polarized antpols represented by the southeast half of each antenna and nn-polarized antpols represented by the northwest half. Outriggers are split from the core and shown at exaggerated size in the right-hand panel. This classification includes ant_metrics, a count of the zeros in the even or odd visibilities, and autocorrelation power, slope, and RFI occupancy. An antenna classified as bad in any classification will be considered bad. An antenna marked as suspect any in any classification will be considered suspect unless it is also classified as bad elsewhere.

In [17]:
if PLOT: array_class_plot()

Perform redundant-baseline calibration¶

In [18]:
def classify_off_grid(reds, all_ants):
    '''Returns AntennaClassification of all_ants where good ants are in reds while bad ants are not.'''
    ants_in_reds = set([ant for red in reds for bl in red for ant in utils.split_bl(bl)])
    on_grid = [ant for ant in all_ants if ant in ants_in_reds]
    off_grid = [ant for ant in all_ants if ant not in ants_in_reds]
    return ant_class.AntennaClassification(good=on_grid, bad=off_grid)

Perform iterative redcal¶

In [19]:
redcal_start = time.time()
rc_settings = {'fc_conv_crit': 1e-6, 'fc_maxiter': 1, 'fc_min_vis_per_ant': 100, 'max_dims': OC_MAX_DIMS,
               'oc_conv_crit': 1e-10, 'gain': 0.4, 'oc_maxiter': OC_MAXITER, 'check_after': OC_MAXITER}

# figure out and filter reds and classify antennas based on whether or not they are on the main grid
reds = redcal.get_reds(hd.data_antpos, pols=['ee', 'nn'], pol_mode='2pol')
reds = redcal.filter_reds(reds, ex_ants=final_class.bad_ants, max_dims=OC_MAX_DIMS, min_dim_size=1)
if OC_SKIP_OUTRIGGERS:
    reds = redcal.filter_reds(reds, ex_ants=[ant for ant in ants if ant[0] >= 320])
redcal_class = classify_off_grid(reds, ants)

# perform first stage of redundant calibration, 
cal = redcal.redundantly_calibrate(data, reds, **rc_settings)
max_dly = np.max(np.abs(list(cal['fc_meta']['dlys'].values())))
med_cspa = {ant: np.median(cal['chisq_per_ant'][ant]) for ant in cal['chisq_per_ant']}
cspa_class = ant_class.antenna_bounds_checker(med_cspa, good=np.array(oc_cspa_good)*2, suspect=np.array(oc_cspa_suspect)*2, bad=(0, np.inf))
redcal_class += cspa_class

# iteratively rerun redundant calibration
for i in range(OC_MAX_RERUN):
    # build RedDataContainer of old visibility solution
    prior_vis = datacontainer.RedDataContainer(cal['v_omnical'], reds)
    
    # refilter reds and update classification to reflect new off-grid ants, if any
    reds = redcal.filter_reds(reds, ex_ants=(final_class + redcal_class).bad_ants, max_dims=OC_MAX_DIMS, min_dim_size=1)
    reds = sorted(reds, key=len, reverse=True)
    redcal_class += classify_off_grid(reds, ants)
    ants_in_reds = set([ant for red in reds for bl in red for ant in utils.split_bl(bl)])    
   
    # re-run redundant calibration using previous solution, updating bad and suspicious antennas
    prior_sol = redcal.RedSol(reds, gains={ant: cal['g_omnical'][ant] for ant in ants_in_reds}, 
                              vis={red[0]: prior_vis[red[0]] for red in reds})    
    cal = redcal.redundantly_calibrate(data, reds, prior_firstcal=prior_sol, prior_sol=prior_sol, **rc_settings)
    med_cspa = {ant: np.median(cal['chisq_per_ant'][ant]) for ant in cal['chisq_per_ant']}
    cspa_class = ant_class.antenna_bounds_checker(med_cspa, good=oc_cspa_good, suspect=oc_cspa_suspect, bad=(0, np.inf))    
    redcal_class += cspa_class
    if len(cspa_class.bad_ants) == 0:
        break  # no new antennas to flag
final_class += redcal_class
print(f'Finished redcal in {(time.time() - redcal_start) / 60:.2f} minutes.')

Finished redcal in 6.57 minutes.

Fix the firstcal delay slope degeneracy using RFI transmitters¶

In [20]:
# find channels clearly contaminated by RFI
not_bad_ants = [ant for ant in final_class.ants if final_class[ant] != 'bad']
chan_flags = np.mean([xrfi.detrend_medfilt(data[utils.join_bl(ant, ant)], Kf=8, Kt=2) for ant in not_bad_ants], axis=(0, 1)) > 5

# hardcoded RFI transmitters and their headings
# channel: frequency (Hz), heading (rad), chi^2
phs_sol = {359: ( 90744018.5546875, 0.7853981, 23.3),
           360: ( 90866088.8671875, 0.7853981, 10.8),
           385: ( 93917846.6796875, 0.7853981, 27.3),
           386: ( 94039916.9921875, 0.7853981, 18.1),
           400: ( 95748901.3671875, 6.0632738, 24.0),
           441: (100753784.1796875, 0.7853981, 21.7),
           442: (100875854.4921875, 0.7853981, 19.4),
           455: (102462768.5546875, 6.0632738, 18.8),
           456: (102584838.8671875, 6.0632738,  8.8),
           471: (104415893.5546875, 0.7853981, 13.3),
           484: (106002807.6171875, 6.0632738, 21.2),
           485: (106124877.9296875, 6.0632738,  4.0),
          1181: (191085815.4296875, 0.7853981, 26.3),
          1182: (191207885.7421875, 0.7853981, 27.0),
          1183: (191329956.0546875, 0.7853981, 25.6),
          1448: (223678588.8671875, 2.6075219, 25.7),
          1449: (223800659.1796875, 2.6075219, 22.6),
          1450: (223922729.4921875, 2.6075219, 11.6),
          1451: (224044799.8046875, 2.6075219,  5.9),
          1452: (224166870.1171875, 2.6075219, 22.6),
          1510: (231246948.2421875, 0.1068141, 23.9)}
rfi_chans = [chan for chan in phs_sol if chan_flags[chan]]
print('Channels used for delay-slope calibration with RFI:', rfi_chans)
rfi_angles = np.array([phs_sol[chan][1] for chan in rfi_chans])
rfi_headings = np.array([np.cos(rfi_angles), np.sin(rfi_angles), np.zeros_like(rfi_angles)])
rfi_chisqs = np.array([phs_sol[chan][2] for chan in rfi_chans])

Channels used for delay-slope calibration with RFI: [359, 360, 385, 386, 400, 441, 442, 455, 456, 471, 484, 485]
In [21]:
# resolve firstcal degeneracy with delay slopes set by RFI transmitters, update cal
RFI_dly_slope_gains = abscal.RFI_delay_slope_cal(reds, hd.antpos, cal['v_omnical'], hd.freqs, rfi_chans, rfi_headings, rfi_wgts=rfi_chisqs**-.5,
                                                 min_tau=-max_dly, max_tau=max_dly, delta_tau=0.1e-9, return_gains=True, gain_ants=cal['g_omnical'].keys())
cal['g_omnical'] = {ant: g * RFI_dly_slope_gains[ant] for ant, g in cal['g_omnical'].items()}
apply_cal.calibrate_in_place(cal['v_omnical'], RFI_dly_slope_gains)

TODO: Perform approximate absolute amplitude calibration using a model of autocorrelations¶

In [22]:
# TODO
In [23]:
def redundant_group_plot():
    fig, axes = plt.subplots(2, 2, figsize=(14, 6), dpi=100, sharex='col', sharey='row', gridspec_kw={'hspace': 0, 'wspace': 0})
    for i, pol in enumerate(['ee', 'nn']):
        reds_here = redcal.get_reds(hd.data_antpos, pols=[pol], pol_mode='1pol')
        red = sorted(redcal.filter_reds(reds_here, ex_ants=final_class.bad_ants), key=len, reverse=True)[0]
        rc_data = {bl: np.array(data[bl]) for bl in red}
        apply_cal.calibrate_in_place(rc_data, cal['g_omnical'])
        for bl in red:
            axes[0, i].plot(hd.freqs/1e6, np.angle(np.mean(rc_data[bl], axis=0)), alpha=.5, lw=.5)
            axes[1, i].semilogy(hd.freqs/1e6, np.abs(np.mean(rc_data[bl], axis=0)), alpha=.5, lw=.5)
        axes[0, i].plot(hd.freqs / 1e6, np.angle(np.mean(cal['v_omnical'][red[0]], axis=0)), lw=2, c='k')
        axes[1, i].semilogy(hd.freqs / 1e6, np.abs(np.mean(cal['v_omnical'][red[0]], axis=0)), lw=2, c='k', label=f'Baseline Group:\n{red[0]}')

        axes[1, i].set_xlabel('Frequency (MHz)')
        axes[1, i].legend(loc='upper right')
    axes[0, 0].set_ylabel('Visibility Phase')
    axes[1, 0].set_ylabel('Visibility Amplitude')
    plt.tight_layout()

Figure 3: Redundant calibration of a single baseline group¶

The results of a redundant-baseline calibration of a single group, the one with the highest redundancy in each polarization after antenna classification and excision based on the above, plus the removal of antennas with high chi^2 per antenna. The black line is the redundant visibility solution. Each thin colored line is a different baseline group. Phases are shown in the top row, amplitudes in the bottom, ee-polarized visibilities in the left column, and nn-polarized visibilities in the right.

In [24]:
if PLOT: redundant_group_plot()

Attempt to calibrate some flagged antennas¶

This attempts to calibrate bad antennas using information from good or suspect antennas without allowing bad antennas to affect their calibration. However antennas flagged for ant_metrics or lots of zeros in the even or odd visibilities are considered beyond saving. Likewise, some antennas would add extra degeneracies (controlled by OC_MAX_DIMS, OC_MIN_DIM_SIZE, and OC_SKIP_OUTRIGGERS) are excluded.

In [25]:
expand_start = time.time()
expanded_reds = redcal.get_reds(hd.data_antpos, pols=['ee', 'nn'], pol_mode='2pol')
expanded_reds = redcal.filter_reds(expanded_reds, ex_ants=(ant_metrics_class + zeros_class).bad_ants, max_dims=OC_MAX_DIMS, min_dim_size=OC_MIN_DIM_SIZE)
if OC_SKIP_OUTRIGGERS:
    expanded_reds = redcal.filter_reds(expanded_reds, ex_ants=[ant for ant in ants if ant[0] >= 320])
nsamples = datacontainer.DataContainer({bl: np.ones_like(data[bl], dtype=float) for bl in data})
redcal.expand_omni_sol(cal, expanded_reds, data, nsamples)
print(f'Finished expanding omni_sol in {(time.time() - expand_start) / 60:.2f} minutes.')

Finished expanding omni_sol in 2.66 minutes.
In [26]:
def array_chisq_plot():
    fig, axes = plt.subplots(1, 2, figsize=(14, 5), dpi=100)
    for ax, pol in zip(axes, ['ee', 'nn']):
        ants_to_plot = set([ant for ant in cal['chisq_per_ant'] if utils.join_pol(ant[1], ant[1]) == pol])
        cspas = [np.median(cal['chisq_per_ant'][ant]) for ant in ants_to_plot]
        xpos = [hd.antpos[ant[0]][0] for ant in ants_to_plot]
        ypos = [hd.antpos[ant[0]][1] for ant in ants_to_plot]
        scatter = ax.scatter(xpos, ypos, s=300, c=cspas, norm=matplotlib.colors.LogNorm(vmin=1, vmax=10))
        for ant in ants_to_plot:
            ax.text(hd.antpos[ant[0]][0], hd.antpos[ant[0]][1], ant[0], va='center', ha='center', fontsize=9,
                    c=('r' if ant in final_class.bad_ants else 'w'))
        plt.colorbar(scatter, ax=ax, extend='both')
        ax.axis('equal')
        ax.set_xlabel('East-West Position (meters)')
        ax.set_ylabel('North-South Position (meters)')
        ax.set_title(f'{pol}-pol $\\chi^2$ / Antenna (Red is Flagged)')
    plt.tight_layout()

Figure 4: chi^2 per antenna across the array¶

This plot shows median (taken over time and frequency) of the normalized chi^2 per antenna. The expectation value for this quantity when the array is perfectly redundant is 1.0. Antennas that are classified as bad for any reason have their numbers shown in red. Some of those antennas were classified as bad during redundant calibration for high chi^2. Some of those antennas were originally excluded from redundant calibration because they were classified as bad earlier for some reason. See here for more details. Note that the color scale saturates at below 1 and above 10.

In [27]:
if PLOT: array_chisq_plot()
In [28]:
def array_class_after_redcal_plot():
    fig, axes = plt.subplots(1, 2, figsize=(14, 6), dpi=100, gridspec_kw={'width_ratios': [2, 1]})
    plot_antclass(hd.antpos, final_class, ax=axes[0], ants=[ant for ant in hd.data_ants if ant < 320], legend=False, title='HERA Core, Post-Redcal')
    plot_antclass(hd.antpos, final_class, ax=axes[1], ants=[ant for ant in hd.data_ants if ant >= 320], radius=50, title='Outriggers')

Figure 5: Summary of antenna classifications after redundant calibration¶

This figure is the same as Figure 2, except that it now includes additional suspect or bad antennas based on redundant calibration. This can include antennas with high chi^2, but it can also include antennas classified as "bad" because they would add extra degeneracies to calibration.

In [29]:
if PLOT: array_class_after_redcal_plot()
In [30]:
to_show = {'Antenna': [f'{ant[0]}{ant[1][-1]}' for ant in ants]}
classes = {'Antenna': [final_class[ant] if ant in final_class else '-' for ant in ants]}
to_show['Dead?'] = [{'good': 'No', 'bad': 'Yes'}[am_totally_dead[ant]] if (ant in am_totally_dead) else '' for ant in ants]
classes['Dead?'] = [am_totally_dead[ant] if (ant in am_totally_dead) else '' for ant in ants]
for title, ac in [('Low Correlation', am_corr),
                  ('Cross-Polarized', am_xpol),
                  ('Even/Odd Zeros', zeros_class),
                  ('Autocorr Power', auto_power_class),
                  ('Autocorr Slope', auto_slope_class),
                  ('RFI in Autos', auto_rfi_class)]:
    to_show[title] = [f'{ac._data[ant]:.2G}' if (ant in ac._data) else '' for ant in ants]
    classes[title] = [ac[ant] if ant in ac else '' for ant in ants]
    
to_show['Redcal chi^2'] = [f'{np.median(cal["chisq_per_ant"][ant]):.3G}' if (ant in cal['chisq_per_ant']) else '-' for ant in ants]
classes['Redcal chi^2'] = [redcal_class[ant] if ant in redcal_class else '' for ant in ants]

df = pd.DataFrame(to_show)
df2 = pd.DataFrame(classes)
colors = {'good': 'darkgreen', 'suspect': 'goldenrod', 'bad': 'maroon'}
df2 = df2.applymap(lambda x: f'background-color: {colors.get(x, None)}')

table = df.style.hide_index() \
                .apply(lambda x: pd.DataFrame(df2.values, columns=x.columns), axis=None) \
                .set_properties(subset=['Antenna'], **{'font-weight': 'bold', 'border-right': "3pt solid black"}) \
                .set_properties(subset=df.columns[1:], **{'border-left': "1pt solid black"}) \
                .set_properties(**{'text-align': 'center', 'color': 'white'}) \
                .set_sticky(axis=1)

/tmp/ipykernel_23460/3939497990.py:22: FutureWarning: this method is deprecated in favour of `Styler.hide(axis='index')`
  table = df.style.hide_index() \

Table 1: Complete summary of per-antenna classifications¶

This table summarizes the results of the various classifications schemes detailed above. As before, green is good, yellow is suspect, and red is bad. The color for each antenna (first column) is the final summary of all other classifications. Antennas missing from redcal χ2χ2 were excluded redundant-baseline calibration, either because they were flagged by ant_metrics or the even/odd zeros check, or because they would add unwanted extra degeneracies.

In [31]:
HTML(table.render())

/tmp/ipykernel_23460/830379045.py:1: FutureWarning: this method is deprecated in favour of `Styler.to_html()`
  HTML(table.render())
Out[31]:
Antenna Dead? Low Correlation Cross-Polarized Even/Odd Zeros Autocorr Power Autocorr Slope RFI in Autos Redcal chi^2
3e No 0.86 0.63 0 7.8 0.15 0.028 1.54
3n No 0.52 0.63 0 7.5 -0.057 0.028 1.4
4e No 0.87 0.64 0 6.6 -0.014 0.062 1.43
4n No 0.52 0.64 0 8.2 0.088 0.043 1.42
5e No 0.87 0.63 0 6.6 -0.017 0.031 1.42
5n No 0.54 0.63 0 11 -0.088 0.027 1.41
7e No 0.87 0.62 0 7.6 0.038 0.029 1.44
7n No 0.53 0.62 1 7 -0.037 0.038 1.44
8e No 0.87 0.63 0 60 -0.0038 0.045 1.47
8n No 0.51 0.63 0 63 -0.0024 0.017 1.44
9e No 0.88 0.64 0 7.5 0.039 0.028 1.55
9n No 0.54 0.64 0 9.1 -0.039 0.053 1.43
10e No 0.87 0.65 0 6.7 0.03 0.038 1.41
10n No 0.52 0.65 0 6.9 -0.026 0.028 1.44
15e No 0.87 0.64 0 7.4 0.035 0.029 1.46
15n No 0.53 0.64 0 8.1 -0.06 0.026 1.44
16e No 0.87 0.63 0 7 0.0078 0.026 1.46
16n No 0.54 0.63 0 7.8 3.9E-05 0.026 1.4
17e No 0.87 0.62 0 6.3 -0.028 0.026 1.52
17n No 0.54 0.62 0 6.9 -0.011 0.026 1.46
18e No 0.85 0.67 0 7.2 -0.068 0.36 1.74
18n No 0.33 0.67 0 9.2 0.12 0.23 1.41
19e No 0.87 0.63 0 7.8 0.021 0.036 1.39
19n No 0.54 0.63 0 8.8 -0.071 0.027 1.39
20e No 0.87 0.63 0 7 -0.014 0.03 1.47
20n No 0.53 0.63 0 8.1 0.057 0.026 1.43
21e No 0.87 0.65 0 7 0.05 0.027 1.44
21n No 0.53 0.65 0 7.9 -0.067 0.028 1.53
22e No 0.69 0.44 0 13 0.86 0.036 2.37
22n No 0.5 0.44 0 25 0.31 0.029 1.91
27e No 0.031 0.0039 0 0.67 0.51 0.098 -
27n No 0.036 0.0039 0 0.64 0.55 0.07 -
28e No 0.44 0.31 0 8.7 0.45 0.021 3.64
28n No 0.16 0.31 0 11 0.69 0.23 -
29e No 0.88 0.62 0 8.3 -0.049 0.029 1.41
29n No 0.54 0.62 0 9.1 -0.055 0.025 1.41
30e No 0.87 0.61 0 6 0.053 0.028 1.39
30n No 0.55 0.61 0 7.8 -0.078 0.026 1.38
31e No 0.88 0.62 0 7.7 -0.097 0.027 1.37
31n No 0.55 0.62 0 8.1 -0.025 0.027 1.36
32e No 0.79 0.47 0 10 0.62 0.035 2.02
32n No 0.49 0.47 0 10 0.73 0.033 2.45
33e No 0.86 0.71 0 6.7 0.019 0.029 1.42
33n No 0.35 0.71 0 7.8 0.057 0.31 1.72
34e No 0.038 0.41 0 2.9 0.57 0.049 -
34n No 0.51 0.41 0 20 0.013 0.025 1.83
35e No 0.84 0.64 0 9.3 0.097 0.047 1.61
35n No 0.51 0.64 0 17 0.012 0.025 1.74
36e No 0.87 0.64 0 7.7 -0.36 0.029 1.53
36n No 0.53 0.64 0 8.8 -0.32 0.027 1.46
37e No 0.87 0.63 0 7.6 -0.035 0.026 1.48
37n No 0.54 0.63 0 9.3 -0.036 0.028 1.44
38e No 0.87 0.62 0 6.7 -0.0044 0.027 1.44
38n No 0.55 0.62 0 7.4 -0.1 0.026 1.41
40e No 0.87 0.61 0 8.1 -0.004 0.028 1.56
40n No 0.54 0.61 1 8.2 0.016 0.026 1.48
41e No 0.88 0.63 0 11 -0.097 0.029 1.8
41n No 0.53 0.63 0 7.1 -0.022 0.026 1.6
42e No 0.88 0.62 0 9.6 -0.041 0.026 1.53
42n No 0.55 0.62 0 7 0.00068 0.025 1.5
43e No 0.027 0.017 1 0.7 0.47 0.053 -
43n No 0.054 0.017 0 26 0.046 0.028 -
44e No 0.041 0.0018 0 9.4 0.23 0.037 -
44n No 0.048 0.0018 0 8.3 -0.016 0.034 -
45e No 0.047 0.0025 0 7.8 -0.082 0.03 -
45n No 0.042 0.0025 0 7.8 -0.073 0.027 -
46e No 0.036 0.02 0 9.8 -0.012 0.027 -
46n No 0.026 0.02 1 0.64 0.57 0.094 -
47e No 0.035 0.42 0 3 0.55 0.031 -
47n No 0.51 0.42 0 21 0.034 0.023 1.68
48e No 0.87 0.66 0 35 0.046 0.024 1.72
48n No 0.52 0.66 0 41 -0.0039 0.022 1.74
49e No 0.86 0.65 0 17 0.011 0.029 1.45
49n No 0.51 0.65 0 35 -0.02 0.023 1.62
50e No 0.87 0.62 0 8 -0.034 0.027 1.74
50n No 0.44 0.62 0 5.5 0.64 0.045 2.41
51e No 0.034 0.38 0 0.34 1 0.21 -
51n No 0.55 0.38 0 6.8 -0.089 0.12 1.38
52e No 0.88 0.62 0 7.6 -0.34 0.028 1.52
52n No 0.55 0.62 0 7.8 -0.32 0.026 1.42
53e No 0.88 0.62 0 6.9 -0.059 0.035 1.61
53n No 0.56 0.62 0 7.2 -0.19 0.027 1.47
54e No 0.041 0.43 0 0.68 0.51 0.051 -
54n No 0.55 0.43 0 8.4 -0.05 0.026 1.57
55e No 0.87 0.61 0 9 0.054 0.027 1.64
55n No 0.54 0.61 0 7.4 0.059 0.026 1.57
56e No 0.87 0.61 0 9.5 0.0022 0.026 1.92
56n No 0.55 0.61 0 11 -0.011 0.026 1.59
57e No 0.74 0.5 0 3.7 0.87 0.065 2.29
57n No 0.55 0.5 0 7.7 -0.012 0.026 1.44
58e No 0.03 0.0042 0 0.69 0.49 0.12 -
58n No 0.025 0.0042 0 0.62 0.57 0.076 -
59e No 0.051 0.0037 0 8.1 0.77 0.027 -
59n No 0.036 0.0037 0 6.7 0.093 0.027 -
60e No 0.026 0.00064 0 0.69 0.52 0.048 -
60n No 0.025 0.00064 1 0.63 0.56 0.041 -
61e No 0.85 0.64 0 15 0.13 0.027 1.42
61n No 0.49 0.64 0 14 0.13 0.024 1.48
62e No 0.87 0.64 0 29 -0.016 0.026 1.47
62n No 0.52 0.64 0 38 -0.053 0.022 1.48
63e No 0.86 0.77 0 29 0.078 0.026 1.43
63n No 0.039 0.77 0 2.9 0.57 0.072 -
64e No 0.85 0.66 0 20 0.045 0.032 1.43
64n No 0.49 0.66 0 32 -0.025 0.023 1.44
65e No 0.87 0.63 0 7.3 -0.0062 0.028 1.48
65n No 0.54 0.63 0 6.7 -0.068 0.026 1.42
66e No 0.87 0.61 0 7 -0.074 0.026 1.52
66n No 0.55 0.61 0 6.7 -0.13 0.027 1.41
67e No 0.87 0.61 0 7.1 -0.036 0.028 1.58
67n No 0.56 0.61 0 8 -0.05 0.026 1.44
68e No 0.87 0.66 0 7.2 0.049 0.027 1.56
68n No 0.027 0.66 0 0.26 1.1 0.24 -
69e No 0.87 0.61 0 9.5 0.022 0.026 1.87
69n No 0.55 0.61 0 7.6 -0.014 0.025 1.5
70e No 0.88 0.6 0 8.8 -0.35 0.026 15.1
70n No 0.56 0.6 0 8.5 -0.062 0.027 10.5
71e No 0.87 0.6 0 7.5 -0.029 0.028 14.5
71n No 0.56 0.6 0 9 -0.012 0.027 10.9
72e No 0.87 0.6 0 7 0.15 0.026 1.69
72n No 0.56 0.6 0 11 -0.072 0.028 1.62
73e No 0.027 0.02 0 0.7 0.48 0.1 -
73n No 0.054 0.02 0 7.6 -0.042 0.026 -
74e No 0.025 0.0094 0 0.66 0.53 0.052 -
74n No 0.038 0.0094 0 0.63 0.56 0.055 -
75e Yes 1.5E+03 0 NAN 0 -
75n Yes 1.5E+03 0 NAN 0 -
77e No 0.78 0.5 0 30 0.68 0.025 2.67
77n No 0.36 0.5 0 24 0.64 0.027 2.22
78e No 0.66 0.43 0 25 0.81 0.025 2.74
78n No 0.5 0.43 0 29 -0.041 0.022 1.37
81e No 0.86 0.64 0 7 -0.053 0.028 1.52
81n No 0.52 0.64 0 7.2 -0.054 0.026 1.49
82e No 0.87 0.62 0 12 -0.033 0.034 1.54
82n No 0.52 0.62 1 5 -0.099 0.026 1.39
83e No 0.88 0.61 0 13 -0.093 0.026 1.49
83n No 0.55 0.61 0 11 -0.081 0.025 1.43
84e No 0.88 0.75 0 8.4 -0.36 0.033 1.61
84n No 0.035 0.75 0 0.27 0.97 0.15 -
85e No 0.87 0.61 0 8.1 -0.048 0.026 1.45
85n No 0.56 0.61 0 8.1 -0.1 0.029 1.4
86e No 0.87 0.62 0 9.5 -0.1 0.028 1.81
86n No 0.51 0.62 0 6.6 0.12 0.028 1.49
87e No 0.88 0.61 0 15 -0.3 0.029 1.52
87n No 0.56 0.61 0 5.2 -0.37 0.027 1.53
88e No 0.87 0.6 0 31 -0.074 0.024 1.57
88n No 0.56 0.6 0 20 -0.18 0.024 1.54
89e No 0.88 0.63 0 15 -0.11 0.028 1.77
89n No 0.53 0.63 0 6.4 -0.088 0.027 1.52
90e No 0.87 0.62 0 6.6 -0.059 0.028 1.6
90n No 0.53 0.62 0 7.5 -0.035 0.027 1.48
91e Yes 1.5E+03 0 NAN 0 -
91n Yes 1.5E+03 0 NAN 0 -
92e No 0.43 0.24 0 12 1.3 0.027 2.88
92n No 0.29 0.24 0 13 1.7 0.026 2.94
93e No 0.86 0.62 0 3.6 0.1 0.034 1.43
93n No 0.53 0.62 0 7 0.014 0.025 1.4
94e No 0.87 0.62 0 7.2 -0.049 0.028 1.41
94n No 0.52 0.62 0 7.6 -0.024 0.025 1.36
98e No 0.85 0.63 0 7 -0.038 0.029 1.51
98n No 0.5 0.63 0 7.8 -0.081 0.027 1.54
99e No 0.83 0.62 0 3.1 0.11 0.024 1.46
99n No 0.52 0.62 0 7.2 -0.078 0.025 1.44
100e No 0.87 0.63 0 8 0.035 0.026 1.52
100n No 0.52 0.63 0 7.1 -0.024 0.028 1.42
101e No 0.88 0.62 0 15 -0.34 0.042 1.53
101n No 0.56 0.62 0 11 -0.4 0.026 1.74
102e No 0.65 0.58 0 0.98 0.37 0.05 1.35
102n No 0.037 0.58 1 0.72 0.57 0.16 -
103e No 0.025 0.004 0 0.49 0.96 0.23 -
103n No 0.028 0.004 0 0.41 0.99 0.24 -
104e No 0.25 -0.27 0 7.7 -0.29 0.026 -
104n No 0.22 -0.27 0 1.4 1.7 0.033 -
105e Yes 1.5E+03 0 NAN 0 -
105n Yes 1.5E+03 0 NAN 0 -
106e No 0.87 0.61 0 7.9 -0.092 0.031 1.48
106n No 0.55 0.61 0 26 -0.17 0.023 1.48
107e No 0.87 0.6 0 7.7 0.15 0.044 1.66
107n No 0.53 0.6 0 8 -0.15 0.08 1.55
108e Yes 1.5E+03 0 NAN 0 -
108n Yes 1.5E+03 0 NAN 0 -
109e No 0.88 0.67 0 8.9 -0.019 0.026 1.53
109n No 0.033 0.67 0 0.67 0.56 0.043 -
110e No 0.77 0.47 0 11 1 0.025 2.51
110n No 0.55 0.47 0 11 -0.077 0.027 1.44
111e No 0.87 0.72 0 7.5 -0.025 0.029 1.5
111n No 0.11 0.72 1 0.68 0.52 0.059 -
112e No 0.87 0.63 0 5.3 0.024 0.027 1.43
112n No 0.53 0.63 0 9.1 -0.04 0.027 1.39
116e No 0.22 -0.28 0 11 0.028 0.028 -
116n No 0.21 -0.28 0 7.5 -0.049 0.027 -
117e No 0.025 0.0021 0 0.68 0.55 0.024 -
117n No 0.029 0.0021 0 0.59 0.62 0.07 -
118e No 0.87 0.62 0 5.3 -0.0081 0.024 1.5
118n No 0.53 0.62 0 6.7 -0.043 0.027 1.41
119e No 0.88 0.64 0 18 -0.081 0.091 1.56
119n No 0.51 0.64 0 6.9 0.00042 0.069 1.43
120e No 0.88 0.75 0 8.7 -0.062 0.027 1.53
120n No 0.03 0.75 0 0.3 0.95 0.22 -
121e No 0.88 0.62 0 7.4 0.012 0.036 1.52
121n No 0.55 0.62 0 7.3 -0.31 0.027 1.41
122e No 0.88 0.62 0 6.6 -0.38 0.028 1.77
122n No 0.55 0.62 0 7.4 -0.37 0.027 1.45
123e No 0.88 0.62 0 12 -0.35 0.028 1.53
123n No 0.55 0.62 0 7.2 -0.42 0.029 1.78
124e No 0.24 -0.27 0 12 -0.045 0.026 -
124n No 0.24 -0.27 0 31 -0.076 0.022 -
125e No 0.87 0.61 0 8.4 -0.15 0.027 1.56
125n No 0.54 0.61 0 11 -0.094 0.025 1.5
126e No 0.85 0.61 0 13 0.36 0.026 1.51
126n No 0.53 0.61 0 8.6 -0.19 0.028 1.63
127e No 0.86 0.62 0 6.7 -0.049 0.028 1.51
127n No 0.54 0.62 0 6.9 -0.04 0.028 1.46
128e No 0.87 0.62 0 7.4 -0.019 0.024 1.76
128n No 0.54 0.62 0 10 0.078 0.026 1.57
129e No 0.87 0.63 0 8 -0.0052 0.026 1.89
129n No 0.53 0.63 0 7.1 -0.062 0.026 1.58
130e No 0.86 0.62 0 7 0.079 0.024 1.55
130n No 0.53 0.62 0 6.8 0.018 0.026 1.41
135e No 0.86 0.66 0 6.7 -0.029 0.026 1.53
135n No 0.5 0.66 0 7.2 -0.016 0.028 1.64
136e No 0.87 0.65 0 7.6 -0.029 0.025 1.88
136n No 0.47 0.65 0 7.8 0.25 0.026 2.02
137e No 0.86 0.63 0 9 -0.035 0.03 1.66
137n No 0.51 0.63 0 8.4 -0.076 0.027 1.58
138e No 0.87 0.63 0 7.9 -0.028 0.027 1.59
138n No 0.53 0.63 0 8.4 -0.079 0.029 1.48
140e No 0.035 0.0022 1 0.72 0.5 0.047 -
140n No 0.034 0.0022 0 0.64 0.58 0.07 -
141e No 0.88 0.64 0 6.1 -0.02 0.024 2.03
141n No 0.49 0.64 0 3.7 0.12 0.51 2.06
142e No 0.44 0.29 0 8.8 0.93 0.022 4.27
142n No 0.032 0.29 0 0.63 0.56 0.058 -
143e No 0.87 0.6 0 7.6 -0.0051 0.025 1.55
143n No 0.54 0.6 0 8.7 -0.059 0.027 1.53
144e No 0.87 0.62 0 6.9 -0.011 0.024 1.58
144n No 0.53 0.62 0 6.9 -0.083 0.027 1.54
145e No 0.03 0.0041 0 0.67 0.53 0.058 -
145n No 0.025 0.0041 0 0.61 0.58 0.13 -
147e Yes 1.5E+03 0 NAN 0 -
147n Yes 1.5E+03 0 NAN 0 -
148e Yes 1.5E+03 0 NAN 0 -
148n Yes 1.5E+03 0 NAN 0 -
149e Yes 1.5E+03 0 NAN 0 -
149n Yes 1.5E+03 0 NAN 0 -
150e No 0.027 0.00067 0 0.68 0.54 0.089 -
150n No 0.028 0.00067 0 0.62 0.57 0.093 -
151e No 0.76 0.55 0 23 0.65 0.068 2.08
151n No 0.5 0.55 0 14 -0.0083 0.025 1.39
152e No 0.87 0.67 0 46 -0.018 0.023 1.34
152n No 0.5 0.67 0 52 -0.026 0.02 1.43
153e No 0.04 0.42 0 3.2 0.53 0.045 -
153n No 0.5 0.42 0 21 0.038 0.024 1.46
154e No 0.86 0.66 0 31 -0.00059 0.025 1.38
154n No 0.49 0.66 0 29 -0.029 0.024 1.44
155e No 0.034 0.0052 0 0.72 0.5 0.08 -
155n No 0.025 0.0052 0 0.65 0.53 0.052 -
156e No 0.88 0.65 0 7.5 -0.014 0.034 1.83
156n No 0.5 0.65 0 7 -0.06 0.029 1.61
157e No 0.87 0.64 0 7.4 -0.03 0.027 1.51
157n No 0.51 0.64 0 9.6 -0.05 0.026 1.54
158e No 0.036 0.37 0 0.66 0.52 0.047 -
158n No 0.53 0.37 0 7.2 -0.013 0.03 1.59
160e No 0.037 0.0042 1 0.69 0.55 0.086 -
160n No 0.039 0.0042 0 0.64 0.6 0.13 -
161e No 0.87 0.62 0 8.1 -0.0066 0.028 1.59
161n No 0.42 0.62 0 11 0.92 0.028 2.66
162e No 0.88 0.62 0 6.4 0.057 0.023 1.61
162n No 0.56 0.62 0 9.5 -0.013 0.026 1.54
163e No 0.87 0.62 0 6.7 -0.082 0.027 1.51
163n No 0.55 0.62 0 8 -0.12 0.026 1.52
164e No 0.87 0.62 0 6.8 0.04 0.025 1.53
164n No 0.54 0.62 0 7.2 -0.056 0.025 1.47
165e No 0.88 0.62 0 15 -0.088 0.044 1.47
165n No 0.55 0.62 0 7.2 -0.11 0.025 1.46
166e No 0.84 0.61 0 8.9 0.38 0.024 1.8
166n No 0.45 0.61 0 9.6 0.52 0.024 2.25
167e No 0.76 0.48 0 14 1.4 0.023 1.84
167n No 0.44 0.48 0 12 1 0.024 2.27
168e No 0.87 0.64 0 8.8 -0.04 0.024 1.64
168n No 0.55 0.64 0 14 -0.053 0.025 1.59
169e No 0.88 0.64 0 9.4 -0.029 0.026 1.63
169n No 0.53 0.64 0 9.9 0.073 0.024 1.51
170e No 0.87 0.63 0 15 0.03 0.025 1.41
170n No 0.53 0.63 0 8.5 0.039 0.024 1.44
171e No 0.86 0.66 0 17 0.0049 0.024 1.39
171n No 0.46 0.66 0 8.8 0.12 0.023 1.39
173e No 0.033 0.0028 0 3.4 0.6 0.15 -
173n No 0.034 0.0028 0 3.1 0.6 0.05 -
176e No 0.87 0.68 0 6.7 -0.057 0.03 1.77
176n No 0.48 0.68 0 6.6 -0.029 0.03 1.52
177e No 0.87 0.65 0 8.6 -0.065 0.028 1.57
177n No 0.5 0.65 0 7 0.032 0.028 1.54
178e No 0.87 0.65 0 7.5 0.035 0.027 1.44
178n No 0.52 0.65 0 7.3 -0.078 0.027 1.42
179e No 0.87 0.65 0 7.8 -0.011 0.026 1.38
179n No 0.53 0.65 0 8.1 -0.085 0.026 1.41
180e No 0.87 0.73 0 9 0.0013 0.026 10.8
180n No 0.21 0.73 1 0.71 0.46 0.067 1.95
181e No 0.041 0.21 0 0.65 0.54 0.07 -
181n No 0.31 0.21 0 18 1.5 0.02 2.74
182e No 0.88 0.74 0 48 -0.033 0.02 1.45
182n No 0.043 0.74 0 0.66 0.55 0.052 -
183e No 0.88 0.63 0 7.1 -0.019 0.027 1.47
183n No 0.55 0.63 0 7.3 -0.033 0.026 1.4
184e No 0.87 0.63 0 6.8 -0.054 0.027 1.59
184n No 0.54 0.63 0 7.8 -0.057 0.025 1.47
185e No 0.87 0.62 0 9.1 -0.1 0.029 1.46
185n No 0.54 0.62 0 7.8 -0.081 0.025 1.41
186e No 0.87 0.62 0 7.5 -0.094 0.035 1.41
186n No 0.54 0.62 0 7.3 -0.1 0.024 1.41
187e No 0.87 0.63 0 7.1 -0.036 0.028 1.59
187n No 0.54 0.63 0 8.5 -0.11 0.025 1.5
189e No 0.86 0.64 0 7 -0.044 0.024 1.98
189n No 0.52 0.64 0 6.6 -0.046 0.024 1.65
190e No 0.71 0.58 0 9.5 1.6 0.024 2.13
190n No 0.032 0.58 0 0.62 0.57 0.11 -
191e No 0.87 0.64 0 7.8 -0.015 0.025 1.42
191n No 0.52 0.64 0 7.5 0.0069 0.025 1.39
192e No 0.83 0.64 0 12 0.21 0.055 1.36
192n No 0.46 0.64 0 83 -0.0047 0.013 1.87
193e No 0.85 0.66 0 83 0.00026 0.011 2.23
193n No 0.5 0.66 0 42 -0.0076 0.021 1.43
200e No 0.047 0.15 0 3.1 0.57 0.059 -
200n No 0.23 0.15 0 37 0.96 0.016 3.57
201e No 0.85 0.65 0 82 -0.00079 0.013 2.59
201n No 0.49 0.65 0 75 -0.0071 0.016 1.73
202e No 0.87 0.65 0 33 0.027 0.021 1.51
202n No 0.47 0.65 0 9.5 0.13 0.035 1.48
203e Yes 1.5E+03 0 NAN 0 -
203n Yes 1.5E+03 0 NAN 0 -
219e Yes 1.5E+03 0 NAN 0 -
219n Yes 1.5E+03 0 NAN 0 -
220e No 0.86 0.64 0 24 0.02 0.022 1.4
220n No 0.51 0.64 0 28 -0.049 0.022 1.44
221e No 0.85 0.63 0 13 0.15 0.025 1.42
221n No 0.52 0.63 0 28 0.0094 0.021 1.41
222e No 0.86 0.64 0 28 0.021 0.021 1.45
222n No 0.51 0.64 0 34 -0.0091 0.021 1.4
237e No 0.85 0.66 0 13 0.073 0.023 1.56
237n No 0.48 0.66 0 22 0.0091 0.023 1.51
238e No 0.87 0.66 0 37 -0.014 0.021 1.35
238n No 0.49 0.66 0 65 -0.0014 0.017 1.38
239e No 0.87 0.68 0 25 0.0024 0.023 1.45
239n No 0.44 0.68 0 9.5 0.085 0.029 1.37
320e No 0.87 0.71 0 30 -0.024 0.024 -
320n No 0.044 0.71 0 1.6 0.59 0.14 -
321e No 0.86 0.72 0 25 -0.037 0.028 -
321n No 0.44 0.72 0 27 -0.063 0.027 -
322e Yes 1.5E+03 0 NAN 0 -
322n Yes 1.5E+03 0 NAN 0 -
323e No 0.63 0.5 0 8.8 0.66 0.052 -
323n No 0.42 0.5 0 41 -0.032 0.026 -
324e No 0.86 0.69 0 35 0.028 0.026 -
324n No 0.44 0.69 0 42 -0.003 0.023 -
325e No 0.87 0.72 0 36 -0.0062 0.028 -
325n No 0.45 0.72 0 21 0.0021 0.03 -
329e No 0.81 0.68 0 7.9 0.25 0.11 -
329n No 0.43 0.68 0 23 0.011 0.062 -
333e No 0.78 0.66 0 7.2 0.27 0.05 -
333n No 0.41 0.66 0 21 0.056 0.031 -
In [32]:
print('Final Ant-Pol Classification:\n\n', final_class)

Final Ant-Pol Classification:

 Jee:
----------
good (94 antpols):
3, 4, 5, 7, 9, 10, 15, 16, 17, 19, 20, 21, 29, 30, 31, 33, 35, 37, 38, 40, 41, 42, 49, 50, 53, 55, 56, 61, 62, 63, 64, 65, 66, 67, 68, 69, 72, 81, 82, 83, 85, 86, 89, 90, 94, 98, 100, 106, 107, 109, 111, 112, 118, 120, 121, 125, 127, 128, 129, 130, 135, 136, 137, 138, 141, 143, 144, 156, 157, 161, 162, 163, 164, 165, 168, 169, 170, 171, 176, 177, 178, 179, 183, 184, 185, 186, 187, 189, 191, 220, 221, 222, 237, 239

suspect (21 antpols):
8, 36, 48, 52, 84, 87, 88, 93, 99, 101, 119, 122, 123, 126, 152, 154, 166, 182, 192, 202, 238

bad (65 antpols):
18, 22, 27, 28, 32, 34, 43, 44, 45, 46, 47, 51, 54, 57, 58, 59, 60, 70, 71, 73, 74, 75, 77, 78, 91, 92, 102, 103, 104, 105, 108, 110, 116, 117, 124, 140, 142, 145, 147, 148, 149, 150, 151, 153, 155, 158, 160, 167, 173, 180, 181, 190, 193, 200, 201, 203, 219, 320, 321, 322, 323, 324, 325, 329, 333


Jnn:
----------
good (93 antpols):
3, 4, 5, 7, 9, 10, 15, 16, 17, 19, 20, 21, 29, 30, 31, 34, 35, 37, 38, 40, 41, 42, 47, 53, 54, 55, 56, 57, 61, 65, 66, 67, 69, 72, 78, 81, 83, 85, 86, 88, 89, 90, 93, 94, 98, 99, 100, 106, 110, 112, 118, 119, 125, 126, 127, 128, 129, 130, 135, 137, 138, 143, 144, 151, 153, 154, 156, 157, 158, 162, 163, 164, 165, 168, 169, 170, 171, 176, 177, 178, 179, 183, 184, 185, 186, 187, 189, 191, 202, 220, 221, 237, 239

suspect (20 antpols):
8, 22, 36, 48, 49, 51, 52, 62, 64, 82, 87, 107, 121, 122, 136, 152, 193, 201, 222, 238

bad (67 antpols):
18, 27, 28, 32, 33, 43, 44, 45, 46, 50, 58, 59, 60, 63, 68, 70, 71, 73, 74, 75, 77, 84, 91, 92, 101, 102, 103, 104, 105, 108, 109, 111, 116, 117, 120, 123, 124, 140, 141, 142, 145, 147, 148, 149, 150, 155, 160, 161, 166, 167, 173, 180, 181, 182, 190, 192, 200, 203, 219, 320, 321, 322, 323, 324, 325, 329, 333

TODO: Find and flag RFI¶

TODO: Perfrom nucal¶

Metadata¶

In [33]:
from hera_cal import __version__
print('hera_cal:', __version__)
from hera_qm import __version__
print('hera_qm:', __version__)

hera_cal: 3.1.5.dev78+gda49a16
hera_qm: 2.0.4.dev9+gdffdef6
In [34]:
print(f'Finished execution in {(time.time() - tstart) / 60:.2f} minutes.')

Finished execution in 11.50 minutes.