Interpolating - Process to L1C

Water Network

To process the water network data, the hypernets processor includes RHYMER (“Reliable processing of HYperspectral MEasurement of Radiance”, version 20190718 on 16/10/2020, written by Quinten Vanhellemont and adapted for the HYPSTAR by Clémence Goyens). RHYMER provides all the required functions to process the above water measurements (currently including Mobley 1999 and 2015 and Ruddick et al. 2016 for the Fresnel correction). RHYMER is written as such that it can easily welcome any additional look-up-tables, processing functions, quality flags, etc.

Note the funtion rhymer_hypstar.process_l1c takes as input the L1A files and not the L1B. Indeed it checks for additional (water-application specific) quality flags. The L1C processor will then average the series (often at the start and end of the sequence) of downwelling radiance and irradiance scans and interpolate these averages to the upwelling radiance scans (standard configuration file requires 6 scans for upwelling radiance). Hence, the final dimensions for the upwelling radiance, downwelling irradiance and radiance and reflectance outputs for the L1C data level is wavelength and upwelling radiance scans (default 6).

  1. Quality check of the scans (flags: temp_variability_irr, temp_variability_rad):

    The first function in the rhymer_hypstar.process_l1c module checks for the variability within a series, i.e. it assigns flags to each scan showing a temporal jump, with the previous and following scan, of more than a given threshold (25% is the default) at 550 nm.

  2. Parse cycle (flags: angles_missing, lu_eq_missing, fresnel_angle_missing, fresnel_default, min_nbred, min_nbrlu, min_nbrlsky,):

    The second function parses the cycle, i.e. it separates downwelling and upwelling radiance, investigate if all required angles are present and if there are coincident upwelling and downwelling radiance measurements for the retrieval of the water leaving radiance.

  3. Wavelength interpolation:

    Next, since the irradiance and radiance measurements have a slight shift in wavelength, first a spectral interpolation is performed to fit the irradiance measurements to the radiance wavelengths.

  4. Averaging downwelling irradiance and radiance series to upwelling radiance scans:

    Downwelling radiance and irradiance (as well as uncertainties) are averaged per series.

  5. Temporal interpolation:

    The (per series) averaged downwelling irradiance and radiance data are interpolated to the timestamp of the upwelling radiance scans. Uncertainties are computed for the interpolation of the downwelling radiance and irradiance measurements, respectively. Timestamp interpolation is done, by default, with a linear interpolation (see https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.interp1d.html)

  6. Ancillary data retrieval:

    All the required parameters for the computation of the water leaving radiance and reflectance, i.e. wind speed (\(ws\)), and the effective fresnel reflectance coefficient (\(\rho(\theta,\theta_0,\Delta\phi,ws)\)) are retrieved. Wind speed may be taken from, e.g. a default value (e.g. 2m/s), in situ measurements, ancilliary datasets such as NCEP Met data. Similarly, the Fresnel reflectance coefficient can be extracted from : (1) different look-up tables (Mobley 1999 (default); 2015), or, set to a default value, estimated as a function of wind speed (e.g. Ruddick et al. 2016). The configuration files (job and processing) determine which option is used for the processing (see Section 6.9, e.g. fresnel_option in Ruddick et al. 2016).

  7. Intermediate L1C surface reflectance:

    L1C processing also includes the retrieval of the water leaving radiance and reflectance for each single upwelling radiance scan. From the above water upwelling radiance \(L_u\) and the above water downwelling radiance \(L_d\), the water leaving radiance \(L_w\) is approximated as:

    \[L_w(\theta,\Delta\phi,\lambda,\theta_0)=L_u(\theta,\Delta\phi,\lambda,\theta_0)-\rho(\theta,\Delta\phi,\lambda,\theta_0,e)L_d(\theta,\Delta\phi,\lambda,\theta_0)\]
    with

    • \(\theta\) being the viewing zenith angle (0° is pointing vertically down, measuring upwelling light and 180° is pointing vertically upward, measuring downwelling light),

    • \(\theta_0\) is the sun zenith angle (equals 0° when the sun is at zenith and 90° when the sun is at sunset), and,

    • \(\Delta\phi\) is the relative azimuth angle between sun and sensor measured with respect to sun and clockwise from sun to target (0° means that the radiance sensors are pointing into the sun glint direction, while 180° corresponds to a viewing azimuth with the sun behind).

    The term \(\rho\) is the air-water interface reflectance coefficient expressed as a function of viewing geometry and sun zenith angle and environmental factors (\(e\)). When the water surface is perfectly flat, \(\rho\) is the Fresnel reflectance and the environmental factor only depends on the relative refractive index of the air-water interface. When the water is not perfectly flat, \(\rho\) needs to account, in addition to the fresnel reflectance, for the geometric effects of the wave facets created by the roughened water surface (often called the “effective Fresnel reflectance coefficient”, Ruddick et al. (2019)). Therefore, \(\rho\) is commonly approximated as a function of the viewing and illumination geometry and wind speed, ws, and can be written as \(\rho(\theta,\theta_0,\Delta\phi,ws)\) (Mobley, 1999 and 2015).

    The water leaving radiance is then converted into water reflectance as follows:

    \[\rho_wnosc =\pi L_w /E_d\]

    with \(E_d\) being the downwelling irradiance. And nosc stands for non similarity corrected reflectance.

  8. Intermediate L1C similarity corrected reflectance (flag: simil_fail):

    Although most acquisition protocols attempt to avoid sun glint, over wind roughened surfaces, sun glint may still be present when measuring the target radiance. Therefore a spectrally flat measurement error, \(\epsilon\), based on the “near infrared (NIR) similarity spectrum” correction, is applied. \(\epsilon\) is estimated using two wavelengths in the NIR Ruddick et al. 2016, where \(\lambda_1\) = 780 nm and \(\lambda_2\) = 870 nm.

    \[\epsilon = [ \alpha\rho_wnosc(\lambda_2)-\rho_wnosc(\lambda_1)]/[(\alpha-1)]\]

    and \(\alpha\) is the similarity spectrum (Ruddick et al. (2016) ratio for the bands used; the default is, \(\alpha(780, 870)\) equals 1/0.523 = 1.912).

    To avoid negative reflectance data, if \(\epsilon\) exceeds a given percentage of the reflectance at a reference wavelength the simil_fail flag is raised (see flags description. The default percentage is 5% and reference wavelength is 670 nm. Next the similarity corrected reflectance product is computed as follows:

    \[\rho_w(\lambda) =\rho_wnosc(\lambda)-\epsilon\]

Land Network

The L1C processing for the land network consists of two interpolation steps that are applied to the irradiance measurements in order to bring them to the same wavelength scale and timestamps as the radiance measurements.

1. Spectral interpolation: The irradiances are spectrally interpolated to the wavelengths of the radiance measurements (which are not identical to the irradiance measurements). Currently, we use a simple linear interpion.

2. Temporal interpolation: Next, we use a similar method to perform a temporal interpolation. In this case, we interpolate the irradiance measurements at the start and end of the sequence, to each of the timestamps of the radiance measurements. A correction is applied to take into account the change in solar zenith angle during the sequence.

The output of the L1C processing is a product with irradiances that now have the same wavelengths and timestamps as the radiance measurements. The radiances in the L1C dataset are unchanged from the L1B dataset.

There are multiple options available for the interpolation. For the temporal interpolation, the default option includes a correction for the change in solar zenith angle throughout the sequence. Prior to the interpolation, the irradiances are divided by the cosine of the solar zenith angle at the time of the irradiance acquisition. After the interpolation, the irradiances are multiplied by the cosine of the solar zenith angle at the timestamps of the radiances. Alternatively, there is also an option to not apply the solar zenith angle correction (i.e. only linear interpolation).

By default, the linear interpolation method is used for both the spectral and temporal interpolations. However, optionally, the hypernets_processor can also be set up to do interpolation following a model. This is done using the interpolation tool within the NPL CoMet toolkit to interpolate between the irradiance wavelengths using a high-resolution reference (comet_maths documentation). The high resolution reference for the spectral irradiance interpolation comes from a clear-sky model, which gives a good first-order approximation of the short scale variability. This model is then scaled to go through the measured irradiance data, while taking into account the spectral response function of the different HYPSTAR® measurements.

The interpolation option using a high resolution model has been implemented, but is not currently operationally used. Further investigations are required to assess whether these alternative interpolation methods lead to sufficient improvement in the performance to justify their significantly slower runtime.