act.qc.read_yaml_supplemental_qc#

act.qc.read_yaml_supplemental_qc(ds, fullpath, variables=None, assessments=None, datetime64=True, time_delim=(';', ',', '|', '\\t'), none_if_empty=True, quiet=False)[source]#

Returns a dictionary converstion of YAML file for flagging data. The dictionary will contain variable names as first key, assessents as second keys containing test description as third key with time as last key. Multiple descriptions are allowed.

Parameters:

  • ds (xarray.Dataset) – Xarray dataset containing data.

  • fullpath (str or pathlib.Path) – Full path to file to read or directory containing YAML files to read. If providing a directory a file with the datastream name ending in .yaml or .yml must exists.

  • variables (str, list of str or None) – Optional, variable names to keep in dictionary. All others will be removed before returning dictionary. If no variables are left will return None.

  • assessments (str, list of str or None) – Optional, assessments categories to keep in dictionary. All others will be removed before returning dictionary.

  • datetime64 (boolean) – Convert the string value list to 2D numpy datetime64 array with first value the start time and second value end time. If the time list in the YAML file is empty, the ‘time’ numpy array will be a single value instead of 2D array and set to numpy datetime64 NaT.

  • time_delim (str or list of str or tuple of str) – Optional, character delimiter to use when parsing times.

  • none_if_empty (boolean) – Return None instead of empty dictionary

  • quiet (boolean) – Suppress information about not finding a YAML file to read.

Returns:

  • Dictionary of [variable names][assessments][description] and time values

  • or if the dictionary is empty after processing options and none_if_empty set

  • to True will return None.

Examples

This example will load the example MET data used for unit testing.

from act.tests import EXAPLE_MET_YAML, EXAMPLE_MET1
from act.io.arm import read_arm_netcdf
from act.qc.add_supplemental_qc import read_yaml_supplemental_qc
ds = read_arm_netcdf(EXAMPLE_MET1, cleanup_qc=True)
result = read_yaml_supplemental_qc(ds, EXAPLE_MET_YAML,
                             variables=['rh_mean'], assessments='Bad')
print(result)

{'rh_mean': {'Bad': {'description': 'Values are bad',
    'time': array([['2020-01-01T00:01:02.000', '2020-01-01T00:03:44.000'],
                   ['2020-01-02T00:01:02.000', '2020-01-02T00:03:44.000']],
                   dtype='datetime64[ms]')}}}