act.qc.QCFilter.datafilter

QCFilter.datafilter(variables=None, rm_assessments=None, rm_tests=None, verbose=False, del_qc_var=True)[source]

Method to apply quality control variables to data variables by changing the data values in the dataset using quality control variables. The data is updated with failing data set to NaN. This can be used to update the data variable in the xarray dataset for use with xarray methods to perform analysis on the data since those methods don’t read the quality control variables.

Parameters:
  • variables (None or str or list of str) – Data variable names to process. If set to None will update all data variables.

  • rm_assessments (str or list of str) – Assessment names listed under quality control varible flag_assessments to exclude from returned data. Examples include [‘Bad’, ‘Incorrect’, ‘Indeterminate’, ‘Suspect’]

  • rm_tests (int or list of int) – Test numbers listed under quality control variable to exclude from returned data. This is the test number (or bit position number) not the mask number.

  • verbose (boolean) – Print processing information.

  • del_qc_var (boolean) – Option to delete quality control variable after processing. Since the data values can not be determined after they are set to NaN and xarray method processing would also process the quality control variables, the default is to remove the quality control data variables.

Examples

from act.io.armfiles import read_netcdf
from act.tests import EXAMPLE_MET1

ds = read_netcdf(EXAMPLE_MET1)
ds.clean.cleanup()

var_name = "atmos_pressure"

ds_1 = ds.nanmean()

ds.qcfilter.add_less_test(var_name, 99, test_assessment="Bad")
ds.qcfilter.datafilter(rm_assessments="Bad")
ds_2 = ds.nanmean()

print("All_data =", ds_1[var_name].values)
All_data = 98.86098
print("Bad_Removed =", ds_2[var_name].values)
Bad_Removed = 99.15148