Quality Level 1¶
- Table of contents
- Quality Level 1
Quality Level 1 checks are performed on data during ESG publication and on metadata during metadata storage in the metafor questionnaire. Quality Level 1 is assigned after passing all QC Level 1 checks on data.
QC Level 1 on Data¶
These checks consist of:
- cmor2 compliance checks by the cmor checker check_CMOR_compliant.py which is part of the cmor2 distribution, e.g.
svn export http://www-pcmdi.llnl.gov/svn/repository/cmor/branches/cmor-1.x
- esg publisher conformance
- name matches the profile: * varid_tableid_modelid_exptid_rid[iid][pid][_startdate-enddate][_suffix][_clim].nc * file path matches DRS requirements
2. Global Attributes: check for validity of required global attributes
- experiment_id, experiment, project_id match tables * parent_experiment_id is valid and different from experiment_id * forcing, frequency, realization, branch_time are valid * creation_date is valid and in right format
- checks for axes names validity * dimensions ordering * checks for required attributes, needed bounds, needed formula_terms, range validity, units (with udunits), type, direction stored, requested_values exist * singleton dimensions are defined via coordinate attribute, not actual dimension * singleton dimension value validity * validity of formula terms * for time axes, is in days since * for time axes, bounds seem ok * for lon/lat axes, checks the grid is an abstract rectangular grid
- name is valid * file contains only 1 variable * checks for optional/required attributes and validity (e.g. CF 'standard_name'), optional/required additional attributes, associated_files defined correctly, type, units (with udunits)
- variable name indicated by file is in the file * file name matches what file says for: * table_id, model_id, exp_id, physics_version, initialization_method, realization, climatology, start and end times, frequency * versions of CMOR and CMOR tables are consistent; table date matches file table date
6. warnings in CMOR2 for:
- size ≥ 2 GB * optional attributes are not defined * global attributes are neither required nor optional
ESG Publisher Conformance¶
- File is readable ('online’ data)
- File format is recognized
- File is of size > 0 bytes
- Discovery data - especially DRS fields - are identifiable and have correct values. If any mandatory fields are missing or invalid, an error is raised and the data cannot be published.
- CF Standard names are valid. A warning is issued if the standard name is missing or unrecognized.
- Coordinate axes are recognizable (definition based on CF conventions) - particularly time. A calendar is defined.
- Time values are monotonic and do not overlap between files. If overlap is discovered, a warning is logged. This is checked when aggregations are generated. It is not considered an error if timepoints are missing.
QC Level 1 on Metadata¶
Mandatory fields in the metafor questionnaire (http://q.cmip5.ceda.ac.uk/cmip5) are checked for completeness; technical validation of CIM-XMLs.