Project

General

Profile

Quality Level 1

Introduction

Quality Level 1 checks are performed on data during ESG publication and on metadata during metadata storage in the metafor questionnaire. Quality Level 1 is assigned after passing all QC Level 1 checks on data.

QC Level 1 on Data

These checks consist of:

  • cmor2 compliance checks by the cmor checker check_CMOR_compliant.py which is part of the cmor2 distribution, e.g.
     svn export http://www-pcmdi.llnl.gov/svn/repository/cmor/branches/cmor-1.x
     

CMOR2 Compliance

1. DRS/Filename:

  • name matches the profile:
  • varid_tableid_modelid_exptid_rid[iid][pid][_startdate-enddate][_suffix][_clim].nc
  • file path matches DRS requirements

2. Global Attributes: check for validity of required global attributes

  • experiment_id, experiment, project_id match tables
  • parent_experiment_id is valid and different from experiment_id
  • forcing, frequency, realization, branch_time are valid
  • creation_date is valid and in right format

3. Axis:

  • checks for axes names validity
  • dimensions ordering
  • checks for required attributes, needed bounds, needed formula_terms, range validity, units (with udunits), type, direction stored, requested_values exist
  • singleton dimensions are defined via coordinate attribute, not actual dimension
  • singleton dimension value validity
  • validity of formula terms
  • for time axes, is in days since
  • for time axes, bounds seem ok
  • for lon/lat axes, checks the grid is an abstract rectangular grid

4. Variable:

  • name is valid
  • file contains only 1 variable
  • checks for optional/required attributes and validity (e.g. CF 'standard_name’), optional/required additional attributes, associated_files defined correctly, type, units (with udunits)

5. cross-checks:

  • variable name indicated by file is in the file
  • file name matches what file says for:
  • table_id, model_id, exp_id, physics_version, initialization_method, realization, climatology, start and end times, frequency
  • versions of CMOR and CMOR tables are consistent; table date matches file table date

6. warnings in CMOR2 for:

  • size ≥ 2 GB
  • optional attributes are not defined
  • global attributes are neither required nor optional

ESG Publisher Conformance

  • File is readable ('online’ data)
  • File format is recognized
  • File is of size > 0 bytes
  • Discovery data - especially DRS fields - are identifiable and have correct values. If any mandatory fields are missing or invalid, an error is raised and the data cannot be published.
  • CF Standard names are valid. A warning is issued if the standard name is missing or unrecognized.
  • Coordinate axes are recognizable (definition based on CF conventions) - particularly time. A calendar is defined.
  • Time values are monotonic and do not overlap between files. If overlap is discovered, a warning is logged. This is checked when aggregations are generated. It is not considered an error if timepoints are missing.

QC Level 1 on Metadata

Mandatory fields in the metafor questionnaire (http://q.cmip5.ceda.ac.uk/cmip5) are checked for completeness; technical validation of CIM-XMLs.