WIS2 Notification Message, Key Performance Indicators

World Meteorological Organization

Date: 2024-10-23

Version: 1.0.0

Document location: https://community.wmo.int/wis-metadata-kpis

Task Team on WIS Metadata (TT-WISMD)^[1]

Expert Team on Metadata Standards (ET-Metadata)^[2]

Standing Committee on Information Management and Technology (SC-IMT)^[3]

Commission for Observation, Infrastructure and Information Systems (INFCOM)^[4]

Table of Contents

1. Overview
2. Key performance indicators
- 2.1. WCMP dataset record identification

1. Overview

1.1. Purpose

This document is intended to define Key Performance Indicators (KPIs) in support of the WIS2 Notification Message (WNM). KPIs provide measurable and valuable quality assessment rules over and above the rulesets put forth by WNM.

The core driver of WNM KPIs is continuous improvement and useability of notificaiton messages as part of the WMO Information System (WIS).^[5]

1.2. Scope

This document is bound to the WNM specification and codelists. All other metadata specifications or representations are not in scope.

1.3. Audience

The target stakeholder audiences for this document include (but are not limited to):

WIS2 Global Services
WIS2 Nodes
WIS2 Global Discovery Catalogues (GDCs)
WIS2 Monitoring

1.4. How to use

The KPIs in this document are designed to help data providers in the production of notificaiton messages, as well as WIS2 Global Services, Monitoring and Nodes to measure the quality of notifications from data providers.

In order to improve quality:

providers should use the KPIs to build into their notification message generation
WIS2 Global Services, Monitoring and Nodes should use the KPIs in order to quality assess notification metadata and provide subsequent feedback to providers

1.5. Scoring

Each KPI assesses a number of criteria asssociated with notification message quality, resulting in a raw score, as well as a percentage. This approach supports weighted rubric scoring.

1.6. Reference implementation

The TT-WISMD maintains pywis-pubsub^[6], as the reference WNM validation utility which includes:

validation against WNM, Annex A: Conformance Class Abstract Test Suite (Normative)
validation against the KPIs described in this document

Documentation on installation, configuration and usage can be found on the pywis-pubsub website.

pywis-pubsub is provided as a resource to the community, under continuous improvement. Contributions are welcome and can be facilited by the WMO.

1.7. Conventions

1.7.1. Symbols and abbreviated terms

Table 1. Symbols and abbreviated terms
Abbreviation	Term
DCPC	Data Collection and Production Centres
GDC	Global Discovery Catalogue
HTML	Hypertext Markup Language
HTTP	Hypertext Transfer Protocol
HTTPS	Hypertext Transfer Protocol Secure
JSON	JavaScript Object Notation
MIME	Multipurpose Internet Mail Extensions
NC	National Centre
OGC	Open Geospatial Consortium
pywis-pubsub	WMO implementation of WNM validation
URL	Uniform Resource Locator
WDC	World Data Centre
WIS	WMO Information System
WMO	World Meteorological Organization

2. Key performance indicators

2.1. WCMP dataset record identification

2.1.1. WCMP properties

properties.metadata_id

2.1.2. Rationale for measurement

The metadata identifier provides linkage to the associated discovery metadata in the GDC and provides traceability to the overall dataset description.

2.1.3. Measurement

Whether WCMP discovery metadata information is available and can be successfully identified.

2.1.4. Guidance to score well on this assessment

Provide an identifier of the associated WCMP dataset record in the Global Discovery Catalogue

2.1.5. Rules

Table 2. Metadata identification implementation rules
Rule	Score
The `properties.metadata_id` property is present	1
The metadata identifier resolves to a valid WCMP record in the Global Discovery Catalogue	1

Total possible score: 2 (100%)

2.1.6. Examples

"properties": {
  ...
  "metadata_id": "urn:wmo:md:ca-eccc-msc:observations.swob"
  ...
}