World Meteorological Organization

Date: 2024-10-17

Version: 1.0.0-DRAFT-2024-10-17

Document location: TBD

Document status: DRAFT

Standing Committee on Information Management and Technology (SC-IMT)[1]

Commission for Observation, Infrastructure and Information Systems (INFCOM)[2]

Copyright © 2024 World Meteorological Organization (WMO)

i. Abstract

WIS2 is comprised of a network of Global Services which provide highly available services for discovery, subscription, notification and download, based on the publication of data by WIS2 Nodes.

Successful operation of WIS2 Global Services will depend on running well-managed IT environments with a very high level of reliability so that all WIS Users and WIS2 Nodes will be able to access and provide the data they need for their duties. The WIS2 Guide defines service levels and performance indicators [3] for Global Services in order to monitor and maintain the health of the network.

This document defines the content, structure, and encoding for WIS2 monitoring and alerting. This standard is an extension of the WIS2 Topic Hierarchy as well as the CloudEvents specification.

WIS2 Monitoring and Alerting topics shall extend the appraoch of the WIS2 Topic Hierarchy. WIS2 monitoring and alerting messages shall be encoded using CloudEvents along with a domain specific model for WIS2.

ii. Keywords

The following are keywords to be used by search engines and document catalogues.

wmo, wis 2.0, weather, climate, water, metadata, pubsub, alert, event, mqp, monitoring, cloudevents, JSON

iii. Security Considerations

TODO

No security considerations have been made for this standard.

1. Scope

This document defines the content, structure, and encoding for WIS2 monitoring and alerting. This standard is an extension of the WIS2 Topic Hierarchy as well as the CloudEvents specification.

This specification defines the conformance requirements for WIS2 Monitoring and Alerting (topic hierarchy and notification message). Annex A defines the abstract test suite.

2. Conformance

Conformance with this standard shall be checked using the tests specified in Annex A (normative) of this document.

The WIS2 Topic Hierarchy defines the topic hierarchy used by WIS message brokers to manage message delivery to subscribers and / or recipients. This standard is an extension of the WIS2 Topic Hiearchy.

CloudEvents is a specification for describing event data in common formats to provide interoperability across services, platforms and systems. This standard is an extension of CloudEvents.

Global Service providers are required to comply with all conformance classes of this specification in support of providing highly available services for discovery, subscription, notification and download of data and metadata within WIS2.

WMO shall publish guidance material to assist data providers in constructing WIS2 Alerting Hierarchy and Alert Messages.

This standard identifies numerous Requirements Classes which define the functional requirements.

The mandatory Requirements Classes for this specification are:

  • "WIS2 Alerting Hierarchy"

  • "WIS2 Event Message Encoding: Core"

  • "WIS2 Event Message Encoding: WCMP2 Executable Test Suite Report"

  • "WIS2 Event Message Encoding: WCMP2 Key Performance Indicator Report"

3. References

  • IETF: RFC-8259 The JavaScript Object Notation (JSON) Data Interchange Format (2016) [4]

  • IETF: RFC 3339: Date and Time on the Internet: Timestamps (2002) [5]

  • W3C: Data on the Web Best Practices, W3C Recommendation (2017) [6]

  • IANA: Link Relation Types (2020) [7]

  • IETF: JSON Schema (2022) [8]

  • CloudEvents: CloudEvents specification (2024) [9]

  • WMO: WIS2 Topic Hierarchy (2022) [10]

  • WMO: WIS2 Notification Message (2022) [11]

4. Terms and definitions

This document uses the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this Standard and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.

This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the 'ModSpec'. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.

The following additional terms and definitions also apply.

4.1. Abbreviated terms

Table 1. Symbols and abbreviated terms
Abbreviation Term

API

Application Programming Interface

DCPC

Data Collection and Production Centres

GDC

Global Discovery Catalogue

GIS

Geographic Information System

GISC

Global Information System Centre

HTML

Hypertext Markup Language

HTTP

Hypertext Transfer Protocol

HTTPS

Hypertext Transfer Protocol Secure

IANA

Internet Assigned Numbers Authority

IETF

Internet Engineering Task Force

ISO

International Organization for Standardization

JSON

JavaScript Object Notation

MQP

Message Queuing Protocol

MQTT

Message Queuing Telemetry Transport

NC

National Centre

NWP

Numerical Weather Prediction

OGC

Open Geospatial Consortium

PubSub

Publish / Subscribe

URI

Uniform Resource Identifier

URL

Uniform Resource Locator

UUID

Universally Unique Identifier

W3C

World Wide Web Consortium

WCMP

WMO Core Metadata Profile

WIS

WMO Information System

WEM

WIS2 Event Message

WATH

WIS2 Alerting Topic Hierarchy

WMO

World Meteorological Organization

WNM

WIS2 notification message

5. Conventions

This section provides details and examples for any conventions used in the document. Examples of conventions are symbols, abbreviations, use of JSON Schema, or special notes regarding how to read the document.

5.1. Identifiers

The normative provisions in this Standard are denoted by the URI:

http://wis.wmo.int/spec/wma/1

All requirements and conformance tests that appear in this document are denoted by partial URIs which are relative to this base.

5.2. Examples

Alerting topics examples provided in this specification are encoded as plain text strings.

Alerting message examples provided in this specification are encoded as JSON.

Complete examples can be found at https://schemas.wmo.int/wma/1.0/examples

5.3. Codelists bundle

Given the WIS2 Alerting Hierarchy extends the WIS2 Topic Hierarchy, no additional codelists bundles are made available given the WTH codelists bundles satisfy the requirements of this specification.

5.4. Schemas

WIS2 Alerting message schemas can be found at https://schemas.wmo.int/wma/1.0

5.5. Schema representation

JSON Schema [12] objects are used throughout this standard to define the structure of metadata records. These schema objects are also typically represented using YAML [13]. YAML is a superset of JSON, and in this standard is regarded as equivalent.

Alerting message instances are always defined as JSON.

5.5.1. Properties

A JSON property represents a key-value pair, where the key is the name of the property and the value is a standard JSON data type.

"myPropertyName": "test123"

6. Introduction

6.1. Motivation

WIS2 Global Services provide high availability capabilities in support of discovery, access and exchange of weather/climate/water/environmental data on WIS2.

Once connected to the WIS2 infrastructure, any Global Service will be monitored by the WIS2 Global Monitor. Monitoring of WIS2 Global Services will allow for detection of service anomalities, interruptions or quality assessments of metadata. These "events" can jeopardize normal WIS2 Operations.

A mechanism to notify on and describe such events is required in support of Global Service communication and corrective action. Using the WIS2 Topic Hiearchy and CloudEvents baselines for this specification provide broad interoperability and low barrier publication and alerting handling in the WIS2 ecosystem and beyond..

6.2. Scenarios

The following scenarios are useful in understanding the drivers and principles that were used in the development of this specification:

  • Global Service service down: a Global Service may cease to operate for any given reason

  • Global Service malfunctioning: a Global Service may fail to function normally (e.g.: Global Cache not providing messages, etc.)

  • WIS2 Node malfunctioning: a WIS2 Node may publish malformed or invalid WIS2 Notification Messages

These scenarios can be realized as planned/expected outages, or occur suddenly, in an unexpected manner.

Those events should be detected, and the Global Services or WIS2 Nodes should be informed to drive corrective action and succssful operation of WIS2.

7. The WIS2 Alerting Hierarchy

The WIS2 Alerting Hierarchy (WAH) provides a mechanism for Global Services to provide reports and alerts to WIS2 Global Services, as well as data/metadata reports for WIS2 Nodes to subscribe to and receive notifications.

7.1. Requirements Class "WIS2 Alerting Hierarchy"

7.1.1. Overview

This Requirements Class provides requirements for the WIS2 Alerting Hierarchy.

Requirements Class

http://www.wmo.int/spec/wma/1/req/alerting-hierarchy

Target type

Topic classification

Dependency

WIS2 Topic Hierarchy

Pre-conditions

Topic levels 2-4 conform to the WIS2 Topic Hierarchy.

Successful operation of the WIS2 infrastructure and monitoring/alerting of same should be information that is made available to all Global Services and WIS2 Nodes, and not designed for communication to external users or data consumers. Global Services need to be able to report information to Global Services and WIS2 Nodes to trigger corrective action.

The WAH is composed of five levels: A fixed channel of monitor, WTH primary topic levels 2 (version) and 3 (system), as well as WTH primary topic level 4 (centre identifier) at two WAH levels:

  • producer: the centre identifier of the entity producing the alert

  • target: the centre identifier of the intended target of the alert

The representation is encoded as a simple text string of values in each topic level separated by a slash (/).

Examples:

monitor/a/wis2/ca-eccc-msc-global-discovery-catalogue/de-dwd

monitor/a/wis2/fr-meteofrance-global-broker/cn-cma

The table below provides an overview of the primary topic levels.

Table 2. WAH primary topic levels
Level Name Description

1

channel

Location of where the data originates from (fixed value of monitor)

2

version

Alphabetical version of the topic hierarchy

3

system

Fixed value of wis2 for WIS2

4

centre-id

Acronym proposed by Member and endorsed by WMO Secretariat, of the entity producing the alert (source)

5

centre-id

Acronym proposed by Member and endorsed by WMO Secretariat, of the intended target of the alert (subject)

7.1.2. Publishing

A simple ruleset is defined for publishing alerts to WAH that enables the clear identification of the alert producer and its intended target, to trigger corrective action.

Requirement 1

/req/alerting-hierarchy/publishing

A

Alerts SHALL NOT be published with a topic that is not defined in this specification.

B

Alerts SHALL be published to exactly level 5.

C

Alert topic level 1 SHALL be named monitor.

D

Alert topic levels 2 and 3 SHALL be defined as per the WIS2 Topic Hierarchy.

E

Alert topic level 4 SHALL be a centre identifier based on the entity producing the alert.

F

Alert topic level 5 SHALL be a centre identifier based on the intended target of the alert.

7.1.3. Management

Primary levels are managed consistently to maintain stability.

Requirement 2

/req/alerting-hierarchy/management

A

Topic levels 2 and 3 SHALL be as specified by the WIS2 Topic Hierarchy.

8. WIS2 Event Message Encoding

Event payloads published via the WIS2 Alerts Hierarchy (WAH) are defined using the CloudEvents specification as a building block.

8.1. Requirements Class "WIS2 Event Message Encoding: Core"

8.1.1. Overview

This Requirements Class provides baseline requirements for all WIS2 event and report types.

CloudEvents provides a standards-based encoding for all event data, and provides mechanisms for extensibility.

Requirements Class

http://www.wmo.int/spec/wma/1/req/event-message-encoding-core

Target type

Event metadata

Dependency

CloudEvents

Pre-conditions

The event message conforms to the CloudEvents specification.

The table below provides an overview of the set of properties that are included in a WIS2 Event Message (WEM).

Table 3. WEM core properties
Property Requirement Description

id

Required

A universally unique identifier (UUID) of the message (see Identifier)

specversion

Required

The CloudEvents specification version (see Version)

source

Required

The centre identifier producing the event (see Source)

type

Required

The event type related to the message (see Type)

subject

Required

The centre identifier of the intended target of the event (see Subject)

time

Required

The date and time of when the notification was published, in RFC3339 format, UTC (see Time)

datacontenttype

Required

The media type of the data content encoding in the event message (application/json) (see Data content type)

dataschema

Required

The JSON schema which is adhered to by the data content encoding in the event message (see Data schema)

data

Required

The event payload as JSON (see Data)

8.1.2. Message size

The WIS2 Event Message Encoding allows for the transmission of event messages in a compact manner and includes the ability to embed content inline as required.

Requirement 3

/req/event-message-encoding-core/message_size

A

A WEM message SHALL NOT exceed 64 000 bytes.

8.1.3. Identifier

A universally unique identifier of the event using the UUID standard (RFC4122). The identifier is generated by the originator of the event.

Example:

"id": "6e1c7f9f-dd6c-48d9-bbc4-aef0625f1fb8"

Requirement 4

/req/event-message-encoding-core/id

A

The id property SHALL be a Universally Unique Identifier (UUID).

8.1.4. Version

The CloudEvents specification version of the event message encoding.

Example:

"specversion": "1.0"

Requirement 5

/req/event-message-encoding-core/version

A

The specversion property SHALL be fixed to "1.0".

8.1.5. Source

The centre identifier of the event producer (as defined in the [wis2-topic-hierarchy]).

Example:

"source": "ca-eccc-msc-global-discovery-catalogue"

Requirement 6

/req/event-message-encoding-core/source

A

The source property SHALL be a valid WIS2 centre identifier.

8.1.6. Type

The type of event related to the event message encoding, using a reverse DNS notation.

TODO: define as a codelist / URI for codes.wmo.int instead?

Example:

"type": "int.wmo.wis.wma.event.wcmp2-ets"

Requirement 7

/req/event-message-encoding-core/type

A

The type property SHALL be encoded using a reverse DNS notation.

A

The type property SHALL begin with int.wmo.wis.wma.event

8.1.7. Subject

The centre identifier of the intended target of the event (as defined in the [wis2-topic-hierarchy]).

Example:

"source": "de-dwd"

Requirement 8

/req/event-message-encoding-core/subject

A

The subject property SHALL be a valid WIS2 centre identifier.

8.1.8. Time

The time property identifies the date/time when the notification was first posted or published by the originator. The date/time is encoded in RFC3339 format with the Coordinated Universal Time (UTC) timezone (Z).

Example:

"time": "2024-10-17T03:42:23Z"

Requirement 9

/req/event-message-encoding-core/time

A

A WEM SHALL provide a time property.

B

The time property SHALL be in RFC3339 format.

C

The time property SHALL be in UTC timezone.

8.1.9. Data content type

The datacontenttype property identifies the media type associated with the event message payload. application/json (JSON) is the required media type for all data specific encodings.

Example:

"datacontenttype": "application/json"

Requirement 10

/req/event-message-encoding-core/datacontenttype

A

The datacontenttype property SHALL be fixed to application/json.

8.1.10. Data schema

The dataschema property identifies the JSON Schema that is adhered to by event message payload. This is the value of a given JSON Schema’s $id property.

Example:

"dataschema": "https://schemas.wmo.int/wcmp/2.0.0/schemas/wcmp2-bundled.json"

Requirement 11

/req/event-message-encoding-core/dataschema

A

The dataschema property SHALL be a URL to a JSON Schema that can be successfully derefenced by validating JSON Schema tools.

8.1.11. Data

The data property provides the event payload in JSON.

Example:

"data": {
    "id": "ab7cd199-ffa3-4909-80be-c78e99791435",
    "report_type": "ets",
    "summary": {
        "PASSED": 12,
        "FAILED": 0,
        "SKIPPED": 0
    },
    "generated_by": "pywcmp 0.10.1 (https://github.com/wmo-im/pywcmp)",
    "tests": [
        {
            "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/conformance",
            "code": "PASSED",
            "message": "Passes given schema is compliant/valid"
        },
        ...
    }
}

Requirement 12

/req/event-message-encoding-core/data

A

The data property SHALL be a JSON encoded payload of a given event.

B

The data property SHALL NOT be escaped representation of JSON.

C

The data property SHALL validate against the JSON Schema specified in the dataschema property.

Annex A: Conformance Class Abstract Test Suite (Normative)

A.1. Conformance Class: WIS2 Alerting Hierarchy

label

http://wis.wmo.int/spec/wma/1/req/alerting-hierarchy

subject

Requirements Class "WIS2 Alerting Hierarchy"

classification

Target Type:Topic Classification

A.1.1. Management

This requirement is not applicable to ATS testing.

A.1.2. Publishing

label

/conf/alerting-hierarchy/publishing

subject

/req/alerting-hierarchy/publishing

test-purpose

Validate that a given topic meets the conventions of WAH.

Split the topic by the / character, into tokens.

Check that there are exactly 5 tokens.

Check that the first token is a value of monitor.

Check that the second token is a valid WTH version value.

Check that the third token is a valid WTH system value of wis2.

Check that the fourth token is a valid centre identifier.

Check that the fifth token is also a valid centre identifier.

A.2. Conformance Class: WIS2 Event Message Encoding: Core

label

http://wis.wmo.int/spec/wma/1/req/event-message-encoding-core

subject

Requirements Class "WIS2 Event Message Encoding: Core"

classification

Target Type:Event Metadata

A.2.1. Message size

label

/conf/event-message-encoding-core/message_size

subject

/req/event-message-encoding-core/message_size

test-purpose

Validate that a WEM has a valid message size.

Check that the size of the complete WEM does not exceed 64000 bytes.

A.2.2. Identifier

label

/conf/event-message-encoding-core/id

subject

/req/event-message-encoding-core/id

test-purpose

Validate that a WEM has a valid identifier.

Check for the existence of an id property in the WEM.

Check that the id property is a valid UUID.

A.2.3. Version

label

/conf/event-message-encoding-core/version

subject

/req/event-message-encoding-core/version

test-purpose

Validate that a WEM has a valid version.

Check for the existence of a specversion property in the WEM.

Check that the specversion property is set to 1.0.

A.2.4. Source

label

/conf/event-message-encoding-core/source

subject

/req/event-message-encoding-core/source

test-purpose

Validate that a WEM has a valid source.

Check for the existence of a source property in the WEM.

Check that the source property is a valid WIS2 centre identifier.

A.2.5. Type

label

/conf/event-message-encoding-core/type

subject

/req/event-message-encoding-core/type

test-purpose

Validate that a WEM has a valid type.

Check for the existence of a type property in the WEM.

Check that the type property begins with int.wmo.wis.wma.event.

A.2.6. Subject

label

/conf/event-message-encoding-core/subject

subject

/req/event-message-encoding-core/subject

test-purpose

Validate that a WEM has a valid subject.

Check for the existence of a subject property in the WEM.

Check that the subject property is a valid WIS2 centre identifier.

A.2.7. Time

label

/conf/event-message-encoding-core/time

subject

/req/event-message-encoding-core/time

test-purpose

Validate that a WEM has a valid identifier.

Check for the existence of an time property.

Check that the time property is in RFC3339 format.

Check that the time property is in the UTC timezone.

A.2.8. Data content type

label

/conf/event-message-encoding-core/datacontenttype

subject

/req/event-message-encoding-core/datacontenttype

test-purpose

Validate that a WEM has a valid data content type.

Check for the existence of a datacontenttype property in the WEM.

Check that the datacontenttype property is set to application/json.

A.2.9. Data schema

label

/conf/event-message-encoding-core/dataschema

subject

/req/event-message-encoding-core/dataschema

test-purpose

Validate that a WEM has a valid data schema.

Check for the existence of a dataschema property in the WEM.

Issue a HTTP GET request on the value of the dataschema property.

Parse the HTTP response.

Ensure the response is a valid JSON Schema.

A.2.10. Data

label

/conf/event-message-encoding-core/data

subject

/req/event-message-encoding-core/data

test-purpose

Validate that a WEM has a valid data payload.

Check for the existence of a data property in the WEM.

Parse the data property as a JSON object.

Validate the parsed JSON object against the JSON Schema defined in the dataschema property.

Annex B: Schemas (Normative)

Note
 Schema documents will only be published on schemas.wmo.int once the standard has been approved.

B.1. WIS2 Event Message Encoding Schema

$schema: https://json-schema.org/draft/2020-12/schema
$id: https://schemas.wmo.int/wma/1/eventMessageEncodingJSON.yaml
title: WIS2 Event Message Encoding
description: WIS2 Event Message Encoding

allOf:
  - $ref: 'https://raw.githubusercontent.com/cloudevents/spec/refs/heads/main/cloudevents/formats/cloudevents.json'
  - required:
      - datacontenttype
      - dataschema
      - subject
      - time
      - data

Annex C: Examples (Informative)

C.1. WIS2 Alerting Hierarchy

Example: Notification from Environment and Climate Change Canada, Meteorological Service of Canada, Global Discovery Catalogue Service, concerning a WCMP2 record from Météo-France (Toulouse)
monitor/a/wis2/ca-eccc-msc-global-discovery-catalogue/fr-meteofrance
Example: Notification from Météo-France (Toulouse), Global Broker Service, concerning a WNM from Servicio Meteorológico Nacional (Argentina)
monitor/a/wis2/fr-meteofrance-global-broker/ar-smn

C.2. WIS2 Event Message Encoding

Example: WCMP2 compliance report event notification from Environment and Climate Change Canada, Meteorological Service of Canada, Global Discovery Catalogue Service, concerning a WCMP2 record from Deutscher Wetterdienst (Germany)
{
    "specversion": "1.0",
    "type": "int.wmo.wis.wma.event.wcmp2-ets",
    "source": "ca-eccc-msc-global-discovery-catalogue",
    "subject": "de-dwd",
    "id": "6e1c7f9f-dd6c-48d9-bbc4-aef0625f1fb8",
    "time": "2024-10-17T05:13:22Z",
    "datacontenttype": "application/json",
    "dataschema": "https://schemas.wmo.int/wma/1.0.0/schemas/wcmp2-ets-bundled.json",
    "data": {
        "id": "f84f34d6-cfb0-4cff-98ec-32f88d0fd7b8",
        "report_type": "ets",
        "summary": {
            "PASSED": 12,
            "FAILED": 0,
            "SKIPPED": 0
        },
        "generated_by": "pywcmp 0.10.1 (https://github.com/wmo-im/pywcmp)",
        "tests": [
            {
                "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/conformance",
                "code": "PASSED",
                "message": "Passes given schema is compliant/valid"
            },
            {
                "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/contacts",
                "code": "PASSED"
            },
            {
                "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/record_created_datetime",
                "code": "PASSED"
            },
            {
                "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/data_policy",
                "code": "PASSED"
            },
            {
                "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/description",
                "code": "PASSED",
                "message": "Passes given schema is compliant/valid"
            },
            {
                "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/extent_geospatial",
                "code": "PASSED"
            },
            {
                "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/extent_temporal",
                "code": "PASSED",
                "message": "Passes given schema is compliant/valid"
            },
            {
                "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/identifier",
                "code": "PASSED"
            },
            {
                "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/links",
                "code": "PASSED"
            },
            {
                "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/themes",
                "code": "PASSED"
            },
            {
                "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/title",
                "code": "PASSED",
                "message": "Passes given schema is compliant/valid"
            },
            {
                "id": "http://wis.wmo.int/spec/wcmp/2/conf/core/type",
                "code": "PASSED"
            }
        ],
        "datetime": "2024-10-02T13:55:00Z",
        "metadata_id": "urn:wmo:md:de-dwd:icon-eps.ALL"
    }
}

Annex D: Bibliography

Annex E: Revision History

Date Release Editor Primary clauses modified Description

2024-10-05

Template

Tom Kralidis

all

initial revision
1. https://community.wmo.int/governance/commission-membership/commission-observation-infrastructures-and-information-systems-infcom/commission-infrastructure-officers/infcom-management-group/standing-committee-information-management-and-technology-sc-imt
2. https://community.wmo.int/governance/commission-membership/infcom
3. https://wmo-im.github.io/wis2-guide/wis2-guide-DRAFT.html#_2_7_2_2_service_levels_performance_indicators_and_fair_usage_policies
4. https://datatracker.ietf.org/doc/html/rfc8259
5. https://datatracker.ietf.org/doc/html/rfc3339
6. https://www.w3.org/TR/dwbp
7. https://www.iana.org/assignments/link-relations/link-relations.xml
8. https://json-schema.org
9. https://cloudevents.io/specification
10. https://github.com/wmo-im/wis2-topic-hierarchy
11. https://github.com/wmo-im/wis2-notification-message
12. https://json-schema.org
13. https://en.wikipedia.org/wiki/YAML