World Meteorological Organization

Date: 2024-03-12

Version: a

Document location: https://community.wmo.int/wis2-topic-hierarchy

Task Team on WIS Metadata (TT-WISMD)[1]

Expert Team on Metadata Standards (ET-Metadata)[2]

Standing Committee on Information Management and Technology (SC-IMT)[3]

Commission for Observation, Infrastructure and Information Systems (INFCOM)[4]

Copyright © 2024 World Meteorological Organization (WMO)

i. Abstract

WIS2 real-time data sharing is based on a message queuing protocol (MQP) supporting a publication/subscription (PubSub) mechanism. A user can subscribe to an MQP broker to receive real-time notifications of the existence of new data.

WIS2 brokers offer a range of topics organised in a hierarchy. Users can select their topics of interest and subscribe to them to receive notifications and download data relevant to their work.

The standard notification message format ensures that the WIS2 ecosystem (data publisher, data user, and global services) is a robust, effective, and unified exchange platform for weather, climate, and water data.

This document defines the structure of the WIS Topic Hierarchy. Topics are utilized by WIS Nodes, Global Broker services, and data/metadata subscribers.

The following are keywords to be used by search engines and document catalogues.

wmo, wis 2.0, weather, climate, water, topic hierarchy, metadata, pubsub, mqp, message queuing protocol

iii. Security Considerations

Based on the WMO Unified Data Policy for the International Exchange of Earth System Data (Resolution 1 (Cg-Ext(2021) [5], exchanged data are classified as core or recommended. Core data is considered fully open and unrestricted with no security considerations. Recommended data may have access control defined.

No security considerations have been made for this standard.

1. Scope

This document defines the topic hierarchy used by WIS message brokers to manage message delivery to subscribers and / or recipients.

This specification defines the conformance requirements for the WIS2 Topic Hierarchy. Annex A defines the abstract test suite.

All other topic structure specifications are not in scope.

2. Conformance

Conformance with this standard shall be checked using the tests specified in Annex A (normative) of this document.

WIS Global Brokers and Nodes are required to comply with all conformance classes of this specification in support of providing MQP services in alignment with the defined topic structure.

WMO shall publish guidance material to assist WIS Global Brokers and Nodes in constructing valid topic structures.

This standard identifies one Requirements Class which defines the functional requirements.

The mandatory Requirements Class for this specification is:

  • "WIS2 Topic Hierarchy Core"

3. References

  • OASIS: MQTT Version 5.0 (2019) [6]

  • OASIS: MQTT Version 3.1.1 (2014) [7]

  • Wikipedia: Publish-subscribe pattern (2023) [8]

  • International Telecommunications Union (ITU): T.50 : International Reference Alphabet (IRA) (Formerly International Alphabet No. 5 or IA5) - Information technology - 7-bit coded character set for information interchange [9]

  • IANA: Root Zone Database (2023) [10]

4. Terms and definitions

This document uses the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this Standard and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.

This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the 'ModSpec'. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.

The following additional terms and definitions also apply.

4.1. Abbreviated terms

Table 1. Symbols and abbreviated terms
Abbreviation Term

DCPC

Data Collection and Production Centres

GISC

Global Information System Centre

HTTP

Hypertext Transfer Protocol

HTTPS

Hypertext Transfer Protocol Secure

IANA

Internet Assigned Numbers Authority

IETF

Internet Engineering Task Force

ISO

International Organization for Standardization

MQP

Message Queuing Protocol

MQTT

Message Queuing Telemetry Transport

NC

National Centre

OGC

Open Geospatial Consortium

PubSub

Publish / Subscribe

PR

Permanent Representative

TLD

Top-level domain

URI

Uniform Resource Identifier

URL

Uniform Resource Locator

W3C

World Wide Web Consortium

WCMP

WMO Core Metadata Profile

WIS

WMO Information System

WMO

World Meteorological Organization

WNM

WIS2 notification message

5. Conventions

This section provides details and examples for any conventions used in the document. Examples of conventions are symbols, abbreviations, or special notes regarding how to read the document.

5.1. Identifiers

The normative provisions in this Standard are denoted by the URI:

http://wis.wmo.int/spec/wth/1

All requirements and conformance tests that appear in this document are denoted by partial URIs which are relative to this base.

5.2. Examples

Examples provided in this specification are encoded as plain text strings.

5.3. Codelists bundle

The WIS2 Topic Hierarchy Notification codelist bundle can be found at https://schemas.wmo.int/wth/1.0/wth.zip. This bundle can be used by tools and applications wishing to browse or validate topic structures.

A browseable codelist can be found at https://codes.wmo.int/wis/topic-hierarchy

6. Introduction

6.1. The WIS Topic Hierarchy

The WIS Topic Hierarchy (WTH) provides a structure used by data providers and WIS Global Services in support of core WIS workflows: publish, discover, subscribe and download.

6.2. Real-time data sharing

WIS real-time data sharing is based on a message queuing protocol (MQP) supporting a publication/subscription mechanism. A user can subscribe to an MQP broker to receive real-time notifications. The notifications can be sent for new or updated data or metadata. The notification message received from the MQP broker contains a URL to download the data. The MQP broker offers a range of topics organised in a hierarchy. The users can select their topics of interest and subscribe to them to receive notifications and download data relevant to their work.

6.3. Data discovery

Users can discover datasets from the Global Discovery Catalogue (GDC). Once a user has identified a dataset of interest, they may subscribe to data notifications and updates using the topic and MQP broker found from the GDC dataset discovery metadata. Datasets in the GDC are made available via the WMO Core Metadata Profile 2 (WCMP2) standard for discovery metadata, which supports a categorisation scheme consistent with the topic hierarchy to provide a seamless search experience compatible with the access modality provided by the MQP broker. In other words, the MQP topic and WIS discovery metadata have the same vocabulary so that discovery, subscription, and download are consistent.

6.4. Structure

The structure of the topic hierarchy underpins the discovery and sharing of data in WIS, and requires standardization across all WIS services to provide consistent filter and access to the user.

Recalling that WIS is designed to support the WMO Unified Data Policy, the topic hierarchy must be aligned with WMO Res. 1 Cg-EXT-21 - Unified Data Policy[11].

Final approval of the WTH updates will go through the WMO fast-track amendment process.[12]

7. The WIS2 Topic Hierarchy

The WIS2 Topic Hierarchy provides a mechanism for users to subscribe to and receive data or metadata notifications. It is documented in discovery metadata records and leveraged by WIS2 brokers.

7.1. Requirements Class Core

7.1.1. Overview

This Core Requirements Class provides requirements for the definition and management of the WIS2 Topic Hierarchy.

Requirements Class

http://wis.wmo.int/spec/wth/1/req/core

Target type

Topic Classification

Dependency

MQTT v5.0

Dependency

MQTT v3.1.1

Pre-conditions

Topics conform to Topic Name requirements of MQTT

The WTH is composed of primary topics (levels 1-7) and sub-discipline specific topics (levels 8 and beyond).

The primary topics apply to all data and resources in WIS. They are relational, meaning that any combination of the values in each level can be used to construct a topic applicable to a notification.

The sub-discipline topics are proposed by domain experts and user communities. These levels are a hierarchical representation of the dataset and the number of levels in this part may vary according to the requirements of various domains.

The representation is encoded as a simple text string of values in each topic level separated by a /. For example, origin/a/wis2/ca-eccc-msc/data/core/weather/surface-based-observations/synop or origin/a/wis2/ca-eccc-msc/data/recommended/atmospheric-composition/experimental/space-based-observation/geostationary/solar-flares.

The table below provides an overview of the primary topic levels.

Table 2. WTH primary topic levels
Level Name Description

1

channel

Location of where the data originates from (data providers are origin and global services cache)

2

version

Alphabetical version of the topic hierarchy, currently: a

3

system

Fixed value of wis2 for WIS2

4

centre-id

Acronym proposed by member and endorsed by WMO Secretariat

5

notification-type

WIS2 notification types (data or metadata)

6

data-policy

Data policy as defined by the WMO Unified Data Policy (core and recommended)

7

earth-system-discipline

Seven high-level categories as defined by the WMO Unified Data Policy, Annex 1: (atmospheric-composition, climate, cryosphere, hydrology, ocean, space-weather, or weather)

7.1.2. Publishing guidelines

For maximum utility and efficient management of topics, it is recommended that data and metadata are published to a detailed level of the topic hierarchy. This helps avoid the "pollution" of messages under the primary topics. Note that each discipline has a sub-discipline topic named experimental for publication to provisional topics.

Requirement 1

/req/core/publishing

A

Data SHALL NOT be published with a topic that is not defined in this specification.

B

Data SHALL be published to at least the level of the sub-discipline topic (level 8 or beyond).

C

Metadata SHALL be published to at least the level of the notification type (metadata).

Recommendation 1

/rec/core/publishing

A

The topic experimental SHOULD be used as a temporary approach until a given sub-discipline topic is approved.

Permission 1

/per/core/publishing

A

Metadata MAY be published at any level at or below the notification type (metadata).

B

Data MAY be published with the experimental topic and include any sub-discipline topics which are not yet approved.

7.1.3. Management

The primary levels and sub-discipline specific levels are managed differently to maintain stability and allow for flexibility.

Requirement 2

/req/core/management

A

Primary topics (levels 1 to 7) SHALL be determined by WMO.

B

Sub-discipline topics (level 8 and beyond) SHALL be proposed by domain experts and user communities.

C

Sub-discipline topics (level 8 and beyond) SHALL be defined using a hierarchical approach.

D

Sub-discipline topics (level 8 and beyond) SHALL be coordinated and integrated by WMO.

Requirement 3

/req/core/releasing

A

The addition of a new centre identifier SHALL trigger an immediate stable release of WTH updates, which is not required to align with the WMO fast-track approval procedure.

B

Immediate stable releases SHALL only contain changes resulting from a new value in the [centre-id] topic.

C

Updates to the primary levels and other major revisions will go through the WMO standard procedure.

D

Updates to the sub-discipline topics (level 8 and beyond) will go through the WMO fast-track approval procedure.[13]

7.1.4. Versioning

The topic hierarchy version helps data providers and data consumers with change management and transition in relation to updates.

Requirement 4

/req/core/versioning

A

A minor version SHALL NOT result in any changes to the version level.

B

A major version SHALL result in a change to the version level (for example, a becomes b).

C

Removal of a topic at any level SHALL result in a major version update.

D

Renaming of a topic at any level SHALL result in a major version update.

E

A change in the structure of the topic hierarchy SHALL result in a major version update.

F

A renaming or removal in the WMO Notification Message encoding SHALL result in a major version update.

G

A new topic SHALL NOT result in any version update.

H

A new centre identifier SHALL NOT result in any version update.

7.1.5. Conventions

All levels of the topic hierarchy are defined in a consistent manner to support a normalized and predictable structure.

Requirement 5

/req/core/conventions

A

Topic level definitions SHALL be lowercase.

B

Topic level definitions SHALL be encoded in IRA T.50[14].

C

Topic level definitions SHALL NOT utilize dots (.).

D

Topic level definitions SHALL utilize dashes (-) to separate words (such as sea-ice).

E

All topic level definitions at a given level SHALL be unique.

F

The topic structure levels imply a fixed sequence and SHALL NOT be re-ordered.

7.1.6. Centre identification

The centre identifier (centre-id) is an acronym as specified by the WIS Centre and endorsed by WMO Secretariat. It is a single identifier comprised of a top-level domain (TLD) and centre name. It represents the data publisher, distributor or issuing centre of a given dataset, data product, data granule or other resource.

Requirement 6

/req/core/centre-id

A

A centre identifier SHALL NOT be used by more than one WIS2 Node or Global Service.

B

A centre identifier SHALL be formatted as tld-centre-name, where:

  • tld is based on TLD as defined by IANA[15] for the relevant country or international organization

  • centre-name is based on a centre name

C

The test TLD SHALL be used only for WIS internal system testing purposes.

Recommendation 2

/rec/core/centre-id

A

Organizations operating with a gov or similar TLD SHOULD use the TLD based on their country to define the TLD component of their centre identifier.

B

International organizations operating with int, org or similar TLD SHOULD reuse these to define the TLD component of their centre identifier.

Permission 2

/per/core/centre-id

A

A centre identifier’s centre-name component MAY contain dashes.

B

Larger organizations providing multiple centres MAY use dashes in the centre-name component to further delineate a centre function (for example, int-org1-nwp, int-org1-ozone).

C

A centre providing a WIS service MAY further qualify the function within the centre-name component (for example, int-org1-global-cache).

D

Organizations wishing to test their WIS2 Node or Global Service MAY provide the -test suffix to their centre identifier (for example, int-org1-test).

Annex A: Conformance Class Abstract Test Suite (Normative)

A.1. Conformance Class: Core

label

http://wis.wmo.int/spec/wth/1/req/core

subject

Requirements Class "core"

classification

Target Type:Topic Classification

A.1.1. Management

This requirement is not applicable to ATS testing.

A.1.2. Versioning

This requirement is not applicable to ATS testing.

A.1.3. Conventions

label

/conf/core/conventions

subject

/req/core/conventions

test-purpose

Validate that a given topic meets the conventions of WTH.

Check that all characters in the topic are lowercase.

Check that all characters in the topic are based on ASCII T.50.

Check that no characters in the topic contain dots (.).

A.1.4. Centre identification

label

/conf/core/centre-id

subject

/req/core/centre-id

test-purpose

Validate that a centre identifier is valid.

Check that the first component of the centre identifier is a valid TLD as defined by IANA[16].

A.1.5. Publishing guidelines

This requirement is not applicable to ATS testing.

Annex B: Examples (Informative)

B.1. WIS2 Topic Hierarchy

Example: WIS Node data publishing (surface weather observations) from Environment and Climate Change Canada, Meteorological Service of Canada
origin/a/wis2/ca-eccc-msc/data/core/weather/surface-based-observations/synop
Example: WIS Node metadata publishing from Deutscher Wetterdienst
cache/a/wis2/de-dwd/metadata/core/wcmp2

Annex C: Bibliography

  • OASIS: MQTT Version 5.0 (2019) [17]

  • OASIS: MQTT Version 3.1.1 (2014) [18]

  • Wikipedia: Publish-subscribe pattern (2023) [19]

  • International Telecommunications Union (ITU): T.50 : International Reference Alphabet (IRA) (Formerly International Alphabet No. 5 or IA5) - Information technology - 7-bit coded character set for information interchange [20]

Annex D: Revision History

Date Release Editor Primary clauses modified Description

2023-09-25

Template

Tom Kralidis

all

initial revision
1. https://community.wmo.int/governance/commission-membership/commission-observation-infrastructures-and-information-systems-infcom/commission-infrastructure-officers/infcom-management-group/standing-committee-information-management-and-technology-sc-imt/expert-team-metadata-0
2. https://community.wmo.int/governance/commission-membership/commission-observation-infrastructures-and-information-systems-infcom/commission-infrastructure-national-representatives/infcom-management-group/standing-committee-information-management-and-technology-sc-imt/et-metadata
3. https://community.wmo.int/governance/commission-membership/commission-observation-infrastructures-and-information-systems-infcom/commission-infrastructure-officers/infcom-management-group/standing-committee-information-management-and-technology-sc-imt
4. https://community.wmo.int/governance/commission-membership/infcom
5. https://library.wmo.int/doc_num.php?explnum_id=11113#page=9
6. https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html
7. http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html
8. https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern
9. https://www.itu.int/rec/T-REC-T.50
10. https://www.iana.org/domains/root/db
11. https://ane4bf-datap1.s3-eu-west-1.amazonaws.com/wmocms/s3fs-public/ckeditor/files/Cg-Ext2021-d04-1-WMO-UNIFIED-POLICY-FOR-THE-INTERNATIONAL-approved_en_0.pdf?4pv38FtU6R4fDNtwqOxjBCndLIfntWeR
12. https://community.wmo.int/en/activity-areas/wis/amendment-processes-wis-manuals-and-guides
16. https://data.iana.org/TLD
17. https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html
18. http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html
19. https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern
20. https://www.itu.int/rec/T-REC-T.50