World Meteorological Organization |
Date: 2024-10-23 |
Version: a |
Document location: https://community.wmo.int/wis2-topic-hierarchy |
Task Team on WIS Metadata (TT-WISMD)[1] |
Expert Team on Metadata Standards (ET-Metadata)[2] |
Standing Committee on Information Management and Technology (SC-IMT)[3] |
Commission for Observation, Infrastructure and Information Systems (INFCOM)[4] |
Copyright © 2024 World Meteorological Organization (WMO) |
i. Abstract
WIS2 real-time data sharing is based on a message queuing protocol (MQP) supporting a publication/subscription (PubSub) mechanism. A user can subscribe to an MQP broker to receive real-time notifications of the existence of new data.
WIS2 brokers offer a range of topics organised in a hierarchy. Users can select their topics of interest and subscribe to them to receive notifications and download data relevant to their work.
The standard notification message format ensures that the WIS2 ecosystem (data publisher, data user, and global services) is a robust, effective, and unified exchange platform for weather, climate, and water data.
This document defines the structure of the WIS Topic Hierarchy. Topics are utilized by WIS Nodes, Global Broker services, and data/metadata subscribers.
The following are keywords to be used by search engines and document catalogues.
wmo, wis 2.0, weather, climate, water, topic hierarchy, metadata, pubsub, mqp, message queuing protocol
iii. Security Considerations
Based on the WMO Unified Data Policy for the International Exchange of Earth System Data (Resolution 1 (Cg-Ext(2021) [5], exchanged data are classified as core or recommended. Core data is considered fully open and unrestricted with no security considerations. Recommended data may have access control defined.
No security considerations have been made for this standard.
1. Scope
This document defines the topic hierarchy used by WIS message brokers to manage message delivery to subscribers and / or recipients.
This specification defines the conformance requirements for the WIS2 Topic Hierarchy. Annex A defines the abstract test suite.
All other topic structure specifications are not in scope.
2. Conformance
Conformance with this standard shall be checked using the tests specified in Annex A (normative) of this document.
WIS Global Brokers and Nodes are required to comply with all conformance classes of this specification in support of providing MQP services in alignment with the defined topic structure.
WMO shall publish guidance material to assist WIS Global Brokers and Nodes in constructing valid topic structures.
This standard identifies one Requirements Class which defines the functional requirements.
The mandatory Requirements Class for this specification is:
-
"WIS2 Topic Hierarchy Core"
3. References
-
OASIS: MQTT Version 5.0 (2019) [6]
-
OASIS: MQTT Version 3.1.1 (2014) [7]
-
Wikipedia: Publish-subscribe pattern (2023) [8]
-
International Telecommunications Union (ITU): T.50 : International Reference Alphabet (IRA) (Formerly International Alphabet No. 5 or IA5) - Information technology - 7-bit coded character set for information interchange [9]
-
IANA: Root Zone Database (2023) [10]
4. Terms and definitions
This document uses the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this Standard and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.
This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the 'ModSpec'. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.
The following additional terms and definitions also apply.
4.1. Abbreviated terms
| Abbreviation | Term |
|---|---|
DCPC |
Data Collection and Production Centres |
GISC |
Global Information System Centre |
HTTP |
Hypertext Transfer Protocol |
HTTPS |
Hypertext Transfer Protocol Secure |
IANA |
Internet Assigned Numbers Authority |
IETF |
Internet Engineering Task Force |
ISO |
International Organization for Standardization |
MQP |
Message Queuing Protocol |
MQTT |
Message Queuing Telemetry Transport |
NC |
National Centre |
OGC |
Open Geospatial Consortium |
PubSub |
Publish / Subscribe |
PR |
Permanent Representative |
TLD |
Top-level domain |
URI |
Uniform Resource Identifier |
URL |
Uniform Resource Locator |
W3C |
World Wide Web Consortium |
WCMP |
WMO Core Metadata Profile |
WIS |
WMO Information System |
WMO |
World Meteorological Organization |
WNM |
WIS2 notification message |
5. Conventions
This section provides details and examples for any conventions used in the document. Examples of conventions are symbols, abbreviations, or special notes regarding how to read the document.
5.1. Identifiers
The normative provisions in this Standard are denoted by the URI:
All requirements and conformance tests that appear in this document are denoted by partial URIs which are relative to this base.
5.3. Codelists bundle
The WIS2 Topic Hierarchy Notification codelist bundle can be found at https://schemas.wmo.int/wth/1.0/wth.zip. This bundle can be used by tools and applications wishing to browse or validate topic structures.
A browseable codelist can be found at https://codes.wmo.int/wis/topic-hierarchy
6. Introduction
6.1. The WIS Topic Hierarchy
The WIS Topic Hierarchy (WTH) provides a structure used by data providers and WIS Global Services in support of core WIS workflows: publish, discover, subscribe and download.
6.2. Real-time data sharing
WIS real-time data sharing is based on a message queuing protocol (MQP) supporting a publication/subscription mechanism. A user can subscribe to an MQP broker to receive real-time notifications. The notifications can be sent for new or updated data or metadata. The notification message received from the MQP broker contains a URL to download the data. The MQP broker offers a range of topics organised in a hierarchy. The users can select their topics of interest and subscribe to them to receive notifications and download data relevant to their work.
6.3. Data discovery
Users can discover datasets from the Global Discovery Catalogue (GDC). Once a user has identified a dataset of interest, they may subscribe to data notifications and updates using the topic and MQP broker found from the GDC dataset discovery metadata. Datasets in the GDC are made available via the WMO Core Metadata Profile 2 (WCMP2) standard for discovery metadata, which supports a categorisation scheme consistent with the topic hierarchy to provide a seamless search experience compatible with the access modality provided by the MQP broker. In other words, the MQP topic and WIS discovery metadata have the same vocabulary so that discovery, subscription, and download are consistent.
6.4. Structure
The structure of the topic hierarchy underpins the discovery and sharing of data in WIS, and requires standardization across all WIS services to provide consistent filter and access to the user.
Recalling that WIS is designed to support the WMO Unified Data Policy, the topic hierarchy must be aligned with WMO Res. 1 Cg-EXT-21 - Unified Data Policy[11].
Final approval of the WTH updates will go through the WMO fast-track amendment process.[12]
7. WIS2 Topic Hierarchy
Note: This section of working draft document is the same as Appendix D in the Manual on the WMO Information System (WMO-No. 1060), Volume II.
The WIS2 Topic Hierarchy (WTH) provides a mechanism for users to subscribe to and receive data or metadata notifications. It is documented in discovery metadata records and leveraged by WIS2 brokers.
The normative provisions in the WTH are denoted by the base Uniform Resoure Identifier (URI) http://wis.wmo.int/spec/wth/1 and requirements are denoted by partial URIs which are relative to this base. Topics, values, and examples are represented with shaded text.
1. Requirements Class Core
1.1 Overview
URI |
|
Target type |
Topic classification |
Dependency |
|
Dependency |
|
Preconditions |
Topics conform to the Topic Name requirements of MQTT |
The WTH is composed of primary topics (levels 1-7) and sub-discipline specific topics (levels 8 and beyond).
The primary topics apply to all data and resources in WIS. They are relational, meaning that any combination of the values in each level can be used to construct a topic applicable to a notification.
The sub-discipline topics are proposed by domain experts and user communities. These levels are a hierarchical representation of the datase, and the number of levels in this part may vary according to the requirements of various domains.
The representation is encoded as a simple text string of values in each topic level, separated by a slash (/).
Examples:
origin/a/wis2/ca-eccc-msc/data/core/weather/surface-based-observations/synop
cache/a/wis2/ca-eccc-msc/data/core/weather/prediction/forecast/medium-range/probabilistic/global
origin/a/wis2/ca-eccc-msc/data/core/weather/surface-based-observations/synop
origin/a/wis2/ca-eccc-msc/data/recommended/atmospheric-composition/experimental/space-based-observation/geostationary/solar-flares
Table 1 provides an overview of the primary topic levels.
| Level | Name | Description |
|---|---|---|
1 |
channel |
Location at which the data originates (data providers are |
2 |
version |
Alphabetical version of the topic hierarchy, currently: |
3 |
system |
Fixed value of |
4 |
centre-id |
Acronym proposed by Member and endorsed by WMO Secretariat |
5 |
notification-type |
WIS2 notification types ( |
6 |
data-policy |
Data policy as defined by the WMO Unified Data Policy (Resolution 1 (Cg-Ext(2021)))[13] ( |
7 |
earth-system-discipline |
Seven high-level categories as defined by the WMO Unified Data Policy (Resolution 1 (Cg-Ext(2021))) - Annex 1 ( |
1.2 Publishing
For maximum utility and efficient management of topics, it is recommended that data is published to a detailed level of the topic hierarchy. This helps avoid the "pollution" of data messages under the primary topics.
Note: Each discipline has a sub-discipline topic named experimental for publication to provisional topics.
Requirement 1 |
/req/core/publishing |
A |
Data SHALL NOT be published with a topic that is not defined in the WTH. |
B |
Data SHALL be published to at least the level of the sub-discipline topic (level 8 or beyond). |
C |
Metadata SHALL be published to at exactly the level of the notification type ( |
Recommendation 1 |
/rec/core/publishing |
A |
The topic |
Permission 1 |
/per/core/publishing |
A |
Data MAY be published with the |
1.3 Management
The primary levels and sub-discipline specific levels are managed differently to maintain stability and allow for flexibility.
Requirement 2 |
/req/core/management |
A |
Primary topics (levels 1 to 7) SHALL be determined by WMO. |
B |
Sub-discipline topics (level 8 and beyond) SHALL be proposed by domain experts and user communities. |
C |
Sub-discipline topics (level 8 and beyond) SHALL be defined using a hierarchical approach. |
D |
Sub-discipline topics (level 8 and beyond) SHALL be coordinated and integrated by WMO. |
Requirement 3 |
/req/core/releasing |
A |
The addition of a new centre identifier SHALL trigger an immediate stable release of WTH updates, which is not required to align with the WMO fast-track approval procedure. |
B |
Immediate stable releases SHALL only contain changes resulting from a new value in the |
C |
Updates to the primary levels and other major revisions will go through the WMO standard procedure. |
D |
Updates to the sub-discipline topics (level 8 and beyond) will go through the WMO fast-track approval procedure (see Rules of Procedure for Technical Commissions (WMO-No. 1240)). |
1.4 Versioning
The topic hierarchy version helps data providers and data consumers with change management and transition in relation to updates.
Requirement 4 |
/req/core/versioning |
A |
A minor version SHALL NOT result in any changes to the version level. |
B |
A major version SHALL result in a change to the version level (for example, |
C |
Removal of a topic at any level SHALL result in a major version update. |
D |
Renaming of a topic at any level SHALL result in a major version update. |
E |
A change in the structure of the topic hierarchy SHALL result in a major version update. |
F |
A renaming or removal in the WMO Notification Message encoding SHALL result in a major version update. |
G |
A new topic SHALL NOT result in any version update. |
H |
A new centre identifier SHALL NOT result in any version update. |
1.5 Conventions
All levels of the topic hierarchy are defined in a consistent manner to support a normalized and predictable structure.
Requirement 5 |
/req/core/conventions |
A |
Topic level definitions SHALL be lowercase. |
B |
Topic level definitions SHALL be encoded in IRA T.50. |
C |
Topic level definitions SHALL NOT utilize dots ( |
D |
Topic level definitions SHALL utilize dashes ( |
E |
All topic level definitions at a given level SHALL be unique. |
F |
The topic structure levels imply a fixed sequence and SHALL NOT be re-ordered. |
1.6 Centre identification
The centre identifier (centre-id) is an acronym, as proposed by the Member and endorsed by the WMO Secretariat. It is a single identifier comprised of a top-level domain (TLD) and centre name. It represents the data publisher, distributor or issuing centre of a given dataset, data product, data granule or other resource.
Requirement 6 |
/req/core/centre-id |
A |
A centre identifier SHALL NOT be used by more than one WIS2 Node or Global Service. |
B |
A centre identifier SHALL be formatted as
|
C |
The |
Recommendation 2 |
/rec/core/centre-id |
A |
Organizations operating with a |
B |
International organizations operating with an |
Permission 2 |
/per/core/centre-id |
A |
A centre identifier’s |
B |
Larger organizations providing multiple centres MAY use dashes in the |
C |
A centre providing a WIS service MAY further qualify the function within the |
D |
Organizations wishing to test their WIS2 Node or Global Service MAY provide the |
2. WIS2 Topic Hierarchy resources
2.1 WMO Codes Registry
| Level | Topic | URI |
|---|---|---|
1 |
channel |
|
2 |
version |
|
3 |
system |
|
4 |
centre-id |
|
5 |
notification-type |
|
6 |
data-policy |
|
7 |
earth-system-discipline |
https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline |
8 |
atmospheric-composition |
https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline/atmospheric-composition |
climate |
https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline/climate |
|
cryosphere |
https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline/cryosphere |
|
hydrology |
https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline/hydrology |
|
ocean |
https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline/ocean |
|
space-weather |
https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline/space-weather |
|
weather |
https://codes.wmo.int/wis/topic-hierarchy/earth-system-discipline/weather |
2.2 WMO schemas server
A zipped directory of all topics is published at https://schemas.wmo.int/wth/a. This bundle can be used by tools and applications wishing to browse or validate topic structures.
Annex A: Conformance Class Abstract Test Suite (Normative)
Conformance Class: Core
- label
- subject
-
Requirements Class "core"
- classification
-
Target Type:Topic Classification
Conventions
- label
-
/conf/core/conventions
- subject
-
/req/core/conventions
- test-purpose
-
Validate that a given topic meets the conventions of WTH.
Check that all characters in the topic are lowercase.
Check that all characters in the topic are based on ASCII T.50.
Check that no characters in the topic contain dots (.).
Centre identification
- label
-
/conf/core/centre-id
- subject
-
/req/core/centre-id
- test-purpose
-
Validate that a centre identifier is valid.
Check that the first component of the centre identifier is a valid TLD as defined by IANA[14].
Annex B: Examples (Informative)
WIS2 Topic Hierarchy
origin/a/wis2/ca-eccc-msc/data/core/weather/surface-based-observations/synopcache/a/wis2/de-dwd/metadataAnnex C: Bibliography
-
OASIS: MQTT Version 5.0 (2019) [15]
-
OASIS: MQTT Version 3.1.1 (2014) [16]
-
Wikipedia: Publish-subscribe pattern (2023) [17]
-
International Telecommunications Union (ITU): T.50 : International Reference Alphabet (IRA) (Formerly International Alphabet No. 5 or IA5) - Information technology - 7-bit coded character set for information interchange [18]