Table of Contents
Preface
All GeoServer-based data products consist of three parts: input (data exchange), output (GeoServer-based OGC Web Services) and VEF integration. This specification covers how output and VEF integration can be configured by the use of the O2A SDI Data Product Configuration GitLab repository. It will contain all GeoServer-hosted layers at AWI but externally hosted OGC web services can also added to have them properly integrated into VEF-based viewers.
After having an initial talk (see SOP), we'll set up your data product within our public Data Product Configuration repository and you'll get access to directly edit it.
Parts of the configuration should only be touched by O2A SDI admins – usually indicated by filenames beginning with admin. – but some parts are designed to be directly maintained by product owners (this could be you!) – usually indicated by filenames beginning with owner. – via Git merge requests (→ guide coming soon).
Multiple filetypes will be used/mentioned here. You will use TOML to configure the metadata (e.g. title and abstract) of your layers and services, SLD for styling your data on the map, Markdown and JSON to configure filter behaviour and metadata presentation in VEF-based viewers.
- Configuration files:
- TOML → https://toml.io
- Layer Styling
- OGC SLD specification → https://www.ogc.org/standard/sld/
- GeoServer SLD reference
- https://docs.geoserver.org/2.21.x/en/user/styling/sld/index.html (release currently used by O2A SDI )
- https://docs.geoserver.org/main/en/user/styling/sld/index.html (current stable release)
- VEF Metadata Templates: Markdown-based → https://spaces.awi.de/x/7QkQH
- VEF Mapping Files: JSON-based
- Gallery Mapping → https://spaces.awi.de/x/TQLeGw
- Filter Mapping (→ documentation coming soon)
Version 0.1
Repository Schema
The repository consists of three "levels" of folders. The first level is the "label" level, holding one folder per public GeoServer instance/label of the O2A SDI (currently: common, platforms, basemaps) and one folder (_other) for OWS hosted differently (O2A SDI ArcGIS Server, external servers). One level down in the folder structure is "service" level, where each folder represents one OWS (e.g. common/marehub, common/media, common/bathymetry, platforms/expedition). One more level down is "layer" level, where each folder represents one OWS layer (e.g. common/media/photos, platforms/expedition/events). Additionaly, there are some fixed folders: _styles within each GeoServer instance/label folder and _resources which may be found on root, label, service and layer level.
_styles folders hold all SLD files for its associated GeoServer instance/label. SLD files are used to define how WMS layers look when rendered for GetMap requests.
_resources folders hold files configuring a layer's behaviour and metadata appearance in VEF-based viewers.
. <repo>
|-- admin.*
|
|-- <label>
| |-- admin.*
| |
| |-- _styles/<SLD files>
| |-- _resources/<resource files>
| |
| `-- <service>
| |-- owner.service.toml
| |-- admin.*
| |
| |-- _resources/<resource files>
| |
| `-- <layer>
| |-- owner.resources.toml
| |-- owner.layer.toml
| |-- admin.*
| |
| `-- _resources
| |-- popup.md
| |-- sidebar.md
| |-- gallery.mapping.json
| `-- filter.mapping.json
|
|-- _resources/<resource files>
|
`-- _other
|
`-- <arbitrary-service-id>
|-- owner.urls
|
|-- _resources/<resource files>
|
`-- <layer>/_resources/<resource files>
_resources folders (on all levels)
A _recources subfolder, independent of its location (root level, label level, service level, layer level), contains files to influence the presentation and behaviour in VEF-based viewers (like the Marine Data viewer). There will be no effect when loading the OWS in any other client (like ArcGIS, QGIS, other mapping apps).
There are three sorts of files supported. VEF Metadata Template take care of metadata presentation in VEF popups and the VEF sidebar. They are recommended for each and every layer. VEF Mapping files take care of frontend-side filter harmonization and media gallery behaviour. These files are only used in special cases. Specific documentation can be found here:
- VEF Metadata Template files (
*.md), to configure presentation of your data in popups and sidebar → https://spaces.awi.de/x/7QkQH - VEF Filter Mapping files (
*.mapping.json) → (documentation coming soon) - VEF Gallery Mapping files (
*.mapping.json) → see https://spaces.awi.de/x/TQLeGw
In general files with specific names inside resources folders will get used automatically. These names are:
popup.md(VEF Metadata Template for popups)sidebar.md(VEF Metadata Template for the sidebar)filter.mapping.json(VEF Filter Mapping)gallery.mapping.json(VEF Gallery Mapping)
For each layer used in VEF-based viewers, the file within its own _recources folder is prioritised over that in its sevice-level _resources folder and so on. This boils down to this priority list:
<label>/<service>/<layer>/_resources/<file-by-convention><label>/<service>/_resources/<file-by-convention><label>/_resources/<file-by-convention>_resources/<file-by-convention>
However, the resources section of the owner.layer.toml (see subsection) file can be used to overwrite this convention with specific filenames (not filepaths) overwriting the convention, resulting in this final priority list:
<label>/<service>/<layer>/_resources/<file-from-layer-configuration-if-existing><label>/<service>/_resources/<file-from-layer-configuration-if-existing><label>/_resources/<file-from-layer-configuration-if-existing>_resources/<file-from-layer-configuration-if-existing><label>/<service>/<layer>/_resources/<file-by-convention><label>/<service>/_resources/<file-by-convention><label>/_resources/<file-by-convention>_resources/<file-by-convention>
File Specs
Admin vs. Owner Files
On any level there might be files prefixed with admin. or owner., like common/media/photos/admin.layer.toml or common/media/owner.service.toml
admin. files are only to be created and edited by SDI admins. owner. files are to be created by admins and can be edited by data product owners to change some parts of the configuration themselves.
SLD files in _styles folders and files in _recources folders can be created and edited by data product owners.
Label Level
A label is a GeoServer instance within the O2A SDI. Currently there are three labels: common, platforms, basemaps. There's also a <label>/_styles subfolder – used to store all SLD files used for all layers of all services within thin GeoServer label/instance – and a <label>/_resources subfolder which can be used to provide resources for representation in VEF-based viewers which should be shared among multiple services and layers of this label/instance to avoid placing copies in each layer's subfolders.
_styles/*.sld
Put your SLD files here. Take care, that the filename matches the actual name of the XML-defined style.
owner.service.toml
A service folder refers to a workspace within a GeoServer instance/label and results in an OWS of the same name. Additionally an OWS has a human-readable title, an abstract and optional keywords – just like a scientific paper. These metadata can be specified in <label>/<service>/owner.service.toml. There's also a <service>/_resources subfolder which can be used to provide resources which should be shared among multiple layers of this service (see section on resource handling).
| section | key | value type | required/optional/default | value description |
|---|---|---|---|---|
title | string | yes | human-readable title of the workspace (used as title for all contained OWS: WMS, WFS, WCS) | |
abstract | string | yes | abstract, describing the service's content. VEF (Marine Data Viewer) supports Markdown URL representation (<http://www.marine-data.de>, [Marine Data](http://www.marine-data.de>)) and HTML (like <br><br>, <a></a>) | |
keywords | string list | no |
owner.layer.toml
A layer folder results in a layer of the same name withing the OWS representet by its parent service-layer folder. A layer has a human-readable title, an abstract and optional keywords – just like a scientific paper. These metadata can be specified in <label>/<service>/<layer>/owner.layer.toml. Additionally default and alternative styles (referencing SLD files in <label>/_styles/) and VEF resources can be specified. If desired the default mechanism of auto-deploying changes to the GeoServer (styles, metadata) and/or VEF (recoures) can be deactivated.
| section | key | value type | required/optional/default | value description |
|---|---|---|---|---|
title | string | yes | human-readable title of the layer | |
abstract | string | yes | abstract, describing the layer's content. VEF (Marine Data Viewer) supports Markdown URL representation (<http://www.marine-data.de>, [Marine Data](http://www.marine-data.de>)) and HTML (like <br><br>, <a></a>) | |
keywords | string list | no | ||
SETTINGS | style_default | string | yes | name of the SLD file (including file extension) to use as default style. The filename needs to match the actual style name. SLD files need to go here: <label>/_styles/*.sld |
style_alternatives | string list | no | names of SLD files (including file extension) to use as alternative styles. The filenames need to match the actual style name. SLD files need to go here: | |
RESOURCES | gallery_mapping | string | no (defaults to gallery.mapping.json) | name of gallery mapping file. Only required if deviating from |
filter_mapping | string | no (defaults to filter.mapping.json) | name of gallery mapping file. Only required if deviating from | |
popup | string | no (defaults to popup.md) | name of popup metadata template file. Only required if deviating from | |
sidebar | string | no (defaults to sidebar.md) | name of sidebar metadata template file. Only required if deviating from | |
AUTODEPLOY | resources | boolean | no (defaults to true) | whether to autodeploy (accepted merge requests will trigger instant updates in AWI-hosted VEF-based viewers, applying changes in resource files) changes for this layer to production. If set to false, triggering updates otherwise is encouraged. Talk to your SDI admins! |
geoserver | boolean | no (defaults to true) | whether to autodeploy (accepted merge requests will trigger instant updates in O2A GeoServers, applying changes in styles, service/layer metadata) changes for this layer in production. If set to false, triggering updates otherwise is encouraged. Talk to your SDI admins! |
_other Folder
The _other subfolder is the place to store resources for services/layers which are not hosted in the GeoServer part of the O2A SDI to be used within VEF-based viewers. This subtree only contains resource files (see above for specs), owner.urls files but neither SLD nor any other configuration files.
Within the _other subfolder each folder refers to one OWS endpoint (which could contain multiple services like WMS/WFS/WCS). The name of the subfolder (_other/<arbitrary-service-id>) can be choosen arbitrarily but should be kind-of human-readable. Inside, there needs to be one file named owner.urls (_other/<arbitrary-service-id>/owner.urls), specifying service URLs – one per line – to which this folder's resources are dedicated. There can be subfolders like on <label>/<service>/ folders and the same resolve priorities apply.
Examples
common/media/photos
Taken from here: https://gitlab.awi.de/software-engineering/sdi/de.awi.sdi.o2a_spatial.dataproducts/-/tree/main/common/media/photos
Folder Structrure
. https://gitlab.awi.de/software-engineering/sdi/de.awi.sdi.o2a_spatial.dataproducts
|
`-- common
|
|-- _styles
| |-- marehubPhotos.sld
| `-- marehubPhotosPreview.sld
|
`-- media
|-- owner.service.toml
|-- admin.service.toml
|
`-- photos
|-- owner.layer.toml
|-- admin.layer.toml
|-- admin.store.toml
|
`-- _resources
|-- popup.md
|-- sidebar.md
|-- gallery.mapping.json
`-- filter.mapping.json
common/media/owner.service.toml
title = "Media OWS" abstract = ''' Media service with photo and video data provided by AWI, Geomar and Hereon. Curated metadata is extracted from image FAIR Digital Objects (iFDOs) (<https://marine-imaging.com/fair/>) or automatically harvested from data products in Pangaea (<https://pangaea.de/>). ''' keywords = [ "WMS", "WFS", "MareHUB", "videos", "photos", "image", "iFDO", "PANGAEA" ]
common/media/photos/owner.layer.toml
title = "Photos" abstract = "Layer with data provided by the three marine research centers AWI, Geomar and Hereon. It includes data collected during marine research projects as well as data enriched for this layer, i.e., technical metadata taken directly from the file, such as EXIF data. Depending on the data set, the provided metadata varies for each data product. Preview photos that are provided via urls are lossy processed to make them accessible for the web. This results in a loss of data information, even if not visible." keywords = [ "MareHUB", "photos", "image", "PANGAEA" ] [SETTINGS] style_default = "marehubPhotos.sld" style_alternatives = [ "marehubPhotosPreview.sld" ] [AUTODEPLOY] resources = false
_other/hcdc-hereon-marehub/
This is how resources – here: metadata templates (*.md) – for an externally hosted WMS could be added. There are no layer subfolders (hence no <layer>/_resources/*.md files) so that all layers of this service would use the the metadata templates provided in _other/hcdc-hereon-marehub/_resources/.
Taken from here: https://gitlab.awi.de/software-engineering/sdi/de.awi.sdi.o2a_spatial.dataproducts/-/tree/main/OTHER/hcdc-hereon-marehub
Folder Structrure
. https://gitlab.awi.de/software-engineering/sdi/de.awi.sdi.o2a_spatial.dataproducts
|
`-- _other
|
`-- hcdc-hereon-marehub
|-- owner.urls
`-- _resources
|-- popup.md
`-- sidebar.md
_other/hcdc-hereon-marehub/owner.urls
https://hcdc.hereon.de/geoserver/MareHub/ows https://hcdc.hereon.de/geoserver/MareHub/wms https://hcdc.hereon.de/geoserver/MareHub/wfs
Overview
Content Tools