| Author | |
|---|---|
| Technical Contacts | |
| Version | 1.1 |
| Date | March 2024 |
Content
GeoServer
The data can be hosted on the AWI GeoServer infrastructure. There are two important stages:
- File format and file exchange
- Layer configuration
For both stages, a coordination with the GeoServer team is necessary. Please consult them if you intend to host your media data with the AWI GeoServer infrastructure to discuss what kind of data product — own layer(s) in own WMS, own layer(s) in existing WMS, integration of own data in existing layer(s) — suits your needs best.
File format and file exchange
In order to import the data into the GeoServer database and to allow the highest possible flexibility for the data provider, the O2A specification for GeoCSV (.sdi.csv) format was agreed upon.
Example files for media data:
_layer_def.csv_layer_def.csvtpangaea#890964#SO242_2_174-1_links-to-images.sdi.csvpangaea#890964#SO242_2_174-1_links-to-images.sdi.join.csvpangaea#818400#PS81_116-3_links-to-photographs.sdi.csvpangaea#818400#PS81_116-3_links-to-photographs.sdi.join.csv
Various options for data exchange are possible, such as shared projects in the cloud, git repositories or OAI-PMH.
For inquiries about exchange location and database ingestion frequency, please reach out to the GeoServer team.
Layer configuration
Besides the database, the GeoServer offers additional layer information, e.g. abstract, title, keywords. Moreover, there are other configuration options that the product owner must decide upon before the final layer can be hosted.
You can find out more about this topic in the O2A Data Product Configuration SOP. The corresponding configuration repository is O2A Spatial Data Products. For access and further permissions, please contact one of the technical contacts.
Provision of web media
Making videos and photos available on the web typically involves a two-step process. First, they need to be transcoded, and then delivered via web server. The subsequent sections present an overview of tools and recommendations that can assist you in this process.
Tools: Encoding of media files
When encoding photos and videos for the web, there is a trade-off between visual quality and data reduction. On the one hand, the media should be reduced in file size to achieve the best possible loading speed. On the other hand, the visual quality should not be degraded to the point of introducing unwanted artifacts. There are also technical parameters to consider that have a direct impact on browser performance and compatibility. To get an idea about the most important technical web video parameters, see Recommendation - Video Proxies Parameters.
The following section provides an overview of tools that have been developed to simplify the creation of web media files and are highly recommended.
MARIVP: Primary Python-based tool for generating preview files for the viewer and media gallery. Provides command line interfaces as well as Python modules to generate photos, videos, thumbnails and covers for the web. For detailed documentation, insights into web encoding strategies and more, see its git repository.
Hotfolder-Encoding: Hotfolder-based video file encoding tool that allows custom encoding configuration. A hotfolder is a folder where media files are temporarily stored, automatically processed (i.e. encoded) and returned, all within a defined structure. For more information and code see the git repository.
ELEMENTS Image Broker: Web-API that brokers access to ELEMENTS content based on custom data fields, i.e. image-uuid. It is based on the implementation of the IIIF image standard with ELEMENTS as data source. For a detailed description and usage, see the git repository.
Recommendation: Hosting of media files
Media files tend to be data intensive. Computing them on demand, as in the case of the image broker, might challenge the infrastructure, especially for high server loads. The following recommendations are intended to reduce load times and contribute to a positive user experience.
Web server: Use state of the art and proven hosting applications, like NGINX or Apache HTTP Server.
HTTP/2: Activate HTTP/2 to improve page loading speed, e.g., via multiple requests over one connection, binary protocol and compression of request headers.
Load balancing: Set up load balancing. Multiple combined media servers allow higher throughput (especially image broker) and provide fault tolerance.
Image broker cache: When using an image broker, it is highly recommended to enable media file caching. This will speed up the playout of images that have already been sent and are still in the cache. It can also help to reduce the amount of processing required when loading multiple images at the same time, i.e. in a media gallery, thus improving the user experience in the browser.
Viewer
Currently, there are two viewer instances hosted in the web: Marine-Data and Earth-Data.
Their development occurs in the git repository Visual Explorer. To provide the full functionality of showing data in the viewer UI, configuration files are required.
These data provisioning configuration files, along with their respective arrangements, are built on a predefined structure. You can find the documentation in the O2A Data Product Configuration SOP.
The repository for sharing your configuration files is located in the O2A Spatial Data Products repository. For access and further permissions, please contact one of the technical contacts.
To visualize media data in the viewer properly, the following configuration files are required.
Popup template
Clicking on a data point on the map triggers the appearance of a popup, which provides an overview of the data. Its style and information is configured via a template file written in markdown. Here, readable text can be mapped to the technical names of the data.
Example:
The Metadata Templates documentation contains a comprehensive list of all functions available for displaying data. In case anything is missing, please get in touch with the viewer team.
Sidebar template
The sidebar template provides an overview of all the data associated with a data point. Besides displaying data in text form, media data can also be configured for playback.
Videos and video streams can be played using the video player. Furthermore, the video player supports video covers that are displayed before video playback begins, and a time-based thumbnail preview when hovering over the seek bar.
Additionally, photos can be integrated in the sidebar. By providing URLs to images in multiple resolutions, responsive image loading can be used. This is further supported by the lightbox that opens when an image is clicked, thus providing an enlarged view in adaptive resolution.
The required functions and all other template functions are available in the Metadata Templates documentation.
Example:
Filter mapping
The viewer enables data filtering. As the naming of the data items may vary across different layers, consistent filtering across multiple layers is provided via filter mapping.
More coming soon...
Gallery
The media gallery provides a simple way to get an overview of all media files, e.g. in a dataset. In this lightbox, photos can be zoomed and viewed more closely, and videos can be played directly in the video player. Besides that, there is a metadata section that displays information from the item in the GeoServer layer. Moreover, it makes it possible to share items in the media gallery via links.
Demo gallery (with predefined configuration and dataset):
https://marine-data.de/?site=gallery
URL scheme
The gallery's content depends on the URL query parameters. The parameters are composed as follows.
| Query Parameter | Query Value | Required | Note |
|---|---|---|---|
| site | gallery | ✔️ | fixed query value |
| ows | <ows_uri> | ✔️ | identifier used to find the gallery configuration in the catalogue (i.e., https://marine-data.de/rest/search/) via the resources.uri field. |
| layer | <layer_name> | ✔️ | depend on the name of the layer in the GeoServer database |
| filter | <cql-filter> | ✔️ | required to select a subset of the layer table, i.e., based on dataset, event, author etc. CQL and ECQL syntax are used (more infos) |
| shareID | <persistent_identifier> | ❌ | if the persistent_identifier field is blank or does not exist in the gallery mapping file, the name field serves as the identifier, or if not present, a combination of location and time is used |
Example:
Gallery mapping
A gallery mapping file needs to be present to enable flexible data structures and provide full functionality inside the gallery. It maps the dataset vocabulary (values in mapping dict) to the gallery vocabulary (keys in mapping dict) and is required for each GeoServer media layer.
All fields inside the mapping file are optional. If any field is missing, its corresponding functionality will not be enabled in the gallery.
A detailed description and overview can be found in the O2A Media Layer - Gallery Mapping documentation.
Like all other configuration files, the gallery mapping file is part of the configuration repository and should be provided in O2A Spatial Data Products.
Metadata section
To prevent redundancy, the sidebar template of the viewer is also used in the media gallery. However, this template usually contains a method to display photos or videos. This method can be disabled in the media gallery as shown in the example file of the sidebar template above.
Overview
Content Tools