Page History
...
The O2A components INGEST and DASHBOARD (and its Data Web Service) use a common data format to exchange time-series data: The NRT (Near Real Time) Data Format.
It uses the SENSOR's parameter codes (URNs, Uniform Resource Names), e.g. vessel:mya_ii:dgps:lat to describe the data columns in the files.
The data is stored in a simple tabular plain-text format which is describe below.
If a sensor cannot provide data NRT Data Format but a binary or plain-text format instead, a driver (converter) will have to be programmed to convert the data to the NRT Data Format.
A small number of drivers is already available (see chapter subchapter Drivers).
The following O2A components are involved in / use NRT Data Format.
Web-app | Web-app’s task |
Sensor (sensor.awi.de) | Provides parameter code as URN, e. g. |
NRT Ingester (ingest.dmawi.de) | Auto-ingests and stores sensor's raw data, executes drivers and forwards to Data-WS |
Data-WS (dashboard.awi.de/data) | Data Web Service having all time-series data and mean values, |
Drivers
...
Format
...
Drivers can be programmed and installed in the NRT Ingester environment. Python is currently the prefered language.
Format version 2
Format version 2 has been introduced on 01-03-2019 and replaces format version 1.
The component Data-WS only uses format version 2 why it is highly recommended to use this as the default format.
Format version 1 is deprecated. It is still supported by NRT Ingester for old ingest jobs but should not be implemented in new projects.
...
- Field separator tabulator „\t“ instead of semicolon „;“
- Unit format changed from “parameterURN;[unit1]” to “parameterURN [unit1]”
- The column „datetime“ has to be the first column and has to be “datetime” instead of “time”
- Quality flag support added
Drivers
| Anchor | ||||
|---|---|---|---|---|
|
Currently the following drivers are available to convert sensor's raw data to NRT Data Format:
| Name | Sensor types | Language |
|---|---|---|
| adcpToNrt | RDI ADCP instruments (format pd0 data format such as contained in files .enc, .enx, ...) | Python |
| ctdToNrt | Sea & Sun Technologies's CTD (format provided by STA acquisition software) | Python |
| ferryboxToNrt | 4h-Jena Ferrybox | Python |
| sbe38ToNrt | Seabird SBE38 temperature probe (format provided by probe itself via serial line) | Python |
| snowbuoyToNrt | Python | |
| toa5ToNrt | Python |
Drivers can be programmed and installed in the NRT Ingester environment. Python is currently the prefered language.
Overview
Content Tools