Versions Compared

Old Version 2

changes.mady.by.user Sebastian Immoor

Saved on

compared with

New Version 3

changes.mady.by.user Sebastian Immoor

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Web-app

Web-app’s task

Sensor (sensor.awi.de)

Provides parameter code as URN, e. g. vessel:mya_ii:dgps:lat

NRT Ingester (ingest.dmawi.de)

Auto-ingests and stores sensor's raw data, executes drivers and forwards to Data-WS

Data-WS (dashboard.awi.de/data)

Data Web Service having all time-series data and mean values,
visualization frontend for time series (DASHBOARD) and REST interfaces for data in/output

Drivers

Currently the following drivers are available to convert sensor's raw data to NRT Data Format:

NameSensor typesLanguage
adcpToNrtRDI ADCP instruments (format pd0 data format such as contained in files .enc, .enx, ...)Python
ctdToNrtSea & Sun Technologies's CTD (format provided by STA acquisition software)Python
ferryboxToNrt4h-Jena FerryboxPython
sbe38ToNrtSeabird SBE38 temperature probe (format provided by probe itself via serial line)Python
snowbuoyToNrt
Python
toa5ToNrt
Python

Drivers can be programmed and installed in the NRT Ingester environment. Python is currently the prefered language.

Format version 2

Format version 2 has been introduced on 01-03-2019 and replaces format version 1.
The component Data-WS only uses format version 2 why it is highly recommended to use this as the default format.
Format version 1 is deprecated. It is still supported by NRT Ingester for old ingest jobs but should not be implemented in new projects.

...

Plain-text table having columns for each parameter and rows for each individual time record.

WhatHow
Field / column separatorTabulator “\t”
End of record markerNew line “\n”
Header

The header must consist of a column named “datetime” and 1-n columns having a valid parameter URN (from Sensor.awi.de)
as column header field. The “datetime” column must always be the first column.

datetime    type:dev:parameter1    type:dev:parameter2

Header - Units

Units may be given in square brackets [], following each parameter URN, separated by a space “ “:

datetime    type:dev:parameter1 [°C]    type:dev:parameter2 []

Units are optional depending on the O2A component and application.

  • Data-WS exports the unit in a data request output (or [] if not defined).
  • Data-WS ignores a passed unit when importing data
  • NRT Ingester passes units to the Data-WS. Only if a new (not existing) parameter URN
    is created in the Data-WS the unit is added to that. Existing parameter URNs and their
    existing units are not updated by a change of the unit.
Quality Flags

Quality flags can be added to each data value. These are numbers from 0 to n defined in document not defined yet.
Each data column needs an additional column titled with the same parameter URN followed by “ (quality_flag)”:

datetime   vessel:mya:temp   vessel:mya:temp (quality_flag)
2019-02-28 15:50:00     56.00000    1
2019-02-28 15:50:01     566.0000    2

Date time format

Expected time zone: UTC

Allowed date formats:

yyyy-mm-dd HH:MM:SS.fff
yyyy-mm-dd HH:MM:SS
yyyy-mm-ddTHH:MM:SS.fff
yyyy-mm-ddTHH:MM:SS

Decimal symbolDot "."
Data values

Decimal values, integer numbers or characters (without containing tabulators!)

34.56556    23    334.43E-2    SAMPLE1

For use of Data-WS only: The Data-WS currently only accepts numbers as input.
Given characters or words (such as “inf”, “NaN”, “SAMPLE1”) will result in an import error.

Error valuesEmpty field (simply give a null string, but do not forget to add
enough tabulators
tabs as data field separator)


Format version 1

Format version 1 is deprecated since 28-02-2019. It is still supported by NRT Ingester for old ingest jobs but should not be implemented in new projects.

...