Versions Compared

Old Version 12

changes.mady.by.user Sebastian Immoor

Saved on

compared with

New Version 13

changes.mady.by.user Brenner Silva

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The O2A components INGEST and DASHBOARD (and its Data Web Service) use a common data format to exchange time-series data: The NRT (Near Real Time) Data Format.
It uses the SENSOR's parameter codes (URNs, Uniform Resource Names), e.g. vessel:mya_ii:dgps:lat to describe the data columns in the files.
The data is stored in a simple tabular plain-text format which is describe below.
If a sensor cannot provide data NRT Data Format but a binary or plain-text format instead, a driver (converter) will have to be programmed to convert the data either to a intermediate NRT RAW Data format or to this NRT Data Format.
A number of drivers is already available (see subchapter Drivers).

The following O2A components are involved in / use NRT Data Format.

...

WhatHow
Character encoding

UTF-8

Field / column separatorTabulator “\t”
End of record markerNew line “\n”
Header

The header must consist of a column named “datetime” and 1-n columns having a valid parameter URN (from Sensor.awi.de)
as column header field. The “datetime” column must always be the first column.

datetime    type:dev:parameter1    type:dev:parameter2

Header - Units

Units may be given in square brackets [], following each parameter URN, separated by a space “ “:

datetime    type:dev:parameter1 [°C]    type:dev:parameter2 []   type:dev:stationname [text]

Units are optional depending on the O2A component and application.

  • Data-WS exports the unit in a data request output (or [] if not defined).
  • Data-WS ignores a passed unit when importing data
  • NRT Ingester passes units to the Data-WS. Only if a new (not existing) parameter URN
    is created in the Data-WS the unit is added to that. Existing parameter URNs and their
    existing units are not updated by a change of the unit.
Quality Flags

Quality flags can be added to each data value. These are numbers from 0 to n defined in document not defined yetthe flagging scheme.
Each data column needs an additional column titled with the same parameter URN followed by “ (quality_flag)”:

datetime   vessel:mya:temp [°C]   vessel:mya:temp (quality_flag)
2019-02-28 15:50:00     56.00000    1
2019-02-28 15:50:01     566.0000    2

Date time format

Expected time zone: UTC

Allowed date formats:

yyyy-mm-dd HH:MM:SS.fff
yyyy-mm-dd HH:MM:SS
yyyy-mm-ddTHH:MM:SS.fff
yyyy-mm-ddTHH:MM:SS

Decimal symbolDot "."
Data values

Decimal values, integer numbers or characters (just no tabulators!)

34.56556    23    334.43E-2    SAMPLE1

For use of Data-WS only: The Data-WS only accepts numbers as input for each column except when its unit is "text"!
For columns which do not have the unit "text" all characters or words (such as “inf”, “NaN”, “SAMPLE1”) will result in an import error.

Error valuesEmpty field (simply give a null string, but do not forget to add tabs as data field separator)

...