Julian Feinauer created IOTDB-286:
-------------------------------------
Summary: Rich Datatypes API / Logical Datatypes
Key: IOTDB-286
URL: https://issues.apache.org/jira/browse/IOTDB-286
Project: Apache IoTDB
Issue Type: New Feature
Reporter: Julian Feinauer
I worked a lot in the automotive industry and there is a broadly adopted open
Standard called ASAM MDF (
[https://www.asam.net/standards/detail/mdf/]
). It is a format which is targeted at the efficient storage but at the same
time it supports VERY complex types (which are often used in automotive
controllers). Take something as simple as a boolean. We could store it as a
boolean (as java bool) in 1 bit. BUT we have overall 4 possibilities: * No
value is available for a timestamp (NULL / nothing stored) * We had a
successful request but the Controller does not know whether true or false (or
had an internal error), this is a bit like Optional.isPresent() == false * True
* False In Java we could model it as a variable Optional<> x which could be
null, Optional.empty(), Optional.of(true), Optional.of(false). Other examples
are discrete values like “ON”, “OFF” (which are handled as “lookup tables” on
integer rows, internally). Or scale formulas like a*x+b which allows to
leverage the precision even for “small” double values or even integers. A
formula but also a “fallback” lookup value like “NV”.
Example:
We know that we measure e.g. a voltage between 3V and 4.2V with a precision of
0.01 or something.
Then its most efficient to store integers and a formula like a * x + b with
e.g. b = 3 and a = 1/100.
So 3V would be stored as x = 0, 3.01V -> x = 1, ... 4.2V as x = 1200.
So we only store 0 to 1200 and no decimals and stuff which would be very easily
compressable I thnk.
See the full Discussion on the mailing list below
https://lists.apache.org/thread.html/b8b0fc1dc854ef5e255f906e85848db73d5c6c20cf0e602a3a8f75e8@%3Cdev.iotdb.apache.org%3E
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
