Hi, The following a redesigned SDT encoding[1]. After discussion, the SDT compression method will be used before encoding. The data will be discarded in memory before flushing to the disk. After compressing data, it will still go through the current encoding process in PageWriter.
Building on top of the current encoding types, we will have more encoding types that support SDT, ie GORiLLA_SDT, PLAIN_SDT, RLE_SDT, etc For the query process, it will remain the same and the returned dataset will contain only compressed data. There will be no SDT decoder. If the user is querying a timestamp that is discarded after SDT or never existed before, he will get an empty set. Or use linear fill keyword. Please let me know if you have any questions or recommendations. Thank you for your time! [1] https://cwiki.apache.org/confluence/display/IOTDB/SDT+encoding Best, Haimei On Fri, Nov 6, 2020 at 5:50 PM Haimei Guo <[email protected]> wrote: > Hi, > > The following is a more detailed design[1] of SDT encoding and query in > IoTDB. The encoding and query methods will consult OSISoft Pi's design. The > document first explains Pi's approach to encoding and query, then shows the > modification of PageWriter, PageReader, SDTEncoder, SDTDecoder. > > Please let me know if you have any questions or recommendations. > Thank you for your time and consideration! > > [1] https://cwiki.apache.org/confluence/display/IOTDB/SDT+encoding > > Best, > Haimei > > >
