Hi, > After discussion, the SDT > compression method will be used before encoding. The data will be discarded > in memory before flushing to the disk. After compressing data, it will > still go through the current encoding process in PageWriter.
+1. > Building on top of the current encoding types, we will have more encoding > types that support SDT, ie GORiLLA_SDT, PLAIN_SDT, RLE_SDT, etc Do we really need to define so many new encoding types? How about adding a new parameter, SDT? i.e, create time series root.a.b.c with datatype=?, encoding=?, compression=? [, loss=SDT?] Not sure which one is better. an evaluation is needed. Best, ----------------------------------- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Haimei Guo <[email protected]> 于2020年11月17日周二 下午5:01写道： > Hi, > > The following a redesigned SDT encoding[1]. After discussion, the SDT > compression method will be used before encoding. The data will be discarded > in memory before flushing to the disk. After compressing data, it will > still go through the current encoding process in PageWriter. > > Building on top of the current encoding types, we will have more encoding > types that support SDT, ie GORiLLA_SDT, PLAIN_SDT, RLE_SDT, etc > > For the query process, it will remain the same and the returned dataset > will contain only compressed data. There will be no SDT decoder. > If the user is querying a timestamp that is discarded after SDT or never > existed before, he will get an empty set. Or use linear fill keyword. > > Please let me know if you have any questions or recommendations. > Thank you for your time! > > [1] https://cwiki.apache.org/confluence/display/IOTDB/SDT+encoding > > Best, > Haimei > > On Fri, Nov 6, 2020 at 5:50 PM Haimei Guo <[email protected]> wrote: > > > Hi, > > > > The following is a more detailed design[1] of SDT encoding and query in > > IoTDB. The encoding and query methods will consult OSISoft Pi's design. > The > > document first explains Pi's approach to encoding and query, then shows > the > > modification of PageWriter, PageReader, SDTEncoder, SDTDecoder. > > > > Please let me know if you have any questions or recommendations. > > Thank you for your time and consideration! > > > > [1] https://cwiki.apache.org/confluence/display/IOTDB/SDT+encoding > > > > Best, > > Haimei > > > > > > >
