> > Do we really need to define so many new encoding types?> > How about adding a new parameter, SDT? > i.e, create time series root.a.b.c with datatype=?, encoding=?, > compression=? [, loss=SDT?]
yes, SDT as a new parameter would be better. Thank you! On Tue, Nov 17, 2020 at 7:49 PM Xiangdong Huang <[email protected]> wrote: > Hi, > > > After discussion, the SDT > > compression method will be used before encoding. The data will be > discarded > > in memory before flushing to the disk. After compressing data, it will > > still go through the current encoding process in PageWriter. > > +1. > > > Building on top of the current encoding types, we will have more encoding > > types that support SDT, ie GORiLLA_SDT, PLAIN_SDT, RLE_SDT, etc > > Do we really need to define so many new encoding types? > How about adding a new parameter, SDT? > i.e, create time series root.a.b.c with datatype=?, encoding=?, > compression=? [, loss=SDT?] > > Not sure which one is better. an evaluation is needed. > > Best, > ----------------------------------- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > > Haimei Guo <[email protected]> 于2020年11月17日周二 下午5:01写道： > > > Hi, > > > > The following a redesigned SDT encoding[1]. After discussion, the SDT > > compression method will be used before encoding. The data will be > discarded > > in memory before flushing to the disk. After compressing data, it will > > still go through the current encoding process in PageWriter. > > > > Building on top of the current encoding types, we will have more encoding > > types that support SDT, ie GORiLLA_SDT, PLAIN_SDT, RLE_SDT, etc > > > > For the query process, it will remain the same and the returned dataset > > will contain only compressed data. There will be no SDT decoder. > > If the user is querying a timestamp that is discarded after SDT or never > > existed before, he will get an empty set. Or use linear fill keyword. > > > > Please let me know if you have any questions or recommendations. > > Thank you for your time! > > > > [1] https://cwiki.apache.org/confluence/display/IOTDB/SDT+encoding > > > > Best, > > Haimei > > > > On Fri, Nov 6, 2020 at 5:50 PM Haimei Guo <[email protected]> wrote: > > > > > Hi, > > > > > > The following is a more detailed design[1] of SDT encoding and query in > > > IoTDB. The encoding and query methods will consult OSISoft Pi's design. > > The > > > document first explains Pi's approach to encoding and query, then shows > > the > > > modification of PageWriter, PageReader, SDTEncoder, SDTDecoder. > > > > > > Please let me know if you have any questions or recommendations. > > > Thank you for your time and consideration! > > > > > > [1] https://cwiki.apache.org/confluence/display/IOTDB/SDT+encoding > > > > > > Best, > > > Haimei > > > > > > > > > > > >
