Hi, Using a new parameter is ok, but how to store the SDT parameter needs to be considered.
Maybe store it in the mlog and the props in MeasurementSchema. Thanks, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 > -----原始邮件----- > 发件人: "Haimei Guo" <[email protected]> > 发送时间: 2020-11-17 20:22:55 (星期二) > 收件人: [email protected] > 抄送: > 主题: Re: SDT Encoding and Query Design > > > > > Do we really need to define so many new encoding types?> > > How about adding a new parameter, SDT? > > i.e, create time series root.a.b.c with datatype=?, encoding=?, > > compression=? [, loss=SDT?] > > > yes, SDT as a new parameter would be better. Thank you! > > > On Tue, Nov 17, 2020 at 7:49 PM Xiangdong Huang <[email protected]> wrote: > > > Hi, > > > > > After discussion, the SDT > > > compression method will be used before encoding. The data will be > > discarded > > > in memory before flushing to the disk. After compressing data, it will > > > still go through the current encoding process in PageWriter. > > > > +1. > > > > > Building on top of the current encoding types, we will have more encoding > > > types that support SDT, ie GORiLLA_SDT, PLAIN_SDT, RLE_SDT, etc > > > > Do we really need to define so many new encoding types? > > How about adding a new parameter, SDT? > > i.e, create time series root.a.b.c with datatype=?, encoding=?, > > compression=? [, loss=SDT?] > > > > Not sure which one is better. an evaluation is needed. > > > > Best, > > ----------------------------------- > > Xiangdong Huang > > School of Software, Tsinghua University > > > > 黄向东 > > 清华大学 软件学院 > > > > > > Haimei Guo <[email protected]> 于2020年11月17日周二 下午5:01写道： > > > > > Hi, > > > > > > The following a redesigned SDT encoding[1]. After discussion, the SDT > > > compression method will be used before encoding. The data will be > > discarded > > > in memory before flushing to the disk. After compressing data, it will > > > still go through the current encoding process in PageWriter. > > > > > > Building on top of the current encoding types, we will have more encoding > > > types that support SDT, ie GORiLLA_SDT, PLAIN_SDT, RLE_SDT, etc > > > > > > For the query process, it will remain the same and the returned dataset > > > will contain only compressed data. There will be no SDT decoder. > > > If the user is querying a timestamp that is discarded after SDT or never > > > existed before, he will get an empty set. Or use linear fill keyword. > > > > > > Please let me know if you have any questions or recommendations. > > > Thank you for your time! > > > > > > [1] https://cwiki.apache.org/confluence/display/IOTDB/SDT+encoding > > > > > > Best, > > > Haimei > > > > > > On Fri, Nov 6, 2020 at 5:50 PM Haimei Guo <[email protected]> wrote: > > > > > > > Hi, > > > > > > > > The following is a more detailed design[1] of SDT encoding and query in > > > > IoTDB. The encoding and query methods will consult OSISoft Pi's design. > > > The > > > > document first explains Pi's approach to encoding and query, then shows > > > the > > > > modification of PageWriter, PageReader, SDTEncoder, SDTDecoder. > > > > > > > > Please let me know if you have any questions or recommendations. > > > > Thank you for your time and consideration! > > > > > > > > [1] https://cwiki.apache.org/confluence/display/IOTDB/SDT+encoding > > > > > > > > Best, > > > > Haimei > > > > > > > > > > > > > > > > > </[email protected]></[email protected]></[email protected]></[email protected]>
