回覆: Enrich Session interfaces

[Jack Tsai]

Hi Jialin,

Thanks for your reply.

I'll implement these deleting interfaces for Session in recent.\

Best,
Tsung Han Tsai



寄件者: Jialin Qiao 
寄件日期: 2019年9月6日 下午 01:35
收件者: dev@iotdb.apache.org 
主旨: Re: Enrich Session interfaces

Hi Tsung Han,

I have checked your PR and give some comments. No big problem, only some 
optimizations.

Besides, before adding the query interface, could you please add the deleting 
storage group interface first? This is what this issue[1] is for. You could 
refer to this PR [2].

To clarify these delete interfaces for others. I list them in the following:

(1) Deleting data in one timeseries: This is what "delete from root.sg1.d1.s1 
where time<=100" statement and DeletePlan do, also the "delete interface" means.
(2) Deleting timeseries, including its data and schema. This is what "delete 
timeseries root.sg1.d1.s1" and MetadataPlan do.
(3) Deleting storage group: deleting all data and timeseries in one storage 
group, including the corresponding storage group processor and system info in 
data/system folder.


[1]https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-188?filter=allopenissues
[2] https://github.com/apache/incubator-iotdb/pull/376

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jack Tsai" 
> 发送时间: 2019-09-05 23:25:39 (星期四)
> 收件人: "dev@iotdb.apache.org" 
> 抄送:
> 主题: 回覆: Enrich Session interfaces
>
> Hi,
>
> I have added the delete interface for the Session API.  I followed the 
> pattern which is similar to the insert interface.
>
> Please have a check whether it is conform to the requirement. I would 
> implement the query interface for Session API if I have more time.
>
> Best,
> Tsung-Han Tsai
> 
> 寄件者: Jialin Qiao 
> 寄件日期: 2019年9月2日 下午 06:48
> 收件者: dev@iotdb.apache.org 
> 主旨: Re: Enrich Session interfaces
>
> Hi Tsung Han,
>
> The new insert interface only inserts one row data, not in batch, for example 
> : (deviceId, time, s1, s2,s3, s1_value, s2_value, s3_value).
>
> Besides, the Session interfaces are only for IoTDB, not TsFile.
>
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
>
> 乔嘉林
> 清华大学 软件学院
>
> > -原始邮件-
> > 发件人: "Jack Tsai" 
> > 发送时间: 2019-09-02 18:44:34 (星期一)
> > 收件人: "dev@iotdb.apache.org" 
> > 抄送:
> > 主题: Re: Enrich Session interfaces
> >
> > Hi,
> >
> > I wonder is the performance of this new kind of insert interface different 
> > from the insert RowBatch interface? Or it is just a more convenient way to 
> > insert batched data?
> >
> > Best regards,
> > Tsung Han Tsai
> >
> > 取得 iOS 版 Outlook
> > 
> > 寄件者: Jialin Qiao 
> > 寄件日期: Monday, September 2, 2019 4:35:55 PM
> > 收件者: dev@iotdb.apache.org 
> > 主旨: Enrich Session interfaces
> >
> > Hi,
> >
> >
> > I got some feedback from our users in Hunan province. Except for 
> > insertBatch, they want an easier interface to insert data. Therefore, I 
> > submit a PR [1] to add an insert interface, which is easier to understand 
> > than the insert RowBatch interface.
> >
> >
> > Moreover, I have created an issue [2] to enrich the Session interfaces, 
> > looking forward to more contributors for the session interfaces.
> >
> >
> > [1] https://github.com/apache/incubator-iotdb/pull/367
> > [2] https://issues.apache.org/jira/browse/IOTDB-191
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院

Re: Add bloom filters to TsFile

[Xiangdong Huang]

+1 for bloom filter!
+1 for implementation (but seems no  license file in the repo...)

By the way, it seems that there are some new variants of bloom filter,
e.g., supporting range query.
I am not sure whether do we need the variants, e.g., for supporting check
whether a timeseries set "root.a.b.*.speed" exist.

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Claude Warren  于2019年9月12日周四 上午12:29写道：

> In my reading of the short message it seems like it would make sense to use
> a bloom filter to determine if the "gear" is in the file.   I have the
> library that I am proposing to move to commons.  It can be found at
>
> https://github.com/Claudenw/BloomFilter/tree/MultiFilter/src/main/java/org/xenei/bloomfilter
>
> Claude
>
> On Tue, Sep 10, 2019 at 3:45 PM Julian Feinauer <
> j.feina...@pragmaticminds.de> wrote:
>
> > Hi,
> >
> > I like the idea. I'm just adding Claude here as we talked yesterday about
> > a bloom filter implementation he has already done.
> >
> > @cla...@apache.org  what do you think? : )
> >
> > Julian
> > --
> > *From:* Tian Jiang 
> > *Sent:* Tuesday, September 10, 2019 5:14:33 AM
> > *To:* dev@iotdb.apache.org 
> > *Subject:* Add bloom filters to TsFile
> >
> >
> >
> > Greetings,
> >
> >
> > The recent readings remind me that the bloom filter is standard equipment
> > in K-VDBs. Although IoTDB is not one of them (at least not typically),
> the
> > bloom filter still helps a lot in various situations. For example, our
> > recent experiments gave us an illusion that the time series in a storage
> > group remains unchanged. However, that is not the case.
> >
> >
> > Naturally, in real situations, the number of time series grows over time,
> > due to reasons like adding new gears. The old files do not contain such a
> > time series. Without the help of bloom filters, we have to check each old
> > file only to find that there is no such time series. To my knowledge,
> this
> > may take a lot of time.
> >
> >
> > So, I suggest we add a bloom filter (or some more efficient one) to each
> > TsFile to help skip unwanted files.
> >
> >
> > | |
> > Tian Jiang
> > |
> > |
> > jt2594...@163.com
> > |
> > 签名由网易邮箱大师定制
> >
>

Re:Solving jira problem (IOTDB-180) Get rid of JSON format in "show timeseries"

[Jialin Qiao]

Hi Tao Yi,

I have reviewed your PR and merged it. Good work!

Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: thss15_yit 
> 发送时间: 2019-09-11 01:53:27 (星期三)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re:Re:Solving jira problem (IOTDB-180) Get rid of JSON format in "show 
> timeseries"
> 
> I have submitted the pull request of this issue [IOTDB-180]. 
> The link of the pull request is 
> https://github.com/apache/incubator-iotdb/pull/387
> Thanks for your checking.
> 
> 
> Tao Yi
> 
> At 2019-09-09 11:30:25, "thss15_yit"  wrote:
> >The JIRA link of this issue is 
> >https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-180?filter=allopenissues
> >
> >
> >
> >
> >
> >
> >
> >
> >在 2019-09-09 11:21:23，"thss15_yit"  写道：
> >>Hi,
> >>I have been working on JIRA issue [IOTDB-180 get rid of JSON format in 
> >> "show timeseries"] these days.
> >>   My plan of dealing with this issue is merging the execution of statement 
> >> "show timeseries" into "show timeseries "，using the functions of 
> >> "show timeseries " to output the table format of the data, and then 
> >> remove some of the useless functions of JSON format.
> >>
> >>
> >>Tao Yi 

Re: Add bloom filters to TsFile

[Claude Warren]

In my reading of the short message it seems like it would make sense to use
a bloom filter to determine if the "gear" is in the file.   I have the
library that I am proposing to move to commons.  It can be found at
https://github.com/Claudenw/BloomFilter/tree/MultiFilter/src/main/java/org/xenei/bloomfilter

Claude

On Tue, Sep 10, 2019 at 3:45 PM Julian Feinauer <
j.feina...@pragmaticminds.de> wrote:

> Hi,
>
> I like the idea. I'm just adding Claude here as we talked yesterday about
> a bloom filter implementation he has already done.
>
> @cla...@apache.org  what do you think? : )
>
> Julian
> --
> *From:* Tian Jiang 
> *Sent:* Tuesday, September 10, 2019 5:14:33 AM
> *To:* dev@iotdb.apache.org 
> *Subject:* Add bloom filters to TsFile
>
>
>
> Greetings,
>
>
> The recent readings remind me that the bloom filter is standard equipment
> in K-VDBs. Although IoTDB is not one of them (at least not typically), the
> bloom filter still helps a lot in various situations. For example, our
> recent experiments gave us an illusion that the time series in a storage
> group remains unchanged. However, that is not the case.
>
>
> Naturally, in real situations, the number of time series grows over time,
> due to reasons like adding new gears. The old files do not contain such a
> time series. Without the help of bloom filters, we have to check each old
> file only to find that there is no such time series. To my knowledge, this
> may take a lot of time.
>
>
> So, I suggest we add a bloom filter (or some more efficient one) to each
> TsFile to help skip unwanted files.
>
>
> | |
> Tian Jiang
> |
> |
> jt2594...@163.com
> |
> 签名由网易邮箱大师定制
>