Re: Questions about Hive Integration

2019-10-13 Thread Jialin Qiao
Hi,

> Suppose IoTDB writes data into hdfs:///usr/iotdb/data/root.sg1
> Now there are two complete tsfiles, and a incomplete tsFile.
> If a user create a hive table referring to this folder, then how many
> tsfiles that Hive will observe?

It depends the implementation. When we create a hive table over a folder. We 
could get all files in that folder, including both complete and incomplete 
files. 
How to construct the table schema and how many data splits to generate depend 
ourselves. (In my opinion, it's better to ignore the incomplete files.)

> Then someone writes data into IoTDB, and finally there are 5 tsfiles, and a
> incomplete tsfile, then how many tsfiles that Hive will observe?

Once a table is created, whether the schema can be updated is beyond my 
knowledge (most probably it can not), maybe @Yuan Tian could give us an answer.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-10-14 08:22:11 (星期一)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Questions about Hive Integration
> 
> Hi,
> 
> What is more important is the 3rd question...
> 
> Suppose IoTDB writes data into hdfs:///usr/iotdb/data/root.sg1
> Now there are two complete tsfiles, and a incomplete tsFile.
> 
> If a user create a hive table referring to this folder, then how many
> tsfiles that Hive will observe?
> 
> Then someone writes data into IoTDB, and finally there are 5 tsfiles, and a
> incomplete tsfile, then how many tsfiles that Hive will observe?
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Jialin Qiao  于2019年10月14日周一 上午8:14写道:
> 
> > Hi,
> >
> > > 1. Hive 3.x needs hadoop 3.x, but our project is based on the hadoop
> > 2.x, and these two are incompatible.
> >
> > We could upgrade the version If users need 3.x. There isn't any particular
> > reason that we based on hadoop-2.x...
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
> > > -原始邮件-
> > > 发件人: "田原" 
> > > 发送时间: 2019-10-13 14:16:32 (星期日)
> > > 收件人: dev@iotdb.apache.org
> > > 抄送:
> > > 主题: Re: Questions about Hive Integration
> > >
> > > Hi,
> > >
> > > 1. Hive 3.x needs hadoop 3.x, but our project is based on the hadoop
> > 2.x, and these two are incompatible.
> > >
> > > 2. Now, we don't support insert operation in hive. Only query operation
> > is supported.
> > >
> > > 3. If there are more than one tsfile in one folder, all the tsfiles in
> > that folder will be pre-read by InputFormat, and then filter them by the
> > table name(i.e. device_id) and the field (i.e. sensor_id). So the result
> > set returned to user will only contains data they want.
> > >
> > >
> > > > -原始邮件-
> > > > 发件人: "Xiangdong Huang" 
> > > > 发送时间: 2019-10-12 14:38:36 (星期六)
> > > > 收件人: dev@iotdb.apache.org
> > > > 抄送:
> > > > 主题: Questions about Hive Integration
> > > >
> > > > Hi,
> > > >
> > > > Today I meets some users from Inspur (a famous Chinese company), and
> > they
> > > > address some requirements about IoTDB with Hive.
> > > >
> > > > 1. First, they use hadoop 3.x.
> > > >
> > > > 2. Can we use Hive to write data back to HDFS with TsFile format?
> > > > e.g., HiveQL: create table . LOCATION HDFS_FILE
> > > >
> > > > 3. If Hive user uses Load command from a IoTDB's data folder, what
> > happens?
> > > > E.g., HiveQL: load data inpath 'iotdb/data/root.sg1/' into table
> > > > tablename...
> > > > Then, which Table files will be observed by Hive? If more TsFiles in
> > the
> > > > folder are generated by IoTDB, can Hive observes them automatically?
> > > >
> > > > Best,
> > > > ---
> > > > Xiangdong Huang
> > > > School of Software, Tsinghua University
> > > >
> > > >  黄向东
> > > > 清华大学 软件学院
> >


Re: Implement Delete Storage Group SQL interface [IOTDB-188]

2019-10-13 Thread Jialin Qiao
Hi,

> The related PR is #440. Multiple storage groups deletion will be suppoted, 
> like "SET STORAGE GROUP TO root.ln.wf01.wt01, root.ln.wf01.wt02, 
> root.ln.wf01.wt03".

As far as I know, we do not support setting multiple storage groups in one sql, 
but it would be great if we support.

> I added and implemented "analyzeMetaDataDeleteFileLevel" method in 
> LogicalGenerator.java, to generate the logical plan of storage group deletion.

'FileLevel' is the ancient version of 'StorageGroup', try to use 'StorageGroup' 
in all places.

Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "康愈圆" 
> 发送时间: 2019-10-12 10:37:36 (星期六)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Implement Delete Storage Group SQL interface [IOTDB-188]
> 
> Hi, all
> 
> The related PR is #440. Multiple storage groups deletion will be suppoted, 
> like "SET STORAGE GROUP TO root.ln.wf01.wt01, root.ln.wf01.wt02, 
> root.ln.wf01.wt03".
> 
> I implemented it by defining the grammer in ANTLR.
> 
> I added and implemented "analyzeMetaDataDeleteFileLevel" method in 
> LogicalGenerator.java, to generate the logical plan of storage group deletion.
> 
> In StorageEngine, "deleteStorageGroup" is also modified. I added "if" 
> statement to check whether the prosessor is null. If the storage group is not 
> in processorMap, it can be null. It happens when a user set a storage group 
> but doesn't insert any data in it. Then if the user tries to delete the 
> storage group, the previous code will throw null pointer exception, which 
> should be aviod.
> 
> Best,
> _
> Yuyuan KANG
> 
> > -原始邮件-
> > 发件人: "康愈圆" 
> > 发送时间: 2019-10-11 19:07:39 (星期五)
> > 收件人: dev@iotdb.apache.org
> > 抄送: 
> > 主题: Implement Delete Storage Group SQL interface [IOTDB-188]
> > 
> > Hi, all
> > 
> > 
> > I'm now working on issue #188. I will implement delete storage group sql 
> > interface.
> > 
> > 
> > 
> > 
> > Best,
> > ___
> > Yuyuan KANG


Re: Questions about Hive Integration

2019-10-13 Thread Jialin Qiao
Hi,

> 1. Hive 3.x needs hadoop 3.x, but our project is based on the hadoop 2.x, and 
> these two are incompatible.

We could upgrade the version If users need 3.x. There isn't any particular 
reason that we based on hadoop-2.x...

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "田原" 
> 发送时间: 2019-10-13 14:16:32 (星期日)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Questions about Hive Integration
> 
> Hi,
> 
> 1. Hive 3.x needs hadoop 3.x, but our project is based on the hadoop 2.x, and 
> these two are incompatible.
> 
> 2. Now, we don't support insert operation in hive. Only query operation is 
> supported.
> 
> 3. If there are more than one tsfile in one folder, all the tsfiles in that 
> folder will be pre-read by InputFormat, and then filter them by the table 
> name(i.e. device_id) and the field (i.e. sensor_id). So the result set 
> returned to user will only contains data they want.
> 
> 
> > -原始邮件-
> > 发件人: "Xiangdong Huang" 
> > 发送时间: 2019-10-12 14:38:36 (星期六)
> > 收件人: dev@iotdb.apache.org
> > 抄送: 
> > 主题: Questions about Hive Integration
> > 
> > Hi,
> > 
> > Today I meets some users from Inspur (a famous Chinese company), and they
> > address some requirements about IoTDB with Hive.
> > 
> > 1. First, they use hadoop 3.x.
> > 
> > 2. Can we use Hive to write data back to HDFS with TsFile format?
> > e.g., HiveQL: create table . LOCATION HDFS_FILE
> > 
> > 3. If Hive user uses Load command from a IoTDB's data folder, what happens?
> > E.g., HiveQL: load data inpath 'iotdb/data/root.sg1/' into table
> > tablename...
> > Then, which Table files will be observed by Hive? If more TsFiles in the
> > folder are generated by IoTDB, can Hive observes them automatically?
> > 
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> > 
> >  黄向东
> > 清华大学 软件学院


[jira] [Created] (IOTDB-251) Improve query data structure in RPC

2019-10-12 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-251:
-

 Summary: Improve query data structure in RPC
 Key: IOTDB-251
 URL: https://issues.apache.org/jira/browse/IOTDB-251
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: Jialin Qiao


Similar to the RowBatch in the write interface, we could use primitive data 
arrays in ResultSet in the RPC query, instead of constructing record each line.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [discussion] Release of IoTDB 0.8.1 RC2

2019-10-11 Thread Jialin Qiao
Hi Kevin,

> Question: The release notes don't mention the speed increase.  I thought 
> there was a substantial speed increase with 0.8.1.  Did I misunderstand that?

The 0.8.1 is just a bug-fix version of 0.8.0, so there isn't any speed 
increasing. Indeed we have a substantial speed increase over 0.8.0, which will 
be contained in the 0.9.0. Looking forward to the release of 0.9.0 :) 

> Question: Should the client script be in an sbin directory?  I would think 
> just bin would be better.  Not a release blocker.

+1, we could change it later. Like Spark did, sbin includes the server script 
and bin includes client or tools script.

Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Kevin A. McGrail" 
> 发送时间: 2019-10-11 22:09:28 (星期五)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [discussion] Release of IoTDB 0.8.1 RC2
> 
> I vote +1 having followed this list:
> 
> downloaded pre-release/iotdb-0.8.1 rc2 binary release from  
> https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.8.1/rc2/apache-iotdb-0.8.1-incubating-bin.zip
> 
> confirmed sha512 matches
> 604a9ba330237766f8f3488a13cde63b573ad005f29a3f9748535a5ad65324b5fc75c25d44f805b44919f2b1405fe4889905f0e31948cf13beade48644d08263
>   apache-iotdb-0.8.1-incubating-bin.zip
> 
> 
> I Looked at the NOTICE, LICENSE and DISCLAIMER
> I loooked at the README.md and the RELEASE_NOTES.md
> 
> Question: The release notes don't mention the speed increase.  I thought 
> there was a substantial speed increase with 0.8.1.  Did I misunderstand that?
> 
> ran start iotdb server
> start client
> 
> Question: Should the client script be in an sbin directory?  I would think 
> just bin would be better.  Not a release blocker.
> 
> Executed the following successfully.
> SET STORAGE GROUP TO root.ln.wf01.wt01;
> CREATE TIMESERIES root.ln.wf01.wt01.status WITH DATATYPE=BOOLEAN, 
> ENCODING=PLAIN
> CREATE TIMESERIES root.ln.wf01.wt01.temperature WITH DATATYPE=FLOAT, 
> ENCODING=PLAIN
> CREATE TIMESERIES root.ln.wf01.wt01.hardware WITH DATATYPE=INT32, 
> ENCODING=PLAIN
> INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)  
> values(1, 1.1, false, 11)
> INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) 
> values(2, 2.2, true, 22)
> INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) 
> values(3, 3.3, false, 33 )
> INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)  
> values(4, 4.4, false, 44)
> INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) 
> values(5, 5.5, false, 55)
> INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)  
> values(6, 1.1, false, 11)
> INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) 
> values(7, 2.2, true, 22)
> INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) 
> values(8, 3.3, false, 33 )
> INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)  
> values(9, 4.4, false, 44)
> INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) 
> values(10, 5.5, false, 55)
> Select * from root;
> 
> I did not do the source release as my system is not working correctly with 
> mvn but based on others feedback, I feel confident in the binary release test.
> 
> Regards,
> KAM 
> 
> On 10/10/2019 9:48 PM, Xiangdong Huang wrote:
> > Hi kevin,
> >
> > Ah, as after the voting in this mailing list, we still need to vote on
> > general@ to get at least 3 +1 from IPMCs.
> > But I noticed that when Julian released 0.8.0 and got more than 3 +1 from
> > IPMCs in dev@iotdb.apache.org mail list,
> > he started a new lazy vote in the general@, which I thought it is because
> > there were 3 IPMCs voting it already.
> > I think it is convenient..
> >
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> >  黄向东
> > 清华大学 软件学院
> >
> >
> > Kevin A. McGrail  于2019年10月11日周五 上午9:01写道:
> >
> >> Are we needed for a release vote?  I usually like to make sure you guys
> >> are doing things correctly and try to leave you to your own devices...
> >>
> >> On 10/10/2019 8:49 PM, Xiangdong Huang wrote:
> >>> Hi,
> >>>
> >>> There are 7 PPMCs voting for the release now. How about IPMC and mentors?
> >>>
> >>> Best,
> >>> ---
> >>> Xiangdong Huang
> >>> School of Software, Tsinghua University
> >>>
> >>>  黄向东
> >>> 清华大学 软件学院
> >>>
> >>>

Re: [VOTE] Apache IoTDB 0.8.1 (incubating) RC2 release

2019-10-11 Thread Jialin Qiao
Hi,

I submitted a PR [1] for it.

[1] https://github.com/apache/incubator-iotdb/pull/447

Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Willem Jiang" 
> 发送时间: 2019-10-12 09:02:00 (星期六)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [VOTE] Apache IoTDB 0.8.1 (incubating) RC2 release
> 
> For the jar in the source release, we could address it by adding below
> line into .gitignore file.
> #skip the maven-wrapper.jar
> .mvn/wrapper/maven-wrapper.jar
> 
> I just create a JIRA[1] for it.
> [1]https://issues.apache.org/jira/browse/IOTDB-250
> 
> Willem Jiang
> 
> Twitter: willemjiang
> Weibo: 姜宁willem
> 
> On Sat, Oct 12, 2019 at 5:21 AM Justin Mclean  
> wrote:
> >
> > Hi,
> >
> > Sorry but it’s -1 (binding) as there is a compiled binary in the source 
> > release [2]
> >
> > If you had used the the work in progress disclaimer that would of been 
> > allowed.
> >
> > I checked:
> > - incubating in name
> > - signature and hashes fine
> > - disclaimer exists
> > - LICENSE is fine ( although it would be good to mention what license the 
> > man wrapper is under)
> > - NOTICE incorrectly has "Copyright 2019 and onwards” in it
> > - the is a jar in the source release this is not allowed [2]
> > - all source files have ASF headers
> >
> > It’s a good idea to use a checklist like this [1] to check the release. If 
> > so you had done so you should have picked up the binary in the source 
> > release and the minor NOTICE issues.
> >
> > Looking at the binary the licensing is not correct as you have multiple 
> > versions of different licenses but only include the text of one of them. 
> > For instance you list two MIT licensed pieces of software but you only have 
> > one MIT license, note that MIT licenses differ and have different copyright 
> > lines. The binary LICENSE file is also missing a number of licenses:
> > airline
> > guava
> > hawtjni
> > jansi
> > tongfei progressbar
> > javax inject
> >
> > Thanks,
> > Justin
> >
> > 1. 
> > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist
> > 2. .mvn/wrapper/maven-wrapper.jar


Re: [IOTDB-208] design of BloomFilter

2019-10-11 Thread Jialin Qiao
Hi,

Glad to see the discussion of design, with some advices:

- The theme should be a short description of the function, such as bloomfilter.
- It's better to roughy describe the issue in the mail. (There is an issue 
report mail about BloomFilter, which you can reply to)
- For convenience and make sure that others could see the design (I'm not sure 
whether others could access the tower, even if you share a public url), it's 
better to put the design in the mail, and in English... 

Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "南京大学软件学院薛恺丰" <827011...@qq.com>
> 发送时间: 2019-10-11 17:17:19 (星期五)
> 收件人: dev 
> 抄送: 
> 主题: [IOTDB-208] design
> 
> Hi,
> I'm working on this issue, and here is my design and thought. 
> https://tower.im/teams/138444/todos/7969/
> If you have some question please discuss with me :D


Re: [DISCUSS] (IOTDB-240) query problem for iotdb filter

2019-10-09 Thread Jialin Qiao
Hi,

+1 for reopen it. The error messages should be consistent in these 4 cases.

--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Lei Rui" 
> 发送时间: 2019-10-09 18:31:21 (星期三)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re:  [DISCUSS] (IOTDB-240) query problem for iotdb filter
> 
> Hi,
> 
> 
> Even settled with response 1, I don't think this issue of inconsistent error 
> messages is solved.
> 
> 
> case 1: select ZT1 from root.demo
> case 2: select ZT1,ZT31 from root.demo 
> case 3: select ZT1,ZT31 from root.demo where ZT1>0
> In the above cases, "root.demo.ZT31" exists and "root.demo.ZT1" doesn't exist.
> The merged pr#423 for this issue checks the existence of every path extracted 
> from the SELECT & FROM clause.
> Therefore, the iotdb-client in case 1&2&3 will now respond with a consistent 
> error message - "Msg: Execute statement error: Path: "root.demo.ZT1" not 
> corresponding any known time series".
> 
> 
> But consider case 4: "select ZT31 from root.demo where ZT1>0" and the path 
> "root.demo.ZT1" in the filter expression doesn't exist. 
> The iotdb-client will respond with the original error message - "Msg: Execute 
> statement error: Index: 0, Size: 0" , which is still inconsistent with case 
> 1&2&3.
> 
> 
> So, I suggest this issue be reopened.
> 
> 
> Lei Rui
> On 10/9/2019 09:34,Jialin Qiao wrote:
> Hi,
> 
> I prefer Response 1.
> 
> When users query some paths that do not exist, it means there are logical 
> errors and we'd better reject the request to let users fix it.
> 
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> -原始邮件-
> 发件人: suyue 
> 发送时间: 2019-10-08 20:54:59 (星期二)
> 收件人: dev@iotdb.apache.org
> 抄送:
> 主题: Re: [DISCUSS] (IOTDB-240) query problem for iotdb filter
> 
> Hi, all.
> 
> I tried to make the following queries in MySQL:(There is a ‘projectID' column 
> in the 'CONFIG' table but no xx column.)
> select projectID, xx from CONFIG;
> Then, I got :[Code: 1054, SQL State: 42S22]  Unknown column 'xx' in 'field 
> list’
> 
> So, I prefer to report errors directly when there are non-existent columns in 
> the query statement, which is Response 1.
> 
> 
> 
> 
> 
> 在 2019年10月8日,下午7:14,Lei Rui  写道:
> 
> Hi all,
> I want to have more discussions on the responses of selecting non-existent 
> timeseries.
> Let's focus on the condition 2 - "select ZT1,ZT31 from root.demo" where 
> root.demo.ZT1 doesn't exist.
> There can be 3 responses as follows.
> Response 1: reject service and return an error message.
> Response 2: provide service by neglecting "ZT1" and directly return the 
> result table of "select ZT31 from root.demo".
> Response 3: provide service by returning the result table with fields under 
> the column "root.demo.ZT1" being null.
> From my side, I definitely disapprove response 3 because of the possible 
> confusion raised about the schema.
> My question is which one do you prefer, response 1 or response 2?
> I myself prefer response 2 to response 1, because response 2 not only conveys 
> clearly the message that "root.demo.ZT1" doesn't exist, but also returns 
> existent data.
> Sincerely,
> Lei Rui
> 
> 
> On 9/26/2019 14:52,南京大学软件学院薛恺丰<827011...@qq.com> wrote:
> Hi~ I'm working on this issue and I think "both condition 2 and 3 should 
> respond with error message" is better
> 
> 
> 
> 
> -- 原始邮件 --
> 发件人: "Yuan Tian (Jira)";
> 发送时间: 2019年9月25日(星期三) 晚上9:56
> 收件人: "dev";
> 
> 主题: [jira] [Created] (IOTDB-240) query problem for iotdb filter
> 
> 
> 
> EJTTianyu created IOTDB-240:
> ---
> 
> Summary: query problem for iotdb filter
> Key: IOTDB-240
> URL: https://issues.apache.org/jira/browse/IOTDB-240
> Project: Apache IoTDB
> Issue Type: Bug
> Reporter: EJTTianyu
> 
> 
> When querying a timeseries that does not exist, iotdb shows different 
> behavior for SQL with or without filter.
> After starting a new iotdb instance, The following steps in iotdb-client 
> below could reproduce the problem.
> Step 1: set storage group to root.demo
> Step 2: CREATE TIMESERIES root.demo.ZT31 WITH DATATYPE=INT64, ENCODING=RLE
> Step 3:insert into root.demo(timestamp,ZT31) values(1,1)
> when using sql:
> 1.select ZT1 from root.demo
> iotdb-client responds with message "Msg: Execute statement error: do not 
> select any existing series"
>

Re: [DISCUSS] (IOTDB-240) query problem for iotdb filter

2019-10-08 Thread Jialin Qiao
Hi,

I prefer Response 1. 

When users query some paths that do not exist, it means there are logical 
errors and we'd better reject the request to let users fix it.

--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: suyue 
> 发送时间: 2019-10-08 20:54:59 (星期二)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [DISCUSS] (IOTDB-240) query problem for iotdb filter
> 
> Hi, all.
> 
>  I tried to make the following queries in MySQL:(There is a ‘projectID' 
> column in the 'CONFIG' table but no xx column.)
> select projectID, xx from CONFIG;
> Then, I got :[Code: 1054, SQL State: 42S22]  Unknown column 'xx' in 'field 
> list’
> 
> So, I prefer to report errors directly when there are non-existent columns in 
> the query statement, which is Response 1.
> 
> 
> 
> 
> 
> > 在 2019年10月8日,下午7:14,Lei Rui  写道:
> > 
> > Hi all,
> > I want to have more discussions on the responses of selecting non-existent 
> > timeseries.
> > Let's focus on the condition 2 - "select ZT1,ZT31 from root.demo" where 
> > root.demo.ZT1 doesn't exist.
> > There can be 3 responses as follows.
> > Response 1: reject service and return an error message.
> > Response 2: provide service by neglecting "ZT1" and directly return the 
> > result table of "select ZT31 from root.demo".
> > Response 3: provide service by returning the result table with fields under 
> > the column "root.demo.ZT1" being null.
> > From my side, I definitely disapprove response 3 because of the possible 
> > confusion raised about the schema.
> > My question is which one do you prefer, response 1 or response 2?
> > I myself prefer response 2 to response 1, because response 2 not only 
> > conveys clearly the message that "root.demo.ZT1" doesn't exist, but also 
> > returns existent data.
> > Sincerely,
> > Lei Rui
> > 
> > 
> > On 9/26/2019 14:52,南京大学软件学院薛恺丰<827011...@qq.com> wrote:
> > Hi~ I'm working on this issue and I think "both condition 2 and 3 should 
> > respond with error message" is better
> > 
> > 
> > 
> > 
> > -- 原始邮件 --
> > 发件人: "Yuan Tian (Jira)";
> > 发送时间: 2019年9月25日(星期三) 晚上9:56
> > 收件人: "dev";
> > 
> > 主题: [jira] [Created] (IOTDB-240) query problem for iotdb filter
> > 
> > 
> > 
> > EJTTianyu created IOTDB-240:
> > ---
> > 
> > Summary: query problem for iotdb filter
> > Key: IOTDB-240
> > URL: https://issues.apache.org/jira/browse/IOTDB-240
> > Project: Apache IoTDB
> > Issue Type: Bug
> > Reporter: EJTTianyu
> > 
> > 
> > When querying a timeseries that does not exist, iotdb shows different 
> > behavior for SQL with or without filter.
> > After starting a new iotdb instance, The following steps in iotdb-client 
> > below could reproduce the problem.
> > Step 1: set storage group to root.demo
> > Step 2: CREATE TIMESERIES root.demo.ZT31 WITH DATATYPE=INT64, ENCODING=RLE
> > Step 3:insert into root.demo(timestamp,ZT31) values(1,1)
> > when using sql:
> > 1.select ZT1 from root.demo
> > iotdb-client responds with message "Msg: Execute statement error: do not 
> > select any existing series"
> > 2.select ZT1,ZT31 from root.demo
> > iotdb-client responds with
> > +---+--+
> > |   Time|root.demo.ZT31|
> > +---+--+
> > |  1970-01-01T08:00:00.001+08:00| 1|
> > +---+--+
> > 3.select ZT1,ZT31 from root.demo where ZT1>0
> > iotdb-client responds with message "Msg: Execute statement error: Index: 0, 
> > Size: 0"
> > 
> > Timeseries ZT1 does not exist in condition 2 and condition 3. However, 
> > without using ZT1 as filter to query  multiple sensors(at least one 
> > exists), iotdb could respond with datapoints. When using ZT1 as filter, the 
> > returned msg confused users.
> > 
> > In my opinion, both condition 2 and 3 should respond with message "Msg: 
> > Execute statement error: do not select any existing series". Or condition 2 
> > responds with datapoints, condition 3 responds with "Msg: Execute statement 
> > error: do not select any existing series".
> > 
> > 
> > 
> > 
> > --
> > This message was sent by Atlassian Jira
> > (v8.3.4#803005)
> 


Re: [VOTE] Apache IoTDB 0.8.1 (incubating) RC2 release

2019-10-08 Thread Jialin Qiao
Hi,

+1 (binding)

I checked:

Binary release:
has NOTICE, LICENSE and DISCLAIMER
Start server (ok)
Start client (ok)
The version printed in client is 0.8.1 (ok)
compatible with 0.8.0, both client-0.8.1 with server-0.8.0 and client-0.8.0 
with server-0.8.1
Example statements are as follow:

```
SET STORAGE GROUP TO root.turbine;
CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=FLOAT, ENCODING=RLE;
insert into root.turbine.d1(timestamp,s0) values(1,24.5);
select * from root;
```

Source release:
has NOTICE, LICENSE and DISCLAIMER
mvn clean package (ok)
test all passed on mac os + jdk1.8(ok)

minor issue: 

filters in TsFileRead in the example of tsfile module are not compatible with 
the file written by TsFileWrite

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-10-05 19:02:47 (星期六)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: [VOTE] Apache IoTDB 0.8.1 (incubating) RC2 release
> 
> Hi all,
> 
> Now I open a new ticket for Apache IoTDB (incubating) 0.8.1 RC2, which fixs
> some issues comparing with RC1. You can find the details at the end of this
> email.
> 
> Apache IoTDB (incubating) 0.8.1 is a bug-fix version from 0.8.0. You can
> get its mainly changes from [5].
> 
> Apache IoTDB (Incubating) 0.8.1 has been staged under [2] and it’s time to
> vote
> on accepting it for release.  All Maven artifacts are available under [1].
> If approved we will seek final release approval from the IPMC.
> Voting will be open for 72hr.
> 
> A minimum of 3 binding +1 votes and more binding +1 than binding -1
> are required to pass.
> 
> Release tag: release/0.8.1
> Hash for the release tag: 37b213b6ab6fc8ca13ab60f21dff099042a0e295
> 
> Per [3] "Before voting +1 PMC/PPMC members are required to download
> the signed source code package, compile it as provided, and test
> the resulting executable on their own platform, along with also
> verifying that the package meets the requirements of the ASF policy
> on releases."
> 
> You can achieve the above by following [4].
> 
> [ ]  +1 accept (indicate what you validated - e.g. performed the non-RM
> items in [4])
> [ ]  -1 reject (explanation required)
> 
> 
> Difference with RC1:
> 
> - PGP signing: there is no established trust chain to any other Apache ...
> - Running IoTDB Server according to information in README.md: MINOR I am
> using OpenJDK 13 and I get an error message that IoTDB only runs on Java 8
> and 11 ... this sort of contradicts the README which states X >= 1.8
> - I would suggest to add an Apache Header to the ".checkstyle" XML file
> - It seems the Hadoop module still contains SNAPSHOT versions: MINOR
> - Able to run the server and client: MINOR Same finding as with the source
> distribution ... not able to run with OpenJDK 13
> - Major bug: The version printed in client is still 0.8.0
> - NOTICE-binary HIVE shows 2008-2018 while NOTICE HIVE shows 2008-2019.
> (Now they are consistent with the notice file at github.com/apache/hive,
> which says 2008-2018)
> 
> [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1005
> [2] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.8.1/rc2
> [3] https://www.apache.org/dev/release.html#approving-a-release
> [4]
> https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
> [5]
> https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.8.1/rc2/RELEASE_NOTES.md
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院


[jira] [Closed] (IOTDB-191) Enrich Session interfaces

2019-10-07 Thread Jialin Qiao (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jialin Qiao closed IOTDB-191.
-
Fix Version/s: 0.9.0
   Resolution: Fixed

> Enrich Session interfaces
> -
>
> Key: IOTDB-191
> URL: https://issues.apache.org/jira/browse/IOTDB-191
> Project: Apache IoTDB
>  Issue Type: New Feature
>    Reporter: Jialin Qiao
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 0.9.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> The module "Session" is an RPC interface for users, which is more efficient 
> than JDBC.
> Currently, it only has 3 interfaces: (1) ser storage group (2) create time 
> series (3) insert batch. 
> Other interfaces such as delete time series, the query could be added.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Closed] (IOTDB-180) Get rid of JSON format in "show timeseries"

2019-10-07 Thread Jialin Qiao (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jialin Qiao closed IOTDB-180.
-
Fix Version/s: 0.9.0
   Resolution: Fixed

> Get rid of JSON format in "show timeseries"
> ---
>
> Key: IOTDB-180
> URL: https://issues.apache.org/jira/browse/IOTDB-180
> Project: Apache IoTDB
>  Issue Type: Wish
>Reporter: Jialin Qiao
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 0.9.0
>
> Attachments: image-2019-08-27-09-56-20-654.png, 
> image-2019-08-27-09-56-28-577.png
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently, "show timeseries" prints result in JSON format, which is not 
> friendly compared to "show timeseries root" in table format.
> !image-2019-08-27-09-56-20-654.png|width=463,height=538!
> !image-2019-08-27-09-56-28-577.png|width=694,height=458!
> Intuitively, I would expect that "show timeseries" works as "show timeseries 
> root".
> Therefore, when users input a "show timeseries", I suggest that we treat it 
> as "show timeseries root" and print results in table format.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Closed] (IOTDB-98) Batch insertion in Server

2019-10-07 Thread Jialin Qiao (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-98?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jialin Qiao closed IOTDB-98.

Resolution: Done

> Batch insertion in Server
> -
>
> Key: IOTDB-98
> URL: https://issues.apache.org/jira/browse/IOTDB-98
> Project: Apache IoTDB
>  Issue Type: Improvement
>    Reporter: Jialin Qiao
>Priority: Minor
>
> Currently, when the client sends a batch of statements (insert/delete) to the 
> server, the server will get each statement from the batch and execute one by 
> one. For each statement, the corresponding FileNodeProcessor (for a storage 
> group) will get a write lock and then execute the statement, which causes 
> unnecessary lock contention. 
> It can be improved by getting a write lock once and executing a batch of 
> statements. We can first consider the scenario that the statements in a batch 
> are all insertions.
> The new process mainly contains 3 steps:
> (1) Group the insert statement by storage group while retaining the original 
> order in the batch.
> (2) For each storage group, get the write lock and execute all statements 
> belong to it.
> (3) Set the return code of each statement to the origin position in the batch.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [VOTE] Apache IoTDB 0.8.1 (incubating) RC1

2019-09-28 Thread Jialin Qiao
Hi,

-1 (binding)

Major bug: The version printed in client is still 0.8.0

Others are all ok, I checked:

Binary release:
has NOTICE, LICENSE and DISCLAIMER
Start server ok
Start client ok
compatible with 0.8.0, both client-0.8.1 with server-0.8.0 and client-0.8.0 
with server-0.8.1
Example statement see below:

```
SET STORAGE GROUP TO root.turbine;
CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=FLOAT, ENCODING=RLE;
insert into root.turbine.d1(timestamp,s0) values(1,24.5);
```

Source release:
has NOTICE, LICENSE and DISCLAIMER
mvn clean package

Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Christofer Dutz" 
> 发送时间: 2019-09-28 21:47:51 (星期六)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: [VOTE] Apache IoTDB 0.8.1 (incubating) RC1
> 
> +1 (binding) 
> However with some minor findings see below.
> 
> Please also automatically carry forward this +1 to the incubator voting
> 
> In preparation I went to the KEYS file and searched for the latest entry, 
> copied the fingerprint of that key and searched for it with my PGP tool. I 
> found the key and imported it. The email address reported matched the one I 
> expected.
> 
> - Validated the SHA512 checksums: OK
> - Validated the PGP signature: OK (However there is no established trust 
> chain to any other Apache ... actually to no-one ... @Xiangdong you really 
> should go to one of the Key-Signing-Events and have your key signed by other 
> Apaches)
> - LICENSE, NOTICE, README, RELEASE_NOTES exist: OK
> - Checked contents of LICENSE, NOTICE, README, RELEASE_NOTES: OK
> - Build IoTDB Server according to information in README.md: OK
> - Running IoTDB Server according to information in README.md: MINOR (I am 
> using OpenJDK 13 and I get an error message that IoTDB only runs on Java 8 
> and 11 ... this sort of contradicts the README which states X >= 1.8)
> - Running IoTDB Server with OpenJDK 11.0.2: OK
> - Build IoTDB Client according to information in README.md: OK
> - Running IoTDB Client according to information in README.md: OK
> - Running a full maven build with all tests (mvn clean install): OK
> - RAT didn’t report anything concerning (All binary files are image files): 
> OK (I would suggest to add an Apache Header to the ".checkstyle" XML file)
>  - It seems the Hadoop module still contains SNAPSHOT versions: MINOR
> - Binary distribution unpacks: OK
> - Able to run the server and client: MINOR (Same finding as with the source 
> distribution ... not able to run with OpenJDK 13)
> - Binary distribution contains: NOTICE and LICENSE that mention additional 
> elements of the binary distribution: OK
> 
> 
> 
> 
> 
> Am 28.09.19, 07:43 schrieb "Xiangdong Huang" :
> 
> Hi all,
> 
> First, thanks to all of you for the support. This is my first time to be 
> as
> a RM of Apache project and I want to thank all guys. It's because all of
> you are so patient to answer my question that I can do this release.
> Special thanks go to our Mentors, our committers, and a lot of documents
> from PLC4X, Apache, Spark, etc..
> 
> Apache IoTDB (incubating) 0.8.1 is a bug-fix version from 0.8.0. You can
> get its mainly changes from [5].
> 
> Apache IoTDB (Incubating) 0.8.1 has been staged under [2] and it’s time to
> vote
> on accepting it for release.  All Maven artifacts are available under [1].
> If approved we will seek final release approval from the IPMC.
> Voting will be open for 72hr.
> 
> A minimum of 3 binding +1 votes and more binding +1 than binding -1
> are required to pass.
> 
> Release tag: release/0.8.1
> Hash for the release tag: b664e8778d1221c4eb92205411e58c392214b3ab
> 
> Per [3] "Before voting +1 PMC/PPMC members are required to download
> the signed source code package, compile it as provided, and test
> the resulting executable on their own platform, along with also
> verifying that the package meets the requirements of the ASF policy
> on releases."
> 
> You can achieve the above by following [4].
> 
> [ ]  +1 accept (indicate what you validated - e.g. performed the non-RM
> items in [4])
> [ ]  -1 reject (explanation required)
> 
> [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1004
> [2] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.8.1/rc1
> [3] https://www.apache.org/dev/release.html#approving-a-release
> [4]
> 
> https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
> [5]
> https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.8.1/rc1/README.md
> 
> Best,
> --
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 


Re: (IOTDB-246) No docs for Hadoop-Connector in IoTDB websites.

2019-09-28 Thread Jialin Qiao
Hi Yuan Tian,

Thanks! A minor suggestion: I notice that your PR only adds one file but with 
36 commits, which contains the hadoop-connector's commits.

Maybe it is because that you checkout this branch from the hadoop-connector, 
not the current master. You can try to pull master and then checkout a new 
branch from master next time.

Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Yuan Tian" 
> 发送时间: 2019-09-28 21:55:56 (星期六)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: (IOTDB-246) No docs for Hadoop-Connector in IoTDB websites.
> 
> Hi,
> 
> I have added the Chinese version for Hadoop-connector docs. The PR link is 
> https://github.com/apache/incubator-iotdb/pull/438 
> <https://github.com/apache/incubator-iotdb/pull/438>.
> 
> Best,
> --
> Yuan Tian
> School of Software, Tsinghua University
> 
> 田原
> 清华大学 软件学院
> 
> 
> > 在 2019年9月28日,下午5:15,Jialin Qiao  写道:
> > 
> > Hi Yuan Tian,
> > 
> > Thanks for your contribution. I have reviewed your PR and approved it.
> > 
> > If you have more time, you can add the Chinese doc :)
> > 
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> > 
> > 乔嘉林
> > 清华大学 软件学院
> > 
> >> -原始邮件-
> >> 发件人: "Yuan Tian" 
> >> 发送时间: 2019-09-28 16:56:37 (星期六)
> >> 收件人: dev@iotdb.apache.org
> >> 抄送: 
> >> 主题: Re: (IOTDB-246) No docs for Hadoop-Connector in IoTDB websites.
> >> 
> >> Hi,
> >> 
> >> I have added the docs for Hadoop-connector in our IoDTB website. The pr 
> >> link is https://github.com/apache/incubator-iotdb/pull/437 
> >> <https://github.com/apache/incubator-iotdb/pull/437>.
> >> 
> >> Best,
> >> --
> >> Yuan Tian
> >> School of Software, Tsinghua University
> >> 
> >> 田原
> >> 清华大学 软件学院
> >> 
> >> 
> >>> 在 2019年9月28日,下午4:42,Yuan Tian  写道:
> >>> 
> >>> Hi,
> >>> 
> >>> I’m working on this issue.
> >>> 
> >>> 
> >>> 
> >>> Best,
> >>> --
> >>> Yuan Tian
> >>> School of Software, Tsinghua University
> >>> 
> >>> 田原
> >>> 清华大学 软件学院
> >>> 
> >>> 
> >>>> 在 2019年9月28日,下午4:41,Yuan Tian (Jira)  写道:
> >>>> 
> >>>> Yuan Tian created IOTDB-246:
> >>>> ---
> >>>> 
> >>>>   Summary: No docs for Hadoop-Connector in IoTDB websites.
> >>>>   Key: IOTDB-246
> >>>>   URL: https://issues.apache.org/jira/browse/IOTDB-246
> >>>>   Project: Apache IoTDB
> >>>>Issue Type: Improvement
> >>>>  Reporter: Yuan Tian
> >>>> 
> >>>> 
> >>>> There are no docs for Hadoop-Connector in IoTDB websites. Users may not 
> >>>> know how to use the function.
> >>>> 
> >>>> 
> >>>> 
> >>>> --
> >>>> This message was sent by Atlassian Jira
> >>>> (v8.3.4#803005)
> >> 
> 


Re: (IOTDB-246) No docs for Hadoop-Connector in IoTDB websites.

2019-09-28 Thread Jialin Qiao
Hi Yuan Tian,

Thanks for your contribution. I have reviewed your PR and approved it.

If you have more time, you can add the Chinese doc :)

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Yuan Tian" 
> 发送时间: 2019-09-28 16:56:37 (星期六)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: (IOTDB-246) No docs for Hadoop-Connector in IoTDB websites.
> 
> Hi,
> 
> I have added the docs for Hadoop-connector in our IoDTB website. The pr link 
> is https://github.com/apache/incubator-iotdb/pull/437 
> <https://github.com/apache/incubator-iotdb/pull/437>.
> 
> Best,
> --
> Yuan Tian
> School of Software, Tsinghua University
> 
> 田原
> 清华大学 软件学院
> 
> 
> > 在 2019年9月28日,下午4:42,Yuan Tian  写道:
> > 
> > Hi,
> > 
> > I’m working on this issue.
> > 
> > 
> > 
> > Best,
> > --
> > Yuan Tian
> > School of Software, Tsinghua University
> > 
> > 田原
> > 清华大学 软件学院
> > 
> > 
> >> 在 2019年9月28日,下午4:41,Yuan Tian (Jira)  写道:
> >> 
> >> Yuan Tian created IOTDB-246:
> >> ---
> >> 
> >>Summary: No docs for Hadoop-Connector in IoTDB websites.
> >>Key: IOTDB-246
> >>URL: https://issues.apache.org/jira/browse/IOTDB-246
> >>Project: Apache IoTDB
> >> Issue Type: Improvement
> >>   Reporter: Yuan Tian
> >> 
> >> 
> >> There are no docs for Hadoop-Connector in IoTDB websites. Users may not 
> >> know how to use the function.
> >> 
> >> 
> >> 
> >> --
> >> This message was sent by Atlassian Jira
> >> (v8.3.4#803005)
> 


Re: Different types of measurements that have same name in one storage group should be supported.

2019-09-27 Thread Jialin Qiao
Hi,

1  for this. Each time series should be independent because we create them 
separately in IoTDB.

This is due to the design of TsFile, maybe we should redesign the TsFile 
structure.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Yuan Tian (Jira)" 
> 发送时间: 2019-09-27 20:32:00 (星期五)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: [jira] [Created] (IOTDB-245) Different types of measurements that have 
> same name in one storage group should be supported.
> 
> Yuan Tian created IOTDB-245:
> ---
> 
>  Summary: Different types of measurements that have same name in 
> one storage group should be supported.
>  Key: IOTDB-245
>  URL: https://issues.apache.org/jira/browse/IOTDB-245
>  Project: Apache IoTDB
>   Issue Type: Improvement
> Reporter: Yuan Tian
> 
> 
> If we have two devices named d1 and d2, and both of them have a sensor named 
> s1, d1.s1 and d2.s1 must have the same type in current version. So if you 
> really have d1.s1 which has float type and d2.s1 which has int32 type, you 
> have to separate them into two different storage groups which is unreasonable.
> 
> So we want to remove the restriction about measurements' type of same name in 
> one storage group.
> 
> 
> 
> --
> This message was sent by Atlassian Jira
> (v8.3.4#803005)


Re: Add interfaces for querying device or timeseries number

2019-09-26 Thread Jialin Qiao
Hi,

>   2.  The "LEVEL" count from "root" no matter what users input:
> count timeseries root.sg1 group by level=2

I prefer this one, in which level is the global level from root. root is the 
level 0.

No matter which one we choose, we should make our documents clear for users and 
give some examples.

Thanks.
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jack Tsai" 
> 发送时间: 2019-09-27 07:41:45 (星期五)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: 回覆: Add interfaces for querying device or timeseries number
> 
> Hi,
> 
> I want to make a discussion about the interfaces about querying the number of 
> timeseries.
> 
> Now, there are two kinds of implementation for users to make a query.
> 
> Example:
> 
> The following querying results are both the timeseries number of root.sg1
> 
>   1.  The "LEVEL" count from the user specified prefix:
> count timeseries root.sg1 group by level=1
>   2.  The "LEVEL" count from "root" no matter what users input:
> count timeseries root.sg1 group by level=2
> 
> If you have any advice, please welcome to discuss with me. I need an 
> instruction to make this decision.
> 
> Best,
> Tsung-Han Tsai
> 
> 寄件者: Jack Tsai 
> 寄件日期: 2019年9月23日 下午 10:35
> 收件者: dev@iotdb.apache.org 
> 主旨: 回覆: Add interfaces for querying device or timeseries number
> 
> Hi,
> 
> I have found the bug inside this querying function which I have initially 
> implemented.
> 
> I did not consider the path condition that users might defined and all the 
> paths start from root. This should be started from the path which has been 
> specified. There is an example below:
> 
> So far, users could only get the timeseries number and the device number from 
> root, which looks like this:
> 
> count timeseries root group by level=3
> 
> and if users enter
> 
> count timeseries root.sg1 group by level=3
> 
> Then it will appear the same answer.
> 
> For this reason, I will change the underlying implementation of this querying 
> interface to improve the users experience.
> 
> Best regards,
> Tsung-Han Tsai
> 
> 寄件者: Jack Tsai 
> 寄件日期: 2019年8月23日 下午 04:00
> 收件者: dev@iotdb.apache.org 
> 主旨: Add interfaces for querying device or timeseries number
> 
> Hi,
> 
> I have finished the implementation of this improvement as adding interfaces 
> for querying device or timeseries number (the corresponding JIRA issue: 
> https://issues.apache.org/jira/browse/IOTDB-174).
> 
> I have created three sql command for users to done such operations mentioned 
> in the issue, which would be shown below.
> 
> Let's take a look with an example. There are 8 timeseries under the root, 
> which are:
> root.vehicle1.d1.s1, root.vehicle1.d1.s2, root.vehicle1.d2.s3, 
> root.vehicle1.d2.s4, root.vehicle2.d3.s5, root.vehicle2.d3.s6, 
> root.vehicle2.d4.s7, root.vehicle2.d4.s8
> 
>   1.  The interface for querying the timeseries number under the specified 
> path.
> COUNT TIMESERIES 
> 
> According to the example above:
> COUNT TIMESERIES root
> 
> Result:
> |Timeseries Num|
> |8   |
> 
>   2.  The interface for querying the node number at the given level in 
> current Metadata Tree (this could be used to query the number of devices)
> COUNT NODES  LEVEL=
> 
> According to the example above:
> COUNT NODES root LEVEL=3
> 
> Result:
> |Node Num|
> |4 |
> 
>   3.  The interface for querying the timeseries number of each node at the 
> given level (this could be used to query the number of sensors under each 
> device)
> COUNT TIMESERIES  GROUP BY LEVEL = 
> 
> According to the example above:
> COUNT TIMESERIES root GROUP BY LEVEL = 3
> 
> Result:
> |Node Path|Node Timeseries Num|
> |root.vehicle.d1|2  |
> |root.vehicle.d2|2  |
> |root.vehicle.d3|2  |
> |root.vehicle.d4|2  |
> 
> These three interfaces could provide users more query options.
> 
> Best regards,
> Tsung-Han Tsai


Re: [Documents] Add JavaDoc API for the official site

2019-09-26 Thread Jialin Qiao
Hi,

> I think they are referring to the Apache license having a block style comment 
> which mean it’s pick up incorrectly as a Java Doc comment. So in that case I 
> think they do want to use a single star comment.

+1

--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Justin Mclean" 
> 发送时间: 2019-09-26 16:04:43 (星期四)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [Documents] Add JavaDoc API for the official site
> 
> Hi,
> 
> > The double star comment block marks it as related to the code block below.
> 
> I think they are referring to teh Apache license having a block style comment 
> which mean it’s pick up incorrectly as a Java Doc comment. So in that case I 
> think they do want to use a single star comment.
> 
> Thanks,
> Justin


Re: [Documents] Add JavaDoc API for the official site

2019-09-25 Thread Jialin Qiao
Hi,

Thanks for your help. This update is accepted.
 
Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Justin Mclean" 
> 发送时间: 2019-09-26 12:52:40 (星期四)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [Documents] Add JavaDoc API for the official site
> 
> Hi,
> 
> > Ah... It is really a big issue... using "/*" rather than "/**" for Apache
> > Rat is correct.
> 
> I don’t think Rat cares which you use. ASF policy certainly doesn’t care. [1]
> 
> Thanks,
> Justin
> 
> 1. https://www.apache.org/legal/src-headers.html#headers


Re: Release of 0.8.1

2019-09-25 Thread Jialin Qiao
Hi,

may be this PR:

* Bug about 'serializedSize' in ChunkGroupMetaData.java
* https://github.com/apache/incubator-iotdb/pull/396/files

--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-09-26 12:37:31 (星期四)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Release of 0.8.1
> 
> Hi,
> 
> I am working on it. Any else PRs that should be involved in 0.8.1?
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Jialin Qiao  于2019年9月17日周二 下午12:53写道:
> 
> > Hi,
> >
> > I went through recent PRs and list those related to bug-fix of 0.8.0.
> >
> > * bug in updating startTime and endTime in TsFileResource
> > * https://github.com/apache/incubator-iotdb/pull/347
> > * bug of Mix unsequence folder in sequence folder
> > * https://github.com/apache/incubator-iotdb/pull/393
> > * bug in tsfile batch write interface
> > * https://github.com/apache/incubator-iotdb/pull/344
> > * https://github.com/apache/incubator-iotdb/pull/385
> > * bug in query with partition info of TsFile
> > * https://github.com/apache/incubator-iotdb/pull/382
> > * bug in tsfile module in example
> > * https://github.com/apache/incubator-iotdb/pull/378
> > * bug of the serialized size of String
> > * https://github.com/apache/incubator-iotdb/pull/377
> > * bug of docs (maven version)
> > * https://github.com/apache/incubator-iotdb/pull/364
> >
> > The 0.8.1 could contain these commits.
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
> > > -原始邮件-
> > > 发件人: "Jialin Qiao (Jira)" 
> > > 发送时间: 2019-09-17 11:53:00 (星期二)
> > > 收件人: dev@iotdb.apache.org
> > > 抄送:
> > > 主题: [jira] [Created] (IOTDB-216) Release of 0.8.1
> > >
> > > Jialin Qiao created IOTDB-216:
> > > -
> > >
> > >  Summary: Release of 0.8.1
> > >  Key: IOTDB-216
> > >  URL: https://issues.apache.org/jira/browse/IOTDB-216
> > >  Project: Apache IoTDB
> > >   Issue Type: Task
> > > Reporter: Jialin Qiao
> > >
> > >
> > > This is a bug-fix version of 0.8.0
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > --
> > > This message was sent by Atlassian Jira
> > > (v8.3.2#803003)
> >


Re: [Documents] Add JavaDoc API for the official site

2019-09-25 Thread Jialin Qiao
Hi,

I think it's ok to fix that. The sooner, the better...

Just remember to merge master in other PRs.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-09-26 12:34:00 (星期四)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [Documents] Add JavaDoc API for the official site
> 
> Hi,
> 
> Ah... It is really a big issue... using "/*" rather than "/**" for Apache
> Rat is correct.
> But it will change all files..
> 
> Any suggestions about what is the time to do that?
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Boris Zhu  于2019年9月26日周四 上午11:39写道:
> 
> > Hi,
> >
> > Yes, I will do it as much as I can. I see many other Apache projects having
> > API docs like spark[1] and flink[2]. API docs are generated from java
> > comments in source code. For example,
> >
> >   /**
> >* constructor of CHUNK_GROUP_FOOTER.
> >*
> >* @param deviceID device ID
> >* @param dataSize data size
> >* @param numberOfChunks number of chunks
> >*/
> >
> > So the real thing to do is to perfect Java-style comments in the source
> > code. When I generate some JavaDoc, facing some problems. For instance, the
> > Apache license shouldn't be in Java-style comments, because JavaDoc tools
> > identified the Java-style comment and didn't skip the Apache license which
> > it should skip. I think a license should be in the block comment, and I
> > create a pr[3] to fix that.
> >
> > best,
> > Boris
> >
> > [1] https://spark.apache.org/docs/latest/api/java/index.html
> > [2] https://ci.apache.org/projects/flink/flink-docs-release-1.9/api/java/
> > [3] https://github.com/apache/incubator-iotdb/pull/420
> >
> > On Thu, Sep 26, 2019 at 10:36 AM Xiangdong Huang 
> > wrote:
> >
> > > Hi Boris,
> > >
> > > Are you interested in supplying the javadoc as possible as you can?
> > > I think some of committers can support help if you have questions when
> > > reading source codes..
> > >
> > > Best,
> > > ---
> > > Xiangdong Huang
> > > School of Software, Tsinghua University
> > >
> > >  黄向东
> > > 清华大学 软件学院
> > >
> > >
> > > Boris Zhu  于2019年9月25日周三 下午3:39写道:
> > >
> > > > Hi,
> > > >
> > > > The official site needs a JavaDoc API page to help users to use IoTDB.
> > I
> > > > deploy a simple JavaDoc API page (only for TsFile module)in my GitHub
> > > > pages[1].
> > > >
> > > > But there are still many problems when I use the JavaDoc tool to
> > generate
> > > > JavaDoc and many Java-style comments need to be perfected.
> > > >
> > > > When you use an IntelliJ idea, you can use the tool in Tools-Generate
> > > > JavaDoc...
> > > >
> > > > Also, open an issue[2]
> > > >
> > > > Best Regards,
> > > > Boris
> > > >
> > > > [1] https://genius-pig.github.io/
> > > > [2] https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-236
> > > >
> > >
> >


[jira] [Created] (IOTDB-238) Remove TS_SessionHandle in TSExecuteStatementReq

2019-09-25 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-238:
-

 Summary: Remove TS_SessionHandle in TSExecuteStatementReq
 Key: IOTDB-238
 URL: https://issues.apache.org/jira/browse/IOTDB-238
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: Jialin Qiao


When we execute a sql statement, the client sends a TSExecuteStatementReq to 
the server. The TSExecuteStatementReq includes a TS_SessionHandle and a sql in 
String. However, the TS_SessionHandle has never been used. We can remove this. 
Then, the TSExecuteStatementReq struct can be simplified to a string.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IOTDB-233) Keep MetadataPlan clear

2019-09-21 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-233:
-

 Summary: Keep MetadataPlan clear
 Key: IOTDB-233
 URL: https://issues.apache.org/jira/browse/IOTDB-233
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: Jialin Qiao


The metadata operator and metadataPlan have both a path and a deletePathList, 
which is confusing. It seems that they are used in different scenarios: 

 private Path path;  (for set storage group, create timeseries)
 private List deletePathList;  (for delete timeseries)

 

It's better to make it clear. Maybe by using more types of physical plans.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Closed] (IOTDB-228) Catch RuntimeException in interting to memtable

2019-09-21 Thread Jialin Qiao (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jialin Qiao closed IOTDB-228.
-
Fix Version/s: 0.9.0
   Resolution: Fixed

> Catch RuntimeException in interting to memtable
> ---
>
> Key: IOTDB-228
> URL: https://issues.apache.org/jira/browse/IOTDB-228
> Project: Apache IoTDB
>  Issue Type: Improvement
>    Reporter: Jialin Qiao
>Priority: Major
> Fix For: 0.9.0
>
>
> When parsing data values into different types in the inserting process, 
> RuntimeException often occurs and will terminate the connection. It's better 
> to catch this and return message to users.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: AW: Reminding of changes in RPC

2019-09-20 Thread Jialin Qiao
Hi,

+1, we will only pick bug-fix commits to 0.8.x, PRs that are not compatible 
with 0.8.0 will go to 0.9. 

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Julian Feinauer" 
> 发送时间: 2019-09-19 23:07:16 (星期四)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: AW: Reminding of changes in RPC
> 
> Totally agree. This should be considered a change not just a fix and thus be 
> in a version 0.9.x but not in 0.8.x
> 
> Julian
> 
> Von meinem Mobiltelefon gesendet
> 
> 
>  Ursprüngliche Nachricht 
> Betreff: Re: Reminding of changes in RPC
> Von: 徐毅
> An: dev@iotdb.apache.org
> Cc:
> 
> Hi,
> 
> 
> Change definitions in rpc.thrift should be careful.
> For example, you have jdbc with 0.8.0 and iotdb with 0.8.1 whose 
> rpc.thrit are a little bit different. It is most likely that you cannot use  
> jdbc with 0.8.0 to connect iotdb with 0.8.1 since they are not compatible.  
> This may confuse our users why two 0.8.x are not compatible?
> 
> 
> Therefore, i would suggest that if there is no great improvement,  the 
> original definitions should be remained.
> 
> 
> Thanks
> XuYi
> 
> 
> On 9/19/2019 20:03,孙泽嵩 wrote:
> Hi,
> 
> 
> I have renamed some variable and struct names in RPC in PR [1]:
> Rename TS_Status to TSStatus;
> Rename TS_StatusType to TSStatusType;
> Use TSStatus to replace TSRPCRsp. (Since it only contains TSStatus field)
> 
> 
> If you are working on JDBC or session, please pull latest master code to 
> avoid more conflicts.
> 
> 
> Besides, if any people changed RPC, remember  to maintain the file 
> `rpc-changelist.md` in service-rpc module. It is helpful for other developers 
> : )
> 
> 
> [1] https://github.com/apache/incubator-iotdb/pull/404
> 
> 
> 
> 
> BR,
> 
> 
> --
> Zesong Sun
> School of Software, Tsinghua University
> 
> 孙泽嵩
> 清华大学 软件学院


Re: [jira] [Created] (IOTDB-232) When I run the Grafana visualization tool, the following exception appears, which I hope can be solved as soon as possible.

2019-09-19 Thread Jialin Qiao
Hi,

I checked the bug and it turned out to be the version of servlet-api is too 
stale. I upgrade it to 3.1.0 and it works.

I submitted a PR[1].

[1] https://github.com/apache/incubator-iotdb/pull/407

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "邢质坦 (Jira)" 
> 发送时间: 2019-09-20 10:27:00 (星期五)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: [jira] [Created] (IOTDB-232) When I run the Grafana visualization tool, 
> the following exception appears, which I hope can be solved as soon as 
> possible.
> 
> 邢质坦 created IOTDB-232:
> -
> 
>  Summary: When I run the Grafana visualization tool, the 
> following exception appears, which I hope can be solved as soon as possible.
>  Key: IOTDB-232
>  URL: https://issues.apache.org/jira/browse/IOTDB-232
>  Project: Apache IoTDB
>   Issue Type: Bug
> Affects Versions: 0.9.0-SNAPSHOT
>  Environment: grafana-server  -v
> Version 6.3.5 (commit: 67bad72, branch: HEAD)
> java -version
> java version "1.8.0_212"
> Java(TM) SE Runtime Environment (build 1.8.0_212-b10)
> Java HotSpot(TM) 64-Bit Server VM (build 25.212-b10, mixed mode)
>  mvn -v
> Apache Maven 3.6.2 (40f52333136460af0dc0d7232c0dc0bcf0d9e117; 
> 2019-08-27T23:06:16+08:00)
> Maven home: /opt/apache-maven-3.6.2
> Java version: 1.8.0_212, vendor: Oracle Corporation, runtime: /opt/jdk8/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "3.10.0-514.26.2.el7.x86_64", arch: "amd64", 
> family: "unix"
> 
> Reporter: 邢质坦
> 
> 
>   .     ___ _ _
>  /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
> ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
>  \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
>   '  || .__|_| |_|_| |_\__, | / / / /
>  =|_|==|___/=/_/_/_/
>  :: Spring Boot ::(v1.5.4.RELEASE)
> 
> 2019-09-20 10:20:55.360  INFO 22582 --- [   main] 
> o.a.i.w.g.TsfileWebDemoApplication   : Starting TsfileWebDemoApplication 
> v0.9.0-SNAPSHOT on izbp12agf08veb00id7ow7z with PID 22582 
> (/opt/incubator-iotdb/grafana/target/iotdb-grafana-0.9.0-SNAPSHOT.war started 
> by root in /opt/incubator-iotdb/grafana/target)
> 2019-09-20 10:20:55.370  INFO 22582 --- [   main] 
> o.a.i.w.g.TsfileWebDemoApplication   : No active profile set, falling 
> back to default profiles: default
> 2019-09-20 10:20:55.606  INFO 22582 --- [   main] 
> ationConfigEmbeddedWebApplicationContext : Refreshing 
> org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@6e5e91e4:
>  startup date [Fri Sep 20 10:20:55 CST 2019]; root of context hierarchy
> 2019-09-20 10:20:59.977  INFO 22582 --- [   main] 
> s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat initialized with port(s): 
> 8998 (http)
> 2019-09-20 10:21:00.013  INFO 22582 --- [   main] 
> o.apache.catalina.core.StandardService   : Starting service [Tomcat]
> 2019-09-20 10:21:00.023  INFO 22582 --- [   main] 
> org.apache.catalina.core.StandardEngine  : Starting Servlet Engine: Apache 
> Tomcat/8.5.15
> 2019-09-20 10:21:02.266 ERROR 22582 --- [cat-startStop-1] 
> org.apache.catalina.core.ContainerBase   : A child container failed during 
> start
> 
> java.util.concurrent.ExecutionException: 
> org.apache.catalina.LifecycleException: Failed to start component 
> [StandardEngine[Tomcat].StandardHost[localhost].TomcatEmbeddedContext[]]
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
> [na:1.8.0_212]
>   at java.util.concurrent.FutureTask.get(FutureTask.java:192) 
> [na:1.8.0_212]
>   at 
> org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:939) 
> ~[tomcat-embed-core-8.5.15.jar!/:8.5.15]
>   at 
> org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:872) 
> [tomcat-embed-core-8.5.15.jar!/:8.5.15]
>   at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) 
> [tomcat-embed-core-8.5.15.jar!/:8.5.15]
>   at 
> org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1419)
>  [tomcat-embed-core-8.5.15.jar!/:8.5.15]
>   at 
> org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1409)
>  [tomcat-embed-core-8.5.15.jar!/:8.5.15]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_212]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  [na:1.8.0_212]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624

Verify if your email is sent successfully

2019-09-19 Thread Jialin Qiao
Hi,


I just want to remind that when you send an email to mail list successfully, 
you will also receive your email. 


If you do not receive your email, others will not either.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

Re: Resolve "input line is too long" in windows os

2019-09-19 Thread Jialin Qiao
Hi,

+1 for not cover system classpath.

In our start shell, you will find we always clear classpath before adding ours:

CLASSPATH=""
for f in ${IOTDB_HOME}/lib/*.jar; do
  CLASSPATH=${CLASSPATH}":"$f
done

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-09-19 14:19:49 (星期四)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Resolve "input line is too long" in windows os
> 
> Hi,
> 
> JVM loads a set of jars according to the order of users declaration.
> 
> If you set classpath=log4j-1.1.jar : log4j-0.9.jar : log4j-1.2.jar, then
> JVM will load log4-1.1.jar first. If there are duplicate files among the 3
> jars, then only the first one will be loaded into JVM.
> 
> That is why we must set it as classpath= %iotdb classpath% : %system
> classpath%.
> 
> Though for current version of IoTDB, you can just set classpath= %iotdb
> classpath%, I'd like to do not cover system classpath, because it is more
> general.
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> JT  于2019年9月19日周四 下午2:01写道:
> 
> > Hi,
> >
> > Should we really include the original CLASSPATH? We may get ourselves into
> > trouble if it contains some incompatible or conflict dependencies with
> > those in IoTDB. For example, if the user has put an old Log4j into
> > CLASSPATH, the system log may fail.
> >
> > Tian Jiang
> >
> > 发送自 Windows 10 版邮件应用
> >
> > 发件人: Lei Rui
> > 发送时间: 2019年9月19日 13:42
> > 收件人: dev@iotdb.apache.org
> > 主题: Re: Resolve "input line is too long" in windows os
> >
> > Hi,
> >
> >
> > Ok then it looks like `start-server.bat` needs changing because now it is
> >
> >
> > ```
> > @REM * CLASSPATH library setting *
> > @REM Ensure that any user defined CLASSPATH variables are not used on
> > startup
> > set CLASSPATH="%IOTDB_HOME%\lib"
> > @REM For each jar in the IOTDB_HOME lib directory call append to build the
> > CLASSPATH variable.
> > set CLASSPATH=%CLASSPATH%;"%IOTDB_HOME%\lib\*"
> > set CLASSPATH=%CLASSPATH%;iotdb.IoTDB
> > goto okClasspath
> > ```
> >
> >
> > Lei Rui
> > On 9/19/2019 13:33,Xiangdong Huang wrote:
> > Hi,
> >
> > set CLASSPATH=%CLASSPATH%;%IOTDB_CLI_HOME%\lib\*
> >
> > Normally, the better way is :
> >
> > set CLASSPATH=%IOTDB_CLI_HOME%\lib\*;%CLASSPATH%
> >
> > It means load %IOTDB_CLI_HOME%\lib\* first, and then %CLASSPATH% that the
> > system original having.
> >
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> > 黄向东
> > 清华大学 软件学院
> >
> >
> > Jialin Qiao  于2019年9月5日周四 上午9:21写道:
> >
> >
> > Hanot iotdb-env.sh, it is in the start-client.bat, import-csv.bat and
> > export-csv.bat...
> >
> > Best
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
> > -原始邮件-
> > 发件人: "Jialin Qiao" 
> > 发送时间: 2019-09-05 09:17:45 (星期四)
> > 收件人: dev@iotdb.apache.org
> > 抄送:
> > 主题: Re:  Resolve "input line is too long" in windows os
> >
> > Hi,
> >
> > Ok, I found this in the current iotdb-env.sh:
> >
> > ```
> > @REM Ensure that any user defined CLASSPATH variables are not used on
> > startup
> > set CLASSPATH=""
> > ```
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
> > -原始邮件-
> > 发件人: "Tian Jiang" 
> > 发送时间: 2019-09-05 09:05:29 (星期四)
> > 收件人: "dev@iotdb.apache.org" 
> > 抄送: "dev@iotdb.apache.org" 
> > 主题: Re:  Resolve "input line is too long" in windows os
> >
> > Hi,
> >
> >
> > I think it is fine, since we are not sure what %CLASSPATH% would be
> > originally, if we depend on some unknown external dependencies, things may
> > get uncontrollable.
> >
> >
> > Tian Jiang
> >
> >
> > | |
> > Tian Jiang
> > |
> > |
> > jt2594...@163.com
> > |
> > 签名由网易邮箱大师定制
> > On 9/5/2019 08:55,Jialin Qiao wrote:
> > Hi Zesong,
> >
> > Thanks for sharing your experience :)
> >
> > I 

[jira] [Created] (IOTDB-228) Catch RuntimeException in interting to memtable

2019-09-18 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-228:
-

 Summary: Catch RuntimeException in interting to memtable
 Key: IOTDB-228
 URL: https://issues.apache.org/jira/browse/IOTDB-228
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: Jialin Qiao


When parsing data values into different types in the inserting process, 
RuntimeException often occurs and will terminate the connection. It's better to 
catch this and return message to users.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IOTDB-225) Support Druid connection pool

2019-09-18 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-225:
-

 Summary: Support Druid connection pool
 Key: IOTDB-225
 URL: https://issues.apache.org/jira/browse/IOTDB-225
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: Jialin Qiao






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: Fix changing to read-only mode when flushing Tsfile on HDFS

2019-09-18 Thread Jialin Qiao
Hi,

Changing to read-only mode may due to meeting error when flushing. 
It can not be solved by changing mode back, because it may happen again.

Do you authorized to flush data to HDFS?

--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Zesong Sun (Jira)" 
> 发送时间: 2019-09-18 16:45:00 (星期三)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: [jira] [Created] (IOTDB-222) Fix changing to read-only mode when flushing 
> Tsfile on HDFS
> 
> Zesong Sun created IOTDB-222:
> 
> 
>  Summary: Fix changing to read-only mode when flushing Tsfile on 
> HDFS
>  Key: IOTDB-222
>  URL: https://issues.apache.org/jira/browse/IOTDB-222
>  Project: Apache IoTDB
>   Issue Type: Bug
> Reporter: Zesong Sun
>  Attachments: WechatIMG20.png
> 
> When flushing Tsfile on HDFS, system changes to read-only mode in some 
> occasion.
> 
> 
> 
> --
> This message was sent by Atlassian Jira
> (v8.3.4#803005)


[jira] [Created] (IOTDB-220) hot-load configuration file

2019-09-17 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-220:
-

 Summary: hot-load configuration file
 Key: IOTDB-220
 URL: https://issues.apache.org/jira/browse/IOTDB-220
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: Jialin Qiao


it's a practical function to hot-load the configuration file without restarting 
IoTDB.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Closed] (IOTDB-97) Try compression in RPC

2019-09-17 Thread Jialin Qiao (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-97?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jialin Qiao closed IOTDB-97.

Fix Version/s: 0.9.0
   Resolution: Fixed

> Try compression in RPC
> --
>
> Key: IOTDB-97
> URL: https://issues.apache.org/jira/browse/IOTDB-97
> Project: Apache IoTDB
>  Issue Type: Improvement
>    Reporter: Jialin Qiao
>Priority: Minor
> Fix For: 0.9.0
>
>
> When executing a batch insertion in JDBC with many insert statements (e.g., 
> >1000), the message to be transferred will be very large. The write 
> throughput may be improved by compressing the message transferred between 
> client and server.
> Currently, thrift is used in RPC. The protocol we use is TBinaryProtocol 
> which does not compact the data that transferred. There are two ways to 
> improve:
> (1) Use TCompactProtocol instead of TBinaryProtocol. There are two classes 
> need to be modified: JDBCService and IoTDBConnection. This protocol will 
> compress the data in each RPC, I'm not sure whether this protocol is better 
> when the size of transferred data is small. 
> (2) We can just compress the data in executeBatch when the batch_size reaches 
> a threshold.  Add a compressed string that stores the SQLs in IoTDBStatement 
> and use a flag to determine whether this field is used.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Created] (IOTDB-218) Using exception instead of returning true/false

2019-09-17 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-218:
-

 Summary: Using exception instead of returning true/false
 Key: IOTDB-218
 URL: https://issues.apache.org/jira/browse/IOTDB-218
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: Jialin Qiao


"boolean insert(InsertPlan insertPlan) throws ProcessorException;"

For example, this is the insert interface in IQueryProcessExecutor, we should 
use exception instead of returning true or false because exception could return 
a message to the client. Otherwise, the client only know an "Execute statement 
error", which is not friendly at all.

 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


Re: Release of 0.8.1

2019-09-16 Thread Jialin Qiao
Hi,

I went through recent PRs and list those related to bug-fix of 0.8.0.

* bug in updating startTime and endTime in TsFileResource
* https://github.com/apache/incubator-iotdb/pull/347
* bug of Mix unsequence folder in sequence folder
* https://github.com/apache/incubator-iotdb/pull/393
* bug in tsfile batch write interface
* https://github.com/apache/incubator-iotdb/pull/344
* https://github.com/apache/incubator-iotdb/pull/385
* bug in query with partition info of TsFile
* https://github.com/apache/incubator-iotdb/pull/382
* bug in tsfile module in example
* https://github.com/apache/incubator-iotdb/pull/378
* bug of the serialized size of String
* https://github.com/apache/incubator-iotdb/pull/377
* bug of docs (maven version)
* https://github.com/apache/incubator-iotdb/pull/364

The 0.8.1 could contain these commits.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jialin Qiao (Jira)" 
> 发送时间: 2019-09-17 11:53:00 (星期二)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: [jira] [Created] (IOTDB-216) Release of 0.8.1
> 
> Jialin Qiao created IOTDB-216:
> -
> 
>  Summary: Release of 0.8.1
>  Key: IOTDB-216
>  URL: https://issues.apache.org/jira/browse/IOTDB-216
>  Project: Apache IoTDB
>       Issue Type: Task
> Reporter: Jialin Qiao
> 
> 
> This is a bug-fix version of 0.8.0
> 
>  
> 
>  
> 
> 
> 
> --
> This message was sent by Atlassian Jira
> (v8.3.2#803003)


[jira] [Created] (IOTDB-217) A new predicate of time

2019-09-16 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-217:
-

 Summary: A new predicate of time
 Key: IOTDB-217
 URL: https://issues.apache.org/jira/browse/IOTDB-217
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: Jialin Qiao


I have encountered an issue that our SQL cannot support well. 

The scenario is: I want to get the data between 9:00am to 12:00am in each day 
last week.

Could we add a new sql to do that? The sql maybe like this:

 

select s0 from root.d0 where time >= now()-1week and time <= now() groupby 
time(1d) innerInterval(9h, 12h)



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Created] (IOTDB-216) Release of 0.8.1

2019-09-16 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-216:
-

 Summary: Release of 0.8.1
 Key: IOTDB-216
 URL: https://issues.apache.org/jira/browse/IOTDB-216
 Project: Apache IoTDB
  Issue Type: Task
Reporter: Jialin Qiao


This is a bug-fix version of 0.8.0

 

 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


Re: One else if branch will never be reached in the method optimize of ExpressionOptimizer

2019-09-16 Thread Jialin Qiao
Hi,

I checked the code, it is NOT a bug, just removing the last else-if clause will 
solve it.

--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-09-17 08:53:17 (星期二)
> 收件人: dev@iotdb.apache.org, xingtan...@apache.org
> 抄送: 
> 主题: Re: [jira] [Created] (IOTDB-210) One else if branch will never be reached 
> in the method optimize of ExpressionOptimizer
> 
> Hi,
> 
> Is it a bug? or just removing the last else-if clause will solve it? Jinrui
> may know more about that.
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Yuan Tian (Jira)  于2019年9月12日周四 上午10:01写道:
> 
> > Yuan Tian created IOTDB-210:
> > ---
> >
> >  Summary: One else if branch will never be reached in the
> > method optimize of ExpressionOptimizer
> >  Key: IOTDB-210
> >  URL: https://issues.apache.org/jira/browse/IOTDB-210
> >  Project: Apache IoTDB
> >   Issue Type: Bug
> > Reporter: Yuan Tian
> >
> >
> > The code has been shown in the following. As we can see, the condition in
> > the last else if, if 'left.getType() == ExpressionType.SERIES &&
> > right.getType() == ExpressionType.SERIES' is true, the condition in the
> > previous else if which is 'left.getType() != ExpressionType.GLOBAL_TIME&&
> > right.getType() != ExpressionType.GLOBAL_TIME' must be true. Thus, the last
> > else if code block will never be executed.
> >
> >
> > {code:java}
> > else if (left.getType() != ExpressionType.GLOBAL_TIME
> >  && right.getType() != ExpressionType.GLOBAL_TIME) {
> > 
> > } else if (left.getType() == ExpressionType.SERIES
> >  && right.getType() == ExpressionType.SERIES) {
> >  ...
> > }{code}
> >
> >
> >
> >
> > --
> > This message was sent by Atlassian Jira
> > (v8.3.2#803003)
> >


Re: Support retention policy

2019-09-16 Thread Jialin Qiao
Hi,

Each storage group has its own retention policy is fine. 
In our configuration, merging can be disabled or at very low frequency. Is it 
better to execute a global (bounded with the storage group) delete plan 
periodically?

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-09-17 08:32:12 (星期二)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [jira] [Created] (IOTDB-205) Support retention policy
> 
> +1 for the feature.
> Cassandra names it as TTL.
> 
> So, how about for each storage group, there is a retention policy. And,
> clean the old data when merging.
> 
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Jialin Qiao (Jira)  于2019年9月9日周一 下午5:13写道:
> 
> > Jialin Qiao created IOTDB-205:
> > -
> >
> >  Summary: Support retention policy
> >  Key: IOTDB-205
> >  URL: https://issues.apache.org/jira/browse/IOTDB-205
> >  Project: Apache IoTDB
> >   Issue Type: New Feature
> > Reporter: Jialin Qiao
> >
> >
> > Many applications use TSDBs to just store recent data for a period of
> > time. For example, 10 days, and older data will be deleted automatically.
> > Influxdb also supports this function called retention policy.
> >
> > It would be great that we also support this.
> >
> >
> >
> > --
> > This message was sent by Atlassian Jira
> > (v8.3.2#803003)
> >


Re: Enrich Session interfaces

2019-09-13 Thread Jialin Qiao
Hi Tsung Han,

For each function, we will add a new sql and a session interface at the same 
time. Session is for developers and sql for data analysts.

Therefore, for issue[1], we will add both. The antlr is upgrading now (all sql 
grammars are changing). Therefore, you can add an interface in Session first. 

PR[2] only finishes some part of deleting storage group of new sql, about 30% 
of all work :) This PR will be closed and you can work on a new one.

Besides, the naming of deleteTimeseries should be delete.

[1]https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-188?filter=allopenissues
[2] https://github.com/apache/incubator-iotdb/pull/376

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jack Tsai" 
> 发送时间: 2019-09-14 00:17:50 (星期六)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: 回覆: Enrich Session interfaces
> 
> Hi Jialin,
> 
> A minor question is that, regarding to issue[1], is it for Session interface? 
> Or it is just the new sql?
> 
> And I don't really understand what is the meaning of the "30%" in the commit 
> message of PR[2]? If PR[2] is already for deleting the storage group, what 
> else do I need to do for it?
> 
> 
> [1]https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-188?filter=allopenissues
> [2] https://github.com/apache/incubator-iotdb/pull/376
> 
> Best,
> Tsung Han Tsai
> 
> 寄件者: Jialin Qiao 
> 寄件日期: 2019年9月6日 下午 01:35
> 收件者: dev@iotdb.apache.org 
> 主旨: Re: Enrich Session interfaces
> 
> Hi Tsung Han,
> 
> I have checked your PR and give some comments. No big problem, only some 
> optimizations.
> 
> Besides, before adding the query interface, could you please add the deleting 
> storage group interface first? This is what this issue[1] is for. You could 
> refer to this PR [2].
> 
> To clarify these delete interfaces for others. I list them in the following:
> 
> (1) Deleting data in one timeseries: This is what "delete from root.sg1.d1.s1 
> where time<=100" statement and DeletePlan do, also the "delete interface" 
> means.
> (2) Deleting timeseries, including its data and schema. This is what "delete 
> timeseries root.sg1.d1.s1" and MetadataPlan do.
> (3) Deleting storage group: deleting all data and timeseries in one storage 
> group, including the corresponding storage group processor and system info in 
> data/system folder.
> 
> 
> [1]https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-188?filter=allopenissues
> [2] https://github.com/apache/incubator-iotdb/pull/376
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> > -原始邮件-
> > 发件人: "Jack Tsai" 
> > 发送时间: 2019-09-05 23:25:39 (星期四)
> > 收件人: "dev@iotdb.apache.org" 
> > 抄送:
> > 主题: 回覆: Enrich Session interfaces
> >
> > Hi,
> >
> > I have added the delete interface for the Session API.  I followed the 
> > pattern which is similar to the insert interface.
> >
> > Please have a check whether it is conform to the requirement. I would 
> > implement the query interface for Session API if I have more time.
> >
> > Best,
> > Tsung-Han Tsai
> > 
> > 寄件者: Jialin Qiao 
> > 寄件日期: 2019年9月2日 下午 06:48
> > 收件者: dev@iotdb.apache.org 
> > 主旨: Re: Enrich Session interfaces
> >
> > Hi Tsung Han,
> >
> > The new insert interface only inserts one row data, not in batch, for 
> > example : (deviceId, time, s1, s2,s3, s1_value, s2_value, s3_value).
> >
> > Besides, the Session interfaces are only for IoTDB, not TsFile.
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
> > > -原始邮件-
> > > 发件人: "Jack Tsai" 
> > > 发送时间: 2019-09-02 18:44:34 (星期一)
> > > 收件人: "dev@iotdb.apache.org" 
> > > 抄送:
> > > 主题: Re: Enrich Session interfaces
> > >
> > > Hi,
> > >
> > > I wonder is the performance of this new kind of insert interface 
> > > different from the insert RowBatch interface? Or it is just a more 
> > > convenient way to insert batched data?
> > >
> > > Best regards,
> > > Tsung Han Tsai
> > >
> > > 取得 iOS 版 Outlook<https://aka.ms/o0ukef>
> > > 
> > > 寄件者: Jialin Qiao 
> > > 寄件日期: Monday, September 2, 2019 4:35:55 PM
> > > 收件者: dev@iotdb.apache.org 
> > > 

Re:Solving jira problem (IOTDB-180) Get rid of JSON format in "show timeseries"

2019-09-12 Thread Jialin Qiao
Hi Tao Yi,

I have reviewed your PR and merged it. Good work!

Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: thss15_yit 
> 发送时间: 2019-09-11 01:53:27 (星期三)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re:Re:Solving jira problem (IOTDB-180) Get rid of JSON format in "show 
> timeseries"
> 
> I have submitted the pull request of this issue [IOTDB-180]. 
> The link of the pull request is 
> https://github.com/apache/incubator-iotdb/pull/387
> Thanks for your checking.
> 
> 
> Tao Yi
> 
> At 2019-09-09 11:30:25, "thss15_yit"  wrote:
> >The JIRA link of this issue is 
> >https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-180?filter=allopenissues
> >
> >
> >
> >
> >
> >
> >
> >
> >在 2019-09-09 11:21:23,"thss15_yit"  写道:
> >>Hi,
> >>I have been working on JIRA issue [IOTDB-180 get rid of JSON format in 
> >> "show timeseries"] these days.
> >>   My plan of dealing with this issue is merging the execution of statement 
> >> "show timeseries" into "show timeseries ",using the functions of 
> >> "show timeseries " to output the table format of the data, and then 
> >> remove some of the useless functions of JSON format.
> >>
> >>
> >>Tao Yi 


[jira] [Created] (IOTDB-205) Support retention policy

2019-09-09 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-205:
-

 Summary: Support retention policy
 Key: IOTDB-205
 URL: https://issues.apache.org/jira/browse/IOTDB-205
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: Jialin Qiao


Many applications use TSDBs to just store recent data for a period of time. For 
example, 10 days, and older data will be deleted automatically. Influxdb also 
supports this function called retention policy.

It would be great that we also support this.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


Listed in Contributors in github

2019-09-08 Thread Jialin Qiao
Hi,

I noticed that some our contributors submitted PRs and get merged, but they are 
not listed in the "Contributors" page in our github. 

It may due to that your github account has not been related with your email you 
used to commit codes. You can try to add your email in your account settings to 
see whether it works...

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

Enable to choose storage in local file system or HDFS

2019-09-08 Thread Jialin Qiao
Hi,

This issue is to let user directly uses spark to read data in IoTDB for 
analyzing.

This function can be done in many ways in IoTDB:

(1) Storing all TsFiles (data files) and other files (system files, WALs) on 
HDFS, then use spark-tsfile to read TsFiles on HDFS.
(2) Storing only TsFiles on HDFS, and other files on local file system, then 
use spark-tsfile to read TsFiles on HDFS.
(3) Storing all files on local file system and let user use 
spark-iotdb-connector to read data from IoTDB, regardless where TsFiles store.

Personally, I prefer the second and the third. If we use the second way, do we 
need the FileFactory for all Files?

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Zesong Sun (Jira)" 
> 发送时间: 2019-08-29 19:34:00 (星期四)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: [jira] [Created] (IOTDB-187) Enable to choose storage in local file 
> system or HDFS
> 
> Zesong Sun created IOTDB-187:
> 
> 
>  Summary: Enable to choose storage in local file system or HDFS
>  Key: IOTDB-187
>  URL: https://issues.apache.org/jira/browse/IOTDB-187
>  Project: Apache IoTDB
>   Issue Type: Improvement
> Reporter: Zesong Sun
> 
> 
> Enable to choose storage in local file system or HDFS
> "is_hdfs_storage=false" by default
> 
> 
> 
> --
> This message was sent by Atlassian Jira
> (v8.3.2#803003)


Re: A new result set format

2019-09-07 Thread Jialin Qiao
Hi Julian,

He is my friend and contacted me offline, because I advertise IoTDB in my 
weChat(like facebook or twitter). 

Next time I will try to let him put issue in the mail list himself :)

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Julian Feinauer" 
> 发送时间: 2019-09-07 13:52:17 (星期六)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: A new result set format
> 
> Hi Jialin,
> 
> perhaps one question about "wanted by users" means (as I didn’t see anything 
> on the list).
> How do these users get in contact with you?
> 
> Julian
> 
> Am 07.09.19, 04:29 schrieb "Jialin Qiao" :
> 
> Hi,
> 
> As described in this issue, a new result set format is wanted by users. 
> I'd like to open a discussion here.
> 
> For simplicity, I refer this format "time, root.sg1.d1.s1, 
> root.sg1.d2.s1" to wide table, and "time, deviceId, s1" as narrow table. 
> 
> This issue is not only about how to organize the results, but also the 
> query process. 
> 
> There are some advantages about narrow table.
> 
> (1) For wide table, we need to open a SeriesReader for each series at the 
> same time, each SeriesReader holds some ChunkMetadatas. For narrow table, we 
> only need to open SeriesReaders for one device at one time, then return 
> results and open SeriesReaders for the next device, which occupies less 
> memory compared to the wide table. 
> (2) Avoid reading all series at once may also improve the query latency.
> 
> There is also a question:
> 
> (1) If we show result in the narrow table format for users, do we need to 
> highlight the concept of table and device? 
> (2) If the answer of the first question is yes, do we need to support 
> sql: "select time, deviceId, s1, s2, s3 from root.sg1 where deviceId=d1"? 
> This may involve a lot of work...
> 
> From my side, I prefer the answers of the two questions are all NO. Then 
> we do not need to change the sql grammar and only use a new query process to 
> organize the result set.
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
>     
> > -原始邮件-
> > 发件人: "Jialin Qiao (Jira)" 
> > 发送时间: 2019-09-07 09:40:00 (星期六)
> > 收件人: dev@iotdb.apache.org
> > 抄送: 
> > 主题: [jira] [Created] (IOTDB-203) A new result set format
> > 
> > Jialin Qiao created IOTDB-203:
> > -----
> > 
> >  Summary: A new result set format
> >  Key: IOTDB-203
> >  URL: https://issues.apache.org/jira/browse/IOTDB-203
> >  Project: Apache IoTDB
> >   Issue Type: New Feature
> > Reporter: Jialin Qiao
> > 
> > 
> > When executing a SQL like "select d1.s1, d2.s1 from root.sg1", the 
> default result set format in IoTDB is 
> > 
> > "time, root.sg1.d1.s1, root.sg1.d2.s1"
> > 
> > 1 , 1, 1
> > 
> > 2, 2, 2
> > 
> > However, some users want to get another format, The results could be 
> grouped by device, then sorted by time.
> > 
> > "time, deviceId, s1".
> > 
> > 1, root.sg1.d1, 1
> > 
> > 2, root.sg1.d2, 2
> > 
> >  
> > 
> > This can be done in the client, but it would be better if we support 
> this format in the server.
> > 
> >  
> > 
> > 
> > 
> > --
> > This message was sent by Atlassian Jira
> > (v8.3.2#803003)
> 
> 


A new result set format

2019-09-06 Thread Jialin Qiao
Hi,

As described in this issue, a new result set format is wanted by users. I'd 
like to open a discussion here.

For simplicity, I refer this format "time, root.sg1.d1.s1, root.sg1.d2.s1" to 
wide table, and "time, deviceId, s1" as narrow table. 

This issue is not only about how to organize the results, but also the query 
process. 

There are some advantages about narrow table.

(1) For wide table, we need to open a SeriesReader for each series at the same 
time, each SeriesReader holds some ChunkMetadatas. For narrow table, we only 
need to open SeriesReaders for one device at one time, then return results and 
open SeriesReaders for the next device, which occupies less memory compared to 
the wide table. 
(2) Avoid reading all series at once may also improve the query latency.

There is also a question:

(1) If we show result in the narrow table format for users, do we need to 
highlight the concept of table and device? 
(2) If the answer of the first question is yes, do we need to support sql: 
"select time, deviceId, s1, s2, s3 from root.sg1 where deviceId=d1"? This may 
involve a lot of work...

From my side, I prefer the answers of the two questions are all NO. Then we do 
not need to change the sql grammar and only use a new query process to organize 
the result set.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jialin Qiao (Jira)" 
> 发送时间: 2019-09-07 09:40:00 (星期六)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: [jira] [Created] (IOTDB-203) A new result set format
> 
> Jialin Qiao created IOTDB-203:
> -
> 
>  Summary: A new result set format
>  Key: IOTDB-203
>  URL: https://issues.apache.org/jira/browse/IOTDB-203
>  Project: Apache IoTDB
>   Issue Type: New Feature
> Reporter: Jialin Qiao
> 
> 
> When executing a SQL like "select d1.s1, d2.s1 from root.sg1", the default 
> result set format in IoTDB is 
> 
> "time, root.sg1.d1.s1, root.sg1.d2.s1"
> 
> 1 , 1, 1
> 
> 2, 2, 2
> 
> However, some users want to get another format, The results could be grouped 
> by device, then sorted by time.
> 
> "time, deviceId, s1".
> 
> 1, root.sg1.d1, 1
> 
> 2, root.sg1.d2, 2
> 
>  
> 
> This can be done in the client, but it would be better if we support this 
> format in the server.
> 
>  
> 
> 
> 
> --
> This message was sent by Atlassian Jira
> (v8.3.2#803003)


[jira] [Created] (IOTDB-203) A new result set format

2019-09-06 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-203:
-

 Summary: A new result set format
 Key: IOTDB-203
 URL: https://issues.apache.org/jira/browse/IOTDB-203
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: Jialin Qiao


When executing a SQL like "select d1.s1, d2.s1 from root.sg1", the default 
result set format in IoTDB is 

"time, root.sg1.d1.s1, root.sg1.d2.s1"

1 , 1, 1

2, 2, 2

However, some users want to get another format, The results could be grouped by 
device, then sorted by time.

"time, deviceId, s1".

1, root.sg1.d1, 1

2, root.sg1.d2, 2

 

This can be done in the client, but it would be better if we support this 
format in the server.

 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


Re: [jira] [Created] (IOTDB-202) Bug in example module

2019-09-06 Thread Jialin Qiao
Hi,

This is because the written data type is not compatible with the filter 
constructed in TsFileRead.

I fixed this bug in this PR [1].

[1] https://github.com/apache/incubator-iotdb/pull/378

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Yuan Tian (Jira)" 
> 发送时间: 2019-09-06 15:14:00 (星期五)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: [jira] [Created] (IOTDB-202) Bug in example module
> 
> Yuan Tian created IOTDB-202:
> ---
> 
>  Summary: Bug in example module
>  Key: IOTDB-202
>  URL: https://issues.apache.org/jira/browse/IOTDB-202
>  Project: Apache IoTDB
>   Issue Type: Bug
> Reporter: Yuan Tian
>  Attachments: WechatIMG111.png
> 
> When I use TsFileWriteWithRowBatch class to write into test.tsfile, both of 
> TsFileRead and TsFileSequenceRead work fine. However, while using 
> TsFileWriteWithTSRecord to write, TsFileSequenceRead still work well, but 
> TsFileRead will crash. After printing some record, it will throw an exception 
> as following:
> 
> !WechatIMG111.png!
> 
>  
> 
>  
> 
> 
> 
> --
> This message was sent by Atlassian Jira
> (v8.3.2#803003)


Re: Enrich Session interfaces

2019-09-05 Thread Jialin Qiao
Hi Tsung Han,

I have checked your PR and give some comments. No big problem, only some 
optimizations. 

Besides, before adding the query interface, could you please add the deleting 
storage group interface first? This is what this issue[1] is for. You could 
refer to this PR [2]. 

To clarify these delete interfaces for others. I list them in the following:

(1) Deleting data in one timeseries: This is what "delete from root.sg1.d1.s1 
where time<=100" statement and DeletePlan do, also the "delete interface" means.
(2) Deleting timeseries, including its data and schema. This is what "delete 
timeseries root.sg1.d1.s1" and MetadataPlan do.
(3) Deleting storage group: deleting all data and timeseries in one storage 
group, including the corresponding storage group processor and system info in 
data/system folder.


[1]https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-188?filter=allopenissues
[2] https://github.com/apache/incubator-iotdb/pull/376

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jack Tsai" 
> 发送时间: 2019-09-05 23:25:39 (星期四)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: 回覆: Enrich Session interfaces
> 
> Hi,
> 
> I have added the delete interface for the Session API.  I followed the 
> pattern which is similar to the insert interface.
> 
> Please have a check whether it is conform to the requirement. I would 
> implement the query interface for Session API if I have more time.
> 
> Best,
> Tsung-Han Tsai
> 
> 寄件者: Jialin Qiao 
> 寄件日期: 2019年9月2日 下午 06:48
> 收件者: dev@iotdb.apache.org 
> 主旨: Re: Enrich Session interfaces
> 
> Hi Tsung Han,
> 
> The new insert interface only inserts one row data, not in batch, for example 
> : (deviceId, time, s1, s2,s3, s1_value, s2_value, s3_value).
> 
> Besides, the Session interfaces are only for IoTDB, not TsFile.
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> > -原始邮件-
> > 发件人: "Jack Tsai" 
> > 发送时间: 2019-09-02 18:44:34 (星期一)
> > 收件人: "dev@iotdb.apache.org" 
> > 抄送:
> > 主题: Re: Enrich Session interfaces
> >
> > Hi,
> >
> > I wonder is the performance of this new kind of insert interface different 
> > from the insert RowBatch interface? Or it is just a more convenient way to 
> > insert batched data?
> >
> > Best regards,
> > Tsung Han Tsai
> >
> > 取得 iOS 版 Outlook<https://aka.ms/o0ukef>
> > 
> > 寄件者: Jialin Qiao 
> > 寄件日期: Monday, September 2, 2019 4:35:55 PM
> > 收件者: dev@iotdb.apache.org 
> > 主旨: Enrich Session interfaces
> >
> > Hi,
> >
> >
> > I got some feedback from our users in Hunan province. Except for 
> > insertBatch, they want an easier interface to insert data. Therefore, I 
> > submit a PR [1] to add an insert interface, which is easier to understand 
> > than the insert RowBatch interface.
> >
> >
> > Moreover, I have created an issue [2] to enrich the Session interfaces, 
> > looking forward to more contributors for the session interfaces.
> >
> >
> > [1] https://github.com/apache/incubator-iotdb/pull/367
> > [2] https://issues.apache.org/jira/browse/IOTDB-191
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院


Re: time series with double datatype insert problem

2019-09-05 Thread Jialin Qiao
Hi,

When I try the following java codes, it always print 0.0.

```
float a = 0f;
System.out.println(a);
a = 0.0f;
System.out.println(a);
double b = 0d;
System.out.println(b);
b = 0.0d;
System.out.println(b);
```

I think it is fine to print 0.0 in the IoTDB Client, as long as it is a float 
value (0f). I wonder how could KairosDB print 0...

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "勾王敏浩" 
> 发送时间: 2019-09-05 15:02:27 (星期四)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: time series with double datatype insert problem
> 
> Hi,
>   Currently, we find that a time series with  DATATYPE=DOUBLE, 
> ENCODING=GORILLA ,when insert a datapoint like (1,0) , it returns (1,0.0). 
> While in KairosDB, it returns (1,0). Do we need to modify this?
> 
> 
> Thank you,
> Wangminhao Gou
> 


Re: Resolve "input line is too long" in windows os

2019-09-04 Thread Jialin Qiao

Hanot iotdb-env.sh, it is in the start-client.bat, import-csv.bat and 
export-csv.bat...

Best
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jialin Qiao" 
> 发送时间: 2019-09-05 09:17:45 (星期四)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re:  Resolve "input line is too long" in windows os
> 
> Hi,
> 
> Ok, I found this in the current iotdb-env.sh:
> 
> ```
> @REM Ensure that any user defined CLASSPATH variables are not used on startup
> set CLASSPATH=""
> ```
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> > -原始邮件-
> > 发件人: "Tian Jiang" 
> > 发送时间: 2019-09-05 09:05:29 (星期四)
> > 收件人: "dev@iotdb.apache.org" 
> > 抄送: "dev@iotdb.apache.org" 
> > 主题: Re:  Resolve "input line is too long" in windows os
> > 
> > Hi,
> > 
> > 
> > I think it is fine, since we are not sure what %CLASSPATH% would be 
> > originally, if we depend on some unknown external dependencies, things may 
> > get uncontrollable.
> > 
> > 
> > Tian Jiang
> > 
> > 
> > | |
> > Tian Jiang
> > |
> > |
> > jt2594...@163.com
> > |
> > 签名由网易邮箱大师定制
> > On 9/5/2019 08:55,Jialin Qiao wrote:
> > Hi Zesong,
> > 
> > Thanks for sharing your experience :)
> > 
> > I just have a question about this line:
> > 
> > set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
> > 
> > Should it be the following command?
> > 
> > set CLASSPATH=%CLASSPATH%;%IOTDB_CLI_HOME%\lib\*
> > 
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> > 
> > 乔嘉林
> > 清华大学 软件学院
> > 
> > -原始邮件-
> > 发件人: "Zesong Sun" 
> > 发送时间: 2019-09-04 22:20:54 (星期三)
> > 收件人: dev 
> > 抄送:
> > 主题: Resolve "input line is too long" in windows os
> > 
> > Hi,
> > 
> > 
> > 
> > 
> > Recently I have encountered a problem when updating maven dependency in 
> > tsfile module: when starting client in windows os, I will get an error: 
> > "The input line is too long."
> > 
> > 
> > It is resulted by limitation of the maximum length of the string that can 
> > be used at the command line: "On computers running Microsoft Windows XP or 
> > later, the maximum length of the string that you can use at the command 
> > prompt is 8191 characters. On computers running Microsoft Windows 2000 or 
> > Windows NT 4.0, the maximum length of the string that you can use at the 
> > command prompt is 2047 characters. This limitation applies to the command 
> > line, individual environment variables (such as the PATH variable) that are 
> > inherited by other processes, and all environment variable expansions. If 
> > you use Command Prompt to run batch files, this limitation also applies to 
> > batch file processing.” [1]
> > 
> > 
> > With the suggestion of @Jialin Qiao, I reduced the dependency (from 
> > hadoop-client to hadoop-common), and local tests are all passed. However, 
> > the tests on Travis CI still failed.
> > 
> > 
> > Thanks to @Tian Jiang, I changed the set CLASSPATH statement in .bat file 
> > of starting client:
> > 
> > 
> > Previous:
> > ```
> > for %%i in ("%IOTDB_CLI_HOME%\lib\*.jar") do call :append "%%i"
> > goto okClasspath
> > 
> > 
> > :append
> > set CLASSPATH=%CLASSPATH%;%1
> > goto :eof
> > 
> > ```
> > 
> > 
> > And  now:
> > ```
> > set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
> > ```
> > 
> > 
> > This avoid too long input line of CLASSPATH, which resolves the problem.
> > 
> > 
> > The error confused me for really a long time... And I think maybe this 
> > solution will help you in the future. If you have any other suggestions and 
> > ideas, welcome to discuss with us : )
> > 
> > 
> > 
> > 
> > [1] 
> > https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation
> > 
> > 
> > 
> > 
> > 
> > BR,
> > --
> > Zesong Sun
> > School of Software, Tsinghua University
> > 
> > 孙泽嵩
> > 清华大学 软件学院


Re: Resolve "input line is too long" in windows os

2019-09-04 Thread Jialin Qiao
Hi,

Ok, I found this in the current iotdb-env.sh:

```
@REM Ensure that any user defined CLASSPATH variables are not used on startup
set CLASSPATH=""
```

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Tian Jiang" 
> 发送时间: 2019-09-05 09:05:29 (星期四)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: "dev@iotdb.apache.org" 
> 主题: Re:  Resolve "input line is too long" in windows os
> 
> Hi,
> 
> 
> I think it is fine, since we are not sure what %CLASSPATH% would be 
> originally, if we depend on some unknown external dependencies, things may 
> get uncontrollable.
> 
> 
> Tian Jiang
> 
> 
> | |
> Tian Jiang
> |
> |
> jt2594...@163.com
> |
> 签名由网易邮箱大师定制
> On 9/5/2019 08:55,Jialin Qiao wrote:
> Hi Zesong,
> 
> Thanks for sharing your experience :)
> 
> I just have a question about this line:
> 
> set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
> 
> Should it be the following command?
> 
> set CLASSPATH=%CLASSPATH%;%IOTDB_CLI_HOME%\lib\*
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> -原始邮件-
> 发件人: "Zesong Sun" 
> 发送时间: 2019-09-04 22:20:54 (星期三)
> 收件人: dev 
> 抄送:
> 主题: Resolve "input line is too long" in windows os
> 
> Hi,
> 
> 
> 
> 
> Recently I have encountered a problem when updating maven dependency in 
> tsfile module: when starting client in windows os, I will get an error: "The 
> input line is too long."
> 
> 
> It is resulted by limitation of the maximum length of the string that can be 
> used at the command line: "On computers running Microsoft Windows XP or 
> later, the maximum length of the string that you can use at the command 
> prompt is 8191 characters. On computers running Microsoft Windows 2000 or 
> Windows NT 4.0, the maximum length of the string that you can use at the 
> command prompt is 2047 characters. This limitation applies to the command 
> line, individual environment variables (such as the PATH variable) that are 
> inherited by other processes, and all environment variable expansions. If you 
> use Command Prompt to run batch files, this limitation also applies to batch 
> file processing.” [1]
> 
> 
> With the suggestion of @Jialin Qiao, I reduced the dependency (from 
> hadoop-client to hadoop-common), and local tests are all passed. However, the 
> tests on Travis CI still failed.
> 
> 
> Thanks to @Tian Jiang, I changed the set CLASSPATH statement in .bat file of 
> starting client:
> 
> 
> Previous:
> ```
> for %%i in ("%IOTDB_CLI_HOME%\lib\*.jar") do call :append "%%i"
> goto okClasspath
> 
> 
> :append
> set CLASSPATH=%CLASSPATH%;%1
> goto :eof
> 
> ```
> 
> 
> And  now:
> ```
> set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
> ```
> 
> 
> This avoid too long input line of CLASSPATH, which resolves the problem.
> 
> 
> The error confused me for really a long time... And I think maybe this 
> solution will help you in the future. If you have any other suggestions and 
> ideas, welcome to discuss with us : )
> 
> 
> 
> 
> [1] 
> https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation
> 
> 
> 
> 
> 
> BR,
> --
> Zesong Sun
> School of Software, Tsinghua University
> 
> 孙泽嵩
> 清华大学 软件学院


Re: Resolve "input line is too long" in windows os

2019-09-04 Thread Jialin Qiao
Hi Zesong,

Thanks for sharing your experience :)

I just have a question about this line:

> set CLASSPATH=%IOTDB_CLI_HOME%\lib\*

Should it be the following command? 

set CLASSPATH=%CLASSPATH%;%IOTDB_CLI_HOME%\lib\* 

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Zesong Sun" 
> 发送时间: 2019-09-04 22:20:54 (星期三)
> 收件人: dev 
> 抄送: 
> 主题: Resolve "input line is too long" in windows os
> 
> Hi,
> 
> 
> 
> 
> Recently I have encountered a problem when updating maven dependency in 
> tsfile module: when starting client in windows os, I will get an error: "The 
> input line is too long."
> 
> 
> It is resulted by limitation of the maximum length of the string that can be 
> used at the command line: "On computers running Microsoft Windows XP or 
> later, the maximum length of the string that you can use at the command 
> prompt is 8191 characters. On computers running Microsoft Windows 2000 or 
> Windows NT 4.0, the maximum length of the string that you can use at the 
> command prompt is 2047 characters. This limitation applies to the command 
> line, individual environment variables (such as the PATH variable) that are 
> inherited by other processes, and all environment variable expansions. If you 
> use Command Prompt to run batch files, this limitation also applies to batch 
> file processing.” [1]
> 
> 
> With the suggestion of @Jialin Qiao, I reduced the dependency (from 
> hadoop-client to hadoop-common), and local tests are all passed. However, the 
> tests on Travis CI still failed.
> 
> 
> Thanks to @Tian Jiang, I changed the set CLASSPATH statement in .bat file of 
> starting client:
> 
> 
> Previous: 
> ```
> for %%i in ("%IOTDB_CLI_HOME%\lib\*.jar") do call :append "%%i"
> goto okClasspath
> 
> 
> :append
> set CLASSPATH=%CLASSPATH%;%1
> goto :eof
> 
> ```
> 
> 
> And  now:
> ```
> set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
> ```
> 
> 
> This avoid too long input line of CLASSPATH, which resolves the problem.
> 
> 
> The error confused me for really a long time... And I think maybe this 
> solution will help you in the future. If you have any other suggestions and 
> ideas, welcome to discuss with us : )
> 
> 
> 
> 
> [1] 
> https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation
> 
> 
> 
> 
> 
> BR,
> --
> Zesong Sun
> School of Software, Tsinghua University
> 
> 孙泽嵩
> 清华大学 软件学院


Re: Create metadata automatically

2019-09-03 Thread Jialin Qiao
Hi Tsung Han,

Thanks! Do you mean the "delete storage group" interface? If so, I suggest 
supporting this interface in Session API first. Our Antlr is upgrading, and the 
sql grammar will be different. After upgrading Altlr to version 4, we can add 
the jdbc api.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jack Tsai" 
> 发送时间: 2019-09-04 00:00:49 (星期三)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: 回覆: Create metadata automatically
> 
> Hi Jialin,
> 
> Thanks for your explanation. I would like to implement this issue to 
> understand more about the writing interface in IoTDB.
> 
> However, due to my time reason recently, I would like to do the delete 
> interface first, which I think it is easier to done it. Then I would 
> implement the query interface when I get more free time.
> 
> Best,
> Tsung Han Tsai
> 
> 寄件者: Jialin Qiao 
> 寄件日期: 2019年9月3日 下午 04:06
> 收件者: dev@iotdb.apache.org 
> 主旨: Re: Create metadata automatically
> 
> Hi,
> 
> You are right, for this statement: insert into root.sg1.d1(time, s1) 
> values(1, "1").
> s1 can be treated as string.
> 
> Thanks,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> > -原始邮件-
> > 发件人: "Xiangdong Huang" 
> > 发送时间: 2019-09-03 15:32:57 (星期二)
> > 收件人: dev@iotdb.apache.org
> > 抄送:
> > 主题: Re: Create metadata automatically
> >
> > Hi,
> >
> > > If the value is none of the (boolean, long, double) type, then it is
> > treated as string.
> >
> > Hm... why the string definition is not  a text which are wrapped by ".
> > Otherwise, how to define a string "true"/"false".
> >
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> >  黄向东
> > 清华大学 软件学院
> >
> >
> > Jialin Qiao  于2019年9月3日周二 上午12:08写道:
> >
> > > Hi,
> > >
> > > Thanks for reminding, boolean type can be inferred from "true" or "false".
> > >
> > > Now the configuration parameters are:
> > >
> > > auto_create_schema=false
> > > default_storage_group_level=2
> > > default_boolean_encoding=RLE
> > > default_long_encoding=RLE
> > > default_double_encoding=GORILLA
> > > default_string_encoding=PLAIN
> > >
> > > If the value is none of the (boolean, long, double) type, then it is
> > > treated as string.
> > >
> > > Best,
> > > --
> > > Jialin Qiao
> > > School of Software, Tsinghua University
> > >
> > > 乔嘉林
> > > 清华大学 软件学院
> > >
> > > > -原始邮件-
> > > > 发件人: "Xiangdong Huang" 
> > > > 发送时间: 2019-09-03 14:52:57 (星期二)
> > > > 收件人: dev@iotdb.apache.org
> > > > 抄送:
> > > > 主题: Re: Create metadata automatically
> > > >
> > > > Hi,
> > > >
> > > > By the way. If so, no "boolean" data type? And why there is no
> > > > default_string_encoding.
> > > >
> > > > Best,
> > > > ---
> > > > Xiangdong Huang
> > > > School of Software, Tsinghua University
> > > >
> > > >  黄向东
> > > > 清华大学 软件学院
> > > >
> > > >
> > > > Xiangdong Huang  于2019年9月2日周一 下午11:50写道:
> > > >
> > > > > Hi,
> > > > >
> > > > > I think it should be called as "schema" rather than "metadata"...
> > > > >
> > > > > Best,
> > > > > ---
> > > > > Xiangdong Huang
> > > > > School of Software, Tsinghua University
> > > > >
> > > > >  黄向东
> > > > > 清华大学 软件学院
> > > > >
> > > > >
> > > > > Jialin Qiao  于2019年9月2日周一 下午11:46写道:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> I created an issue about creating metadata automatically [1].
> > > > >>
> > > > >> To achieve this, we can introduce four configuration parameters:
> > > > >>
> > > > >> auto_create_metadata=false
> > > > >> default_storage_group_level=2
> > > > >> default_int_encoding=R

[jira] [Created] (IOTDB-197) JXM connection failure

2019-09-03 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-197:
-

 Summary: JXM connection failure
 Key: IOTDB-197
 URL: https://issues.apache.org/jira/browse/IOTDB-197
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: Jialin Qiao
 Attachments: image-2019-09-04-10-40-22-547.png

When I try to use jvisualvm to connect IoTDB, the connection cannot be 
established. 

A potential solution is to set the -Djava.rmi.server.hostname=ip. We can 
dynamically get the ip address and set it to IOTDB_JMX_OPTS in iotdb-env.sh.

!image-2019-09-04-10-40-22-547.png!



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


Document of master

2019-09-03 Thread Jialin Qiao
Hi,



Our last release is 0.8.0, and we make a lot of changes in master (we treat 
master as dev).

Some users want to use the functions in master. However, when they request the 
documents, I just can't give a URL to them, maybe it's better to open a 
document for master in our website.

Currently, we have 0.8.0 and latest documents in our website. We could leave 
0.8.0 as default branch and rename the latest to master.


What do you think?


Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

Re: Create metadata automatically

2019-09-03 Thread Jialin Qiao
Hi,

You are right, for this statement: insert into root.sg1.d1(time, s1) values(1, 
"1").
s1 can be treated as string.

Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-09-03 15:32:57 (星期二)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Create metadata automatically
> 
> Hi,
> 
> > If the value is none of the (boolean, long, double) type, then it is
> treated as string.
> 
> Hm... why the string definition is not  a text which are wrapped by ".
> Otherwise, how to define a string "true"/"false".
> 
> Best,
> -------
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Jialin Qiao  于2019年9月3日周二 上午12:08写道:
> 
> > Hi,
> >
> > Thanks for reminding, boolean type can be inferred from "true" or "false".
> >
> > Now the configuration parameters are:
> >
> > auto_create_schema=false
> > default_storage_group_level=2
> > default_boolean_encoding=RLE
> > default_long_encoding=RLE
> > default_double_encoding=GORILLA
> > default_string_encoding=PLAIN
> >
> > If the value is none of the (boolean, long, double) type, then it is
> > treated as string.
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
> > > -原始邮件-
> > > 发件人: "Xiangdong Huang" 
> > > 发送时间: 2019-09-03 14:52:57 (星期二)
> > > 收件人: dev@iotdb.apache.org
> > > 抄送:
> > > 主题: Re: Create metadata automatically
> > >
> > > Hi,
> > >
> > > By the way. If so, no "boolean" data type? And why there is no
> > > default_string_encoding.
> > >
> > > Best,
> > > ---
> > > Xiangdong Huang
> > > School of Software, Tsinghua University
> > >
> > >  黄向东
> > > 清华大学 软件学院
> > >
> > >
> > > Xiangdong Huang  于2019年9月2日周一 下午11:50写道:
> > >
> > > > Hi,
> > > >
> > > > I think it should be called as "schema" rather than "metadata"...
> > > >
> > > > Best,
> > > > ---
> > > > Xiangdong Huang
> > > > School of Software, Tsinghua University
> > > >
> > > >  黄向东
> > > > 清华大学 软件学院
> > > >
> > > >
> > > > Jialin Qiao  于2019年9月2日周一 下午11:46写道:
> > > >
> > > >> Hi,
> > > >>
> > > >> I created an issue about creating metadata automatically [1].
> > > >>
> > > >> To achieve this, we can introduce four configuration parameters:
> > > >>
> > > >> auto_create_metadata=false
> > > >> default_storage_group_level=2
> > > >> default_int_encoding=RLE
> > > >> default_floating_encoding=GORILLA
> > > >>
> > > >> Supposing we receive an insert statement "insert into
> > root.sg1.d1(time,
> > > >> s1, s2) values(1, 1, 1)".
> > > >> When the "auto_create_metadata" is set to true and the user didn't
> > create
> > > >> metadata before, we can set storage group to root.sg1 (because the
> > > >> default_storage_group_level is 2) and create two timeseries
> > > >> "root.sg1.d1.s1" and "root.sg1.d1.s2".
> > > >>
> > > >> The datatype can be inferred from the value type. If it is "1", we
> > treat
> > > >> it as long. If it is a "floating number", we treat it as double.
> > > >>
> > > >> I'd like to have your suggestions.
> > > >>
> > > >> [1] https://issues.apache.org/jira/browse/IOTDB-193
> > > >>
> > > >> Best,
> > > >> --
> > > >> Jialin Qiao
> > > >> School of Software, Tsinghua University
> > > >>
> > > >> 乔嘉林
> > > >> 清华大学 软件学院
> > > >>
> > > >> > -原始邮件-
> > > >> > 发件人: "Jialin Qiao (Jira)" 
> > > >> > 发送时间: 2019-09-03 13:12:00 (星期二)
> > > >> > 收件人: dev@iotdb.apache.org
> > > >> > 抄送:
> > > >> > 主题: [jira] [Created] (IOTDB-193) Create metadata a

Re: Create metadata automatically

2019-09-03 Thread Jialin Qiao
Hi,

Thanks for reminding, boolean type can be inferred from "true" or "false".

Now the configuration parameters are:

auto_create_schema=false
default_storage_group_level=2
default_boolean_encoding=RLE
default_long_encoding=RLE
default_double_encoding=GORILLA
default_string_encoding=PLAIN

If the value is none of the (boolean, long, double) type, then it is treated as 
string.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-09-03 14:52:57 (星期二)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Create metadata automatically
> 
> Hi,
> 
> By the way. If so, no "boolean" data type? And why there is no
> default_string_encoding.
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Xiangdong Huang  于2019年9月2日周一 下午11:50写道:
> 
> > Hi,
> >
> > I think it should be called as "schema" rather than "metadata"...
> >
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> >  黄向东
> > 清华大学 软件学院
> >
> >
> > Jialin Qiao  于2019年9月2日周一 下午11:46写道:
> >
> >> Hi,
> >>
> >> I created an issue about creating metadata automatically [1].
> >>
> >> To achieve this, we can introduce four configuration parameters:
> >>
> >> auto_create_metadata=false
> >> default_storage_group_level=2
> >> default_int_encoding=RLE
> >> default_floating_encoding=GORILLA
> >>
> >> Supposing we receive an insert statement "insert into root.sg1.d1(time,
> >> s1, s2) values(1, 1, 1)".
> >> When the "auto_create_metadata" is set to true and the user didn't create
> >> metadata before, we can set storage group to root.sg1 (because the
> >> default_storage_group_level is 2) and create two timeseries
> >> "root.sg1.d1.s1" and "root.sg1.d1.s2".
> >>
> >> The datatype can be inferred from the value type. If it is "1", we treat
> >> it as long. If it is a "floating number", we treat it as double.
> >>
> >> I'd like to have your suggestions.
> >>
> >> [1] https://issues.apache.org/jira/browse/IOTDB-193
> >>
> >> Best,
> >> --
> >> Jialin Qiao
> >> School of Software, Tsinghua University
> >>
> >> 乔嘉林
> >> 清华大学 软件学院
> >>
> >> > -原始邮件-
> >> > 发件人: "Jialin Qiao (Jira)" 
> >> > 发送时间: 2019-09-03 13:12:00 (星期二)
> >> > 收件人: dev@iotdb.apache.org
> >> > 抄送:
> >> > 主题: [jira] [Created] (IOTDB-193) Create metadata automatically in the
> >> insertion
> >> >
> >> > Jialin Qiao created IOTDB-193:
> >> > -
> >> >
> >> >  Summary: Create metadata automatically in the insertion
> >> >  Key: IOTDB-193
> >> >  URL: https://issues.apache.org/jira/browse/IOTDB-193
> >> >  Project: Apache IoTDB
> >> >   Issue Type: New Feature
> >> > Reporter: Jialin Qiao
> >> >
> >> >
> >> > Setting storage group and creating time series can be troublesome when
> >> using IoTDB in applications.
> >> >
> >> > Let's look at the typical scenes:
> >> >
> >> > The devices send messages to Kafka. Each message includes: factoryId,
> >> deviceId, timestamp, s1_value, s2_value, s3_value.
> >> >
> >> > The consumer of Kafka needs to consume the messages and organize the
> >> data into an insert statement: "insert into root.factoryId.deviceId(time,
> >> s1, s2, s3) values(timestamp, s1_value, s2_value, s3_value)"
> >> >
> >> > Before inserting the data into IoTDB, the consumer needs to judge
> >> whether the storage group and time series are created before, which is not
> >> convenient. Typically, there are 3 ways to do that:
> >> >  # Caching the storage group and timeseries created before in the
> >> consumer
> >> >  # Query from IoTDB whether a storage group and timeseries exists
> >> >  # Directly create metadata regardless of whether created before
> >> >
> >> > Could IoTDB support creating metadata (at least time series)
> >> automatically when receiving an insertion?
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > This message was sent by Atlassian Jira
> >> > (v8.3.2#803003)
> >>
> >


Create metadata automatically

2019-09-03 Thread Jialin Qiao
Hi,

I created an issue about creating metadata automatically [1].

To achieve this, we can introduce four configuration parameters: 

auto_create_metadata=false
default_storage_group_level=2
default_int_encoding=RLE
default_floating_encoding=GORILLA

Supposing we receive an insert statement "insert into root.sg1.d1(time, s1, s2) 
values(1, 1, 1)". 
When the "auto_create_metadata" is set to true and the user didn't create 
metadata before, we can set storage group to root.sg1 (because the 
default_storage_group_level is 2) and create two timeseries "root.sg1.d1.s1" 
and "root.sg1.d1.s2".

The datatype can be inferred from the value type. If it is "1", we treat it as 
long. If it is a "floating number", we treat it as double.

I'd like to have your suggestions.

[1] https://issues.apache.org/jira/browse/IOTDB-193

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jialin Qiao (Jira)" 
> 发送时间: 2019-09-03 13:12:00 (星期二)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: [jira] [Created] (IOTDB-193) Create metadata automatically in the 
> insertion
> 
> Jialin Qiao created IOTDB-193:
> -
> 
>  Summary: Create metadata automatically in the insertion
>  Key: IOTDB-193
>  URL: https://issues.apache.org/jira/browse/IOTDB-193
>  Project: Apache IoTDB
>   Issue Type: New Feature
> Reporter: Jialin Qiao
> 
> 
> Setting storage group and creating time series can be troublesome when using 
> IoTDB in applications.
> 
> Let's look at the typical scenes:
> 
> The devices send messages to Kafka. Each message includes: factoryId, 
> deviceId, timestamp, s1_value, s2_value, s3_value. 
> 
> The consumer of Kafka needs to consume the messages and organize the data 
> into an insert statement: "insert into root.factoryId.deviceId(time, s1, s2, 
> s3) values(timestamp, s1_value, s2_value, s3_value)"
> 
> Before inserting the data into IoTDB, the consumer needs to judge whether the 
> storage group and time series are created before, which is not convenient. 
> Typically, there are 3 ways to do that:
>  # Caching the storage group and timeseries created before in the consumer
>  # Query from IoTDB whether a storage group and timeseries exists
>  # Directly create metadata regardless of whether created before
> 
> Could IoTDB support creating metadata (at least time series) automatically 
> when receiving an insertion?
> 
>  
> 
> 
> 
> --
> This message was sent by Atlassian Jira
> (v8.3.2#803003)


[jira] [Created] (IOTDB-193) Create metadata automatically in the insertion

2019-09-02 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-193:
-

 Summary: Create metadata automatically in the insertion
 Key: IOTDB-193
 URL: https://issues.apache.org/jira/browse/IOTDB-193
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: Jialin Qiao


Setting storage group and creating time series can be troublesome when using 
IoTDB in applications.

Let's look at the typical scenes:

The devices send messages to Kafka. Each message includes: factoryId, deviceId, 
timestamp, s1_value, s2_value, s3_value. 

The consumer of Kafka needs to consume the messages and organize the data into 
an insert statement: "insert into root.factoryId.deviceId(time, s1, s2, s3) 
values(timestamp, s1_value, s2_value, s3_value)"

Before inserting the data into IoTDB, the consumer needs to judge whether the 
storage group and time series are created before, which is not convenient. 
Typically, there are 3 ways to do that:
 # Caching the storage group and timeseries created before in the consumer
 # Query from IoTDB whether a storage group and timeseries exists
 # Directly create metadata regardless of whether created before

Could IoTDB support creating metadata (at least time series) automatically when 
receiving an insertion?

 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


Re: Enrich Session interfaces

2019-09-02 Thread Jialin Qiao
Hi Tsung Han,

The new insert interface only inserts one row data, not in batch, for example : 
(deviceId, time, s1, s2,s3, s1_value, s2_value, s3_value).

Besides, the Session interfaces are only for IoTDB, not TsFile. 

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jack Tsai" 
> 发送时间: 2019-09-02 18:44:34 (星期一)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: Enrich Session interfaces
> 
> Hi,
> 
> I wonder is the performance of this new kind of insert interface different 
> from the insert RowBatch interface? Or it is just a more convenient way to 
> insert batched data?
> 
> Best regards,
> Tsung Han Tsai
> 
> 取得 iOS 版 Outlook<https://aka.ms/o0ukef>
> 
> 寄件者: Jialin Qiao 
> 寄件日期: Monday, September 2, 2019 4:35:55 PM
> 收件者: dev@iotdb.apache.org 
> 主旨: Enrich Session interfaces
> 
> Hi,
> 
> 
> I got some feedback from our users in Hunan province. Except for insertBatch, 
> they want an easier interface to insert data. Therefore, I submit a PR [1] to 
> add an insert interface, which is easier to understand than the insert 
> RowBatch interface.
> 
> 
> Moreover, I have created an issue [2] to enrich the Session interfaces, 
> looking forward to more contributors for the session interfaces.
> 
> 
> [1] https://github.com/apache/incubator-iotdb/pull/367
> [2] https://issues.apache.org/jira/browse/IOTDB-191
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院


Re: [IoTDB-189] question about the feature of compatibility of HDFS

2019-09-02 Thread Jialin Qiao
Hi,

I remember that the sync module will rename the file after merge. More details 
about sync could be given by @Tianan.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-09-02 13:28:50 (星期一)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [IoTDB-189] question about the feature of compatibility of HDFS
> 
> Hi,
> 
> Look forward to the performance comparison.
> 
> > Currently, renaming only occurs in two case
> > (1) updating the version files for each storage group periodically in
> system/storage_group_x/version-xxx
> > (2) updating the compression-ratio file when dynamically adjusting
> parameters
> 
> How about the sync module? Seems there is a file for recording the file
> list (well, the sync module execution is not frequent)
> 
> Best,
> 
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Jialin Qiao  于2019年9月1日周日 下午8:00写道:
> 
> > Hi,
> >
> > > Ah... I remember that there is a file whose name is "version-xxx", in
> > which
> > the "xxx" is a counter, and will be updated if the version difference
> > between the disk and memory >= 50.
> >
> > Yes, each storage group has a file named version-xxx under
> > system/storage_group_xxx folder.
> >
> > > I am not sure whether it is correct that "renaming is not very frequent"
> > because each ChunkGroup has a version (am I right?)..
> >
> > Right, each ChunkGroup will increase this version, and each new data file
> > will increase this version.
> >
> > Currently, renaming only occurs in two case
> > (1) updating the version files for each storage group periodically in
> > system/storage_group_x/version-xxx
> > (2) updating the compression-ratio file when dynamically adjusting
> > parameters
> >
> > It's hard to say whether it is frequent or not, we can compare the
> > performance between local file system and HDFS after supporting HDFS.
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
> > > -原始邮件-
> > > 发件人: "Xiangdong Huang" 
> > > 发送时间: 2019-09-01 23:10:08 (星期日)
> > > 收件人: dev@iotdb.apache.org
> > > 抄送:
> > > 主题: Re: [IoTDB-189] question about the feature of compatibility of HDFS
> > >
> > > Hi,
> > >
> > > > since renaming is not very frequent, maybe we do not need to worry
> > about
> > > performance.
> > >
> > > Ah... I remember that there is a file whose name is "version-xxx", in
> > which
> > > the "xxx" is a counter, and will be updated if the version difference
> > > between the disk and memory >= 50.
> > >
> > > I am not sure whether it is correct that "renaming is not very frequent"
> > > because each ChunkGroup has a version (am I right?)..
> > >
> > > But, I am not saying that writing on HDFS is bad, I just want to remind
> > you
> > > that reducing renaming operations, or proving that renaming on HDFS is
> > not
> > > a time consuming operation.
> > >
> > > Best,
> > > ---
> > > Xiangdong Huang
> > > School of Software, Tsinghua University
> > >
> > >  黄向东
> > > 清华大学 软件学院
> > >
> > >
> > > Zesong Sun  于2019年8月31日周六 下午8:28写道:
> > >
> > > > Hi,
> > > >
> > > >
> > > > This issue is a sub-task of IoTDB-187 [1], which enables users to
> > choose
> > > > whether files are storied in local file system or HDFS.
> > > >
> > > >
> > > > In this sub-task, I intend to use an encapsulated "File" to replace
> > > > existed Java File in our project.
> > > >
> > > >
> > > > Currently in our design, all files could be written on HDFS.
> > > >
> > > >
> > > > HDFS supports truncating files, and since renaming is not very
> > frequent,
> > > > maybe we do not need to worry about performance.
> > > >
> > > >
> > > > If anyone has any other ideas, please discuss with me.
> > > >
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IOTDB-187
> > > >
> > > >
> >

Enrich Session interfaces

2019-09-02 Thread Jialin Qiao
Hi,


I got some feedback from our users in Hunan province. Except for insertBatch, 
they want an easier interface to insert data. Therefore, I submit a PR [1] to 
add an insert interface, which is easier to understand than the insert RowBatch 
interface. 


Moreover, I have created an issue [2] to enrich the Session interfaces, looking 
forward to more contributors for the session interfaces.


[1] https://github.com/apache/incubator-iotdb/pull/367
[2] https://issues.apache.org/jira/browse/IOTDB-191

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

[jira] [Created] (IOTDB-191) Enrich Session interfaces

2019-09-02 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-191:
-

 Summary: Enrich Session interfaces
 Key: IOTDB-191
 URL: https://issues.apache.org/jira/browse/IOTDB-191
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: Jialin Qiao


The module "Session" is an RPC interface for users, which is more efficient 
than JDBC.

Currently, it only has 3 interfaces: (1) ser storage group (2) create time 
series (3) insert batch. 

Other interfaces such as delete time series, the query could be added.

 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


Re: [IoTDB-189] question about the feature of compatibility of HDFS

2019-09-01 Thread Jialin Qiao
Hi,

> Ah... I remember that there is a file whose name is "version-xxx", in which
the "xxx" is a counter, and will be updated if the version difference
between the disk and memory >= 50.

Yes, each storage group has a file named version-xxx under 
system/storage_group_xxx folder.

> I am not sure whether it is correct that "renaming is not very frequent"
because each ChunkGroup has a version (am I right?)..

Right, each ChunkGroup will increase this version, and each new data file will 
increase this version.

Currently, renaming only occurs in two case 
(1) updating the version files for each storage group periodically in 
system/storage_group_x/version-xxx
(2) updating the compression-ratio file when dynamically adjusting parameters

It's hard to say whether it is frequent or not, we can compare the performance 
between local file system and HDFS after supporting HDFS.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-09-01 23:10:08 (星期日)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [IoTDB-189] question about the feature of compatibility of HDFS
> 
> Hi,
> 
> > since renaming is not very frequent, maybe we do not need to worry about
> performance.
> 
> Ah... I remember that there is a file whose name is "version-xxx", in which
> the "xxx" is a counter, and will be updated if the version difference
> between the disk and memory >= 50.
> 
> I am not sure whether it is correct that "renaming is not very frequent"
> because each ChunkGroup has a version (am I right?)..
> 
> But, I am not saying that writing on HDFS is bad, I just want to remind you
> that reducing renaming operations, or proving that renaming on HDFS is not
> a time consuming operation.
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Zesong Sun  于2019年8月31日周六 下午8:28写道:
> 
> > Hi,
> >
> >
> > This issue is a sub-task of IoTDB-187 [1], which enables users to choose
> > whether files are storied in local file system or HDFS.
> >
> >
> > In this sub-task, I intend to use an encapsulated "File" to replace
> > existed Java File in our project.
> >
> >
> > Currently in our design, all files could be written on HDFS.
> >
> >
> > HDFS supports truncating files, and since renaming is not very frequent,
> > maybe we do not need to worry about performance.
> >
> >
> > If anyone has any other ideas, please discuss with me.
> >
> >
> > [1] https://issues.apache.org/jira/browse/IOTDB-187
> >
> >
> > --
> > Zesong Sun
> > School of Software, Tsinghua University
> >
> > 孙泽嵩
> > 清华大学 软件学院
> >
> >
> >
> >
> >
> >
> >
> > -- 原始邮件 --
> > 发件人: "Xiangdong Huang";
> > 发送时间: 2019年8月31日(星期六) 晚上10:50
> > 收件人: "dev";
> >
> > 主题: [IoTDB-189] question about the feature of compatibility of HDFS
> >
> >
> >
> > Hi,
> >
> > This issue only has a title.. without any description...
> >
> > I think it is for letting IoTDB write TsFiles on HDFS directly.
> >
> > My question is, is only TsFile written on HDFS? or all files (e.g., wal,
> > system data file)?
> >
> > Besides, if all files are on HDFS, do we need to avoid using File name as a
> > mark or a lock (i.e., create a file and then rename it) for performance
> > consideration?
> >
> > (I am on my trip and then I realize clearly that current info on issue is
> > not enough for guys who are on the internet)
> >
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> >  黄向东
> > 清华大学 软件学院


[jira] [Created] (IOTDB-188) Delete Storage Group

2019-08-29 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-188:
-

 Summary: Delete Storage Group
 Key: IOTDB-188
 URL: https://issues.apache.org/jira/browse/IOTDB-188
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: Jialin Qiao


add “delete storage group xxx” interface, which delete all data and metadata of 
one storage group



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


Re: [BUILD-FAILURE]: Job 'IoTDB-Pipeline/master [master] [163]'

2019-08-27 Thread Jialin Qiao
Hi,

I have fixed it.

--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-08-27 23:49:51 (星期二)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [BUILD-FAILURE]: Job 'IoTDB-Pipeline/master [master] [163]'
> 
> Hi,
> 
> > [ERROR]
> /home/jenkins/jenkins-slave/workspace/IoTDB-Pipeline_master/jdbc/src/test/java/org/apache/iotdb/jdbc/IoTDBQueryResultSetTest.java:[133,5]
> cannot find symbol
> 
> This is a critical error Is that caused by your merged PR? @Jialin QIao.
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Apache Jenkins Server  于2019年8月27日周二 下午11:42写道:
> 
> > BUILD-FAILURE: Job 'IoTDB-Pipeline/master [master] [163]':
> >
> > Check console output at "https://builds.apache.org/job/IoTDB-Pipeline/job/master/163/;>IoTDB-Pipeline/master
> > [master] [163]"


[jira] [Closed] (IOTDB-179) Add create matadata interfaces to Session

2019-08-27 Thread Jialin Qiao (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jialin Qiao closed IOTDB-179.
-
Fix Version/s: 0.9.0
   Resolution: Fixed

> Add create matadata interfaces to Session
> -
>
> Key: IOTDB-179
> URL: https://issues.apache.org/jira/browse/IOTDB-179
> Project: Apache IoTDB
>  Issue Type: New Feature
>    Reporter: Jialin Qiao
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.9.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Only insertBatch() is supported in Session, we need more efficient 
> interfaces. 
> The first are "set storage group to XXX" and "create timeseries..."



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Created] (IOTDB-181) Remove "first_value" and "last_value" in TsDigest

2019-08-26 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-181:
-

 Summary: Remove "first_value" and "last_value" in TsDigest
 Key: IOTDB-181
 URL: https://issues.apache.org/jira/browse/IOTDB-181
 Project: Apache IoTDB
  Issue Type: Improvement
        Reporter: Jialin Qiao


In TsDigest, we have five statistics: min_value, max_value, sum_value, 
first_value, last_value.

When filtering page or chunk, we use min_value and max_value for a filter like 
"s1>10" or "s1<20".

The sum value can be used in aggregation query "select sum(s1) from root...".

However, it is difficult for me to come up with a scenario that uses 
"first_value" and "last_value" to filter data, and they are never used at all...

So why not remove them?



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Created] (IOTDB-180) Get rid of JSON format in "show timeseries"

2019-08-26 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-180:
-

 Summary: Get rid of JSON format in "show timeseries"
 Key: IOTDB-180
 URL: https://issues.apache.org/jira/browse/IOTDB-180
 Project: Apache IoTDB
  Issue Type: Wish
Reporter: Jialin Qiao
 Attachments: image-2019-08-27-09-56-20-654.png, 
image-2019-08-27-09-56-28-577.png

Currently, "show timeseries" prints result in JSON format, which is not 
friendly compared to "show timeseries root" in table format.

!image-2019-08-27-09-56-20-654.png|width=463,height=538!

!image-2019-08-27-09-56-28-577.png|width=694,height=458!

Intuitively, I would expect that "show timeseries" works as "show timeseries 
root".

Therefore, when users input a "show timeseries", I suggest that we treat it as 
"show timeseries root" and print results in table format.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Created] (IOTDB-179) Add create matadata interfaces to Session

2019-08-26 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-179:
-

 Summary: Add create matadata interfaces to Session
 Key: IOTDB-179
 URL: https://issues.apache.org/jira/browse/IOTDB-179
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: Jialin Qiao


Only insertBatch() is supported in Session, we need more efficient interfaces. 

The first are "set storage group to XXX" and "create timeseries..."



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Closed] (IOTDB-173) Add rpc interface to insert data

2019-08-26 Thread Jialin Qiao (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jialin Qiao closed IOTDB-173.
-

> Add rpc interface to insert data
> 
>
> Key: IOTDB-173
> URL: https://issues.apache.org/jira/browse/IOTDB-173
> Project: Apache IoTDB
>  Issue Type: Improvement
>    Reporter: Jialin Qiao
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.9.0
>
> Attachments: image-2019-08-23-12-20-32-821.png
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I used Jprofiler to profile the executBatch jdbc interface when inserting 
> data, and found 70% of the time cost is parsing SQL by Antlr3. 
>  !image-2019-08-23-12-20-32-821.png! 
> To accelerate the insertion speed, a new interface that bypasses SQL is 
> needed.
> One way is to add an RPC interface for batch insert, and I am working on that.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


Re: Add rpc interface to insert data

2019-08-23 Thread Jialin Qiao
Hi,

I'm not robot...  Here is the original content:

To resolve the issue (Add rpc interface to insert data) [1], I add a 
insertBatch interface in rpc.thrift and a new class named Session as client. 
Currently it only support insertBatch(), but we will add more interfaces (such 
as create timeseries, set storage group), which is a more efficient way 
compared to JDBC.

I have submitted a PR [2] to master. Anyone interested is welcome to join or 
review.

[1] 
https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-173?filter=allopenissues
[2] https://github.com/apache/incubator-iotdb/pull/351

--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-08-23 20:00:09 (星期五)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Add rpc interface to insert data
> 
> Hi Jialin,
> 
> It is quite an optimization for insertion only workloads. I will test its
> performance improvement.
> 
> Hope we can implement PreparedStatement in JDBC/server asap so that we can
> support more..
> 
> By the way, are you a robot because you cannot pass the verification code
> detection... :P
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Jialin Qiao  于2019年8月23日周五 下午7:44写道:
> 
> > Hi,
> >
> > 用户您好,您的访问过于频繁,为确认本次访问为正常用户行为,需要您协助验证。
> >
> > 验证码:
> >
> > 换一张验证码错误,请重新输入
> >
> > 提交后没解决问题?欢迎反馈
> >
> > 提交
> >
> >
> > To resolve the issue (Add rpc interface to insert data) [1], I add a
> > insertBatch interface in rpc.thrift and a new class named Session as
> > client. Currently it only support insertBatch(), but we will add more
> > interfaces (such as create timeseries, set storage group), which is a more
> > efficient way compared to JDBC.
> >
> >
> > I have submitted a PR [2] to master. Anyone interested is welcome to join
> > or review.
> >
> >
> > [1]
> > https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-173?filter=allopenissues
> > [2] https://github.com/apache/incubator-iotdb/pull/351
> >
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院


Add rpc interface to insert data

2019-08-23 Thread Jialin Qiao
Hi,

用户您好,您的访问过于频繁,为确认本次访问为正常用户行为,需要您协助验证。

验证码:

换一张验证码错误,请重新输入

提交后没解决问题?欢迎反馈

提交


To resolve the issue (Add rpc interface to insert data) [1], I add a 
insertBatch interface in rpc.thrift and a new class named Session as client. 
Currently it only support insertBatch(), but we will add more interfaces (such 
as create timeseries, set storage group), which is a more efficient way 
compared to JDBC.


I have submitted a PR [2] to master. Anyone interested is welcome to join or 
review.


[1] 
https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-173?filter=allopenissues
[2] https://github.com/apache/incubator-iotdb/pull/351


Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

[jira] [Created] (IOTDB-173) Add rpc interface to insert data

2019-08-22 Thread Jialin Qiao (Jira)
Jialin Qiao created IOTDB-173:
-

 Summary: Add rpc interface to insert data
 Key: IOTDB-173
 URL: https://issues.apache.org/jira/browse/IOTDB-173
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: Jialin Qiao
 Attachments: image-2019-08-23-12-20-32-821.png

I used Jprofiler to profile the executBatch jdbc interface when inserting data, 
and found 70% of the time cost is parsing SQL by Antlr3. 

 !image-2019-08-23-12-20-32-821.png! 

To accelerate the insertion speed, a new interface that bypasses SQL is needed.
One way is to add an RPC interface for batch insert, and I am working on that.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


Re: New reports

2019-08-22 Thread Jialin Qiao
Hi Kaifeng,

Welcome! I have implemented a form "time, deviceId, s1, s2, s3..." on version 
0.7.0, which is in my repo: https://github.com/qiaojialin/tsfile-spark-connector

You will find something to use :)

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Willem Jiang" 
> 发送时间: 2019-08-21 16:55:27 (星期三)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: New reports
> 
> Welcome.
> If you have any questions about the iotdb, you can just ask question here.
> I think there are bunch of developers will help you with that.
> Please go through this developer guide[1] to setup the development 
> environment.
> Looking forward your PR :)
> 
> [1]http://iotdb.apache.org/#/Development
> 
> Willem Jiang
> 
> Twitter: willemjiang
> Weibo: 姜宁willem
> 
> On Wed, Aug 21, 2019 at 3:54 PM 南京大学软件学院薛恺丰 <827011...@qq.com> wrote:
> >
> > Dear everybody:
> > Hi~ I'm a fresh man to come to this project. My name is Xue Kaifeng and I'm 
> > working on changing the form of spark dataframe. Hope I can learn something 
> > amazing from all of you and I will try my best to make some contribution to 
> > iotdb :–)
> > Best Regards,
> > Xue Kaifeng


Re: newcomer

2019-08-15 Thread Jialin Qiao
Hi Yuxin,

Welcome to join us! I have reviewed your PR and approved it.
Hope to see more contributions from you :)

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-08-16 00:03:42 (星期五)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: newcomer
> 
> Hi Yuxin,
> 
> Welcome to the community. Hope IoTDB can be used in FreeWheel in the
> future. :D
> 
> I just finished a simple example project [1] to show how to collect data,
> store data in IoTDB,
> query data from IoTDB, analyze TsFile of IoTDB by Spark, and visualize data
> by Grafana.
> Maybe it can help you to understand how to use IoTDB preliminarily.
> 
> [1] https://github.com/jixuan1989/iotdb-tutorial
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> zyx  于2019年8月15日周四 下午11:26写道:
> 
> > Hi,
> >
> >
> >
> >
> > I'm Yuxin Zhang from FreeWheel. I am interested in IoTDB and starting to
> > learn and contribute to it. When I followed the quickstart, I found the
> > TsFileSequenceRead is not easy to use because of lack of default
> > parameters. Therefore, I submitted a PR[1] to resolve it. Looking forward
> > to see a better IoTDB in the future!
> >
> >
> >
> >
> > [1] https://github.com/apache/incubator-iotdb/pull/340
> >
> >
> >
> >
> > Best,
> >
> > Yuxin Zhang


Re: An easier way to create time series.

2019-08-14 Thread Jialin Qiao
Hi,

I think source or datasource is good, and it's better to use, or at least add 
the tags and fields, because many TSDBs use these conceptual module.

Some feasible schema organization ways and "select * from the table" results:

(1) Each type of datasource is a table, which has a time column, some tag 
columns and some field columns.

Table: sourceType
time tag1,  field1, field2
1, device1, 1, 1
2, device1, 2, 2
2, device2, 2, 2

(2) Each datasource is a table with some tags. Each table has a time column, 
and some field columns. (Tags of one datasource may be not changed, so just see 
it as metadata.)

Table: source1(tag1=device1)
time, field1, field2
1, 1, 1
2, 2, 2

Table: source2(tag1=device2)
time, field1, field2
2, 2, 2


Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Julian Feinauer" 
> 发送时间: 2019-08-14 16:10:22 (星期三)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: An easier way to create time series.
> 
> Hi,
> 
> let me stick in hier also.
> One of the things which was at first a bit "unfamiliar" for me was this 
> device focus.
> It’s a bit to "one-dimensional" in my perspective.
> 
> Personally, I quite like how Influx does it that you have a name and can 
> attach tags and fields to it.
> And even if we do not do it that way I would prefer to name it a bit 
> differently as "series" or "measurement" or "source".
> Device is a bit specific and just sounds odd, from a users perspective.
> 
> I think it was good to keep it that way for 0.8.0.
> But for the next release we are open to break things a bit.
> 
> What do others think?
> 
> Julian
> 
> Am 14.08.19, 04:52 schrieb "Tian Jiang" :
> 
> Maybe starting from a sugar, we can add some improvements gradually. 
> Currently, I think making timeseries creation easier should be enough. Please 
> share if you have some fancy ideas that can go with the introduction of 
> "device".
> 
> Tian Jiang
> 
> 
> At 2019-08-14 10:44:14, "Xiangdong Huang"  wrote:
> >Hi,
> >
> >Looks fine for me.
> >
> >One question, is it just a language syntax sugar, or we can as well as
> >improve the schema management? Any idea?
> >
> >Best,
> >---
> >Xiangdong Huang
> >School of Software, Tsinghua University
> >
> > 黄向东
> >清华大学 软件学院
> >
> >
> >Tian Jiang  于2019年8月14日周三 上午10:37写道:
> >
> >> Greetings,
> >>
> >>
> >> In the present version, it is a little trouble some to create a set
> >> timeseries that has the same measurements. On the other hand, although 
> we
> >> use the conception "device" in the code, it is not properly abstracted.
> >>
> >> Expected usage:
> >>
> >> Using IoTDB in a more relational way:
> >>
> >> CREATE DEVICE TEMPLATE vehicle (speed DOUBLE PLAIN, direction DOUBLE
> >> PLAIN, temperature DOUBLE PLAIN, fuel DOUBLE PLAIN)
> >>
> >> If all datatypes(or encodings) are the same, you can write the equal 
> form:
> >>
> >> CREATE DEVICE TEMPLATE vehicle MEASUREMENTS (speed, direction,
> >> temperature, fuel) DATATYPE DOUBLE ENCODING PLAIN
> >>
> >> Then you will be able to create time series in an easier way:
> >>
> >> CREATE DEVICE (vehicle) root.sg1.vehicle1
> >>
> >> Which equals:
> >>
> >> CREATE TIMESERIES root.sg1.vehicle1.speed WITH
> >> DATATYPE=DOUBLE,ENCODING=PLAIN
> >>
> >> CREATE TIMESERIES root.sg1.vehicle1.direction WITH
> >> DATATYPE=DOUBLE,ENCODING=PLAIN
> >>
> >> CREATE TIMESERIES root.sg1.vehicle1.fuel WITH
> >> DATATYPE=DOUBLE,ENCODING=PLAIN
> >>
> >> CREATE TIMESERIES root.sg1.vehicle1.temperature WITH
> >> DATATYPE=DOUBLE,ENCODING=PLAIN
> >>
> >> I hope this will narrow the gap between using IoTDB and traditional
> >> relation databases.
> >> Jira link:
> >> 
> https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-163?filter=allopenissues
> >>
> >>
> >> Tian Jiang
> 
> 


Re: [jira] [Created] (IOTDB-163) Support create device template and create device.

2019-08-13 Thread Jialin Qiao
Hi Tian,

I like this abstraction. I support to expose some interfaces for users to 
manipulate the "device". 

Further, maybe we could introduce the "database", "table", "column" concepts in 
IoTDB.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Tian Jiang (JIRA)" 
> 发送时间: 2019-08-14 10:23:00 (星期三)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: [jira] [Created] (IOTDB-163) Support create device template and create 
> device.
> 
> Tian Jiang created IOTDB-163:
> 
> 
>  Summary: Support create device template and create device.
>  Key: IOTDB-163
>  URL: https://issues.apache.org/jira/browse/IOTDB-163
>  Project: Apache IoTDB
>   Issue Type: Improvement
> Reporter: Tian Jiang
> 
> 
> In the present version, it is a little trouble some to create a set 
> timeseries that has the same measurements. On the other hand, although we use 
> the conception "device" in the code, it is not properly abstracted.
> 
> Expected usage:
> 
> Using IoTDB in a more _*relational*_ way:
> 
> *CREATE DEVICE TEMPLATE vehicle (speed DOUBLE PLAIN, direction* *DOUBLE 
> PLAIN, temperature* *DOUBLE PLAIN, fuel* *DOUBLE PLAIN**)*
> 
> If all datatypes(or encodings) are the same, you can write the equal form:
> 
> *CREATE DEVICE TEMPLATE vehicle MEASUREMENTS (speed, direction, temperature, 
> fuel) DATATYPE DOUBLE ENCODING PLAIN*
> 
> Then you will be able to create time series in an easier way:
> 
> *CREATE DEVICE (vehicle) root.sg1.vehicle1*
> 
> Which equals:
> 
> *CREATE TIMESERIES root.sg1.vehicle1.speed WITH 
> DATATYPE=DOUBLE,ENCODING=PLAIN*
> 
> *CREATE TIMESERIES root.sg1.vehicle1.direction WITH 
> DATATYPE=DOUBLE,ENCODING=PLAIN*
> 
> *CREATE TIMESERIES root.sg1.vehicle1.fuel WITH DATATYPE=DOUBLE,ENCODING=PLAIN*
> 
> *CREATE TIMESERIES root.sg1.vehicle1.temperature WITH 
> DATATYPE=DOUBLE,ENCODING=PLAIN*
> 
> I ** hope this will narrow the gap of using IoTDB and traditional relation 
> databases.
> 
> 
> 
> --
> This message was sent by Atlassian JIRA
> (v7.6.14#76016)


Re: [VOTE] Apache IoTDB (Incubating) 0.8.0 RC3

2019-08-10 Thread Jialin Qiao
Hi,

+1 (binding)

I checked:

- Build server and client (mvn clean package -pl server -am 
-Dmaven.test.skip=true, mvn clean package -pl client -am -Dmaven.test.skip=true)
- All tests are passed on my laptop: macOS-10.14.5, jdk-1.8.0_112
- The zip file in server/target, client/target are good.
- The bin.zip works well.

The statements I used to test the binaries are:

SET STORAGE GROUP TO root.turbine;
CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=FLOAT, ENCODING=RLE;
insert into root.turbine.d1(timestamp,s0) values(1,24.5);
insert into root.turbine.d1(timestamp,s0) values(2,25.3);
insert into root.turbine.d1(timestamp,s0) values(3,25.8);
insert into root.turbine.d1(timestamp,s0) values(4,1000);
insert into root.turbine.d1(timestamp,s0) values(5,1000);
insert into root.turbine.d1(timestamp,s0) values(6,1000);
insert into root.turbine.d1(timestamp,s0) values(7,25.9);
insert into root.turbine.d1(timestamp,s0) values(8,25.8);
select s0 from root.turbine.d1;
delete from root.turbine.d1.s0 where time < 5;
select s0 from root.turbine.d1;

Minor issues: 
- When I run the tests, there are some compilation errors because of lack of 
generated sources codes, so I add a note in the readme and submit a PR [1] to 
rel/0.8.

[1] https://github.com/apache/incubator-iotdb/pull/335

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Christofer Dutz" 
> 发送时间: 2019-08-10 18:06:36 (星期六)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: [VOTE] Apache IoTDB (Incubating) 0.8.0 RC3
> 
> +1 (binding) Chris
> 
> Details of what I checked:
> 
> Source-Release
> - Signatures: OK
> - Hashes: OK
> - LICENSE, NOTICE, README, RELEASE_NOTES exist: OK
>   - README refers to wrong paths after the latest changes in the build 
> (client/target/iotdb-client-0.8.0 instead of client/cli ... same for server)
> - RAT: OK
>   - The .checkstyle file doesn't have an Apache Header (minor issue)
>   - There seem to be some log files included in the source-bundle 
> (client/logs/* and tsfile/logs/*) they don't have Apache headers, but I think 
> this is just a minor thing (In general the source-release shouldn't contain 
> log files)
>   - The only binaries found are pictures
> - Building:
>   - server as in the readme (mvn clean package -pl server -am 
> -Dmaven.test.skip=true): OK
>   - client as in the readme (mvn clean package -pl client -am 
> -Dmaven.test.skip=true): OK
>   - full build with all tests (mvn clean verify): OK
> - Running Server: OK
> - Running Client: OK
> - No reference of SNAPSHOT versions: OK
>   - Only the Hadoop pom and the Dockerfile still reference SNAPSHOT 
> versions (minor)
> 
> Binary distribution:
> - Signatures: OK
> - Hashes: OK
> - Running Server: OK
> - Running Client: OK
> 
> 
> 
> Am 06.08.19, 16:56 schrieb "Julian Feinauer" :
> 
> Hi all,
> 
> thanks to cdutz for finding some major issues in RC2 and fixing them we 
> are able to already provide RC3 now.
> 
> Important: All PMCs please note that you have to CHECK the release 
> artifacts [1] (see below) before being allowed to vote “+1”, see [3,4].
> 
> Apache IoTDB (Incubating) 0.8.0 has been staged under [1] and it’s time 
> to vote
> on accepting it for release.  All Maven artifacts are available under [2].
> If approved we will seek final release approval from the IPMC.
> Voting will be open for 72hr.
> 
> A minimum of 3 binding +1 votes and more binding +1 than binding -1
> are required to pass.
> 
> Release tag: release/0.8.0
> Hash for the release tag: 2f4da03b05d1c063eaaca622c68de86abe35de22
> 
> Per [3] "Before voting +1 [P]PMC members are required to download
> the signed source code package, compile it as provided, and test
> the resulting executable on their own platform, along with also
> verifying that the package meets the requirements of the ASF policy
> on releases."
> 
> You can achieve the above by following [4].
> 
> [ ]  +1 accept (indicate what you validated - e.g. performed the non-RM 
> items in [4])
> [ ]  -1 reject (explanation required)
> 
> [1] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.8.0/rc3
> [2] https://repository.apache.org/content/repositories/orgapacheiotdb-1003
> [3] https://www.apache.org/dev/release.html#approving-a-release
> [4] 
> https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
> 
> 
> 


Re: [VOTE] Apache IoTDB (Incubating) 0.8.0 RC1

2019-07-28 Thread Jialin Qiao
─ iotdb-server-0.8.0-SNAPSHOT.jar
│   ├── istack-commons-runtime-3.0.6.jar
│   ├── jansi-1.17.1.jar
│   ├── javassist-3.24.0-GA.jar
│   ├── javax.activation-api-1.2.0.jar
│   ├── javax.annotation-api-1.3.2.jar
│   ├── javax.inject-1.jar
│   ├── jaxb-api-2.4.0-b180725.0427.jar
│   ├── jaxb-runtime-2.4.0-b180725.0644.jar
│   ├── jline-2.14.5.jar
│   ├── jline-3.10.0.jar
│   ├── jsr305-3.0.2.jar
│   ├── junit-4.12.jar
│   ├── libthrift-0.9.3.jar
│   ├── logback-classic-1.1.11.jar
│   ├── logback-core-1.1.11.jar
│   ├── mockito-all-1.10.19.jar
│   ├── mockito-core-2.23.0.jar
│   ├── objenesis-3.0.1.jar
│   ├── powermock-api-mockito2-2.0.2.jar
│   ├── powermock-api-support-2.0.2.jar
│   ├── powermock-core-2.0.2.jar
│   ├── powermock-module-junit4-2.0.2.jar
│   ├── powermock-module-junit4-common-2.0.2.jar
│   ├── powermock-reflect-2.0.2.jar
│   ├── progressbar-0.7.3.jar
│   ├── service-rpc-0.8.0-SNAPSHOT.jar
│   ├── slf4j-api-1.7.12.jar
│   ├── slf4j-api-1.7.22.jar
│   ├── snappy-java-1.0.5-M1.jar
│   ├── stax-ex-1.8.jar
│   ├── tsfile-0.8.0-SNAPSHOT.jar
│   └── txw2-2.4.0-b180725.0644.jar
├── licenses
│   ├── CDDL\ License\ 1.1
│   ├── EDL\ License\ 1.0
│   ├── Eclipse\ Public\ License\ 1.0\ (EPL-1.0)
│   ├── Eclipse\ Public\ License\ version\ 2.0\ (EPL-2.0)
│   ├── The\ 2-Clause\ BSD\ License
│   ├── The\ 3-Clause\ BSD\ License
│   └── The\ MIT\ License
├── sbin
│   ├── start-client.bat
│   ├── start-client.sh
│   ├── start-server.bat
│   ├── start-server.sh
│   ├── stop-server.bat
│   └── stop-server.sh
└── tools
├── export-csv.bat
├── export-csv.sh
├── import-csv.bat
├── import-csv.sh
├── memory-tool.bat
├── memory-tool.sh
├── start-WalChecker.bat
├── start-WalChecker.sh
├── start-sync-client.bat
├── start-sync-client.sh
├── stop-sync-client.bat
└── stop-sync-client.sh


Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Christofer Dutz" 
> 发送时间: 2019-07-27 23:15:25 (星期六)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: [VOTE] Apache IoTDB (Incubating) 0.8.0 RC1
> 
> I intentionally didn’t read all the other vote emails as I wanted to do this 
> unbiassed … so please forgive for repeating to report things.
> 
> 
> 
> +1 (for the source-release) / -1 for the binary distribution (binding)
> 
> 
> 
> The source-release looks quite good (except the version set in the “hadoop” 
> module and some minor issues)
> 
> The binary distribution looks empty … it doesn’t seem to contain any jars at 
> all … guess this is not how it should be?
> 
> 
> 
> 
> 
> 
> 
> Details:
> 
> 
> 
>   *   Signatures (ASC) (src and bin) ok and relate to apache email address: OK
>   *   SHA512 hashes match (src and bin): OK
>   *   Source Bundle:
>  *   LICENSE, NOTICE, README, RELEASE_NOTES exist in source bundle: OK 
> (However I would prefer the binary versions not to be included)
>  *   check the contents of LICENSE, NOTICE, README.md, RELEASE_NOTES.md
> *   LICENSE: Path of MavenDownloaderWrapper.java is wrong
> *   NOTICE: Year in the notice could need updating … we’re no longer 
> 2018
> *   NOTICE: IANAL but: I don’t think the source bundle actually 
> includes crypto code. If it’s just a dependency, it doesn’t need to be listed.
> *   NOTICE: What does the “SE” in “an open-source Java SE” mean? 
> Should this be “SW” or better “software”?
> *   README.md: When building a Java JRE might not be enough … a JDK 
> could be required (Not checked)
> *   README.md: Should add the possibility of downloading the source 
> bundle from the website
> *   README.md: The structure of the project isn’t as described in the 
> file (Line 84ff)
> *   README.md: Couldn’t find bin/start-server.sh or .bat (Line 104ff) 
> same applies for the start-client.sh or .bat
> *   README.md: Could not build according to maven command as there is 
> no “iotdb” module (After a check in the history, I noticed there was some 
> renaming … used “server” instead and that worked).
> *   README.md: Build output looks different due to renaming (Line 
> 130ff)
> *   REDEME.md: Structure of output looks different (server/iotdb 
> instead of iotdb/iotdb) and later on there’s no “bin” direcrtory (line 144ff)
> *   README.md: Should we link to downloads at the domain tsfile.org 
> (Line 156)
> *   README.md: Could not build client due to renaming (used “client” 
> instead of “iotdb-cleient” (Lines 201ff)
> *   README.md: Stopped validating the content in detail after being 
> able to login to the local server with the cli :-)
> *   RELEASE_NOTES: Just saying it’s incompatible with 0.7.0 is a 
> little frustrating as some

Re: [VOTE] Apache IoTDB (Incubating) 0.8.0 RC1

2019-07-27 Thread Jialin Qiao
Hi,

Thanks, I submit a PR[1] to update the copyright of MIT and BSD licenses. 

The licenses and related software licenses are as follows (I update the 
copyright according to these urls.):

MIT License

org.slf4j:slf4j-api [2]
me.tongfei:progressbar:0.7.3 [3]

BSD 2-Clause

jline:jline:2.14.5 [4]

BSD 3-Clause

org.antlr:antlr-runtime:3.5.2 [5]


[1] https://github.com/apache/incubator-iotdb/pull/318/files
[2]  https://github.com/qos-ch/slf4j/blob/master/slf4j-api/LICENSE.txt
[3]  https://github.com/ctongfei/progressbar/blob/master/LICENSE
[4] https://jline.github.io/jline2/license.html
[5] https://www.antlr.org/license.html

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Justin Mclean" 
> 发送时间: 2019-07-27 11:59:35 (星期六)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [VOTE] Apache IoTDB (Incubating) 0.8.0 RC1
> 
> Hi,
> 
> Thanks for fixing all of those minor things.
> 
> >> - Most permissive licenses say the full text of the license needs to be 
> >> included, that includes the copyright line for BSD and MIT licensed 
> >> software, I’m not sure that have generic license test in license covers 
> >> that.
> > 
> > Actually, we have a "licenses" folder under the root which contains full 
> > text of the licenses.
> 
> Except it doesn’t have the actual licenses fro each bit of software just some 
> generic ones. With ALv2 it doesn’t;t matter, but with BSD and MIT it does due 
> to the copyright line.
> 
> Thanks,
> Justin


Re: [VOTE] Apache IoTDB (Incubating) 0.8.0 RC1

2019-07-26 Thread Jialin Qiao
Hi,

+1 (binding)


I fixed the problems that Justin mentioned in the PR [1], which is already 
merged into rel/0.8 and cherry-picked to master. The followings are my 
modifications:

> - NOTICE may include a little too much text.  I not seen an export control 
> placed in NOTICE before, and you only need to include the bit from Hive 
> notice that are relevant [1]

fixed, and the export control notice is removed.

> - All source files have ASF headers. However this file [4] incorrectly has an 
> ASF header. It’s 3rd party software and should not use the ASF header.

removed the license and ASF header

> In the README.md
> - Under "Installation from source code” it point the user to GitHub. The user 
> should be pointed to the projects download page, we don’t want to encourage 
> users to use unreleased software.

I add the projects download url.

> - The text displayed under "If successful, you will see the the following 
> text in the terminal:” is not correct

fixed

> - The instruction for compiling are not correct “mvn XXX -pl iotdb-cli” and 
> “mvn XXX -pl iotdb” don’t work.

fixed

> - The script to start and stop the server have incorrect paths 

fixed

> When starting the server I get this error, but I assume that just my setup or 
> environment. Can someone else confirm that once compiled the source release 
> they can start the server?
> Error: Could not find or load main class org.apache.iotdb.db.service.IoTDB

not reproduced, may be it is caused by the above incorrect path and commands. 
Now you can try again.

> For the binary convenience release:
> - The binary notice has way too much information in it. For instance you only 
> need to include the bit from Hive notice that are relevant [1], logbook [2] 
> has no notice file so there no need to include anything about it in the 
> NOTICE file [3]. There no need to mention licensing information, that goes in 
> LICENCE

fixed

> - Most permissive licenses say the full text of the license needs to be 
> included, that includes the copyright line for BSD and MIT licensed software, 
> I’m not sure that have generic license test in license covers that.

Actually, we have a "licenses" folder under the root which contains full text 
of the licenses.

[1] https://github.com/apache/incubator-iotdb/pull/314


Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Justin Mclean" 
> 发送时间: 2019-07-27 07:56:31 (星期六)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [VOTE] Apache IoTDB (Incubating) 0.8.0 RC1
> 
> Hi,
> 
> +1 (binding)
> 
> I checked:
> - incubating in artefact names
> - DISCLAIMER exists
> - LICENSE is good
> - NOTICE may include a little too much text.  I not seen an export control 
> placed in NOTICE before, and you only need to include the bit from Hive 
> notice that are relevant [1]
> - All source files have ASF headers. However this file [4] incorrectly has an 
> ASF header. It’s 3rd party software and should not use the ASF header.
> - No unexpected binary files
> - Can compile from source
> 
> One minor thing in license is this path 
> "./mvn/wrapper/MavenWrapperDownloader.java” should be 
> "./.mvn/wrapper/MavenWrapperDownloader.java"
> 
> In the README.md
> - Under "Installation from source code” it point the user to GitHub. The user 
> should be pointed to the projects download page, we don’t want to encourage 
> users to use unreleased software.
> - The text displayed under "If successful, you will see the the following 
> text in the terminal:” is not correct
> - The instruction for compiling are not correct “mvn XXX -pl iotdb-cli” and 
> “mvn XXX -pl iotdb” don’t work.
> - The script to start and stop the server have incorrect paths 
> 
> When starting the server I get this error, but I assume that just my setup or 
> environment. Can someone else confirm that once compiled the source release 
> they can start the server?
> Error: Could not find or load main class org.apache.iotdb.db.service.IoTDB
> 
> For the binary convenience release:
> - The binary notice has way too much information in it. For instance you only 
> need to include the bit from Hive notice that are relevant [1], logbook [2] 
> has no notice file so there no need to include anything about it in the 
> NOTICE file [3]. There no need to mention licensing information, that goes in 
> LICENCE
> - Most permissive licenses say the full text of the license needs to be 
> included, that includes the copyright line for BSD and MIT licensed software, 
> I’m not sure that have generic license test in license covers that.
> 
> Thanks,
> Justin
> 
> 1. https://github.com/apache/hive/blob/master/NOTICE
> 2. https://github.com/qos-ch/logback
> 3. http://www.apache.org/dev/licensing-howto.html#mod-notice
> 4. ./.mvn/wrapper/MavenWrapperDownloader.java


Re: Next steps for release 0.8.0-rc1

2019-07-26 Thread Jialin Qiao
Hi Chris,

Glad to see you back! Our PR [1] separates the license with license-binary, 
updates the artifact in pom and adds "incubating" as the prefix of the artefact 
name. You can check and work in this PR.

[1] https://github.com/apache/incubator-iotdb/pull/300

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-07-26 15:47:29 (星期五)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Next steps for release 0.8.0-rc1
> 
> Hi Chris,
> 
> I think It could not be better if there is a PR. The PR makes it easier to
> let us know how to do the same thing in the future..
> 
> Best
> 
> Christofer Dutz 于2019年7月26日 周五下午3:39写道:
> 
> > Hi Justin,
> >
> > As I see it, it was producing a source-release without the "incubating" in
> > it ... I just whipped up a patch for that and one that also generated the
> > required sha512 hashes.
> > Can I just commit that to the release branch, or do we have to do the full
> > PR procedure?
> >
> > Chris
> >
> >
> > Am 26.07.19, 09:16 schrieb "Justin Mclean" :
> >
> > Hi,
> >
> > > I am sure I can help with all the issues you are having. Could you
> > please make a list that I can work off?
> >
> > I think everything been sort, but you could take a look at the product
> > artefact to see if you think it has been.
> >
> > Thanks,
> > Justin
> >
> > --
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院


Re: Next steps for release 0.8.0-rc1

2019-07-25 Thread Jialin Qiao
Hi,

I have fixed three problems in PR [1]

(1) change the "incubator" to "incubating"
(2) add DISCLAIMER file and package it into the tar.gz file
(3) change the tar.gz file name to incubating-iotdb-0.8.0-release.tar.gz

[1] https://github.com/apache/incubator-iotdb/pull/300

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Justin Mclean" 
> 发送时间: 2019-07-26 08:25:08 (星期五)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Next steps for release 0.8.0-rc1
> 
> HI,
> 
> > Thanks, I have fix both of them. Now the root artificatID is called
> > incubator-iotdb-parent.
> 
> It needs to be “incubating" not “incubator”. See [1] and [2].
> 
> Thanks,
> Justin
> 
> 1. https://incubator.apache.org/policy/incubation.html#releases
> 2. https://incubator.apache.org/cookbook/#podling_releases


Re: Very frequent commits to rel/0.8

2019-07-23 Thread Jialin Qiao
Hi,

I create a new module called build for generating the tar.gz file using 
maven-assembly-plugin. The PR is [1]. 

After executing "mvn clean package", a "build-0.8.0-release.tar.gz" will 
generated under build/target. Is this file ok?

I haven't found how to change the file name to "iotdb-0.8.0-release.tar.gz", 
may be renaming the build module to iotdb... Or is it necessary to change the 
name?

Besides, After building the tar.gz file, I think copying "sbin, conf, tools and 
lib" folders to the target folder may be needless, so I remove this function in 
this PR.

[1] https://github.com/apache/incubator-iotdb/pull/295

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Julian Feinauer" 
> 发送时间: 2019-07-23 16:45:37 (星期二)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: Very frequent commits to rel/0.8
> 
> Thank you!
> 
> Am 23.07.19, 10:06 schrieb "Jialin Qiao" :
> 
> Hi Julian,
> 
> I like this idea :)  It's really troublesome to delete them one by one
> 
> I submitted a PR[1] to rel/0.8 to do so.
> 
> [1] https://github.com/apache/incubator-iotdb/pull/293
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> > -原始邮件-
> > 发件人: "Julian Feinauer" 
> > 发送时间: 2019-07-23 15:43:07 (星期二)
> > 收件人: "dev@iotdb.apache.org" 
> > 抄送: 
> > 主题: Re: Very frequent commits to rel/0.8
> > 
> > Hi Jialin,
> > 
>     > great, thanks for the summary.
> > As these are artifacts it would be best to copy them to "target/" I 
> think.
> > Is this okay for you?
> > 
> > Julian
> > 
> > Am 23.07.19, 09:41 schrieb "Jialin Qiao" :
> > 
> > Hi,
> > 
> > All known bugs and absent documents are supplemented in rel/0.8, 
> and the modules haven been renamed. Now we can package the tar.gz file. 
> > 
> > Currently, when execute mvn package, the "sbin, conf, lib, tools" 
> folders in iotdb-server and "sbin, lib, tools" folders in iotdb-client will 
> be copied to the "conf, sbin, lib, tools" in the root dir. (This is done by 
> maven-antrun-plugin, and I'm not sure whether copying them to the root dir is 
> needed.)
> > 
> > I'm not familiar with the package, it would be great if Chris could 
> offer some help :)
> > 
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> > 
> > 乔嘉林
> > 清华大学 软件学院
> > 
> > > -原始邮件-
> > > 发件人: "Julian Feinauer" 
> > > 发送时间: 2019-07-22 21:24:12 (星期一)
> > > 收件人: "dev@iotdb.apache.org" 
> > > 抄送: 
> > > 主题: Re: Very frequent commits to rel/0.8
> > > 
> > > Hey,
> > > 
> > > we should extend the package phase to finally combining all of 
> those inside one tar.gz file or something which is then moved to target/ as 
> artefact.
> > > 
> > > Regarding the Voting Process:
> > > First, a vote is held on this list.
> > > If the RC passes this vote (I will send some details on what one 
> should do to cast a Vote, but you can always refer to [1,2]) then a second 
> vote will be held on gene...@incubator.apache.org which is, formally 
> speaking, the "real" vote.
> > > If this vote also passes then we move our artefacts to the 
> released repos and they are "online".
> > > 
> > > Finally, we have to finish download links on the homepage and 
> then we can send out an announce email which goes over all official channels 
> fo the ASF and push maven artefacts to maven central.
> > > 
> > > Then its done : )
> > > 
> > > Julian
> > > 
> > > 
> > > [1] https://www.apache.org/foundation/voting.html
> > > [2] https://www.apache.org/legal/release-policy.html
> > > 
> > > Am 22.07.19, 15:08 schrieb "Xiangdong Huang" :
> > > 
> > > Hi,
> > > 
> > > > PS.: But after that is done I would start the fir

Re: Licensing / Notice / etc

2019-07-23 Thread Jialin Qiao
Hi,

I found one :D

https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-45?filter=allissues

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jialin Qiao" 
> 发送时间: 2019-07-23 16:35:31 (星期二)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Licensing / Notice / etc
> 
> Hi Julian,
> 
> We do not have such issues.
> 
> Xinyi and I checked and updated the LICENSE and NOTICE in March. From then 
> on, we do not bring in additional dependencies.
> 
> Logically, these two files should be ok, but one more check is good. 
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> > -原始邮件-
> > 发件人: "Julian Feinauer" 
> > 发送时间: 2019-07-23 15:44:44 (星期二)
> > 收件人: "dev@iotdb.apache.org" 
> > 抄送: 
> > 主题: Licensing / Notice / etc
> > 
> > Hi all,
> > 
> > just another think which is VERY important fort he release ist he 
> > correctness of all LICENSE files, NOTICE file and all that.
> > So are there any open issues or do we have a good feeling about all that?
> > 
> > @Justin: Could you perhaps assist us a bit with a quick check (to avoid 
> > that we cast a Vote and cancel it again due to “obvious” things).
> > 
> > Thanks!
> > Julian


Re: Licensing / Notice / etc

2019-07-23 Thread Jialin Qiao
Hi Julian,

We do not have such issues.

Xinyi and I checked and updated the LICENSE and NOTICE in March. From then on, 
we do not bring in additional dependencies.

Logically, these two files should be ok, but one more check is good. 

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Julian Feinauer" 
> 发送时间: 2019-07-23 15:44:44 (星期二)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Licensing / Notice / etc
> 
> Hi all,
> 
> just another think which is VERY important fort he release ist he correctness 
> of all LICENSE files, NOTICE file and all that.
> So are there any open issues or do we have a good feeling about all that?
> 
> @Justin: Could you perhaps assist us a bit with a quick check (to avoid that 
> we cast a Vote and cancel it again due to “obvious” things).
> 
> Thanks!
> Julian


Re: Very frequent commits to rel/0.8

2019-07-23 Thread Jialin Qiao
Hi Julian,

I like this idea :)  It's really troublesome to delete them one by one

I submitted a PR[1] to rel/0.8 to do so.

[1] https://github.com/apache/incubator-iotdb/pull/293

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Julian Feinauer" 
> 发送时间: 2019-07-23 15:43:07 (星期二)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: Very frequent commits to rel/0.8
> 
> Hi Jialin,
> 
> great, thanks for the summary.
> As these are artifacts it would be best to copy them to "target/" I think.
> Is this okay for you?
> 
> Julian
> 
> Am 23.07.19, 09:41 schrieb "Jialin Qiao" :
> 
> Hi,
> 
> All known bugs and absent documents are supplemented in rel/0.8, and the 
> modules haven been renamed. Now we can package the tar.gz file. 
> 
> Currently, when execute mvn package, the "sbin, conf, lib, tools" folders 
> in iotdb-server and "sbin, lib, tools" folders in iotdb-client will be copied 
> to the "conf, sbin, lib, tools" in the root dir. (This is done by 
> maven-antrun-plugin, and I'm not sure whether copying them to the root dir is 
> needed.)
> 
> I'm not familiar with the package, it would be great if Chris could offer 
> some help :)
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> > -原始邮件-
> > 发件人: "Julian Feinauer" 
> > 发送时间: 2019-07-22 21:24:12 (星期一)
> > 收件人: "dev@iotdb.apache.org" 
> > 抄送: 
> > 主题: Re: Very frequent commits to rel/0.8
> > 
> > Hey,
> > 
> > we should extend the package phase to finally combining all of those 
> inside one tar.gz file or something which is then moved to target/ as 
> artefact.
> > 
> > Regarding the Voting Process:
> > First, a vote is held on this list.
> > If the RC passes this vote (I will send some details on what one should 
> do to cast a Vote, but you can always refer to [1,2]) then a second vote will 
> be held on gene...@incubator.apache.org which is, formally speaking, the 
> "real" vote.
> > If this vote also passes then we move our artefacts to the released 
> repos and they are "online".
> > 
> > Finally, we have to finish download links on the homepage and then we 
> can send out an announce email which goes over all official channels fo the 
> ASF and push maven artefacts to maven central.
> > 
> > Then its done : )
> > 
> > Julian
> > 
> > 
> > [1] https://www.apache.org/foundation/voting.html
> > [2] https://www.apache.org/legal/release-policy.html
> > 
> > Am 22.07.19, 15:08 schrieb "Xiangdong Huang" :
> > 
> > Hi,
> > 
> > > PS.: But after that is done I would start the first release 
> candidate and
> > we can move to the next phase which is Checking the RC and Voting : 
> )
> > 
> > Where it occurs, on this mailing list or 
> gene...@incubator.apache.org?
> > 
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> > 
> >  黄向东
> > 清华大学 软件学院
> > 
> > 
> > Julian Feinauer  于2019年7月22日周一 
> 下午9:01写道:
> > 
> > > Hey,
> > >
> > > before finally preparing the first RC the only thing thats 
> missing from my
> > > side is the packaging oft he binaries.
> > > As Chris already said, he would offer some help on the maven 
> assembly
> > > plugin.
> > >
> > > Background is that it’s a "delicate" topic in the ASF to provide 
> bundled
> > > artefacts alongside sources (we do not do "binary releases").
> > > So at least one wants an automated and reproducibale way to 
> ensure that
> > > everyone gets the same binary artefacts and ideally one can check 
> those (if
> > > only valid things are inside and such).
> > >
> > > But if someone else wants to do the assembly stuff, feel free : )
> > > I'm personally lacking skills and experience with that kind of 
> stuff, so I
> > > have to rely on other volunteers : )
>  

Re: Very frequent commits to rel/0.8

2019-07-23 Thread Jialin Qiao
Hi,

All known bugs and absent documents are supplemented in rel/0.8, and the 
modules haven been renamed. Now we can package the tar.gz file. 

Currently, when execute mvn package, the "sbin, conf, lib, tools" folders in 
iotdb-server and "sbin, lib, tools" folders in iotdb-client will be copied to 
the "conf, sbin, lib, tools" in the root dir. (This is done by 
maven-antrun-plugin, and I'm not sure whether copying them to the root dir is 
needed.)

I'm not familiar with the package, it would be great if Chris could offer some 
help :)

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Julian Feinauer" 
> 发送时间: 2019-07-22 21:24:12 (星期一)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: Very frequent commits to rel/0.8
> 
> Hey,
> 
> we should extend the package phase to finally combining all of those inside 
> one tar.gz file or something which is then moved to target/ as artefact.
> 
> Regarding the Voting Process:
> First, a vote is held on this list.
> If the RC passes this vote (I will send some details on what one should do to 
> cast a Vote, but you can always refer to [1,2]) then a second vote will be 
> held on gene...@incubator.apache.org which is, formally speaking, the "real" 
> vote.
> If this vote also passes then we move our artefacts to the released repos and 
> they are "online".
> 
> Finally, we have to finish download links on the homepage and then we can 
> send out an announce email which goes over all official channels fo the ASF 
> and push maven artefacts to maven central.
> 
> Then its done : )
> 
> Julian
> 
> 
> [1] https://www.apache.org/foundation/voting.html
> [2] https://www.apache.org/legal/release-policy.html
> 
> Am 22.07.19, 15:08 schrieb "Xiangdong Huang" :
> 
> Hi,
> 
> > PS.: But after that is done I would start the first release candidate 
> and
> we can move to the next phase which is Checking the RC and Voting : )
> 
> Where it occurs, on this mailing list or gene...@incubator.apache.org?
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Julian Feinauer  于2019年7月22日周一 下午9:01写道:
> 
> > Hey,
> >
> > before finally preparing the first RC the only thing thats missing from 
> my
> > side is the packaging oft he binaries.
> > As Chris already said, he would offer some help on the maven assembly
> > plugin.
> >
> > Background is that it’s a "delicate" topic in the ASF to provide bundled
> > artefacts alongside sources (we do not do "binary releases").
> > So at least one wants an automated and reproducibale way to ensure that
> > everyone gets the same binary artefacts and ideally one can check those 
> (if
> > only valid things are inside and such).
> >
> > But if someone else wants to do the assembly stuff, feel free : )
> > I'm personally lacking skills and experience with that kind of stuff, 
> so I
> > have to rely on other volunteers : )
> >
> > Julian
> >
> > PS.: But after that is done I would start the first release candidate 
> and
> > we can move to the next phase which is Checking the RC and Voting : )
> >
> > Am 22.07.19, 14:43 schrieb "Xiangdong Huang" :
> >
> > Hi,
> >
> > Both PR #282 and #274 are merged.
> >
> > What is the next? Wait for #287 (rename source folder)?
> >
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> >  黄向东
> > 清华大学 软件学院
> >
> >
> > Xiangdong Huang  于2019年7月22日周一 下午4:33写道:
> >
> > > Hi,
> > >
> > > +1 for  DO NOT push more new features into rel/0.8.
> > >
> > > Best,
> > > ---
> > > Xiangdong Huang
> > > School of Software, Tsinghua University
> > >
> > >  黄向东
> > > 清华大学 软件学院
> > >
> > >
> > > Julian Feinauer  于2019年7月22日周一
> > 下午4:24写道:
> > >
> > >> Hi Xiangdong and Jialin,
> > >>
> >  

Re: Rename modules

2019-07-22 Thread Jialin Qiao
Hi,

This idea is excellent! So, the target folders and artifactIds can be as 
follows:

tsfile : tsfile
service-rpc : service-rpc
jdbc : iotdb-jdbc
client : iotdb-client
server: iotdb-server
example : iotdb-examples
grafana : iotdb-grafana
spark-tsfile : spark-tsfile   (we may add spark-iotdb-connector module in the 
future, so it could be distinguished from spark-tsfile)

If this structure is ok, I will resubmit this PR :)

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Julian Feinauer" 
> 发送时间: 2019-07-22 20:09:36 (星期一)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: Rename modules
> 
> Hey,
> 
> logically the PR iss mall but many changed files : )
> 
> The only thing I would take to consider is whether we really do want those 
> names for folders.
> In other projects you usually make the short name for the folder and longer 
> for the artifact, this means folders would be like
> 
> For example the "server" module would be in the folder "iotdb/server" but 
> with artifactId "iotdb-server".
> I like those shorter folder names when navigating with the cmd.
> 
> So I suggest to rename iotdb to "server" but artifactId to "iotdb-server".
> 
> What do others think?
> 
> Julian
> 
> PS.: Oh and I saw that Travis fails.. so does that imply any build-breaking 
> adjustments that we need to do?
> 
> 
> 
> Am 22.07.19, 14:03 schrieb "Xiangdong Huang" :
> 
> Hi,
> 
> It is a big change because at least 2/3 files are renamed...
>     
> Best.
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Jialin Qiao  于2019年7月22日周一 下午7:43写道:
> 
> > Hi all,
> >
> > IoTDB project has the following modules: tsfile, service-rpc, jdbc, 
> iotdb,
> > grafana, service-rpc, iotdb-cli, example and spark, in which "iotdb" and
> > "spark" look strange.
> >
> > Therefore, I rename iotdb to iotdb-server and spark to spark-tsfile by
> > "git mv" for retaining the commit history and submit a PR (
> > https://github.com/apache/incubator-iotdb/pull/287) to rel/0.8.
> >
> > I believe this makes the modules clearer... What do you think?
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
> 
> 


Rename modules

2019-07-22 Thread Jialin Qiao
Hi all,

IoTDB project has the following modules: tsfile, service-rpc, jdbc, iotdb, 
grafana, service-rpc, iotdb-cli, example and spark, in which "iotdb" and 
"spark" look strange.

Therefore, I rename iotdb to iotdb-server and spark to spark-tsfile by "git mv" 
for retaining the commit history and submit a PR 
(https://github.com/apache/incubator-iotdb/pull/287) to rel/0.8.
 
I believe this makes the modules clearer... What do you think?

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院


Re: Very frequent commits to rel/0.8

2019-07-22 Thread Jialin Qiao
Hi Julian,

Thanks for the introduction. 

Most of the commits are from PRs that supplement documents of version 0.8.0. 
However, these PRs went to master.

From now on, I will change the target of document-related PRs to rel/0.8.0. 

Besides, there are two PRs that should be merged to rel/0.8.0. 

1. about generating release folder:   
https://github.com/apache/incubator-iotdb/pull/282
2. about TsFile documents:https://github.com/apache/incubator-iotdb/pull/274

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Julian Feinauer" 
> 发送时间: 2019-07-22 15:01:14 (星期一)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Very frequent commits to rel/0.8
> 
> Hi al,
> 
> I just checked the git logs and observerd that there are very frequent 
> commits to the branch “rel/0.8”, which should not be the case.
> So I’ll try to briefly summarize the release process or how the branching 
> model works.
> 
> The branch rel/0.8 is solely for the preparation for the release. So NO new 
> feature is allowed to come in.
> All we do in this branch is to clean up things, fix possible bugs (in 
> preparation for the release), finish release documents or artifacts and those 
> things.
> 
> Everything else goes to master.
> 
> So there usually is only one direction, only cherry-picks from rel/0.8 to 
> master are allowed the other direction NOT.
> There are very few circumstances where one wants to cherry pick from master 
> to a rel branch.
> 
> So please try to focus on the master branch with development.
> Perhaps it also helps to notify me on PRs that should go to rel/0.8.
> 
> If there are questions please feel free to ask : )
> 
> Julian


Re: Binary Release of IoTDB

2019-07-19 Thread Jialin Qiao
Hi,

Nothing good comes easily :)

After reorganizing the tools, deduplicating scripts and removing grafana 
related folders, structure 4 here comes:

(Structure 4):
.
├──  LICENSE
├──  NOTICE
├──  RELEASE_NOTES
│
├──  bin
│  ├──  start-client.bat
│  ├──  start-client.sh
│  ├──  start-server.bat
│  ├──  start-server.sh
│  ├──  stop-server.bat
│  └──  stop-server.sh
│
├──  conf
│  ├──  iotdb-engine.properties
│  ├──  iotdb-env.bat
│  ├──  iotdb-env.sh
│  ├──  iotdb-sync-client.properties
│  ├──  logback.xml
│  └──  tsfile-format.properties
│
├──  lib
│  └──  *.jar
│
├──  licenses
│  └──  *.license
│
└──  tools
   ├──  export-csv.bat
   ├──  export-csv.sh
   ├──  import-csv.bat
   ├──  import-csv.sh
   ├──  start-WalChecker.bat
   ├──  start-WalChecker.sh
   ├──  memory-tool.bat
   ├──  memory-tool.sh
   ├──  start-sync-client.bat
   ├──  start-sync-client.sh
   ├──  stop-sync-client.bat
   └──  stop-sync-client.sh


Thanks, 
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-07-18 20:13:52 (星期四)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Binary Release of IoTDB
> 
> Hi all,
> 
> So we have made a consensus :).
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Julian Feinauer  于2019年7月18日周四 下午4:55写道:
> 
> > I also agree with structure 3.
> >
> > Am 18.07.19, 10:39 schrieb "Xiangdong Huang" :
> >
> > +1 for structure 3.
> >
> > But it needs some work to fix current scripts (both the start-*.sh and
> > the
> > maven package plugin)...
> >
> > By the way, I find that now when we are using `mvn package` with
> > "maven-dependency-plugin:copy-dependencies" plugin, the jars whose
> > lifecycle should be as "test" are copied into the lib folder... (e.g.,
> > powermock-*.jar)
> >
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> >  黄向东
> > 清华大学 软件学院
> >
> >
> > Jialin Qiao  于2019年7月18日周四 上午11:38写道:
> >
> > > Hi,
> > >
> > > I think structure 3 is better than 1 and 2.
> > >
> > > However, what makes the bin folder a mess is some tools script.
> > >
> > > Could we move walcheck and sync related scripts from "server" to a
> > folder
> > > named "tools", also csv import/export scripts from client to "tools"?
> > >
> > > Besides, it's better to combine "run-client.bat" with
> > "start-client.bat".
> > >
> > > Best
> > > --
> > > Jialin Qiao
> > > School of Software, Tsinghua University
> > >
> > > 乔嘉林
> > > 清华大学 软件学院
> > >
> > > > -原始邮件-
> > > > 发件人: "RUI, LEI" <1010953...@qq.com>
> > > > 发送时间: 2019-07-18 11:24:50 (星期四)
> > > > 收件人: dev 
> > > > 抄送:
> > > > 主题: 回复: Binary Release of IoTDB
> > > >
> > > > Hi, I'm here to suggest another structure like this :)
> > > >
> > > >
> > > > (Structure 3):
> > > > .
> > > > ├── LICENSE
> > > > ├── NOTICE
> > > > ├── changes.txt
> > > > │
> > > > ├── bin
> > > > │   ├── client
> > > > │   │   ├── export-csv.bat
> > > > │   │   ├── export-csv.sh
> > > > │   │   ├── import-csv.bat
> > > > │   │   ├── import-csv.sh
> > > > │   │   ├── run-client.bat
> > > > │   │   ├── start-client.bat
> > > > │   │   └── start-client.sh
> > > > │   └── server
> > > > │├── start-WalChecker.bat
> > > > │├── start-WalChecker.sh
> > > > │├── start-server.bat
> > > > │├── start-server.sh
> > > > │├── start-sync-client.bat
> > > > │├── start-sync-client.sh
> > > > │├── stop-server.bat
> > > > │├── stop-server.sh
> > > > │├── stop-sync-client.bat
> 

Re: Binary Release of IoTDB

2019-07-17 Thread Jialin Qiao
Hi,

I think structure 3 is better than 1 and 2.

However, what makes the bin folder a mess is some tools script.

Could we move walcheck and sync related scripts from "server" to a folder named 
"tools", also csv import/export scripts from client to "tools"?

Besides, it's better to combine "run-client.bat" with "start-client.bat".

Best
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "RUI, LEI" <1010953...@qq.com>
> 发送时间: 2019-07-18 11:24:50 (星期四)
> 收件人: dev 
> 抄送: 
> 主题: 回复: Binary Release of IoTDB
> 
> Hi, I'm here to suggest another structure like this :)
> 
> 
> (Structure 3):
> .
> ├── LICENSE
> ├── NOTICE
> ├── changes.txt
> │
> ├── bin
> │   ├── client
> │   │   ├── export-csv.bat
> │   │   ├── export-csv.sh
> │   │   ├── import-csv.bat
> │   │   ├── import-csv.sh
> │   │   ├── run-client.bat
> │   │   ├── start-client.bat
> │   │   └── start-client.sh
> │   └── server
> │├── start-WalChecker.bat
> │├── start-WalChecker.sh
> │├── start-server.bat
> │├── start-server.sh
> │├── start-sync-client.bat
> │├── start-sync-client.sh
> │├── stop-server.bat
> │├── stop-server.sh
> │├── stop-sync-client.bat
> │└── stop-sync-client.sh
> │
> ├── conf
> │   ├── error_info_cn.properties
> │   ├── error_info_en.properties
> │   ├── iotdb-engine.properties
> │   ├── iotdb-env.bat
> │   ├── iotdb-env.sh
> │   ├── iotdb-sync-client.properties
> │   ├── logback.xml
> │   └── tsfile-format.properties
> │
> ├──  lib
> │   ├── client
> │   │   └── *.jar
> │   ├── server
> │   │   └── *.jar
> 
> │   └── common
> │   └── *.jar
> 
> │
> ├── licenses
> │   └── LICENCES
> │
> └── grafana-connector
>  ├── bin
>  │   ├── start-grafana-connector.bat
>  │   └── start-grafana-connector.sh
>  └── iotdb-grafana-0.8.0-SNAPSHOT.war
>  
> 
> 
> 
> 
> -- 原始邮件 --
> 发件人: "Justin Mclean";
> 发送时间: 2019年7月18日(星期四) 上午10:38
> 收件人: "dev";
> 
> 主题: Re: Binary Release of IoTDB
> 
> 
> 
> Hi,
> There should be no need to vote on something like this try to reach
> consensus by discussion.
> Thanks.
> Justin
> 
> On Thu, 18 Jul 2019, 12:35 Xiangdong Huang,  wrote:
> 
> > Hi,
> >
> > any other opinion?
> >
> > We need to make a decision asap
> >
> > Because there is some divergence, do we need a vote?
> >
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> >  黄向东
> > 清华大学 软件学院
> >
> >
> > Jialin Qiao  于2019年7月17日周三 下午3:39写道:
> >
> > > Hi,
> > >
> > > I prefer the first structure that assembles all scripts in the "bin"
> > > folder and all jars in the "lib" folder.
> > >
> > > Suppose I am a user, I would expect that the structure is as clear and
> > > simple as possible.
> > >
> > > Thanks,
> > > --
> > > Jialin Qiao
> > > School of Software, Tsinghua University
> > >
> > > 乔嘉林
> > > 清华大学 软件学院
> > >
> > > > -原始邮件-
> > > > 发件人: "Xiangdong Huang" 
> > > > 发送时间: 2019-07-17 14:18:10 (星期三)
> > > > 收件人: dev@iotdb.apache.org
> > > > 抄送:
> > > > 主题: Re: Binary Release of IoTDB
> > > >
> > > > Hi,
> > > >
> > > > Though I also think the second structure is more clear, many databases
> > > > projects use the structure 1... e.g, Cassandra.
> > > >
> > > > When using structure 2, there are some jars both in the client/lib/ and
> > > the
> > > > server/lib/, which will enlarge the binary file.
> > > >
> > > > Do we need to extract them out and put them into another folder?
> > > Something
> > > > like:
> > > > .
> > > > ├── client
> > > > │   └── lib
> > > > ├── common
> > > > │   └── lib
> > > > └── server
> > > > └── lib
> > > >
> > > > Best,
> > > > ---
> > > > Xiangdong Huang
> > > > School of Software, Tsinghua University
> > > >
> > > >  黄向东
> > > > 清华大学 软件学院
> > > >
> > > >
> > > > Julian Feinauer  于201

Re: Memory estimation tool

2019-07-17 Thread Jialin Qiao
Hi Tianan,

Nice work, This is very useful! 

Could you add an example (command) about how to use this tool?

Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "李天安" 
> 发送时间: 2019-07-17 14:57:51 (星期三)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Memory estimation tool
> 
> Hi, 
> 
> 
> Recently, I participated in the development of the dynamic parameter adapter 
> module[https://github.com/apache/incubator-iotdb/pull/232], which can 
> dynamically adjust system parameters according to load conditions. When we 
> have the above module, I think we can develop a cooler thing: the memory 
> estimation tool. When users provide the number of storage groups and 
> timeseries according to the workload, the memory estimation tool can give the 
> minimum memory for writing to meet the workload.
> 
> 
> I think this tool is very useful and I have started to develop it 
> [https://github.com/apache/incubator-iotdb/pull/256].
> 
> 
> Anyone interested in this tool can discuss it here together.
> 
> 
> Best Regards,
> —
> Tianan Li
> School of Software, Tsinghua University
> 
> 
> 


Re: Binary Release of IoTDB

2019-07-17 Thread Jialin Qiao
Hi,

I prefer the first structure that assembles all scripts in the "bin" folder and 
all jars in the "lib" folder. 

Suppose I am a user, I would expect that the structure is as clear and simple 
as possible.

Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-07-17 14:18:10 (星期三)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Binary Release of IoTDB
> 
> Hi,
> 
> Though I also think the second structure is more clear, many databases
> projects use the structure 1... e.g, Cassandra.
> 
> When using structure 2, there are some jars both in the client/lib/ and the
> server/lib/, which will enlarge the binary file.
> 
> Do we need to extract them out and put them into another folder? Something
> like:
> .
> ├── client
> │   └── lib
> ├── common
> │   └── lib
> └── server
> └── lib
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Julian Feinauer  于2019年7月16日周二 下午11:27写道:
> 
> > Hi,
> >
> > I would prefer structure 2 and I really like it.
> > And we should add a readme.txt with short usage instructions.
> >
> > Julian
> >
> > Am 16.07.19, 13:58 schrieb "Xiangdong Huang" :
> >
> > Hi,
> >
> > I think the structure of the binaries can be:
> >
> > (Structure 1):
> > .
> > ├── LICENSE
> > ├── NOTICE
> > ├── bin
> > │   ├── export-csv.bat
> > │   ├── export-csv.sh
> > │   ├── import-csv.bat
> > │   ├── import-csv.sh
> > │   ├── run-client.bat
> > │   ├── start-WalChecker.bat
> > │   ├── start-WalChecker.sh
> > │   ├── start-client.bat
> > │   ├── start-client.sh
> > │   ├── start-grafana-connector.bat
> > │   ├── start-grafana-connector.sh
> > │   ├── start-server.bat
> > │   ├── start-server.sh
> > │   ├── start-sync-client.bat
> > │   ├── start-sync-client.sh
> > │   ├── stop-server.bat
> > │   ├── stop-server.sh
> > │   ├── stop-sync-client.bat
> > │   └── stop-sync-client.sh
> > ├── changes.txt
> > ├── conf
> > │   ├── error_info_cn.properties
> > │   ├── error_info_en.properties
> > │   ├── iotdb-engine.properties
> > │   ├── iotdb-env.bat
> > │   ├── iotdb-env.sh
> > │   ├── iotdb-sync-client.properties
> > │   ├── logback.xml
> > │   └── tsfile-format.properties
> > ├── lib
> > │   └── *.jar
> > └── licenses
> > └── LICENCES
> >
> > (Structure 2):
> > .
> > ├── LICENSE
> > ├── NOTICE
> > ├── changes.txt
> > ├── client
> > │   ├── bin
> > │   │   ├── export-csv.bat
> > │   │   ├── export-csv.sh
> > │   │   ├── import-csv.bat
> > │   │   ├── import-csv.sh
> > │   │   ├── run-client.bat
> > │   │   ├── start-client.bat
> > │   │   └── start-client.sh
> > │   └── lib
> > │   └──  *.jar
> > ├── grafana-connector
> > │   ├── bin
> > │   │   ├── start-grafana-connector.bat
> > │   │   └── start-grafana-connector.sh
> > │   └── iotdb-grafana-0.8.0-SNAPSHOT.war
> > ├── licenses
> > │   └── LICENCES
> > └── server
> > ├── bin
> > │   ├── start-WalChecker.bat
> > │   ├── start-WalChecker.sh
> > │   ├── start-server.bat
> > │   ├── start-server.sh
> > │   ├── start-sync-client.bat
> > │   ├── start-sync-client.sh
> > │   ├── stop-server.bat
> > │   ├── stop-server.sh
> > │   ├── stop-sync-client.bat
> > │   └── stop-sync-client.sh
> > ├── conf
> > │   ├── error_info_cn.properties
> > │   ├── error_info_en.properties
> > │   ├── iotdb-engine.properties
> > │   ├── iotdb-env.bat
> > │   ├── iotdb-env.sh
> > │   ├── iotdb-sync-client.properties
> > │   ├── logback.xml
> > │   └── tsfile-format.properties
> > └── lib
> > └──  *.jar
> >
> > In structure 1, CLI and server scripts  are in the same folder, so
> > does the
> > libs. In structure 2, CLI and server module are seperated.
> >
> > The first one has smaller size because the 

Re: Volunteer as Release Manager for release 0.8

2019-07-16 Thread Jialin Qiao
Hi,

I have updated the RELEASE_NOTES and submit a PR to rel/0.8: 
https://github.com/apache/incubator-iotdb/pull/250/files

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Julian Feinauer" 
> 发送时间: 2019-07-16 16:17:58 (星期二)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: Volunteer as Release Manager for release 0.8
> 
> Hi Jialin,
> 
> of course it is.
> Adding Jira is just a service to point readers (or people who did a jira) 
> directly to the issue and detailed description.
> But if you look at early PLC4X releases, we basically had non issues at all __
> 
> Julian
> 
> Am 16.07.19, 10:16 schrieb "Jialin Qiao" :
> 
> Hi Julian,
> 
> I will submit the PR about updating RELEASE_NOTES to rel/0.8.
> 
> Just one question, some PRs are not associated with Jira issues, is 
> "listing them in the bottom of the corresponding list without a Jira number" 
> ok?
> 
> for example:
> 
> Bug Fixes
> -
> 
> IOTDB-7 OpenFileNumUtilTest failed
> fix time zone bug in different region (<---this one does not 
> match a Jira issue)
> 
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> > -原始邮件-
> > 发件人: "Julian Feinauer" 
> > 发送时间: 2019-07-16 15:50:15 (星期二)
> > 收件人: "dev@iotdb.apache.org" 
> > 抄送: 
> > 主题: Re: Volunteer as Release Manager for release 0.8
> > 
> > Hi all,
> > 
> > so I created a release branch now called rel/0.8, see [1].
> > Master branch is now set to the new development version 0.9.0-SNAPSHOT.
> > 
> > This means, that all PRs should be rebased (which should be no problem 
> in 99% of cases).
> > 
> > So all changes we do regarding the release should go into rel/0.8.
> > If we fix  bugs there they should be merged into rel/0.8 BUT also 
> cherry-picked into master.
> > 
> > @ Jialin: if you assist with the RELEASE_NOTES file you should do it in 
> rel/0.8.
> > 
> > So all PRs keep going to master and all future development and only 
> things regarding the release will go into the rel/0.8 branch.
> > If we do major changes there they will be merged back to master 
> regularily and we keep the branch open to support "minor" or "hotfix" 
> releases.
> > 
> > Best
> > Julian
> > 
> > [1] https://github.com/apache/incubator-iotdb/tree/rel/0.8
> > 
> > Am 16.07.19, 09:43 schrieb "Julian Feinauer" 
> :
> > 
> > Yes, I can spin up a release branch now and then everybody who does 
> things for "release stabilization" should work there or point the PRs to 
> there (like updating release notes and stuff).
> > 
> > Am 16.07.19, 09:32 schrieb "Xiangdong Huang" :
> > 
> > Hi,
> > 
> > So, do we need to confirm the source codes and create a release 
> branch now?
> > 
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> > 
> >  黄向东
> > 清华大学 软件学院
> > 
> > 
> >     Julian Feinauer  于2019年7月16日周二 
> 下午12:54写道:
> > 
> > > Hey Jialin,
> > >
> > > that is a very kind offer!
> > > Some time ago I already created a "RELEASE_NOTES" File (in 
> the root) and
> > > added in what was there up till then.
> > > But as this was like 2 or 3 months ago it needs to be 
> overworked.
> > > So feel free to extend the file.
> > >
> > > Julian
> > >
> > > Am 16.07.19, 04:32 schrieb "Jialin Qiao" 
> :
> > >
> > > Hi,
> > >
> > > It's very exciting that we are going to release the first 
> version.
> > >
> > > PR #245, #232 have been merged successfully. Therefore, 
> we could start
> > > to release.
> > >
> > > If needed, I will

Re: Volunteer as Release Manager for release 0.8

2019-07-16 Thread Jialin Qiao
Hi Julian,

I will submit the PR about updating RELEASE_NOTES to rel/0.8.

Just one question, some PRs are not associated with Jira issues, is "listing 
them in the bottom of the corresponding list without a Jira number" ok?

for example:

Bug Fixes
-

IOTDB-7 OpenFileNumUtilTest failed
fix time zone bug in different region (<---this one does not match a 
Jira issue)


Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Julian Feinauer" 
> 发送时间: 2019-07-16 15:50:15 (星期二)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: Volunteer as Release Manager for release 0.8
> 
> Hi all,
> 
> so I created a release branch now called rel/0.8, see [1].
> Master branch is now set to the new development version 0.9.0-SNAPSHOT.
> 
> This means, that all PRs should be rebased (which should be no problem in 99% 
> of cases).
> 
> So all changes we do regarding the release should go into rel/0.8.
> If we fix  bugs there they should be merged into rel/0.8 BUT also 
> cherry-picked into master.
> 
> @ Jialin: if you assist with the RELEASE_NOTES file you should do it in 
> rel/0.8.
> 
> So all PRs keep going to master and all future development and only things 
> regarding the release will go into the rel/0.8 branch.
> If we do major changes there they will be merged back to master regularily 
> and we keep the branch open to support "minor" or "hotfix" releases.
> 
> Best
> Julian
> 
> [1] https://github.com/apache/incubator-iotdb/tree/rel/0.8
> 
> Am 16.07.19, 09:43 schrieb "Julian Feinauer" :
> 
> Yes, I can spin up a release branch now and then everybody who does 
> things for "release stabilization" should work there or point the PRs to 
> there (like updating release notes and stuff).
> 
> Am 16.07.19, 09:32 schrieb "Xiangdong Huang" :
> 
> Hi,
> 
> So, do we need to confirm the source codes and create a release 
> branch now?
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Julian Feinauer  于2019年7月16日周二 
> 下午12:54写道:
> 
> > Hey Jialin,
> >
> > that is a very kind offer!
> > Some time ago I already created a "RELEASE_NOTES" File (in the 
> root) and
> > added in what was there up till then.
> > But as this was like 2 or 3 months ago it needs to be overworked.
> > So feel free to extend the file.
> >
> > Julian
> >
> > Am 16.07.19, 04:32 schrieb "Jialin Qiao" 
> :
> >
> > Hi,
> >
> > It's very exciting that we are going to release the first 
> version.
> >
> > PR #245, #232 have been merged successfully. Therefore, we 
> could start
> > to release.
> >
> > If needed, I will traverse the PRs and Issues to give a changes
> > (major) list from 0.7.0 to 0.8.0.
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
> > > -原始邮件-
> > > 发件人: "Julian Feinauer" 
> > > 发送时间: 2019-07-15 21:48:54 (星期一)
> > > 收件人: "dev@iotdb.apache.org" 
> > > 抄送:
> > > 主题: Re: Volunteer as Release Manager for release 0.8
> > >
> > > Hey Xiangdong,
> > >
> > > I also think that the "changelog" will be huge.
> > > But I'm unsure if we have to write everything as "change" 
> because
> > this is the "first" release with regards to the ASF so I'm unsure 
> if we
> > would point to 0.7 as "previous" release.
> > >
> > > @cdutz, @jmclean what is your point on that, how do other 
> podlings
> > handle that?
> > >
> > > Julian
> > >
> > > Am 15.07.19, 15:45 schrieb "Xiangdong Huang" 
> :
> > >
> > > Hi,
> > >
> > >  

  1   2   >