A simple tool to visualize logs

[Tian Jiang]

Greetings,




Usually, the very first thing we will do on finding a bug is to search the 
logs. Logs play a vital role in debugging especially in some environment that 
attaching a debugger is impossible. In such circumstance, logs will hopefully 
become the only information sources for the developers.

 

However, a single log, which is just a string, is easy to understand. But when 
it comes to mining information from thousands of logs or even more,  getting 
lost is nearly unavoidable, since humans have a much limited memory for exact 
truth compared to computers. From time to time, I forget what I have read 
before and I must go back to review the previous logs, as a result, progress is 
made very slowly. Reading several strings is easy, but when we have thousands, 
there must be some better way to present them than raw text.

 

So, I keep thinking it would be much better if we can make the logs into plots. 
Of course there must some existing tools, but they are often powerful but too 
heavy (like Kibana) , or specialized for web or other logs (like LogStalgia). 
Having a fantastic web interface is great, but a simple but handy suit us 
better. What I want is something light-weighted, stand-alone and highly 
customized.

 

As a result, I developed a simple tool that can visualize (plot) logs generated 
by IoTDB (with some modification, it can be applied to other type of logs, too) 
and generate report. I designed a simple GUI which provides full 
functionalities and a command line tool to fast generate reports. The 
attachment contains an example report I generated from one of my experiments, 
which reveal interesting things like how the size of memtables converges over 
time.

 

I may have missed some tools that are more powerful or easier to use. If you 
know any, please inform me and I shall see what I can learn from them.



| |
Tian Jiang
|
|
jt2594...@163.com
|
签名由网易邮箱大师定制

Re: A serious hidden but simple bug caused by object abnormal publishing

[Xiangdong Huang]

Hi,

The same question.

> so there must be no overlap between the two parameters

Ok, we need to have a check before startup.

> Therefore, I suspected that it was an abnormal publishing of the sequence
folder list and the unsequence folder list.

What abnormal "publishing" means?

> I tracked all the places where it was possible to use the two objects and
found that it will returned the object directly when I require them from
multi-file directory manager.

the two objects?

The bug seems critical and the solution may just be create a new Java
Object (which overrides the equals() and hashCode()? I guess.), but the
description of the bug is really not so good...

Maybe you can show how to reproduce the bug in the issue description and
then analyze why. And, finally submit a PR just for the issue (rather than
along with the sync module. As you know, the sync module PR may be blocked
for a long time to wait committers to review it, but the bug-fix PR can be
merged quickly because there is no too many code modifications).

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


徐毅  于2019年9月4日周三 上午5:19写道：

> Hi,
>
>
> After reading your explanation, I’m still confused about your problem.
> First, what's the meaning of 'object abnormal publishing’? Can you add some
> example codes to define your problem? Besides, you have fixed this problem
> in your sync pr, I suggest you create an individual branch to solve this
> problem since this problem is a common case.
>
>
> By the way, these are few grammar mistakes in your explanation, try to
> avoid them next time. :)
>
>
> Thanks
> XuYi
>
>
>
>
> On 09/4/2019 16:46，李天安 wrote：
> Hi,
> Recently，in developing sync module, I found a very important bug in the
> process of unit testing, which will seriously affect the normal operation
> of the system. During the unit test of sync module, I found that when I
> retrieved the next sequence folder from the multi-file directory manager, I
> may get an unseuqence folder, which made me confused and began to track
> down the problem.
>
>
> In IoTDB, sequence folders and unsequence folders are specified as user
> parameters, and currently the name of the sequence folder is "sequence" and
> the name of unsequence folder is “unsequence”, so there must be no overlap
> between the two parameters. Therefore, I suspected that it was an abnormal
> publishing of the sequence folder list and the unsequence folder list. I
> tracked all the places where it was possible to use the two objects and
> found that it wil returned the object directly when I require them from
> multi-file directory manager. It is further found that merging the two
> objects acquired in Storage Group Processor causes problems.
>
>
> The problem is very simple and hidden but serious. The reason why it has
> not been found is that this method of incorrectly modifying the contents of
> the list is currently only used in the delete module, which is nearly
> unused. Now that this problem has been solved in sync pr, I just want to
> share with you here. The issue of variable publishing still needs to be
> paid attention to. In addition, people interested in sync module can focus
> on https://issues.apache.org/jira/browse/IOTDB-198.
>
>
>
>
> Best Regards,
> —
> Tianan Li
> School of Software, Tsinghua University
>
> 李天安
> 清华大学 软件学院

Re: Resolve "input line is too long" in windows os

[Jialin Qiao]

Hanot iotdb-env.sh, it is in the start-client.bat, import-csv.bat and 
export-csv.bat...

Best
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jialin Qiao" 
> 发送时间: 2019-09-05 09:17:45 (星期四)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re:  Resolve "input line is too long" in windows os
> 
> Hi,
> 
> Ok, I found this in the current iotdb-env.sh:
> 
> ```
> @REM Ensure that any user defined CLASSPATH variables are not used on startup
> set CLASSPATH=""
> ```
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> > -原始邮件-
> > 发件人: "Tian Jiang" 
> > 发送时间: 2019-09-05 09:05:29 (星期四)
> > 收件人: "dev@iotdb.apache.org" 
> > 抄送: "dev@iotdb.apache.org" 
> > 主题: Re:  Resolve "input line is too long" in windows os
> > 
> > Hi,
> > 
> > 
> > I think it is fine, since we are not sure what %CLASSPATH% would be 
> > originally, if we depend on some unknown external dependencies, things may 
> > get uncontrollable.
> > 
> > 
> > Tian Jiang
> > 
> > 
> > | |
> > Tian Jiang
> > |
> > |
> > jt2594...@163.com
> > |
> > 签名由网易邮箱大师定制
> > On 9/5/2019 08:55，Jialin Qiao wrote：
> > Hi Zesong,
> > 
> > Thanks for sharing your experience :)
> > 
> > I just have a question about this line:
> > 
> > set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
> > 
> > Should it be the following command?
> > 
> > set CLASSPATH=%CLASSPATH%;%IOTDB_CLI_HOME%\lib\*
> > 
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> > 
> > 乔嘉林
> > 清华大学 软件学院
> > 
> > -原始邮件-
> > 发件人: "Zesong Sun" 
> > 发送时间: 2019-09-04 22:20:54 (星期三)
> > 收件人: dev 
> > 抄送:
> > 主题: Resolve "input line is too long" in windows os
> > 
> > Hi,
> > 
> > 
> > 
> > 
> > Recently I have encountered a problem when updating maven dependency in 
> > tsfile module: when starting client in windows os, I will get an error: 
> > "The input line is too long."
> > 
> > 
> > It is resulted by limitation of the maximum length of the string that can 
> > be used at the command line: "On computers running Microsoft Windows XP or 
> > later, the maximum length of the string that you can use at the command 
> > prompt is 8191 characters. On computers running Microsoft Windows 2000 or 
> > Windows NT 4.0, the maximum length of the string that you can use at the 
> > command prompt is 2047 characters. This limitation applies to the command 
> > line, individual environment variables (such as the PATH variable) that are 
> > inherited by other processes, and all environment variable expansions. If 
> > you use Command Prompt to run batch files, this limitation also applies to 
> > batch file processing.” [1]
> > 
> > 
> > With the suggestion of @Jialin Qiao, I reduced the dependency (from 
> > hadoop-client to hadoop-common), and local tests are all passed. However, 
> > the tests on Travis CI still failed.
> > 
> > 
> > Thanks to @Tian Jiang, I changed the set CLASSPATH statement in .bat file 
> > of starting client:
> > 
> > 
> > Previous:
> > ```
> > for %%i in ("%IOTDB_CLI_HOME%\lib\*.jar") do call :append "%%i"
> > goto okClasspath
> > 
> > 
> > :append
> > set CLASSPATH=%CLASSPATH%;%1
> > goto :eof
> > 
> > ```
> > 
> > 
> > And  now:
> > ```
> > set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
> > ```
> > 
> > 
> > This avoid too long input line of CLASSPATH, which resolves the problem.
> > 
> > 
> > The error confused me for really a long time... And I think maybe this 
> > solution will help you in the future. If you have any other suggestions and 
> > ideas, welcome to discuss with us : )
> > 
> > 
> > 
> > 
> > [1] 
> > https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation
> > 
> > 
> > 
> > 
> > 
> > BR,
> > --
> > Zesong Sun
> > School of Software, Tsinghua University
> > 
> > 孙泽嵩
> > 清华大学 软件学院

Re: Resolve "input line is too long" in windows os

[Jialin Qiao]

Hi,

Ok, I found this in the current iotdb-env.sh:

```
@REM Ensure that any user defined CLASSPATH variables are not used on startup
set CLASSPATH=""
```

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Tian Jiang" 
> 发送时间: 2019-09-05 09:05:29 (星期四)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: "dev@iotdb.apache.org" 
> 主题: Re:  Resolve "input line is too long" in windows os
> 
> Hi,
> 
> 
> I think it is fine, since we are not sure what %CLASSPATH% would be 
> originally, if we depend on some unknown external dependencies, things may 
> get uncontrollable.
> 
> 
> Tian Jiang
> 
> 
> | |
> Tian Jiang
> |
> |
> jt2594...@163.com
> |
> 签名由网易邮箱大师定制
> On 9/5/2019 08:55，Jialin Qiao wrote：
> Hi Zesong,
> 
> Thanks for sharing your experience :)
> 
> I just have a question about this line:
> 
> set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
> 
> Should it be the following command?
> 
> set CLASSPATH=%CLASSPATH%;%IOTDB_CLI_HOME%\lib\*
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> -原始邮件-
> 发件人: "Zesong Sun" 
> 发送时间: 2019-09-04 22:20:54 (星期三)
> 收件人: dev 
> 抄送:
> 主题: Resolve "input line is too long" in windows os
> 
> Hi,
> 
> 
> 
> 
> Recently I have encountered a problem when updating maven dependency in 
> tsfile module: when starting client in windows os, I will get an error: "The 
> input line is too long."
> 
> 
> It is resulted by limitation of the maximum length of the string that can be 
> used at the command line: "On computers running Microsoft Windows XP or 
> later, the maximum length of the string that you can use at the command 
> prompt is 8191 characters. On computers running Microsoft Windows 2000 or 
> Windows NT 4.0, the maximum length of the string that you can use at the 
> command prompt is 2047 characters. This limitation applies to the command 
> line, individual environment variables (such as the PATH variable) that are 
> inherited by other processes, and all environment variable expansions. If you 
> use Command Prompt to run batch files, this limitation also applies to batch 
> file processing.” [1]
> 
> 
> With the suggestion of @Jialin Qiao, I reduced the dependency (from 
> hadoop-client to hadoop-common), and local tests are all passed. However, the 
> tests on Travis CI still failed.
> 
> 
> Thanks to @Tian Jiang, I changed the set CLASSPATH statement in .bat file of 
> starting client:
> 
> 
> Previous:
> ```
> for %%i in ("%IOTDB_CLI_HOME%\lib\*.jar") do call :append "%%i"
> goto okClasspath
> 
> 
> :append
> set CLASSPATH=%CLASSPATH%;%1
> goto :eof
> 
> ```
> 
> 
> And  now:
> ```
> set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
> ```
> 
> 
> This avoid too long input line of CLASSPATH, which resolves the problem.
> 
> 
> The error confused me for really a long time... And I think maybe this 
> solution will help you in the future. If you have any other suggestions and 
> ideas, welcome to discuss with us : )
> 
> 
> 
> 
> [1] 
> https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation
> 
> 
> 
> 
> 
> BR,
> --
> Zesong Sun
> School of Software, Tsinghua University
> 
> 孙泽嵩
> 清华大学 软件学院

Re: Resolve "input line is too long" in windows os

[Tian Jiang]

Hi,


I think it is fine, since we are not sure what %CLASSPATH% would be originally, 
if we depend on some unknown external dependencies, things may get 
uncontrollable.


Tian Jiang


| |
Tian Jiang
|
|
jt2594...@163.com
|
签名由网易邮箱大师定制
On 9/5/2019 08:55，Jialin Qiao wrote：
Hi Zesong,

Thanks for sharing your experience :)

I just have a question about this line:

set CLASSPATH=%IOTDB_CLI_HOME%\lib\*

Should it be the following command?

set CLASSPATH=%CLASSPATH%;%IOTDB_CLI_HOME%\lib\*

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

-原始邮件-
发件人: "Zesong Sun" 
发送时间: 2019-09-04 22:20:54 (星期三)
收件人: dev 
抄送:
主题: Resolve "input line is too long" in windows os

Hi,




Recently I have encountered a problem when updating maven dependency in tsfile 
module: when starting client in windows os, I will get an error: "The input 
line is too long."


It is resulted by limitation of the maximum length of the string that can be 
used at the command line: "On computers running Microsoft Windows XP or later, 
the maximum length of the string that you can use at the command prompt is 8191 
characters. On computers running Microsoft Windows 2000 or Windows NT 4.0, the 
maximum length of the string that you can use at the command prompt is 2047 
characters. This limitation applies to the command line, individual environment 
variables (such as the PATH variable) that are inherited by other processes, 
and all environment variable expansions. If you use Command Prompt to run batch 
files, this limitation also applies to batch file processing.” [1]


With the suggestion of @Jialin Qiao, I reduced the dependency (from 
hadoop-client to hadoop-common), and local tests are all passed. However, the 
tests on Travis CI still failed.


Thanks to @Tian Jiang, I changed the set CLASSPATH statement in .bat file of 
starting client:


Previous:
```
for %%i in ("%IOTDB_CLI_HOME%\lib\*.jar") do call :append "%%i"
goto okClasspath


:append
set CLASSPATH=%CLASSPATH%;%1
goto :eof

```


And  now:
```
set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
```


This avoid too long input line of CLASSPATH, which resolves the problem.


The error confused me for really a long time... And I think maybe this solution 
will help you in the future. If you have any other suggestions and ideas, 
welcome to discuss with us : )




[1] 
https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation





BR,
--
Zesong Sun
School of Software, Tsinghua University

孙泽嵩
清华大学 软件学院

Re: Resolve "input line is too long" in windows os

[Jialin Qiao]

Hi Zesong,

Thanks for sharing your experience :)

I just have a question about this line:

> set CLASSPATH=%IOTDB_CLI_HOME%\lib\*

Should it be the following command? 

set CLASSPATH=%CLASSPATH%;%IOTDB_CLI_HOME%\lib\* 

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Zesong Sun" 
> 发送时间: 2019-09-04 22:20:54 (星期三)
> 收件人: dev 
> 抄送: 
> 主题: Resolve "input line is too long" in windows os
> 
> Hi,
> 
> 
> 
> 
> Recently I have encountered a problem when updating maven dependency in 
> tsfile module: when starting client in windows os, I will get an error: "The 
> input line is too long."
> 
> 
> It is resulted by limitation of the maximum length of the string that can be 
> used at the command line: "On computers running Microsoft Windows XP or 
> later, the maximum length of the string that you can use at the command 
> prompt is 8191 characters. On computers running Microsoft Windows 2000 or 
> Windows NT 4.0, the maximum length of the string that you can use at the 
> command prompt is 2047 characters. This limitation applies to the command 
> line, individual environment variables (such as the PATH variable) that are 
> inherited by other processes, and all environment variable expansions. If you 
> use Command Prompt to run batch files, this limitation also applies to batch 
> file processing.” [1]
> 
> 
> With the suggestion of @Jialin Qiao, I reduced the dependency (from 
> hadoop-client to hadoop-common), and local tests are all passed. However, the 
> tests on Travis CI still failed.
> 
> 
> Thanks to @Tian Jiang, I changed the set CLASSPATH statement in .bat file of 
> starting client:
> 
> 
> Previous: 
> ```
> for %%i in ("%IOTDB_CLI_HOME%\lib\*.jar") do call :append "%%i"
> goto okClasspath
> 
> 
> :append
> set CLASSPATH=%CLASSPATH%;%1
> goto :eof
> 
> ```
> 
> 
> And  now:
> ```
> set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
> ```
> 
> 
> This avoid too long input line of CLASSPATH, which resolves the problem.
> 
> 
> The error confused me for really a long time... And I think maybe this 
> solution will help you in the future. If you have any other suggestions and 
> ideas, welcome to discuss with us : )
> 
> 
> 
> 
> [1] 
> https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation
> 
> 
> 
> 
> 
> BR,
> --
> Zesong Sun
> School of Software, Tsinghua University
> 
> 孙泽嵩
> 清华大学 软件学院

Re: Incubator Report is due today

[Justin Mclean]

Hi,

Thanks for submitting the report but can you please remove the unnecessary 
formatting, there no need for random word to be bolded as it makes it hard to 
read. Some people will be looking at the raw text and not what is shown in the 
wiki.

Thanks,
Justin

Re: Incubator Report is due today

[Kevin A. McGrail]

I've signed off on the report as well.

Any progress on plans around 0.9.0?

On 9/4/2019 12:23 AM, Xiangdong Huang wrote:
> Hi,
>
> Done for that. Thanks for notification.
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Justin Mclean  于2019年9月3日周二 下午4:39写道：
>
>> Hi,
>>
>> And ignore that, I even provided feedback on the draft. Don't forget to
>> submit it!
>>
>> Thanks,
>> Justin
>>
-- 
Kevin A. McGrail
kmcgr...@apache.org

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Resolve "input line is too long" in windows os

[Zesong Sun]

Hi,




Recently I have encountered a problem when updating maven dependency in tsfile 
module: when starting client in windows os, I will get an error: "The input 
line is too long."


It is resulted by limitation of the maximum length of the string that can be 
used at the command line: "On computers running Microsoft Windows XP or later, 
the maximum length of the string that you can use at the command prompt is 8191 
characters. On computers running Microsoft Windows 2000 or Windows NT 4.0, the 
maximum length of the string that you can use at the command prompt is 2047 
characters. This limitation applies to the command line, individual environment 
variables (such as the PATH variable) that are inherited by other processes, 
and all environment variable expansions. If you use Command Prompt to run batch 
files, this limitation also applies to batch file processing.?? [1]


With the suggestion of @Jialin Qiao, I reduced the dependency (from 
hadoop-client to hadoop-common), and local tests are all passed. However, the 
tests on Travis CI still failed.


Thanks to @Tian Jiang, I changed the set CLASSPATH statement in .bat file of 
starting client:


Previous: 
```
for %%i in ("%IOTDB_CLI_HOME%\lib\*.jar") do call :append "%%i"
goto okClasspath


:append
set CLASSPATH=%CLASSPATH%;%1
goto :eof

```


And  now:
```
set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
```


This avoid too long input line of CLASSPATH, which resolves the problem.


The error confused me for really a long time... And I think maybe this solution 
will help you in the future. If you have any other suggestions and ideas, 
welcome to discuss with us : )




[1] 
https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation





BR,
--
Zesong Sun
School of Software, Tsinghua University

??
 

A serious hidden but simple bug caused by object abnormal publishing

[李天安]

Hi,
Recently，in developing sync module, I found a very important bug in the 
process of unit testing, which will seriously affect the normal operation of 
the system. During the unit test of sync module, I found that when I retrieved 
the next sequence folder from the multi-file directory manager, I may get an 
unseuqence folder, which made me confused and began to track down the problem.


 In IoTDB, sequence folders and unsequence folders are specified as user 
parameters, and currently the name of the sequence folder is "sequence" and the 
name of unsequence folder is “unsequence”, so there must be no overlap between 
the two parameters. Therefore, I suspected that it was an abnormal publishing 
of the sequence folder list and the unsequence folder list. I tracked all the 
places where it was possible to use the two objects and found that it wil 
returned the object directly when I require them from multi-file directory 
manager. It is further found that merging the two objects acquired in Storage 
Group Processor causes problems.


The problem is very simple and hidden but serious. The reason why it has 
not been found is that this method of incorrectly modifying the contents of the 
list is currently only used in the delete module, which is nearly unused. Now 
that this problem has been solved in sync pr, I just want to share with you 
here. The issue of variable publishing still needs to be paid attention to. In 
addition, people interested in sync module can focus on 
https://issues.apache.org/jira/browse/IOTDB-198.




Best Regards,
—
Tianan Li
School of Software, Tsinghua University

李天安
清华大学 软件学院

Re: Create metadata automatically

[Jack Tsai]

Hi,

Sorry for the misunderstanding. I think this mail got to the wrong place. I do 
mean the delete interface for IoTDB Session API.

Best,
Tsung Han Tsai

取得 iOS 版 Outlook

寄件者: Jialin Qiao 
寄件日期: Wednesday, September 4, 2019 10:45:39 AM
收件者: dev@iotdb.apache.org 
主旨: Re: Create metadata automatically

Hi Tsung Han,

Thanks! Do you mean the "delete storage group" interface? If so, I suggest 
supporting this interface in Session API first. Our Antlr is upgrading, and the 
sql grammar will be different. After upgrading Altlr to version 4, we can add 
the jdbc api.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jack Tsai" 
> 发送时间: 2019-09-04 00:00:49 (星期三)
> 收件人: "dev@iotdb.apache.org" 
> 抄送:
> 主题: 回覆: Create metadata automatically
>
> Hi Jialin,
>
> Thanks for your explanation. I would like to implement this issue to 
> understand more about the writing interface in IoTDB.
>
> However, due to my time reason recently, I would like to do the delete 
> interface first, which I think it is easier to done it. Then I would 
> implement the query interface when I get more free time.
>
> Best,
> Tsung Han Tsai
> 
> 寄件者: Jialin Qiao 
> 寄件日期: 2019年9月3日 下午 04:06
> 收件者: dev@iotdb.apache.org 
> 主旨: Re: Create metadata automatically
>
> Hi,
>
> You are right, for this statement: insert into root.sg1.d1(time, s1) 
> values(1, "1").
> s1 can be treated as string.
>
> Thanks,
> --
> Jialin Qiao
> School of Software, Tsinghua University
>
> 乔嘉林
> 清华大学 软件学院
>
> > -原始邮件-
> > 发件人: "Xiangdong Huang" 
> > 发送时间: 2019-09-03 15:32:57 (星期二)
> > 收件人: dev@iotdb.apache.org
> > 抄送:
> > 主题: Re: Create metadata automatically
> >
> > Hi,
> >
> > > If the value is none of the (boolean, long, double) type, then it is
> > treated as string.
> >
> > Hm... why the string definition is not  a text which are wrapped by ".
> > Otherwise, how to define a string "true"/"false".
> >
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> >  黄向东
> > 清华大学 软件学院
> >
> >
> > Jialin Qiao  于2019年9月3日周二 上午12:08写道：
> >
> > > Hi,
> > >
> > > Thanks for reminding, boolean type can be inferred from "true" or "false".
> > >
> > > Now the configuration parameters are:
> > >
> > > auto_create_schema=false
> > > default_storage_group_level=2
> > > default_boolean_encoding=RLE
> > > default_long_encoding=RLE
> > > default_double_encoding=GORILLA
> > > default_string_encoding=PLAIN
> > >
> > > If the value is none of the (boolean, long, double) type, then it is
> > > treated as string.
> > >
> > > Best,
> > > --
> > > Jialin Qiao
> > > School of Software, Tsinghua University
> > >
> > > 乔嘉林
> > > 清华大学 软件学院
> > >
> > > > -原始邮件-
> > > > 发件人: "Xiangdong Huang" 
> > > > 发送时间: 2019-09-03 14:52:57 (星期二)
> > > > 收件人: dev@iotdb.apache.org
> > > > 抄送:
> > > > 主题: Re: Create metadata automatically
> > > >
> > > > Hi,
> > > >
> > > > By the way. If so, no "boolean" data type? And why there is no
> > > > default_string_encoding.
> > > >
> > > > Best,
> > > > ---
> > > > Xiangdong Huang
> > > > School of Software, Tsinghua University
> > > >
> > > >  黄向东
> > > > 清华大学 软件学院
> > > >
> > > >
> > > > Xiangdong Huang  于2019年9月2日周一 下午11:50写道：
> > > >
> > > > > Hi,
> > > > >
> > > > > I think it should be called as "schema" rather than "metadata"...
> > > > >
> > > > > Best,
> > > > > ---
> > > > > Xiangdong Huang
> > > > > School of Software, Tsinghua University
> > > > >
> > > > >  黄向东
> > > > > 清华大学 软件学院
> > > > >
> > > > >
> > > > > Jialin Qiao  于2019年9月2日周一 下午11:46写道：
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> I created an issue about creating metadata automatically [1].
> > > > >>
> > > > >> To achieve this, we can introduce four configuration parameters:
> > > > >>
> > > > >> auto_create_metadata=false
> > > > >> default_storage_group_level=2
> > > > >> default_int_encoding=RLE
> > > > >> default_floating_encoding=GORILLA
> > > > >>
> > > > >> Supposing we receive an insert statement "insert into
> > > root.sg1.d1(time,
> > > > >> s1, s2) values(1, 1, 1)".
> > > > >> When the "auto_create_metadata" is set to true and the user didn't
> > > create
> > > > >> metadata before, we can set storage group to root.sg1 (because the
> > > > >> default_storage_group_level is 2) and create two timeseries
> > > > >> "root.sg1.d1.s1" and "root.sg1.d1.s2".
> > > > >>
> > > > >> The datatype can be inferred from the value type. If it is "1", we
> > > treat
> > > > >> it as long. If it is a "floating number", we treat it as double.
> > > > >>
> > > > >> I'd like to have your suggestions.
> > > > >>
> > > > >> [1] https://issues.apache.org/jira/browse/IOTDB-193
> > > > >>
> > > > >> Best,
> > > > >> --
> > > > >> Jialin Qiao
> > > > >> School of Software, Tsinghua University
> > > > >>
> > > > >> 乔嘉林
> > > > >> 

[jira] [Created] (IOTDB-198) Reimplementation sync module

[litianan (Jira)]

litianan created IOTDB-198:
--

 Summary: Reimplementation sync module
 Key: IOTDB-198
 URL: https://issues.apache.org/jira/browse/IOTDB-198
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: litianan
 Fix For: 0.9.0


After updating the engine, the sync module needs to be re-implemented, and the 
function is the same as that of the original version. In addition, the logic of 
file loading at the receiving end needs to be redesigned, and the sender needs 
to support choosing storage groups whose data files to be synced.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)