Re: How to use the new table schema of Spark-Connector
Hi, The website page http://iotdb.apache.org/#/Tools/Spark does not link to the latest https://github.com/apache/incubator-iotdb/blob/master/docs/Documentation/UserGuide/9-Tools-spark.md. I'm afraid that the website is not based on the master branch. Can someone validate that? Yes I think adding the java version example is user-friendly. Sincerely, Lei Rui On 9/11/2019 09:13<827011...@qq.com> wrote?? Hi, Thanks for trying this new feature. The document of this feature is in /docs/Documentation/UserGuide/9-Tools-spark but only in scala. Should I add java version? They will be quite similar. Best, Kaifeng Xue -- -- ??: "Xiangdong Huang"; : 2019??9??11??(??) 2:50 ??: "dev"; : How to use the new table schema of Spark-Connector Hi, As PR[1] has been merged, but I can not find how to use the new feature from the iotdb.apache.org or even spark/Readme.md. What is more, even I read the introduction about this PR, I can only get how to use it in scala, but how to use it in Java? [1] https://github.com/apache/incubator-iotdb/pull/303 Best, --- Xiangdong Huang School of Software, Tsinghua University ??
Re: [IoTDB-164] Convert Path(String) to ID(Long)
Hi, I am also confused about the "six times worse" when creating timeseries. Could you propose a draft pull request to IoTDB so that we can have a look at your code implementation? Sincerely, Lei Rui On 9/11/2019 09:25，安彦哲 wrote： Hi, I'm trying to solve [IoTDB-164](https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-164?filter=allopenissues). To accomplish the task, I've modified the structure of MTree and replace Path(String) of several Maps with ID(Long), including latestTimeForEachDevice and latestFlushedTimeForEachDevice in the org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.java memTableMap in the org.apache.iotdb.db.engine.memtable.AbstractMemTable.java Then, I've carried out a simple experiment: create 200,000 timeseries in the same storage group insert 10 tuples() into each of timeseries The result shows that Step 1: the original version costs 25138.8ms while the modified one costs 177753ms. It concludes that the modified version performs six times worse than the original version when creating timeseries. Step 2: the original version costs 213662.8ms while the modified one costs 194271.2ms. It concludes that the modified version performs nine percent better than the original version when inserting values. In addition, we don't usually create too many timeseries, while insertion is the most frequently used operation in IoTDB. I'm confused whether we should apply the modified version. If anyone knows more about this topic, please inform me. Best, --- Yanzhe An School of Software, Tsinghua University | | 安彦哲 | | thss15_a...@163.com | 签名由网易邮箱大师定制
Re: Confused about one read method in TsFileInput interface
Hi,
- `TsFileInput.read(ByteBuffer dst, long position)` does not modify this
TsFileInput's position.
It works just as the annotation depicts.
If you do a simple experiment using the `class DefaultTsFileInput implements
TsFileInput` to
1) position(), 2) read(ByteBuffer dst, long position), 3) position(),
you will find that the two positions before and after read are the same.
- `class HDFSInput implements TsFileInput` in the tsfile-spark module
I encountered this similar question when implementing tsfile-spark-connector
before.
As you can see in the `class HDFSInput implements TsFileInput` in the
tsfile-spark module,
I use a plain method to realize the `read(ByteBuffer dst, long position)`
method.
Sincerely,
Lei Rui
On 9/10/2019 00:21，Yuan Tian wrote：
Hi, Boris
Yes, like what I said, the method without position signature acts like the
inputStream that you can’t manually change the position where you want to read.
Instead, you can only read by order. The position will automatically move
forward while you read data from the stream.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
在 2019年9月10日，上午12:09，Boris Zhu 写道：
I check-in java doc and it says the file position is updated with the
number of bytes actually read. You can see below
* Reads a sequence of bytes from this channel into the given buffer.
*
* Bytes are read starting at this channel's current file position, and
* then the file position is updated with the number of bytes actually
* read. Otherwise this method behaves exactly as specified in the {@link
* ReadableByteChannel} interface.
*/
public abstract int read(ByteBuffer dst) throws IOException;
you can also see it in
https://docs.oracle.com/javase/8/docs/api/java/nio/channels/FileChannel.html#read-java.nio.ByteBuffer-
best,
Boris zhu
On Tue, Sep 10, 2019 at 12:00 AM Yuan Tian wrote:
Hi, Boris
If the position is not specified, I think the position should be modified.
There are two methods in TsFileInput interface that don’t have position
parameter in their function signature.
1.
int read(ByteBuffer dst) throws IOException;
2.
int read() throws IOException;
Actually, in the corresponding methods in the FileChannel, they really
don’t modify the position which you can verify by reading the java doc of
that class. Perhaps, the reason for that is when you use the method without
position parameter, you just want to read from it by order like using a
input stream.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
在 2019年9月9日，下午11:46，Boris Zhu 写道：
I agree with you that shouldn’t modify the position of TsFileInput when
having already specified the position. Sorry for the misunderstanding.
But
if not specified, should it be changed for hdfs or local?
Best,
Boris zhu
On Mon, Sep 9, 2019 at 11:35 PM Yuan Tian
wrote:
In fact, I am working on the hadoop-connector, so I should create a
HDFSInput that implements TsFileInput interface for reading tsfile from
HDFS.
And actually, in DefaultTsFileInput, this method is unsupported which
you
can see in the following.
@Override
public int read(byte[] b, int off, int len) throws IOException {
throw new UnsupportedOperationException();
}
Not all read methods in DefaultTsFileInput modify the position, such as
'int read(ByteBuffer dst, long position)’.
So it seems that if you specify the position where you want to start in
the method signature, the method shouldn’t modify the position of
TsFileInput which is consistent with FileChannel’s interface.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
在 2019年9月9日，下午11:19，Boris Zhu 写道：
Hi, tian
DefaultTsFileInput is a class to implement TsFileInput
interface. DefaultTsFileInput's method read actually change the
position
of
itself because it uses java.io.FileChannel, and we can use a "position"
method to change its position back. But I'm also confusing that why
can't
change the position of TsFileInput or modify this TsFileInput.
Best,
Boris zhu
On Mon, Sep 9, 2019 at 10:55 PM Yuan Tian
wrote:
Hi,
While implementing one read method(shown in the following) in
TsFileInput
interface, I don’t know whether to modify the position of TsFileInput.
/**
* read an array of byte from the Input.
*
* @param b -array of byte
* @param off -offset of the Input
* @param len -length
*/
int read(byte[] b, int off, int len) throws IOException;
I think this method should not modify this TsFileInput and it should
be
specified in the comments.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
Re: [jira] [Created] (IOTDB-223) TsFile skeleton viewer
Hi, Yes, adding a script is more user-friendly. I'll find time to do that. By the way, do you guys have any opinions on this tool names, which one is better? "TsFile skeleton viewer" or "TsFile sketch tool"? Lei Rui On 9/19/2019 13:48，Xiangdong Huang wrote： Hi, I like the idea, and I notice the PR has been merged. A suggestion: as you consider the skeleton viewer as a tool. how about moving the java file into "tool" package, and add a script to use it? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Lei Rui (Jira) 于2019年9月19日周四 上午12:42写道： Lei Rui created IOTDB-223: - Summary: TsFile skeleton viewer Key: IOTDB-223 URL: https://issues.apache.org/jira/browse/IOTDB-223 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Resolve "input line is too long" in windows os
Hi,
Ok then it looks like `start-server.bat` needs changing because now it is
```
@REM * CLASSPATH library setting *
@REM Ensure that any user defined CLASSPATH variables are not used on startup
set CLASSPATH="%IOTDB_HOME%\lib"
@REM For each jar in the IOTDB_HOME lib directory call append to build the
CLASSPATH variable.
set CLASSPATH=%CLASSPATH%;"%IOTDB_HOME%\lib\*"
set CLASSPATH=%CLASSPATH%;iotdb.IoTDB
goto okClasspath
```
Lei Rui
On 9/19/2019 13:33，Xiangdong Huang wrote：
Hi,
set CLASSPATH=%CLASSPATH%;%IOTDB_CLI_HOME%\lib\*
Normally, the better way is :
set CLASSPATH=%IOTDB_CLI_HOME%\lib\*;%CLASSPATH%
It means load %IOTDB_CLI_HOME%\lib\* first, and then %CLASSPATH% that the
system original having.
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Jialin Qiao 于2019年9月5日周四 上午9:21写道：
Hanot iotdb-env.sh, it is in the start-client.bat, import-csv.bat and
export-csv.bat...
Best
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
-原始邮件-
发件人: "Jialin Qiao"
发送时间: 2019-09-05 09:17:45 (星期四)
收件人: dev@iotdb.apache.org
抄送:
主题: Re: Resolve "input line is too long" in windows os
Hi,
Ok, I found this in the current iotdb-env.sh:
```
@REM Ensure that any user defined CLASSPATH variables are not used on
startup
set CLASSPATH=""
```
Best,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
-原始邮件-
发件人: "Tian Jiang"
发送时间: 2019-09-05 09:05:29 (星期四)
收件人: "dev@iotdb.apache.org"
抄送: "dev@iotdb.apache.org"
主题: Re: Resolve "input line is too long" in windows os
Hi,
I think it is fine, since we are not sure what %CLASSPATH% would be
originally, if we depend on some unknown external dependencies, things may
get uncontrollable.
Tian Jiang
| |
Tian Jiang
|
|
jt2594...@163.com
|
签名由网易邮箱大师定制
On 9/5/2019 08:55，Jialin Qiao wrote：
Hi Zesong,
Thanks for sharing your experience :)
I just have a question about this line:
set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
Should it be the following command?
set CLASSPATH=%CLASSPATH%;%IOTDB_CLI_HOME%\lib\*
Best,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
-原始邮件-
发件人: "Zesong Sun"
发送时间: 2019-09-04 22:20:54 (星期三)
收件人: dev
抄送:
主题: Resolve "input line is too long" in windows os
Hi,
Recently I have encountered a problem when updating maven dependency
in tsfile module: when starting client in windows os, I will get an error:
"The input line is too long."
It is resulted by limitation of the maximum length of the string that
can be used at the command line: "On computers running Microsoft Windows XP
or later, the maximum length of the string that you can use at the command
prompt is 8191 characters. On computers running Microsoft Windows 2000 or
Windows NT 4.0, the maximum length of the string that you can use at the
command prompt is 2047 characters. This limitation applies to the command
line, individual environment variables (such as the PATH variable) that are
inherited by other processes, and all environment variable expansions. If
you use Command Prompt to run batch files, this limitation also applies to
batch file processing.” [1]
With the suggestion of @Jialin Qiao, I reduced the dependency (from
hadoop-client to hadoop-common), and local tests are all passed. However,
the tests on Travis CI still failed.
Thanks to @Tian Jiang, I changed the set CLASSPATH statement in .bat
file of starting client:
Previous:
```
for %%i in ("%IOTDB_CLI_HOME%\lib\*.jar") do call :append "%%i"
goto okClasspath
:append
set CLASSPATH=%CLASSPATH%;%1
goto :eof
```
And now:
```
set CLASSPATH=%IOTDB_CLI_HOME%\lib\*
```
This avoid too long input line of CLASSPATH, which resolves the
problem.
The error confused me for really a long time... And I think maybe this
solution will help you in the future. If you have any other suggestions and
ideas, welcome to discuss with us : )
[1]
https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation
BR,
--
Zesong Sun
School of Software, Tsinghua University
孙泽嵩
清华大学 软件学院
Re: [jira] [Created] (IOTDB-229) Inconsistent usage of Marker in TsFile
Hi, > I do not know why the mail does not show the description... Sorry it is because I didn't know how to add the detail description when opening an issue on JIRA. Thankfully, I now know how to do that. Below is the raw description: ``` 1) Structure ChunkMetaData.offsetOfChunkHeader refers to the position of Marker_1, while TsDeviceMetadataIndex refers to the position AFTER Marker_2. This may impose unnecessary learning burdens on users. As suggested by Jialin Qiao, an ideal case may be that ChunkMetaData.offsetOfChunkHeader refers to the position AFTER Marker_1 too. 2) Annotation The parameter `boolean markerRead` is used across multiple methods. I found two annotations of it. One annotates "read marker (boolean type)" and the other annotates "Whether the marker of the CHUNK_HEADER has been read". When `markerRead` is FALSE, the former usage will add the current read offset by 1 while the latter usage will read the byte of the Marker. Although they both push forward the read process, the two different meanings of `markerRead` is a bit of a counter-intuitive. A solution is changing the name of the parameter from `markerRead` to `isMarkerExist`, and then when `isMarkerExist` is TRUE, the former usage can choose to add the current read offset by 1 to skip it while the latter usage can choose to read the byte of the Marker. Remind to maintain the tsfile/format-changelist.md if changes are made. ``` > If it is true, then the reader do not need to read 1 byte. Ok then just take care when using it. Lei Rui On 9/19/2019 14:12，Xiangdong Huang wrote： Hi, I do not know why the mail does not show the description... As suggested by Jialin Qiao, an ideal case may be that ChunkMetaData.offsetOfChunkHeader refers to the position AFTER Marker_1 too. +1. A solution is changing the name of the parameter from `markerRead` to `isMarkerExist` The correct meaning of the parameter should be `has the Marker been read`. If it is true, then the reader do not need to read 1 byte. I suggest to make the inconsistency to the above meaning. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Lei Rui (Jira) 于2019年9月19日周四 下午1:27写道： Lei Rui created IOTDB-229: - Summary: Inconsistent usage of Marker in TsFile Key: IOTDB-229 URL: https://issues.apache.org/jira/browse/IOTDB-229 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [jira] [Created] (IOTDB-229) Inconsistent usage of Marker in TsFile
+1, generally the javadoc is not incorrect but can be written more clearly. Lei Rui On 9/19/2019 14:36，Xiangdong Huang wrote： Hi, Ok then just take care when using it. No, I think we need to fix the javadoc. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Lei Rui 于2019年9月19日周四 下午2:31写道： Hi, I do not know why the mail does not show the description... Sorry it is because I didn't know how to add the detail description when opening an issue on JIRA. Thankfully, I now know how to do that. Below is the raw description: ``` 1) Structure ChunkMetaData.offsetOfChunkHeader refers to the position of Marker_1, while TsDeviceMetadataIndex refers to the position AFTER Marker_2. This may impose unnecessary learning burdens on users. As suggested by Jialin Qiao, an ideal case may be that ChunkMetaData.offsetOfChunkHeader refers to the position AFTER Marker_1 too. 2) Annotation The parameter `boolean markerRead` is used across multiple methods. I found two annotations of it. One annotates "read marker (boolean type)" and the other annotates "Whether the marker of the CHUNK_HEADER has been read". When `markerRead` is FALSE, the former usage will add the current read offset by 1 while the latter usage will read the byte of the Marker. Although they both push forward the read process, the two different meanings of `markerRead` is a bit of a counter-intuitive. A solution is changing the name of the parameter from `markerRead` to `isMarkerExist`, and then when `isMarkerExist` is TRUE, the former usage can choose to add the current read offset by 1 to skip it while the latter usage can choose to read the byte of the Marker. Remind to maintain the tsfile/format-changelist.md if changes are made. ``` If it is true, then the reader do not need to read 1 byte. Ok then just take care when using it. Lei Rui On 9/19/2019 14:12，Xiangdong Huang wrote： Hi, I do not know why the mail does not show the description... As suggested by Jialin Qiao, an ideal case may be that ChunkMetaData.offsetOfChunkHeader refers to the position AFTER Marker_1 too. +1. A solution is changing the name of the parameter from `markerRead` to `isMarkerExist` The correct meaning of the parameter should be `has the Marker been read`. If it is true, then the reader do not need to read 1 byte. I suggest to make the inconsistency to the above meaning. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Lei Rui (Jira) 于2019年9月19日周四 下午1:27写道： Lei Rui created IOTDB-229: - Summary: Inconsistent usage of Marker in TsFile Key: IOTDB-229 URL: https://issues.apache.org/jira/browse/IOTDB-229 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] [VOTE] Apache IoTDB 0.8.1 (incubating) RC1
Hi Chris,
I get it. Thank you for the explanation.
Lei Rui
On 9/30/2019 19:49，Christofer Dutz wrote：
Hi Lei,
the last sentence is there because it's copied from the Apache PLC4X release
guidelines.
There we have one module that doesn't like not having the git repo metadata
available.
Guess it's the Karaf or the Kafka module.
You can simply ignore this for IoTDB ;-)
Chris
Am 30.09.19, 09:14 schrieb "Lei Rui" :
Hi all, my vote is +1 (binding) PPMC Vote.
Thanks to Feinauer, I followed [1] and checked:
1. download staged artifacts. Check their signature and hashes.
✔ [RM] verify the reported signature is for an "apache.org" address ("gpg: Good
signature from ...")
✔ [RM] verify the reported hashes
I use the tool [2] to automatically finish this step.
2. extract src bundle
✔ cd downloaded-iotdb-0.8.1rc1
✔ unzip 0.8.1/rc1/apache-iotdb-0.8.1-incubating-source-release.zip
✔ verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES files in the
extracted source bundle
✔ [RM] verify the staged source README, RELEASE_NOTE files correspond to those
in the extracted source bundle
3. staged src bundle items: content, can compile & test
✔ a. cd apache-iotdb-0.8.1-incubating
✔ b. check the contents of LICENSE, NOTICE, README, RELEASE_NOTES
❗ c. UNPACK the binary artefact and check the contents of LICENSE-binary and
NOTICE-binary
✔ d. build from directions in README with an empty maven local repo
❗ e. [RM] review target/rat.txt (though the build should fail if RAT
constraints aren't met)
❗ f. Search for SNAPSHOT references
✔ g. ./mvnw install
✔ h. ./mvnw site:site
Btw, I don't understand the sentence "NOTE: can't do this in a source-bundle
(only in a repo) " as noted in the step h [1].
Therefore, in my practice, I run ./mvnw install and ./mvnw site:site under the
extracted src bundle got from the previous step "unzip
0.8.1/rc1/apache-iotdb-0.8.1-incubating-source-release.zip".
Details of three ❗ items:
❗ c. UNPACK the binary artefact and check the contents of LICENSE-binary and
NOTICE-binary
NOTICE-binary HIVE shows 2008-2018 while NOTICE HIVE shows 2008-2019.
❗ e. [RM] review target/rat.txt (though the build should fail if RAT
constraints aren't met)
I didn't find the 'rat.txt' in the download staged artifacts. I don't know if
it is right.
❗ f. Search for SNAPSHOT references
I found ./hadoop/pom.xml reference 0.8.0-SNAPSHOT. The hadoop module is
excluded from the build.
By the way, there are some issues in the README.md:
The referenced wbesite https://iotdb.apache.org/#/Download does not have 0.8.1
version yet. Maybe we can update the website quickly after this release.
Out-of-date information in the Configure chapter: Now we have two more
configuration files: logback-tool.xml and iotdb-sync-client.properties.
Sincerely,
Lei Rui
---
[1]
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[2] https://github.com/apache/incubator-iotdb/pull/441
On 9/30/2019 09:18，Willem Jiang wrote：
Hi Xiangdong,
You may close this vote before starting a new vote.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Sun, Sep 29, 2019 at 7:17 PM Xiangdong Huang wrote:
Hi,
As Chris and Jialin pointed out some issues, I will release RC2 to fix them
later.
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Jialin Qiao 于2019年9月29日周日 上午11:00写道：
Hi,
-1 (binding)
Major bug: The version printed in client is still 0.8.0
Others are all ok, I checked:
Binary release:
has NOTICE, LICENSE and DISCLAIMER
Start server ok
Start client ok
compatible with 0.8.0, both client-0.8.1 with server-0.8.0 and
client-0.8.0 with server-0.8.1
Example statement see below:
```
SET STORAGE GROUP TO root.turbine;
CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=FLOAT, ENCODING=RLE;
insert into root.turbine.d1(timestamp,s0) values(1,24.5);
```
Source release:
has NOTICE, LICENSE and DISCLAIMER
mvn clean package
Thanks,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
-原始邮件-
发件人: "Christofer Dutz"
发送时间: 2019-09-28 21:47:51 (星期六)
收件人: "dev@iotdb.apache.org"
抄送:
主题: Re: [VOTE] Apache IoTDB 0.8.1 (incubating) RC1
+1 (binding)
However with some minor findings see below.
Please also automatically carry forward this +1 to the incubator voting
In preparation I went to the KEYS file and searched for the latest
entry, copied the fingerprint of that key and searched for it with my PGP
tool. I found the key and imported it. The email address reported matched
the one I expected.
- Validated the SHA512 checksums: OK
- Validated the PGP signature: OK (However there is no established trust
chain to any other Apache ... actually to no-one ... @Xiangdong you really
should go to one of the Key-Signing-Events and have your key signed by
other Apaches)
- LICENSE, NOTICE, README, RELEASE_NOTES
Re: [DISCUSS] [VOTE] Apache IoTDB 0.8.1 (incubating) RC1
Hi Chris,
Thank you for answering this question.
I then execute "mvn clean apache-rat:check" and it builds successfully with
'rat.txt' created under the target directories.
So I guess the step of check 3.e [RM] review target/rat.txt passes.
Lei Rui
On 10/1/2019 13:25，Christofer Dutz wrote：
And the rat.txt is created during the build. So you won't see it in the
downloaded archive unless you build. And running rat manually is probably a
good idea as you never know which exclusions were defined and would prevent rat
from finding stuff.
Chris
Holen Sie sich Outlook für Android<https://aka.ms/ghei36>
____________
From: Lei Rui
Sent: Tuesday, October 1, 2019 2:11:38 AM
To: dev@iotdb.apache.org
Subject: Re: [DISCUSS] [VOTE] Apache IoTDB 0.8.1 (incubating) RC1
Hi Chris,
I get it. Thank you for the explanation.
Lei Rui
On 9/30/2019 19:49，Christofer Dutz wrote：
Hi Lei,
the last sentence is there because it's copied from the Apache PLC4X release
guidelines.
There we have one module that doesn't like not having the git repo metadata
available.
Guess it's the Karaf or the Kafka module.
You can simply ignore this for IoTDB ;-)
Chris
Am 30.09.19, 09:14 schrieb "Lei Rui" :
Hi all, my vote is +1 (binding) PPMC Vote.
Thanks to Feinauer, I followed [1] and checked:
1. download staged artifacts. Check their signature and hashes.
✔ [RM] verify the reported signature is for an "apache.org" address ("gpg: Good
signature from ...")
✔ [RM] verify the reported hashes
I use the tool [2] to automatically finish this step.
2. extract src bundle
✔ cd downloaded-iotdb-0.8.1rc1
✔ unzip 0.8.1/rc1/apache-iotdb-0.8.1-incubating-source-release.zip
✔ verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES files in the
extracted source bundle
✔ [RM] verify the staged source README, RELEASE_NOTE files correspond to those
in the extracted source bundle
3. staged src bundle items: content, can compile & test
✔ a. cd apache-iotdb-0.8.1-incubating
✔ b. check the contents of LICENSE, NOTICE, README, RELEASE_NOTES
❗ c. UNPACK the binary artefact and check the contents of LICENSE-binary and
NOTICE-binary
✔ d. build from directions in README with an empty maven local repo
❗ e. [RM] review target/rat.txt (though the build should fail if RAT
constraints aren't met)
❗ f. Search for SNAPSHOT references
✔ g. ./mvnw install
✔ h. ./mvnw site:site
Btw, I don't understand the sentence "NOTE: can't do this in a source-bundle
(only in a repo) " as noted in the step h [1].
Therefore, in my practice, I run ./mvnw install and ./mvnw site:site under the
extracted src bundle got from the previous step "unzip
0.8.1/rc1/apache-iotdb-0.8.1-incubating-source-release.zip".
Details of three ❗ items:
❗ c. UNPACK the binary artefact and check the contents of LICENSE-binary and
NOTICE-binary
NOTICE-binary HIVE shows 2008-2018 while NOTICE HIVE shows 2008-2019.
❗ e. [RM] review target/rat.txt (though the build should fail if RAT
constraints aren't met)
I didn't find the 'rat.txt' in the download staged artifacts. I don't know if
it is right.
❗ f. Search for SNAPSHOT references
I found ./hadoop/pom.xml reference 0.8.0-SNAPSHOT. The hadoop module is
excluded from the build.
By the way, there are some issues in the README.md:
The referenced wbesite https://iotdb.apache.org/#/Download does not have 0.8.1
version yet. Maybe we can update the website quickly after this release.
Out-of-date information in the Configure chapter: Now we have two more
configuration files: logback-tool.xml and iotdb-sync-client.properties.
Sincerely,
Lei Rui
---
[1]
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[2] https://github.com/apache/incubator-iotdb/pull/441
On 9/30/2019 09:18，Willem Jiang wrote：
Hi Xiangdong,
You may close this vote before starting a new vote.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Sun, Sep 29, 2019 at 7:17 PM Xiangdong Huang wrote:
Hi,
As Chris and Jialin pointed out some issues, I will release RC2 to fix them
later.
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Jialin Qiao 于2019年9月29日周日 上午11:00写道：
Hi,
-1 (binding)
Major bug: The version printed in client is still 0.8.0
Others are all ok, I checked:
Binary release:
has NOTICE, LICENSE and DISCLAIMER
Start server ok
Start client ok
compatible with 0.8.0, both client-0.8.1 with server-0.8.0 and
client-0.8.0 with server-0.8.1
Example statement see below:
```
SET STORAGE GROUP TO root.turbine;
CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=FLOAT, ENCODING=RLE;
insert into root.turbine.d1(timestamp,s0) values(1,24.5);
```
Source release:
has NOTICE, LICENSE and DISCLAIMER
mvn clean package
Thanks,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
-原始邮件-
发件人: "Christofer Dut
Re: [jira] [Created] (IOTDB-244) wrong duplicated columns query result
Hi，
I find the bug and the cause is that there is divergence between the server and
the client in treating queries with duplicated columns.
For example, I query "select s0,s0,s0,s1 from root.vehicle.d0",
The server returns the result with duplicated fields as queried.
[root.vehicle.d0.s0, root.vehicle.d0.s0, root.vehicle.d0.s0, root.vehicle.d0.s1]
However, the client assumes that the returned result has been deduplicated,
[root.vehicle.d0.s0, root.vehicle.d0.s1]
thus using a structure called `columnInfoMap` in the `IoTDBQueryResultSet`.
Before fixing this bug, we should decide who handles the duplication?
The server returns a complete result and the client just need to take,
or the server returns deduplicated result and the client do additional map task?
Lei Rui
On 9/27/2019 16:09，Lei Rui (Jira) wrote：
Lei Rui created IOTDB-244:
-
Summary: wrong duplicated columns query result
Key: IOTDB-244
URL: https://issues.apache.org/jira/browse/IOTDB-244
Project: Apache IoTDB
Issue Type: Bug
Reporter: Lei Rui
I use the following sql to insert data,
{code:java}
SET STORAGE GROUP TO root.vehicle.d0
SET STORAGE GROUP TO root.vehicle.d1
CREATE TIMESERIES root.vehicle.d0.s0 WITH DATATYPE=INT32, ENCODING=RLE
CREATE TIMESERIES root.vehicle.d0.s1 WITH DATATYPE=INT64, ENCODING=RLE
CREATE TIMESERIES root.vehicle.d0.s2 WITH DATATYPE=FLOAT, ENCODING=RLE
CREATE TIMESERIES root.vehicle.d0.s3 WITH DATATYPE=TEXT, ENCODING=PLAIN
CREATE TIMESERIES root.vehicle.d0.s4 WITH DATATYPE=BOOLEAN, ENCODING=PLAIN
CREATE TIMESERIES root.vehicle.d1.s0 WITH DATATYPE=INT32, ENCODING=RLEinsert
into root.vehicle.d0(timestamp,s0) values(1,101)
insert into root.vehicle.d0(timestamp,s0) values(2,198)
insert into root.vehicle.d0(timestamp,s0) values(100,99)
insert into root.vehicle.d0(timestamp,s0) values(101,99)
insert into root.vehicle.d0(timestamp,s0) values(102,80)
insert into root.vehicle.d0(timestamp,s0) values(103,99)
insert into root.vehicle.d0(timestamp,s0) values(104,90)
insert into root.vehicle.d0(timestamp,s0) values(105,99)
insert into root.vehicle.d0(timestamp,s0) values(106,99)
insert into root.vehicle.d0(timestamp,s0) values(2,1)
insert into root.vehicle.d0(timestamp,s0) values(50,1)
insert into root.vehicle.d0(timestamp,s0) values(1000,2)insert into
root.vehicle.d0(timestamp,s1) values(1,1101)
insert into root.vehicle.d0(timestamp,s1) values(2,198)
insert into root.vehicle.d0(timestamp,s1) values(100,199)
insert into root.vehicle.d0(timestamp,s1) values(101,199)
insert into root.vehicle.d0(timestamp,s1) values(102,180)
insert into root.vehicle.d0(timestamp,s1) values(103,199)
insert into root.vehicle.d0(timestamp,s1) values(104,190)
insert into root.vehicle.d0(timestamp,s1) values(105,199)
insert into root.vehicle.d0(timestamp,s1) values(2,4)
insert into root.vehicle.d0(timestamp,s1) values(50,5)
insert into root.vehicle.d0(timestamp,s1) values(1000,5)insert into
root.vehicle.d0(timestamp,s2) values(1000,5)
insert into root.vehicle.d0(timestamp,s2) values(2,2.22)
insert into root.vehicle.d0(timestamp,s2) values(3,3.33)
insert into root.vehicle.d0(timestamp,s2) values(4,4.44)
insert into root.vehicle.d0(timestamp,s2) values(102,10.00)
insert into root.vehicle.d0(timestamp,s2) values(105,11.11)
insert into root.vehicle.d0(timestamp,s2) values(1000,1000.11)insert into
root.vehicle.d0(timestamp,s3) values(60,'a')
insert into root.vehicle.d0(timestamp,s3) values(70,'b')
insert into root.vehicle.d0(timestamp,s3) values(80,'c')
insert into root.vehicle.d0(timestamp,s3) values(101,'d')
insert into root.vehicle.d0(timestamp,s3) values(102,'f')insert into
root.vehicle.d1(timestamp,s0) values(1,999)
insert into root.vehicle.d1(timestamp,s0) values(1000,888)insert into
root.vehicle.d0(timestamp,s1) values(2000-01-01T08:00:00+08:00, 100)
insert into root.vehicle.d0(timestamp,s3) values(2000-01-01T08:00:00+08:00,
'good')insert into root.vehicle.d0(timestamp,s4) values(100, false)
insert into root.vehicle.d0(timestamp,s4) values(100, true)
{code}
and then I query as follows:
{code:java}
IoTDB> select s0,s0,s0,s1 from root.vehicle.d0
+---+--+--+--+--+
|
Time|root.vehicle.d0.s0|root.vehicle.d0.s0|root.vehicle.d0.s0|root.vehicle.d0.s1|
+---+--+--+--+--+
| 1970-01-01T08:00:00.001+08:00| 101| 101|
101| 101|
| 1970-01-01T08:00:00.002+08:00| 1| 1|
1| 1|
| 1970-01-01T08:00:00.050+08:00| 1| 1|
1| 1|
| 1970-01-01T08:00:00.100+08:00|99|99|
99|99|
| 197
probably a bug in the release tool
Hi all,
Since we are having the next release on the go, I try to use the release tool
[1] by courtesy of Julian.
I encounter an error "Error: Not a X.Y.Z version number "0.8.0" when I execute
"bash download_staged_release.sh 0.8.0 2".
I check the code and find the following two functions in common.sh seem to
return the wrong results:
```
function checkVerNum() { # $1: X.Y.Z returns true/false (0/1)
if [ `echo $1 | grep -o -E '^\d+\.\d+\.\d+$'` ]; then
return 0
else
return 1
fi
}
function checkRcNum() { # $1: rc-num returns true/false (0/1)
if [ `echo $1 | grep -o -E '^\d+$'` ] && [ $1 != 0 ]; then
return 0
else
return 1
fi
}
```
I flip the results and "bash download_staged_release.sh 0.8.0 2" can run
successfully.
Sincerely,
Lei Rui
[1] https://github.com/apache/incubator-iotdb/tree/tooling-for-release/tools
Re: probably a bug in the release tool
Hi all,
I have just proposed a pull request [1] to fix the release tool and it works on
my computer now.
This tool was adapted from PLC4X for IoTDB thanks to Julian.
This tool can help us go through the first step of validating a staged release
[2].
That is, it can download staged artifacts and check their signature and hashes.
To use this tool, run the following command:
```
bash download_staged_release.sh 0.8.1 1
```
`0.8.1` is the version number. `1` is the rc number.
[1] https://github.com/apache/incubator-iotdb/pull/441
[2]
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
Sincerely,
Lei Rui
On 9/29/2019 20:54，Lei Rui wrote：
Hi all,
Since we are having the next release on the go, I try to use the release tool
[1] by courtesy of Julian.
I encounter an error "Error: Not a X.Y.Z version number "0.8.0" when I execute
"bash download_staged_release.sh 0.8.0 2".
I check the code and find the following two functions in common.sh seem to
return the wrong results:
```
function checkVerNum() { # $1: X.Y.Z returns true/false (0/1)
if [ `echo $1 | grep -o -E '^\d+\.\d+\.\d+$'` ]; then
return 0
else
return 1
fi
}
function checkRcNum() { # $1: rc-num returns true/false (0/1)
if [ `echo $1 | grep -o -E '^\d+$'` ] && [ $1 != 0 ]; then
return 0
else
return 1
fi
}
```
I flip the results and "bash download_staged_release.sh 0.8.0 2" can run
successfully.
Sincerely,
Lei Rui
[1] https://github.com/apache/incubator-iotdb/tree/tooling-for-release/tools
Re: [VOTE] Apache IoTDB 0.8.1 (incubating) RC1
Hi all, my vote is +1 (binding) PPMC Vote.
Thanks to Feinauer, I followed [1] and checked:
1. download staged artifacts. Check their signature and hashes.
✔ [RM] verify the reported signature is for an "apache.org" address ("gpg:
Good signature from ...")
✔ [RM] verify the reported hashes
I use the tool [2] to automatically finish this step.
2. extract src bundle
✔ cd downloaded-iotdb-0.8.1rc1
✔ unzip 0.8.1/rc1/apache-iotdb-0.8.1-incubating-source-release.zip
✔ verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES files in the
extracted source bundle
✔ [RM] verify the staged source README, RELEASE_NOTE files correspond to those
in the extracted source bundle
3. staged src bundle items: content, can compile & test
✔ a. cd apache-iotdb-0.8.1-incubating
✔ b. check the contents of LICENSE, NOTICE, README, RELEASE_NOTES
❗ c. UNPACK the binary artefact and check the contents of LICENSE-binary and
NOTICE-binary
✔ d. build from directions in README with an empty maven local repo
❗ e. [RM] review target/rat.txt (though the build should fail if RAT
constraints aren't met)
❗ f. Search for SNAPSHOT references
✔ g. ./mvnw install
✔ h. ./mvnw site:site
Btw, I don't understand the sentence "NOTE: can't do this in a source-bundle
(only in a repo) " as noted in the step h [1].
Therefore, in my practice, I run ./mvnw install and ./mvnw site:site
under the extracted src bundle got from the previous step "unzip
0.8.1/rc1/apache-iotdb-0.8.1-incubating-source-release.zip".
Details of three ❗ items:
❗ c. UNPACK the binary artefact and check the contents of LICENSE-binary and
NOTICE-binary
NOTICE-binary HIVE shows 2008-2018 while NOTICE HIVE shows 2008-2019.
❗ e. [RM] review target/rat.txt (though the build should fail if RAT
constraints aren't met)
I didn't find the 'rat.txt' in the download staged artifacts. I don't know if
it is right.
❗ f. Search for SNAPSHOT references
I found ./hadoop/pom.xml reference 0.8.0-SNAPSHOT. The hadoop module is
excluded from the build.
By the way, there are some issues in the README.md:
The referenced wbesite https://iotdb.apache.org/#/Download does not have 0.8.1
version yet. Maybe we can update the website quickly after this release.
Out-of-date information in the Configure chapter: Now we have two more
configuration files: logback-tool.xml and iotdb-sync-client.properties.
Sincerely,
Lei Rui
---
[1]
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[2] https://github.com/apache/incubator-iotdb/pull/441
On 9/30/2019 09:18，Willem Jiang wrote：
Hi Xiangdong,
You may close this vote before starting a new vote.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Sun, Sep 29, 2019 at 7:17 PM Xiangdong Huang wrote:
Hi,
As Chris and Jialin pointed out some issues, I will release RC2 to fix them
later.
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Jialin Qiao 于2019年9月29日周日 上午11:00写道：
Hi,
-1 (binding)
Major bug: The version printed in client is still 0.8.0
Others are all ok, I checked:
Binary release:
has NOTICE, LICENSE and DISCLAIMER
Start server ok
Start client ok
compatible with 0.8.0, both client-0.8.1 with server-0.8.0 and
client-0.8.0 with server-0.8.1
Example statement see below:
```
SET STORAGE GROUP TO root.turbine;
CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=FLOAT, ENCODING=RLE;
insert into root.turbine.d1(timestamp,s0) values(1,24.5);
```
Source release:
has NOTICE, LICENSE and DISCLAIMER
mvn clean package
Thanks,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
-原始邮件-
发件人: "Christofer Dutz"
发送时间: 2019-09-28 21:47:51 (星期六)
收件人: "dev@iotdb.apache.org"
抄送:
主题: Re: [VOTE] Apache IoTDB 0.8.1 (incubating) RC1
+1 (binding)
However with some minor findings see below.
Please also automatically carry forward this +1 to the incubator voting
In preparation I went to the KEYS file and searched for the latest
entry, copied the fingerprint of that key and searched for it with my PGP
tool. I found the key and imported it. The email address reported matched
the one I expected.
- Validated the SHA512 checksums: OK
- Validated the PGP signature: OK (However there is no established trust
chain to any other Apache ... actually to no-one ... @Xiangdong you really
should go to one of the Key-Signing-Events and have your key signed by
other Apaches)
- LICENSE, NOTICE, README, RELEASE_NOTES exist: OK
- Checked contents of LICENSE, NOTICE, README, RELEASE_NOTES: OK
- Build IoTDB Server according to information in README.md: OK
- Running IoTDB Server according to information in README.md: MINOR (I
am using OpenJDK 13 and I get an error message that IoTDB only runs on Java
8 and 11 ... this sort of contradicts the README which states X >= 1.8)
- Running IoTDB Server with OpenJDK 11.0.2: OK
- Build IoT
Re: [DISCUSS] Release 0.9.0
You should remove duplicated changes in the former versions. For example, "IOTDB-172fix a bug of TsFileResource updateTime" has already been listed in 0.8.1. Besides, "[IOTDB-165][TsFile] Fix a example" does not only fix a bug, but also change the TsFile structure, which is unfortunately reflected neither from the JIRA issue title nor the pr name. From this example, we can also see that it is necessary to name our pr as accurate as possible. And the release manager better double check if there is inconsistency between the JIRA issue and the linked pr. Lei Rui On 10/29/2019 17:11，Jialin Qiao wrote： Hi, As 0.8.1 has been released, the release of 0.9.0 could start. I would like to do the release manager of 0.9.0, and I gathered the change list: ## New Features * IOTDB-143Compaction of data file * IOTDB-205Support storage-group-level Time To Live (TTL) * IoTDB 226Hive connect TsFile * IOTDB-188Delete storage group * IOTDB-253support time expression * IOTDB-239Add interface for showing devices * IOTDB-249enable lowercase in create_timeseries sql * IOTDB-203add "group by device" function for narrow table display * IOTDB-193Create schema automatically when inserting * IOTDB-241Add query and non query interface in session * IOTDB-223Add a TsFile sketch tool * IOTDB-158add metrics web service * IOTDB-234Refactor TsFile storage on HDFS * IOTDB-199Add a log visulization tool * IoTDB-174Add interfaces for querying device or timeseries number * IOTDB-173add batch write interface in session * IOTDB-151support number format in timeseries path * Spark-iotdb-connector * generate cpp, go, and python thrift files under service-rpc * display cache hit rate through jconsole * support time < 0: Fix initial value of minTimestamp to Long.MIN_VALUE in ChunkBuffer * Add interface (Delete timeseries) in session * Add a tool to print tsfileResources (each device's start and end time) * Support watermark feature * Add micro and nano timestamp precision ## Incompatible changes * RPC is incompatible, you can not use client-0.8.0 to connect with server-0.9.0 or use client-0.9.8 to connect with server-0.8.0. * Server is backward compatible, server-0.9.0 could run on data folder of 0.8.0. The data file will be upgraded background. https://github.com/apache/incubator-iotdb/pull/467 ## Miscellaneous changes * IOTDB-258Add documents for Query History Visualization Tool and Shared Storage Architecture * IOTDB-233keep metadata plan clear * IOTDB-267reduce IO operations in deserializing chunk header * IOTDB-265Re-adjust the threshold size of memtable * IOTDB-251improve TSQueryDataSet structure in RPC * IOTDB-221Add a python client example * IOTDB-180Get rid of JSON format in "show timeseries" * IOTDB-161Add ErrorCode of different response errors * IOTDB-160External sort * IOTDB-153further limit fetchSize to speed up LIMIT query * reconstruct antlrv3 grammar to improve performance * Tooling for release * Modified Decoder and SequenceReader to support old version of TsFile * Remove jdk constrain of jdk8 and 11 ## Known Issues * IOTDB-20Need to support UPDATE ## Bug Fixes * IOTDB-266 NullPoint exception when reading not existed devices using ReadOnlyTsFile * IOTDB-264restart failure due to WAL replay error * IOTDB-165[TsFile] Fix a example * IOTDB-240fix unknown time series in where clause * IOTDB-244fix bug when querying with duplicated columns * IOTDB-174Fix querying timeseries interface cannot make a query by the specified path prefix * IOTDB-195using String.getBytes(utf-9).length to replace string.length() in ChunkGroupMetadata for supporting Chinese * IOTDB-211use "%IOTDB_HOME%\lib\*" to refers to all .jar files in the directory in start-server.bat * IOTDB-172fix a bug of TsFileResource updateTime * fix start-walchecker scripts for leting user define the wal folder Is there any missing? Thanks, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: [DISCUSS] Release 0.9.0
Fix: there are two merged prs linked to "[IOTDB-165][TsFile] Fix a example". One is "[IOTDB-165][TsFile] Fix a example", and the other one is "[IOTDB-165][TsFile] Delete a current version and add a number version and an exception". Actually the IOTDN-165 JIRA title is "compatibility considersion", not "Fix a example". On 10/30/2019 14:22，Lei Rui wrote： You should remove duplicated changes in the former versions. For example, "IOTDB-172fix a bug of TsFileResource updateTime" has already been listed in 0.8.1. Besides, "[IOTDB-165][TsFile] Fix a example" does not only fix a bug, but also change the TsFile structure, which is unfortunately reflected neither from the JIRA issue title nor the pr name. From this example, we can also see that it is necessary to name our pr as accurate as possible. And the release manager better double check if there is inconsistency between the JIRA issue and the linked pr. Lei Rui On 10/29/2019 17:11，Jialin Qiao wrote： Hi, As 0.8.1 has been released, the release of 0.9.0 could start. I would like to do the release manager of 0.9.0, and I gathered the change list: ## New Features * IOTDB-143Compaction of data file * IOTDB-205Support storage-group-level Time To Live (TTL) * IoTDB 226Hive connect TsFile * IOTDB-188Delete storage group * IOTDB-253support time expression * IOTDB-239Add interface for showing devices * IOTDB-249enable lowercase in create_timeseries sql * IOTDB-203add "group by device" function for narrow table display * IOTDB-193Create schema automatically when inserting * IOTDB-241Add query and non query interface in session * IOTDB-223Add a TsFile sketch tool * IOTDB-158add metrics web service * IOTDB-234Refactor TsFile storage on HDFS * IOTDB-199Add a log visulization tool * IoTDB-174Add interfaces for querying device or timeseries number * IOTDB-173add batch write interface in session * IOTDB-151support number format in timeseries path * Spark-iotdb-connector * generate cpp, go, and python thrift files under service-rpc * display cache hit rate through jconsole * support time < 0: Fix initial value of minTimestamp to Long.MIN_VALUE in ChunkBuffer * Add interface (Delete timeseries) in session * Add a tool to print tsfileResources (each device's start and end time) * Support watermark feature * Add micro and nano timestamp precision ## Incompatible changes * RPC is incompatible, you can not use client-0.8.0 to connect with server-0.9.0 or use client-0.9.8 to connect with server-0.8.0. * Server is backward compatible, server-0.9.0 could run on data folder of 0.8.0. The data file will be upgraded background. https://github.com/apache/incubator-iotdb/pull/467 ## Miscellaneous changes * IOTDB-258Add documents for Query History Visualization Tool and Shared Storage Architecture * IOTDB-233keep metadata plan clear * IOTDB-267reduce IO operations in deserializing chunk header * IOTDB-265Re-adjust the threshold size of memtable * IOTDB-251improve TSQueryDataSet structure in RPC * IOTDB-221Add a python client example * IOTDB-180Get rid of JSON format in "show timeseries" * IOTDB-161Add ErrorCode of different response errors * IOTDB-160External sort * IOTDB-153further limit fetchSize to speed up LIMIT query * reconstruct antlrv3 grammar to improve performance * Tooling for release * Modified Decoder and SequenceReader to support old version of TsFile * Remove jdk constrain of jdk8 and 11 ## Known Issues * IOTDB-20Need to support UPDATE ## Bug Fixes * IOTDB-266 NullPoint exception when reading not existed devices using ReadOnlyTsFile * IOTDB-264restart failure due to WAL replay error * IOTDB-165[TsFile] Fix a example * IOTDB-240fix unknown time series in where clause * IOTDB-244fix bug when querying with duplicated columns * IOTDB-174Fix querying timeseries interface cannot make a query by the specified path prefix * IOTDB-195using String.getBytes(utf-9).length to replace string.length() in ChunkGroupMetadata for supporting Chinese * IOTDB-211use "%IOTDB_HOME%\lib\*" to refers to all .jar files in the directory in start-server.bat * IOTDB-172fix a bug of TsFileResource updateTime * fix start-walchecker scripts for leting user define the wal folder Is there any missing? Thanks, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: [DISCUSS] Release 0.9.0
Yes, you miss "modify print function in AbstractClient"https://github.com/apache/incubator-iotdb/pull/322 :) On 10/29/2019 17:11，Jialin Qiao wrote： Hi, As 0.8.1 has been released, the release of 0.9.0 could start. I would like to do the release manager of 0.9.0, and I gathered the change list: ## New Features * IOTDB-143Compaction of data file * IOTDB-205Support storage-group-level Time To Live (TTL) * IoTDB 226Hive connect TsFile * IOTDB-188Delete storage group * IOTDB-253support time expression * IOTDB-239Add interface for showing devices * IOTDB-249enable lowercase in create_timeseries sql * IOTDB-203add "group by device" function for narrow table display * IOTDB-193Create schema automatically when inserting * IOTDB-241Add query and non query interface in session * IOTDB-223Add a TsFile sketch tool * IOTDB-158add metrics web service * IOTDB-234Refactor TsFile storage on HDFS * IOTDB-199Add a log visulization tool * IoTDB-174Add interfaces for querying device or timeseries number * IOTDB-173add batch write interface in session * IOTDB-151support number format in timeseries path * Spark-iotdb-connector * generate cpp, go, and python thrift files under service-rpc * display cache hit rate through jconsole * support time < 0: Fix initial value of minTimestamp to Long.MIN_VALUE in ChunkBuffer * Add interface (Delete timeseries) in session * Add a tool to print tsfileResources (each device's start and end time) * Support watermark feature * Add micro and nano timestamp precision ## Incompatible changes * RPC is incompatible, you can not use client-0.8.0 to connect with server-0.9.0 or use client-0.9.8 to connect with server-0.8.0. * Server is backward compatible, server-0.9.0 could run on data folder of 0.8.0. The data file will be upgraded background. https://github.com/apache/incubator-iotdb/pull/467 ## Miscellaneous changes * IOTDB-258Add documents for Query History Visualization Tool and Shared Storage Architecture * IOTDB-233keep metadata plan clear * IOTDB-267reduce IO operations in deserializing chunk header * IOTDB-265Re-adjust the threshold size of memtable * IOTDB-251improve TSQueryDataSet structure in RPC * IOTDB-221Add a python client example * IOTDB-180Get rid of JSON format in "show timeseries" * IOTDB-161Add ErrorCode of different response errors * IOTDB-160External sort * IOTDB-153further limit fetchSize to speed up LIMIT query * reconstruct antlrv3 grammar to improve performance * Tooling for release * Modified Decoder and SequenceReader to support old version of TsFile * Remove jdk constrain of jdk8 and 11 ## Known Issues * IOTDB-20Need to support UPDATE ## Bug Fixes * IOTDB-266 NullPoint exception when reading not existed devices using ReadOnlyTsFile * IOTDB-264restart failure due to WAL replay error * IOTDB-165[TsFile] Fix a example * IOTDB-240fix unknown time series in where clause * IOTDB-244fix bug when querying with duplicated columns * IOTDB-174Fix querying timeseries interface cannot make a query by the specified path prefix * IOTDB-195using String.getBytes(utf-9).length to replace string.length() in ChunkGroupMetadata for supporting Chinese * IOTDB-211use "%IOTDB_HOME%\lib\*" to refers to all .jar files in the directory in start-server.bat * IOTDB-172fix a bug of TsFileResource updateTime * fix start-walchecker scripts for leting user define the wal folder Is there any missing? Thanks, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: [DISCUSS] Release 0.9.0
Hi Jialin, How is everything going? I looked into the v0.9.0 changelist again and I am confused about IOTDB-174, which appears in both "New Features" and "Bug Fixes". The description of IOTDB-174 is more like "New Features"- "Add interfaces for querying device or timeseries number ". But the title of the pr#435 related to IOTDB-174 is more like "Bug Fixes" - "Fix querying timeseries interface cannot make a query by the specified path prefix". Usually a pr is intended to do one thing at a time. So I think we better clarify whether the pr fixes a bug or adds a new feature, or both indeed. Sincerely, Lei Rui On 10/31/2019 21:06，Jialin Qiao wrote： Hi Tianan Li and Lei Rui, I will update the change list according to your feedback. Thanks, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 -原始邮件- 发件人: "Lei Rui" 发送时间: 2019-10-30 14:30:41 (星期三) 收件人: "dev@iotdb.apache.org" 抄送: 主题: Re: [DISCUSS] Release 0.9.0 Fix: there are two merged prs linked to "[IOTDB-165][TsFile] Fix a example". One is "[IOTDB-165][TsFile] Fix a example", and the other one is "[IOTDB-165][TsFile] Delete a current version and add a number version and an exception". Actually the IOTDN-165 JIRA title is "compatibility considersion", not "Fix a example". On 10/30/2019 14:22，Lei Rui wrote： You should remove duplicated changes in the former versions. For example, "IOTDB-172fix a bug of TsFileResource updateTime" has already been listed in 0.8.1. Besides, "[IOTDB-165][TsFile] Fix a example" does not only fix a bug, but also change the TsFile structure, which is unfortunately reflected neither from the JIRA issue title nor the pr name. From this example, we can also see that it is necessary to name our pr as accurate as possible. And the release manager better double check if there is inconsistency between the JIRA issue and the linked pr. Lei Rui On 10/29/2019 17:11，Jialin Qiao wrote： Hi, As 0.8.1 has been released, the release of 0.9.0 could start. I would like to do the release manager of 0.9.0, and I gathered the change list: ## New Features * IOTDB-143Compaction of data file * IOTDB-205Support storage-group-level Time To Live (TTL) * IoTDB 226Hive connect TsFile * IOTDB-188Delete storage group * IOTDB-253support time expression * IOTDB-239Add interface for showing devices * IOTDB-249enable lowercase in create_timeseries sql * IOTDB-203add "group by device" function for narrow table display * IOTDB-193Create schema automatically when inserting * IOTDB-241Add query and non query interface in session * IOTDB-223Add a TsFile sketch tool * IOTDB-158add metrics web service * IOTDB-234Refactor TsFile storage on HDFS * IOTDB-199Add a log visulization tool * IoTDB-174Add interfaces for querying device or timeseries number * IOTDB-173add batch write interface in session * IOTDB-151support number format in timeseries path * Spark-iotdb-connector * generate cpp, go, and python thrift files under service-rpc * display cache hit rate through jconsole * support time < 0: Fix initial value of minTimestamp to Long.MIN_VALUE in ChunkBuffer * Add interface (Delete timeseries) in session * Add a tool to print tsfileResources (each device's start and end time) * Support watermark feature * Add micro and nano timestamp precision ## Incompatible changes * RPC is incompatible, you can not use client-0.8.0 to connect with server-0.9.0 or use client-0.9.8 to connect with server-0.8.0. * Server is backward compatible, server-0.9.0 could run on data folder of 0.8.0. The data file will be upgraded background. https://github.com/apache/incubator-iotdb/pull/467 ## Miscellaneous changes * IOTDB-258Add documents for Query History Visualization Tool and Shared Storage Architecture * IOTDB-233keep metadata plan clear * IOTDB-267reduce IO operations in deserializing chunk header * IOTDB-265Re-adjust the threshold size of memtable * IOTDB-251improve TSQueryDataSet structure in RPC * IOTDB-221Add a python client example * IOTDB-180Get rid of JSON format in "show timeseries" * IOTDB-161Add ErrorCode of different response errors * IOTDB-160External sort * IOTDB-153further limit fetchSize to speed up LIMIT query * reconstruct antlrv3 grammar to improve performance * Tooling for release * Modified Decoder and SequenceReader to support old version of TsFile * Remove jdk constrain of jdk8 and 11 ## Known Issues * IOTDB-20Need to support UPDATE ## Bug Fixes * IOTDB-266 NullPoint exception when reading not existed devices using ReadOnlyTsFile * IOTDB-264restart failure due to WAL replay error * IOTDB-165[TsFile] Fix a example * IOTDB-240fix unknown time series in where clause * IOTDB-244fix bug when querying with duplicated columns * IOTDB-174Fix querying timeseries interface cannot make a query by the s
Re: [VOTE] Apache IoTDB 0.9.0 (incubating) RC1 release
Hi, my vote is -1(PPMC). The reason is that I ran "./mvn.sh install" and the build failed: ``` [ERROR] Errors: [ERROR] SeqTsFileRecoverTest.test:138 » NullPointer [ERROR] UnseqTsFileRecoverTest.test:147 » NullPointer ``` Sincerely Lei Rui On 11/14/2019 16:23，Jialin Qiao wrote： Hi all, Thanks to all of you for releasing. As a new release manager, I don't meet many troubles in the release process. Big thanks for Xiangdong's release document: (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=130027555) Apache IoTDB (incubating) 0.9.0 is a major version with many exciting features. You can get the release notes in [5]. Apache IoTDB (Incubating) 0.9.0 has been staged under [2] and it’s time to vote on accepting it for release. All Maven artifacts are available under [1]. If approved we will seek final release approval from the IPMC. Voting will be open for 72hr. A minimum of 3 binding +1 votes and more binding +1 than binding -1 are required to pass. Release tag: release/0.9.0 Hash for the release tag: 4d5e3b3beb52a1e74422f95ddb652805bfef90a1 Per [3] "Before voting +1 PMC/PPMC members are required to download the signed source code package, compile it as provided, and test the resulting executable on their own platform, along with also verifying that the package meets the requirements of the ASF policy on releases." You can achieve the above by following [4]. [ ] +1 accept (indicate what you validated - e.g. performed the non-RM items in [4]) [ ] -1 reject (explanation required) [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1007 [2] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.0/rc1 [3] https://www.apache.org/dev/release.html#approving-a-release [4] https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release [5] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.0/rc1/RELEASE_NOTES.md Thanks, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: A SQL to Query a Group of Devices Seperately
Hi Tsai, +1 for your idea. It sounds good to me. Best, Lei Rui On 11/23/2019 09:42，Jack Tsai wrote： Hi all, I am recently working on this issue: https://issues.apache.org/jira/browse/IOTDB-305, which is about resolving the problem while users want to query something like below as the issue reporter Lei Rui said: select * from root.sg.d1,root.sg.d2 where s1=1 (a demo conception) / select * from root.sg.* where s1=1 (a demo conception) There is no specific sql that could satisfy this kind of query requirement. The only one which is similar to this concept is using the “group by device” statement. However, it also cannot return the appropriate result the users need as what Lei Rui mentioned in the issue. Now, I plan to edit the implement logic of the “group by device” statement. The original implement method, which is like using the “and” statement to form the condition part of the sql as mentioned in this issue, cannot well suited for regular users’ requirements. In conclusion, the original function of the “group by device” will be abandoned, and it will be replaced by the one which mentioned in this issue to satisfy users query. I’m not sure whether this is OK for you guys or the project. So if you got any advice, please welcome to discuss with me. Best regards, Jack Tsai
Re: A SQL to Query a Group of Devices Seperately
Hi,
I want to share some of my thoughts about the to-be-changed ,
after I took a look at the introduction of GROUP BY in the Microsoft SQL Docs
[1] and
ISO/IEC 9075-2:2003 2 ("SQL/Foundation") .
Basically I agree with the semantic changes of as Tsai
described in the email.
I try to sum up: it is how interacts with the SELECT
statement that is to be changed.
The original interacts mainly with the FROM clause.
The new will additionally interact with the WHERE clause by
making WHERE conditions function within separate devices.
Example query:
```
SELECT * FROM root.sg.d2,root.sg.d1 WHERE s1=1 <(new) group by device>
```
According to my understanding, the result will be the same as the following
conceptual query that returns a union of two separate queries:
```
SELECT * FROM root.sg.d2 WHERE s1=1
UNION ALL
SELECT * FROM root.sg.d1 WHERE s1=1
ORDER BY device_name (p.s. or just keep the same order as in the FROM clause.
This can be left for more discussion.)
```
------
Best,
Lei Rui
[1]
https://docs.microsoft.com/en-us/sql/t-sql/queries/select-group-by-transact-sql?view=sql-server-ver15#syntax
On 11/23/2019 13:37，Xiangdong Huang wrote：
Hi,
The term "group by device" is fine for ok.
Or, we can I think classify the queries into 3 part:
1. return data points time series by time series, while in each
timeseries, the data points are ordered by the timestamp. (Or, totally
ordered by timestamp for all timeseries)
2. Join all time series that belong to the same device on the timestamp, in
Relational SQL, looks like d1.s1 join d1.s2 on d1.s1.time=d1.s2.time join
d1.s3 on d1.s1.time=d1.s3.time ...
3. Join all the time series in the database, which is what IoTDB now
supports.
This issue IOTDB-305 is for solving the 2nd query.
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Lei Rui 于2019年11月23日周六 下午12:02写道：
Hi Tsai,
+1 for your idea. It sounds good to me.
Best,
Lei Rui
On 11/23/2019 09:42，Jack Tsai wrote：
Hi all,
I am recently working on this issue:
https://issues.apache.org/jira/browse/IOTDB-305, which is about resolving
the problem while users want to query something like below as the issue
reporter Lei Rui said:
select * from root.sg.d1,root.sg.d2 where s1=1 (a demo
conception)
/ select * from root.sg.* where s1=1 (a demo conception)
There is no specific sql that could satisfy this kind of query
requirement. The only one which is similar to this concept is using the
“group by device” statement. However, it also cannot return the appropriate
result the users need as what Lei Rui mentioned in the issue.
Now, I plan to edit the implement logic of the “group by device”
statement. The original implement method, which is like using the “and”
statement to form the condition part of the sql as mentioned in this issue,
cannot well suited for regular users’ requirements.
In conclusion, the original function of the “group by device” will be
abandoned, and it will be replaced by the one which mentioned in this issue
to satisfy users query. I’m not sure whether this is OK for you guys or the
project. So if you got any advice, please welcome to discuss with me.
Best regards,
Jack Tsai
Re: [VOTE] Apache IoTDB 0.9.0 (incubating) RC4 release
My vote is +1 (PPMC binding).
1. download staged artifacts. Check their signature and hashes.
✔ [RM] verify the reported signature is for an "apache.org" address ("gpg: Good
signature from ...")
✔ [RM] verify the reported hashes
2. extract src bundle
✔ verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES files in the
extracted source bundle
✔ [RM] verify the staged source README, RELEASE_NOTE files correspond to those
in the extracted source bundle
3. staged src bundle items: content, can compile & test
✔ check the contents of LICENSE, NOTICE, README, RELEASE_NOTES
✔ UNPACK the binary artefact and check the contents of LICENSE-binary and
NOTICE-binary
✔ build from directions in README with an empty maven local repo: mvn clean
package -DskipTests -Dmaven.repo.local=../.m2
✔ [RM] review target/rat.txt (though the build should fail if RAT constraints
aren't met)
- mvn clean apache-rat:check
- Find and files containing binary content with this command: find .
-type f -name 'rat.txt' -exec grep -l " B " {} \;
- Then review for B (binary) content in those files:
grep " B " ./grafana/target/rat.txt
grep " B " ./server/target/rat.txt
I find:
B
/home/ruilei/tools/downloaded-iotdb-0.9.0rc4/0.9.0/rc4/apache-iotdb-0.9.0-incubating/grafana/img/add_data_source.png
B
/home/ruilei/tools/downloaded-iotdb-0.9.0rc4/0.9.0/rc4/apache-iotdb-0.9.0-incubating/grafana/img/edit_data_source.png
B
/home/ruilei/tools/downloaded-iotdb-0.9.0rc4/0.9.0/rc4/apache-iotdb-0.9.0-incubating/grafana/img/add_graph.png
B
/home/ruilei/tools/downloaded-iotdb-0.9.0rc4/0.9.0/rc4/apache-iotdb-0.9.0-incubating/server/src/main/resources/iotdb/ui/static/iotdb-logo.png
✔ Search for SNAPSHOT references: find . -type f -name 'pom.xml' -exec grep -l
"SNAPSHOT" {} \;
✔ ./mvnw.sh install
✔ ./mvnw.sh site:site
Best,
Lei Rui
On 11/21/2019 12:41，Jialin Qiao wrote：
Hi all,
Now I open a new ticket for Apache IoTDB (incubating) 0.9.0 RC4, which fixs
some issues comparing with RC3. You can find the details at the end of this
email.
Apache IoTDB (incubating) 0.9.0 is a major version with many exciting features.
You can get the release notes in [5].
Apache IoTDB (Incubating) 0.9.0 has been staged under [2] and it’s time to
vote on accepting it for release. All Maven artifacts are available under [1].
If approved we will seek final release approval from the IPMC.
Voting will be open for 72hr.
A minimum of 3 binding +1 votes and more binding +1 than binding -1 are
required to pass.
Release tag: release/0.9.0
Hash for the release tag: eedfec84462c0f574f9d6ade5ac53801cb3bc869
Per [3] "Before voting +1 PMC/PPMC members are required to download the
signed source code package, compile it as provided, and test the resulting
executable on their own platform, along with also verifying that the package
meets the requirements of the ASF policy on releases."
Difference with RC3:
Remove the .tsfile
You can achieve the above by following [4].
[ ] +1 accept (indicate what you validated - e.g. performed the non-RM items
in [4])
[ ] -1 reject (explanation required)
[1] https://repository.apache.org/content/repositories/orgapacheiotdb-1016
[2] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.0/rc4
[3] https://www.apache.org/dev/release.html#approving-a-release
[4]
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[5]
https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.0/rc4/RELEASE_NOTES.md
Thanks,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
Re: [jira] [Created] (IOTDB-347) Bug in limit clause
Hi,
it's not a bug :)
Note that LIMIT has been removed to the server side, and
IoTDBQueryResultSetTest here is only faking a result from the server.
Lei Rui
On 12/9/2019 21:40，Yuan Tian (Jira) wrote：
Yuan Tian created IOTDB-347:
---
Summary: Bug in limit clause
Key: IOTDB-347
URL: https://issues.apache.org/jira/browse/IOTDB-347
Project: Apache IoTDB
Issue Type: Bug
Reporter: Yuan Tian
The test function `testQuery` in `IoTDBQueryResultSetTest.java` construct data
as following
{code:java}
Object[][] input = {
{2L, 2.22F, 4L, null,},
{3L, 3.33F, null, null,},
{4L, 4.44F, null, null,},
{50L, null, 5L, null,},
{100L, null, 199L, null,},
{101L, null, 199L, null,},
{103L, null, 199L, null,},
{105L, 11.11F, 199L, 3,},
{1000L, 1000.11F, 5L, 2,}};
{code}
and query sql is
{code:java}
String testSql = "select *,s1,s0,s2 from root.vehicle.d0 where s1 > 190 or s2 <
10.0 "
+ "limit 20 offset 1 slimit 4 soffset 2";
{code}
with the offset 1, we should skip the `\{2L, 2.22F, 4L, null,}`, but the
expected answer doesn't do that:
{code:java}
String standard =
"Time,root.vehicle.d0.s2,root.vehicle.d0.s1,root.vehicle.d0.s0,root.vehicle.d0.s2,\n"
+ "2,2.22,4,null,2.22,\n"
+ "3,3.33,null,null,3.33,\n"
+ "4,4.44,null,null,4.44,\n"
+ "50,null,5,null,null,\n"
+ "100,null,199,null,null,\n"
+ "101,null,199,null,null,\n"
+ "103,null,199,null,null,\n"
+ "105,11.11,199,3,11.11,\n"
+ "1000,1000.11,5,2,1000.11,\n";
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Code refactoring of Query
Hi, +1 with Jialin Qiao. I think the ongoing query refactor work will cover the following points: 1) refactor the ordinary query, by a) changing all operators in query plans to be batch operators b) replanning the reading of data from storage, taking into consideration the interactions both between the sequence and overflow data and within overflow data 2) refactor the aggregate and groupBy queries, by a) abstracting new aggregate operators or interfaces to enable both point, page and chunk level aggregate reading b) replanning the reading of data from storage, taking into consideration the interactions both between the sequence and overflow data and within overflow data Lastly, sorry @atoiLiu I don't quite get what you mean. `Limit` query is already realized I think. Best, Lei Rui On 12/10/2019 22:37，atoiLiu wrote： hi, Offset command is sometimes found in SQL, so hopefully it will also provide a jump query 在 2019年12月10日，下午7:16，Jialin Qiao 写道： Hi, Code refactoring is inevitable when building a large system. The read/write of TsFile, storage engine of the server have been refactored. Now, it's time to refactor the query engine in the server. Currently, the query is in a tuple-at-a-time manner. In the meantime, the interface is in chaos. To improve the query speed, a batch-at-a-time(vectorization) iteration is needed. I have opened a branch with Lei Rui: f_batch_reader. The existing interfaces are simplified and IBatchReader is added. Welcome to work on this branch for query optimization. Thanks, — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: [VOTE] Apache IoTDB 0.8.1 (incubating) RC3 release
Hi, my vote is +1 (binding) PPMC vote.
I followed [1] and checked:
1. download staged artifacts. Check their signature and hashes.
✔[RM] verify the reported signature is for an "apache.org" address ("gpg:
Good signature from ...")
✔[RM] verify the reported hashes
2. extract src bundle
✔cd downloaded-iotdb-0.8.1rc1
✔unzip 0.8.1/rc1/apache-iotdb-0.8.1-incubating-source-release.zip
✔verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES files in
the extracted source bundle
✔[RM] verify the staged source README, RELEASE_NOTE files correspond to
those in the extracted source bundle
3. staged src bundle items: content, can compile & test
✔a. cd apache-iotdb-0.8.1-incubating
❗ b. check the contents of LICENSE, NOTICE, README, RELEASE_NOTES
❗ c. UNPACK the binary artefact and check the contents of LICENSE-binary
and NOTICE-binary
✔d. build from directions in README with an empty maven local repo
❗ e. [RM] review target/rat.txt (though the build should fail if RAT
constraints aren't met)
- Find and files containing binary content with this command: find .
-type f -name 'rat.txt' -exec grep -l " B " {} \;
- Then review for B (binary) content in those files: grep " B "
target/rat.txt
❗ f. Search for SNAPSHOT references: find . -type f -name 'pom.xml' -exec
grep -l "SNAPSHOT" {} \;
✔g. ./mvnw install
✔h. ./mvnw site:site
Details of three ❗ items:
b. check the contents of LICENSE, NOTICE, README, RELEASE_NOTES
- LICENSE: There is a place of duplication:
```
The following class is modified from Maven Wrapper, which is under
Apache License 2.0:
.mvn/wrapper/MavenWrapperDownloader.java
The following class is modified from Maven Wrapper
./.mvn/wrapper/MavenWrapperDownloader.java
```
- NOTICE: "Copyright 2019 and onwards" is not incorrect, as Justin has
pointed out.
c. UNPACK the binary artefact and check the contents of LICENSE-binary and
NOTICE-binary
- LICENSE-binary: I checked the licenses/ directory and I wonder what
software the license "Eclipse Public License version 2.0 (EPL-2.0)" corresponds
to.
- NOTICE-binary: "Copyright 2019 and onwards"
e. [RM] review target/rat.txt (though the build should fail if RAT constraints
aren't met)
I executed the following commands:
1. mvn clean apache-rat:check
2. Find and files containing binary content with this command: find .
-type f -name 'rat.txt' -exec grep -l " B " {} \;
3. Then review for B (binary) content in those files: grep " B "
./grafana/target/rat.txt
And I found the following png marked as "B":
B
/home/rl/test/downloaded-iotdb-0.8.1rc3/0.8.1/rc3/apache-iotdb-0.8.1-incubating/grafana/img/add_data_source.png
B
/home/rl/test/downloaded-iotdb-0.8.1rc3/0.8.1/rc3/apache-iotdb-0.8.1-incubating/grafana/img/edit_data_source.png
B
/home/rl/test/downloaded-iotdb-0.8.1rc3/0.8.1/rc3/apache-iotdb-0.8.1-incubating/grafana/img/add_graph.png
Is that normal?
f. Search for SNAPSHOT references: find . -type f -name 'pom.xml' -exec grep -l
"SNAPSHOT" {} \;
I found ./hadoop/pom.xml reference 0.8.1-SNAPSHOT.
[1]https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[2]https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist
Sincerely,
Lei Rui
On 10/15/2019 17:14，Tian Jiang wrote：
Hi,
+1 (PPMC vote)
The followings are successfully verified:
Source release:
1. ASC sinature
2. Hashes
3. LICENSE, NOTICE, README, RELEASE_NOTES
4. RAT
5. mvn clean package
6. start server and client
7. execute basic operations and reboot
Binary distribution:
1. ASC sinature
2. Hashes
3. start server and client
4. execute basic operations and reboot
Best,
Tian Jiang
On 10/15/2019 15:57，李天安 wrote：
Hi,
+1 (PPMC vote)
I have checked following items:
Source release:
1. Running RAT(mvn apache-rat:check) (Correct)
2. Verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES files in the
extracted source bundle (Correct)
3. Compile and build code with command: mvn clean package. (Correct)
4. Verifying the signature (ASC) (Correct)
5. Verifying the hashes (SHA512) (Correct)
Binary distribution:
1. Verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES files
(Correct)
2. Verifying the signature (ASC) (Correct)
3. Verifying the hashes (SHA512) (Correct)
Functionality：
1. Run server and client script: sh start-server.sh, start-client.sh -h
127.0.0.1 -p 6667 -u root -pw root (Correct)
2. REBOOT
3. Execuet some basic operations: (Correct)
1) SET STORAGE GROUP TO root.vehicle
2) CREATE TIMESERIES root.vehicle.d0.s0 WITH DATATYPE=INT32, ENCODING=RLE
3) insert into root.vehicle.d0(timestamp,s0) values(10,100)
4) insert into ro
Re: Who can review this pr?
Agree. The plugin for Antlr4 in IDEA is really easy to use. Thank you for your work. I can review this pr this week. Lei Rui On 11/27/2019 10:19，Boris Zhu wrote： It is about upgrading antlr3 to antlr4, I don't think it's a good idea to use a software that is no longer maintained. On Wed, Nov 27, 2019 at 9:16 AM Boris Zhu wrote: Hi, Can someone help me with reviewing this pr? https://github.com/apache/incubator-iotdb/pull/564 Best Boris
Re: [VOTE] Apache IoTDB 0.9.0 (incubating) RC5 release
Hi, my vote is +1 (PPMC binding).
I followed [1] and checked:
1. download staged artifacts. Check their signature and hashes.
✔ [RM] verify the reported signature is for an "apache.org" address ("gpg: Good
signature from ...")
✔ [RM] verify the reported hashes
2. extract src bundle
✔ verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES files in the
extracted source bundle
✔ [RM] verify the staged source README, RELEASE_NOTE files correspond to those
in the extracted source bundle
3. staged src bundle items: content, can compile & test
✔ check the contents of LICENSE, NOTICE, README, RELEASE_NOTES
✔ UNPACK the binary artefact and check the contents of LICENSE-binary and
NOTICE-binary
✔ build from directions in README with an empty maven local repo: mvn clean
package -DskipTests -Dmaven.repo.local=../.m2
✔ [RM] review target/rat.txt (though the build should fail if RAT constraints
aren't met)
- mvn clean apache-rat:check
- Find and files containing binary content with this command: find .
-type f -name 'rat.txt' -exec grep -l " B " {} \;
- Then review for B (binary) content in those files:
grep " B " ./grafana/target/rat.txt
grep " B " ./server/target/rat.txt
I find:
B
/home/ruilei/tools/downloaded-iotdb-0.9.0rc4/0.9.0/rc4/apache-iotdb-0.9.0-incubating/grafana/img/add_data_source.png
B
/home/ruilei/tools/downloaded-iotdb-0.9.0rc4/0.9.0/rc4/apache-iotdb-0.9.0-incubating/grafana/img/edit_data_source.png
B
/home/ruilei/tools/downloaded-iotdb-0.9.0rc4/0.9.0/rc4/apache-iotdb-0.9.0-incubating/grafana/img/add_graph.png
B
/home/ruilei/tools/downloaded-iotdb-0.9.0rc4/0.9.0/rc4/apache-iotdb-0.9.0-incubating/server/src/main/resources/iotdb/ui/static/iotdb-logo.png
✔ Search for SNAPSHOT references: find . -type f -name 'pom.xml' -exec grep -l
"SNAPSHOT" {} \;
✔ ./mvnw.sh install
✔ ./mvnw.sh site:site
Best,
Lei Rui
[1]https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
On 11/23/2019 18:56，Jialin Qiao wrote：
Hi all,
Now I open a new ticket for Apache IoTDB (incubating) 0.9.0 RC5, which fixs
some issues comparing with RC4. You can find the details at the end of this
email.
Apache IoTDB (incubating) 0.9.0 is a major version with many exciting features.
You can get the release notes in [5].
Apache IoTDB (Incubating) 0.9.0 has been staged under [2] and it’s time to
vote on accepting it for release. All Maven artifacts are available under [1].
If approved we will seek final release approval from the IPMC.
Voting will be open for 72hr.
A minimum of 3 binding +1 votes and more binding +1 than binding -1 are
required to pass.
Release tag: release/0.9.0
Hash for the release tag: 257df94d57c29c628b1ab0d6a2fdd20ba7df9ca4
Per [3] "Before voting +1 PMC/PPMC members are required to download the
signed source code package, compile it as provided, and test the resulting
executable on their own platform, along with also verifying that the package
meets the requirements of the ASF policy on releases."
Difference with RC4:
- Fix windows start problem
- Fix deadlock between merge and query
You can achieve the above by following [4].
[ ] +1 accept (indicate what you validated - e.g. performed the non-RM items
in [4])
[ ] -1 reject (explanation required)
[1] https://repository.apache.org/content/repositories/orgapacheiotdb-1018
[2] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.0/rc5
[3] https://www.apache.org/dev/release.html#approving-a-release
[4]
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[5]
https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.0/rc5/RELEASE_NOTES.md
Thanks,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
[DISCUSS] Apache IoTDB 0.9.0 (incubating) RC5 release
Hi Tian Jiang, I can't see the picture you attached. Lei Rui On 11/25/2019 16:51，Tian Jiang wrote： Hi, +1 (PPMC vote) The followings are successfully verified: Source release: 1. ASC sinature 2. Hashes 3. LICENSE, NOTICE, README, RELEASE_NOTES 4. RAT 5. mvn clean package 6. start server and client 7. execute basic operations and reboot 8. python client Binary distribution: 1. ASC sinature 2. Hashes 3. start server and client 4. execute basic operations and reboot 5. python client Minor issues: 1. I can’t use tools/download_staged_release.sh and have to manually verify something. Did I mistake the usage? Best, Tian Jiang | | Tian Jiang | | jt2594...@163.com | 签名由网易邮箱大师定制 On 11/25/2019 12:15，Xiangdong Huang wrote： +1. I checked: - mvn apache-rat:check. - DISCLAIMER LICENSE NOTICE exist and the content is ok. - No *.jar, no *.tsfile, no *.ts binary files in the source-release.zip file - compiling the source codes on WinOS successfully, all UT passed. - can run start-server.bat and start-client.bat on WinOS Minor issues: - Document: no readme for tools/upgrade/offline-upgrade.sh, After the releasing, we need add a document on the website for the tool (or on the github, or on the confluence. Either is ok.) - As Kevin said and discussions before, the client scripts should be moved to bin out of sbin. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Jialin Qiao 于2019年11月23日周六 下午6:57写道： Hi all, Now I open a new ticket for Apache IoTDB (incubating) 0.9.0 RC5, which fixs some issues comparing with RC4. You can find the details at the end of this email. Apache IoTDB (incubating) 0.9.0 is a major version with many exciting features. You can get the release notes in [5]. Apache IoTDB (Incubating) 0.9.0 has been staged under [2] and it’s time to vote on accepting it for release. All Maven artifacts are available under [1]. If approved we will seek final release approval from the IPMC. Voting will be open for 72hr. A minimum of 3 binding +1 votes and more binding +1 than binding -1 are required to pass. Release tag: release/0.9.0 Hash for the release tag: 257df94d57c29c628b1ab0d6a2fdd20ba7df9ca4 Per [3] "Before voting +1 PMC/PPMC members are required to download the signed source code package, compile it as provided, and test the resulting executable on their own platform, along with also verifying that the package meets the requirements of the ASF policy on releases." Difference with RC4: - Fix windows start problem - Fix deadlock between merge and query You can achieve the above by following [4]. [ ] +1 accept (indicate what you validated - e.g. performed the non-RM items in [4]) [ ] -1 reject (explanation required) [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1018 [2] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.0/rc5 [3] https://www.apache.org/dev/release.html#approving-a-release [4] https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release [5] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.0/rc5/RELEASE_NOTES.md Thanks, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: 回复: [DISCUSS] Apache IoTDB 0.9.0 (incubating) RC5 release
Hi Tian Jiang, I remember that Jialin Qiao encountered the same problem before, so I guess it may be a special bug on MacOS, since I can use this tool on both Windows and Linux. Does someone else with MacOS encounter this problem? Lei Rui 在2019年11月25日 19:31，JT 写道： Sorry, it seems something was wrong with the file server. I can not fully recall it, but I used it with “./download_staged_release.sh 0.9.0 5” and it probably said “not a X.Y.Z version number “0.9.0” ”. Tian Jiang 发件人: Lei Rui 发送时间: 2019年11月25日 19:09 收件人: dev@iotdb.apache.org 主题: [DISCUSS] Apache IoTDB 0.9.0 (incubating) RC5 release Hi Tian Jiang, I can't see the picture you attached. Lei Rui On 11/25/2019 16:51，Tian Jiang wrote： Hi, +1 (PPMC vote) The followings are successfully verified: Source release: 1. ASC sinature 2. Hashes 3. LICENSE, NOTICE, README, RELEASE_NOTES 4. RAT 5. mvn clean package 6. start server and client 7. execute basic operations and reboot 8. python client Binary distribution: 1. ASC sinature 2. Hashes 3. start server and client 4. execute basic operations and reboot 5. python client Minor issues: 1. I can’t use tools/download_staged_release.sh and have to manually verify something. Did I mistake the usage? Best, Tian Jiang | | Tian Jiang | | jt2594...@163.com | 签名由网易邮箱大师定制 On 11/25/2019 12:15，Xiangdong Huang wrote： +1. I checked: - mvn apache-rat:check. - DISCLAIMER LICENSE NOTICE exist and the content is ok. - No *.jar, no *.tsfile, no *.ts binary files in the source-release.zip file - compiling the source codes on WinOS successfully, all UT passed. - can run start-server.bat and start-client.bat on WinOS Minor issues: - Document: no readme for tools/upgrade/offline-upgrade.sh, After the releasing, we need add a document on the website for the tool (or on the github, or on the confluence. Either is ok.) - As Kevin said and discussions before, the client scripts should be moved to bin out of sbin. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Jialin Qiao 于2019年11月23日周六 下午6:57写道： Hi all, Now I open a new ticket for Apache IoTDB (incubating) 0.9.0 RC5, which fixs some issues comparing with RC4. You can find the details at the end of this email. Apache IoTDB (incubating) 0.9.0 is a major version with many exciting features. You can get the release notes in [5]. Apache IoTDB (Incubating) 0.9.0 has been staged under [2] and it’s time to vote on accepting it for release. All Maven artifacts are available under [1]. If approved we will seek final release approval from the IPMC. Voting will be open for 72hr. A minimum of 3 binding +1 votes and more binding +1 than binding -1 are required to pass. Release tag: release/0.9.0 Hash for the release tag: 257df94d57c29c628b1ab0d6a2fdd20ba7df9ca4 Per [3] "Before voting +1 PMC/PPMC members are required to download the signed source code package, compile it as provided, and test the resulting executable on their own platform, along with also verifying that the package meets the requirements of the ASF policy on releases." Difference with RC4: - Fix windows start problem - Fix deadlock between merge and query You can achieve the above by following [4]. [ ] +1 accept (indicate what you validated - e.g. performed the non-RM items in [4]) [ ] -1 reject (explanation required) [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1018 [2] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.0/rc5 [3] https://www.apache.org/dev/release.html#approving-a-release [4] https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release [5] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.0/rc5/RELEASE_NOTES.md Thanks, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: [jira] [Created] (IOTDB-315) Can't start iotdb on windows
I checked in 0.9.0-RC4 and yes...it can't start on windows. Lei Rui On 11/22/2019 16:20，Xiangdong Huang wrote： Ops, is 0.9.0-RC4 also involved? --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Lei Rui (Jira) 于2019年11月22日周五 下午4:18写道： Lei Rui created IOTDB-315: - Summary: Can't start iotdb on windows Key: IOTDB-315 URL: https://issues.apache.org/jira/browse/IOTDB-315 Project: Apache IoTDB Issue Type: Bug Reporter: Lei Rui D:\new\incubator-iotdb\server\target\iotdb-server-0.9.0-SNAPSHOT\sbin>.\start-server.bat Starting IoTDB If you want to change this configuration, please check conf/iotdb-env.sh(Unix or OS X, if you use Windows, check conf/iotdb-env.bat). 此时不应有 (。 I use master branch. commit id=def4daf2addef0062fe04f698a825301ffa343bc -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] Apache IoTDB 0.9.0 (incubating) RC4 release
Hi,
because of the issue IOTDB-315 "Can't start iotdb on windows" just found, I
change my vote to -1 (PPMC binding).
Best,
Lei Rui
On 11/22/2019 11:12，Tian Jiang wrote：
Hi,
+1 (PPMC vote)
The followings are successfully verified:
Source release:
1. ASC sinature
2. Hashes
3. LICENSE, NOTICE, README, RELEASE_NOTES
4. RAT
5. mvn clean package
6. start server and client
7. execute basic operations and reboot
8. python client
Binary distribution:
1. ASC sinature
2. Hashes
3. start server and client
4. execute basic operations and reboot
5. python client
Best,
Tian Jiang
| |
Tian Jiang
|
|
jt2594...@163.com
|
签名由网易邮箱大师定制
On 11/22/2019 09:44，Lei Rui wrote：
My vote is +1 (PPMC binding).
1. download staged artifacts. Check their signature and hashes.
✔ [RM] verify the reported signature is for an "apache.org" address ("gpg: Good
signature from ...")
✔ [RM] verify the reported hashes
2. extract src bundle
✔ verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES files in the
extracted source bundle
✔ [RM] verify the staged source README, RELEASE_NOTE files correspond to those
in the extracted source bundle
3. staged src bundle items: content, can compile & test
✔ check the contents of LICENSE, NOTICE, README, RELEASE_NOTES
✔ UNPACK the binary artefact and check the contents of LICENSE-binary and
NOTICE-binary
✔ build from directions in README with an empty maven local repo: mvn clean
package -DskipTests -Dmaven.repo.local=../.m2
✔ [RM] review target/rat.txt (though the build should fail if RAT constraints
aren't met)
- mvn clean apache-rat:check
- Find and files containing binary content with this command: find . -type f
-name 'rat.txt' -exec grep -l " B " {} \;
- Then review for B (binary) content in those files:
grep " B " ./grafana/target/rat.txt
grep " B " ./server/target/rat.txt
I find:
B
/home/ruilei/tools/downloaded-iotdb-0.9.0rc4/0.9.0/rc4/apache-iotdb-0.9.0-incubating/grafana/img/add_data_source.png
B
/home/ruilei/tools/downloaded-iotdb-0.9.0rc4/0.9.0/rc4/apache-iotdb-0.9.0-incubating/grafana/img/edit_data_source.png
B
/home/ruilei/tools/downloaded-iotdb-0.9.0rc4/0.9.0/rc4/apache-iotdb-0.9.0-incubating/grafana/img/add_graph.png
B
/home/ruilei/tools/downloaded-iotdb-0.9.0rc4/0.9.0/rc4/apache-iotdb-0.9.0-incubating/server/src/main/resources/iotdb/ui/static/iotdb-logo.png
✔ Search for SNAPSHOT references: find . -type f -name 'pom.xml' -exec grep -l
"SNAPSHOT" {} \;
✔ ./mvnw.sh install
✔ ./mvnw.sh site:site
Best,
Lei Rui
On 11/21/2019 12:41，Jialin Qiao wrote：
Hi all,
Now I open a new ticket for Apache IoTDB (incubating) 0.9.0 RC4, which fixs
some issues comparing with RC3. You can find the details at the end of this
email.
Apache IoTDB (incubating) 0.9.0 is a major version with many exciting features.
You can get the release notes in [5].
Apache IoTDB (Incubating) 0.9.0 has been staged under [2] and it’s time to
vote on accepting it for release. All Maven artifacts are available under [1].
If approved we will seek final release approval from the IPMC.
Voting will be open for 72hr.
A minimum of 3 binding +1 votes and more binding +1 than binding -1 are
required to pass.
Release tag: release/0.9.0
Hash for the release tag: eedfec84462c0f574f9d6ade5ac53801cb3bc869
Per [3] "Before voting +1 PMC/PPMC members are required to download the
signed source code package, compile it as provided, and test the resulting
executable on their own platform, along with also verifying that the package
meets the requirements of the ASF policy on releases."
Difference with RC3:
Remove the .tsfile
You can achieve the above by following [4].
[ ] +1 accept (indicate what you validated - e.g. performed the non-RM items
in [4])
[ ] -1 reject (explanation required)
[1] https://repository.apache.org/content/repositories/orgapacheiotdb-1016
[2] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.0/rc4
[3] https://www.apache.org/dev/release.html#approving-a-release
[4]
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[5]
https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.0/rc4/RELEASE_NOTES.md
Thanks,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
Re: [jira] [Created] (IOTDB-240) query problem for iotdb filter
Hi all, I want to have more discussions on the responses of selecting non-existent timeseries. Let's focus on the condition 2 - "select ZT1,ZT31 from root.demo" where root.demo.ZT1 doesn't exist. There can be 3 responses as follows. Response 1: reject service and return an error message. Response 2: provide service by neglecting "ZT1" and directly return the result table of "select ZT31 from root.demo". Response 3: provide service by returning the result table with fields under the column "root.demo.ZT1" being null. From my side, I definitely disapprove response 3 because of the possible confusion raised about the schema. My question here is which one do you prefer, response 1 or response 2? I myself prefer response 2 to response 1, because response 2 returns existent data and meanwhile conveys the message that "root.demo.ZT1" doesn't exist clearly. Sincerely, Lei Rui On 9/26/2019 14:52<827011...@qq.com> wrote?? Hi~ I'm working on this issue and I think "both condition 2 and 3 should respond with error message" is better -- -- ??: "Yuan Tian (Jira)"; : 2019??9??25??(??) 9:56 ??: "dev"; : [jira] [Created] (IOTDB-240) query problem for iotdb filter EJTTianyu created IOTDB-240: --- Summary: query problem for iotdb filter Key: IOTDB-240 URL: https://issues.apache.org/jira/browse/IOTDB-240 Project: Apache IoTDB Issue Type: Bug Reporter: EJTTianyu When querying a timeseries that does not exist, iotdb shows different behavior for SQL with or without filter. After starting a new iotdb instance, The following steps in iotdb-client below could reproduce the problem. Step 1: set storage group to root.demo Step 2: CREATE TIMESERIES root.demo.ZT31 WITH DATATYPE=INT64, ENCODING=RLE Step 3:insert into root.demo(timestamp,ZT31) values(1,1) when using sql: 1.select ZT1 from root.demo iotdb-client responds with message "Msg: Execute statement error: do not select any existing series" 2.select ZT1,ZT31 from root.demo iotdb-client responds with +---+--+ | Time|root.demo.ZT31| +---+--+ | 1970-01-01T08:00:00.001+08:00| 1| +---+--+ 3.select ZT1,ZT31 from root.demo where ZT1>0 iotdb-client responds with message "Msg: Execute statement error: Index: 0, Size: 0" Timeseries ZT1 does not exist in condition 2 and condition 3. However, without using ZT1 as filter to query multiple sensors(at least one exists), iotdb could respond with datapoints. When using ZT1 as filter, the returned msg confused users. In my opinion, both condition 2 and 3 should respond with message "Msg: Execute statement error: do not select any existing series". Or condition 2 responds with datapoints, condition 3 responds with "Msg: Execute statement error: do not select any existing series". -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: A new result set format
up by device" is equal to "select s0,s1 from root.sg.* group by device". # REALIZATION OF GROUP BY DEVICE - Code Realization 1. sql lexer & parser related classes: TSLexer, TSParser 2. sql -> logical operator related classes: QueryOperator, LogicalGenerator 3. logical opeator -> optimized logical operator related classes: ConcatPathOptimizer 4. optimized logical operator -> physical plan (core step) related classes: QueryPlan, PhysicalGenerator 5. physical plan -> executeQueryStatement & client related classes: QueryPlan, TSServiceImpl, SQLConstant 6. physical plan -> fetchResults & server related classes: QueryPlan, AbstractQueryProcessorExecutor, DeviceIterateDataSet, QueryDataSet 7. For table display format related classes: AbstractClient # USER GUIDANCE OF GROUP BY DEVICE GroupbyDeviceClause : GROUP BY DEVICE Rules: 1. Both uppercase and lowercase are ok. Correct example: select * from root.sg1 group by device Correct example: select * from root.sg1 GROUP BY DEVICE 2. GroupbyDeviceClause can only be used at the end of a query statement. Correct example: select * from root.sg1 where time > 10 group by device Wrong example: select * from root.sg1 group by device where time > 10 3. The paths of the SELECT clause can only be single level. In other words, the paths of the SELECT clause can only be measurements or STAR, without DOT. Correct example: select s0,s1 from root.sg1.* group by device Correct example: select s0,s1 from root.sg1.d0, root.sg1.d1 group by device Correct example: select * from root.sg1.* group by device Correct example: select * from root group by device Correct example: select s0,s1,* from root.*.* group by device Wrong example: select d0.s1, d0.s2, d1.s0 from root.sg1 group by device Wrong example: select *.s0, *.s1 from root.* group by device Wrong example: select *.*.* from root group by device 4. The data types of the same measurement column should be the same across devices. Note that when it comes to aggregated paths, the data type of the measurement column will reflect the aggregation function rather than the original timeseries. Correct example: select s0 from root.sg1.d0,root.sg1.d1 group by device root.sg1.d0.s0 and root.sg1.d1.s0 are both INT32. Correct example: select count(s0) from root.sg1.d0,root.sg1.d1 group by device count(root.sg1.d0.s0) and count(root.sg1.d1.s0) are both INT64. Wrong example: select s0 from root.sg1.d0, root.sg2.d3 group by device root.sg1.d0.s0 is INT32 while root.sg2.d3.s0 is FLOAT. 5. The display principle of the result table is that only when the column (or row) has existing data will the column (or row) be shown, with nonexistent cells being null. For example, "select s0,s1,s2 from root.sg.d0, root.sg.d1, root.sg.d2 group by device". Suppose that the actual existing timeseries are as follows: - root.sg.d0.s0 - root.sg.d0.s1 - root.sg.d1.s0 Then the header of the result table will be: [Time, Device, s0, s1]. And you could expect a table like: | Time | Device | s0 | s1 | | 1 |root.sg.d0| 20 | 2.5| | 2 |root.sg.d0| 23 | 3.1| | 1 |root.sg.d1| 12 |null| | 2 |root.sg.d1| 19 |null| Note that the cells of measurement 's0' and device 'root.sg.d1' are all null. Also note that the column of 's2' and the rows of 'root.sg.d2' are not existent. 6. The duplicated devices in the prefix paths are neglected. For example, "select s0,s1 from root.sg.d0,root.sg.d0,root.sg.d1 group by device" is equal to "select s0,s1 from root.sg.d0,root.sg.d1 group by device". For example. "select s0,s1 from root.sg.*,root.sg.d0 group by device" is equal to "select s0,s1 from root.sg.* group by device". 7. The duplicated measurements in the suffix paths are not neglected. For example, "select s0,s0,s1 from root.sg.* group by device" is not equal to "select s0,s1 from root.sg.* group by device". 8. More correct examples: - select * from root.vehicle group by device - select s0,s0,s1 from root.vehicle.* group by device - select s0,s1 from root.vehicle.* limit 10 offset 1 group by device - select * from root.vehicle slimit 10 soffset 2 group by device - select * from root.vehicle where time > 10 group by device - select * from root.vehicle where root.vehicle.d0.s0>0 group by device - select count(*) from root.vehicle group by device - select sum(*) from root.vehicle GROUP BY (20ms,0,[2,50]) group by device - select * from root.vehicle where time = 3 Fill(int32[previous, 5ms]) group by device --- Sincerely, Lei Rui On 9/8/2019 04:36，Julian Feinauer wrote： Hi, I think this is one oft he most important discussion to make it easily accessible to users. As Xiangdong states, we have to make it comfortable for users to map to relational schemes. And I agree that there are use cases for both, so probably really just provide both. Julian Am 07.09.19, 13:18 schrieb "Xiangdong Huang" : Hi, Glad
Re: [DISCUSS] Release of 0.9.1
Hi, I want to challenge myself to be the release manager of 0.9.1. Regards, Lei Rui On 12/17/2019 20:47，Jialin Qiao wrote： Hi, Change list could be inferred from merged PRs in https://github.com/apache/incubator-iotdb/pulse The second, as far as I know, there is no need to do more patches, cause I already cherry-pick all the bug-fix commits, but double-check is better. Thanks, Jialin Qiao Xiangdong Huang 于2019年12月17日周二 下午8:40写道： Hi, +1 for releasing 0.9.1. Two concerns: Actually, when I merge the bug-fix PRs, I also cherry-pick them into rel/0.9. Therefore, this release will be easier than 0.9.0... So, how to maintain the change list? The second is similar, how many patches should be involved into 0.9.1? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Jialin Qiao 于2019年12月17日周二 下午8:27写道： Hi, After releasing 0.9.0, we fixed some bugs that will impact the user experience. It's time to release 0.9.1, a bug-fix version of 0.9.0. Is there anyone who wants to be the release manager? Actually, when I merge the bug-fix PRs, I also cherry-pick them into rel/0.9. Therefore, this release will be easier than 0.9.0... Thanks, — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 -- — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: [DISCUSS] Release of 0.9.1
Hi Jialin, That's nice. Thank you. Lei Rui On 12/18/2019 14:25，Jialin Qiao wrote： Hi, That's very kind of you :) I'd like to offer help and share my experience if you meet trouble when releasing. Thanks, Jialin Qiao Lei Rui 于2019年12月18日周三 下午2:13写道： Hi, I want to challenge myself to be the release manager of 0.9.1. Regards, Lei Rui On 12/17/2019 20:47，Jialin Qiao wrote： Hi, Change list could be inferred from merged PRs in https://github.com/apache/incubator-iotdb/pulse The second, as far as I know, there is no need to do more patches, cause I already cherry-pick all the bug-fix commits, but double-check is better. Thanks, Jialin Qiao Xiangdong Huang 于2019年12月17日周二 下午8:40写道： Hi, +1 for releasing 0.9.1. Two concerns: Actually, when I merge the bug-fix PRs, I also cherry-pick them into rel/0.9. Therefore, this release will be easier than 0.9.0... So, how to maintain the change list? The second is similar, how many patches should be involved into 0.9.1? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Jialin Qiao 于2019年12月17日周二 下午8:27写道： Hi, After releasing 0.9.0, we fixed some bugs that will impact the user experience. It's time to release 0.9.1, a bug-fix version of 0.9.0. Is there anyone who wants to be the release manager? Actually, when I merge the bug-fix PRs, I also cherry-pick them into rel/0.9. Therefore, this release will be easier than 0.9.0... Thanks, — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 -- — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 -- — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
[CALL FOR VOTES] Release Apache IoTDB 0.9.1
Hi, After the vote letter titled "[VOTE] Release Apache IoTDB 0.9.1" was sent to gene...@incubator.apache.org on Jan 3, two +1 votes have been cast by Justin Mclean and Willem Jiang. We may need more votes, since a minimum of 3 binding +1 votes and more binding +1 than binding -1 are required to pass. @Christofer Dutz @Kevin A. McGrail @Julian Feinauer Regards, Lei Rui
Re: [VOTE] Apache IoTDB (incubating) 0.9.1 RC1
Thank you Julian. Regards, Lei Rui On 1/9/2020 18:30，Xiangdong Huang wrote： Hi Julian, Welcome back~. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Julian Feinauer 于2020年1月9日周四 下午6:28写道： Hi, sorry for being late to the Party... My vote is +1 (binding, IPMC Vote) I did: - check out from svn - Check README, RELEASE_NOTES - Check Signatures, Hashes and Keys file (first release of Lei Rui) - Run "mvn clean install" on source release Minor Issue: With German Locale it does NOT work as one test fails due to decimal comma vs decimal point. But on US localeit works, so I think its okay - No unexpected binaries - Checked LICENSE and NOTICE Best Julian Am 03.01.20, 04:43 schrieb "Justin Mclean" : Hi, Do you think that when bring up a release VOTE, we need to add the PR of update the website in VOTE thread? The changes of the PR would be similar with No. You should only update the website after the IPMC vote has passes, the file put into the Apache mirrors and you waited 24 hours or so for the release to propagate. Thanks, Justin
[DISCUSS] When Release 0.10.0
Hi, It has been almost two months since the release tag of `release/0.9.0` was created on Nov 23, 2019. 1. New changes that have taken place: Refer to the temporary release note for `0.10.0-SNAPSHOT`, auto-generated by JIRA [1]. (about 16 bug fixes, 4 new features, 13 improvements) 2. Upcoming new changes: As far as I know, the develop team has been working towards two directions recently: 1. the distributed version of IoTDB 2. the refactor of both physical design and query processing Shall we withhold the release of 0.10.0 until the two upcoming changes are finished? What do you think? Regards, Lei Rui [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12322720=12346616
[DISCUSS] Apache IoTDB (incubating) 0.9.1 RC1
Thank Justin for the quick reply. Regards, Lei Rui On 1/3/2020 11:43，Justin Mclean wrote： Hi, Do you think that when bring up a release VOTE, we need to add the PR of update the website in VOTE thread? The changes of the PR would be similar with No. You should only update the website after the IPMC vote has passes, the file put into the Apache mirrors and you waited 24 hours or so for the release to propagate. Thanks, Justin
Re: [DISCUSS] Enable github issue?
I would suggest a formal vote. Regards, Lei Rui On 1/3/2020 16:41，Jialin Qiao wrote： Hi, In the discussion about enabling github issues, 5 people support while no other voices. Jialin Qiao Minhao Gouwang Xiangdong Huang Tianci Zhu Dawei Liu I think there is no need to call for a vote. Meanwhile, Jira is also used and we could carry the issues to Jira through some tools. Could someone help enable the github issues? Thanks, Jialin Qiao 于2020年1月2日周四 下午4:29写道： Hi, Let's continue this discussion or make a decision? Jialin Qiao 于2019年12月31日周二 下午7:28写道： Hi, +1 for managing issues in github. +0 for using Jira. Thanks Jialin Qiao -- — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 -- — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: [DISCUSS] Apache IoTDB (incubating) 0.9.1 RC1
@jincheng sun Thanks for your kind sharing. Regards, Lei Rui On 1/3/2020 16:01，jincheng sun wrote： I see. Thanks for your feedback! Lei Rui & Justin! As I know, Both Apache Beam[1] and Apache Flink[2] carry website updated PRs when bring up the release votes. This approach benefits Apache Beam and Apache Flink as the update changes of Beam and Flink are a little more complicated compare with Apache IoTDB, this approach speeds up the entire publishing process. However, the current update of the IoTDB website is a little bit easy, So, this approach has not brought obvious benefits. So, I am fine with keep current process of IoTDB. [1] https://lists.apache.org/thread.html/2fa0dd0e8710e62deab2e25c6081ed94636448f123d6f91ad190aa21%40%3Cdev.beam.apache.org%3E [2] https://lists.apache.org/thread.html/716337faba0160e8382543c8eb10bfa34895efc2419e94a1f3cc8d19%40%3Cdev.flink.apache.org%3E Best, Jincheng Lei Rui 于2020年1月3日周五 下午2:06写道： Hi, No need +1. I don't think it's a good practice, because if the released file doesn't exist, how can people review a PR like [1] where there are download links in it waiting to be checked, e.g., https://www.apache.org/dyn/closer.cgi/incubator/iotdb/0.9.0-incubating/apache-iotdb-0.9.0-incubating-source-release.zip . Regards, Lei Rui [1] https://github.com/apache/incubator-iotdb-website/commit/dc4d0226985c976561944068fd012e5cdfab4339 On 1/3/2020 13:52，jincheng sun wrote： No. You should only update the website after the IPMC vote has passes, the file put into the Apache mirrors and you waited 24 hours or so for the release to propagate. I mean that prepare the PR when bring up the VOTE, then people can review the PR ASAP. we can merge it when vote has passed and 24 hours later after the release . Best, Jincheng Lei Rui 于2020年1月3日周五 下午12:22写道： Thank Justin for the quick reply. Regards, Lei Rui On 1/3/2020 11:43，Justin Mclean wrote： Hi, Do you think that when bring up a release VOTE, we need to add the PR of update the website in VOTE thread? The changes of the PR would be similar with No. You should only update the website after the IPMC vote has passes, the file put into the Apache mirrors and you waited 24 hours or so for the release to propagate. Thanks, Justin
Re: [DISCUSS] Apache IoTDB (incubating) 0.9.1 RC1
Hi, No need +1. I don't think it's a good practice, because if the released file doesn't exist, how can people review a PR like [1] where there are download links in it waiting to be checked, e.g., https://www.apache.org/dyn/closer.cgi/incubator/iotdb/0.9.0-incubating/apache-iotdb-0.9.0-incubating-source-release.zip. Regards, Lei Rui [1] https://github.com/apache/incubator-iotdb-website/commit/dc4d0226985c976561944068fd012e5cdfab4339 On 1/3/2020 13:52，jincheng sun wrote： No. You should only update the website after the IPMC vote has passes, the file put into the Apache mirrors and you waited 24 hours or so for the release to propagate. I mean that prepare the PR when bring up the VOTE, then people can review the PR ASAP. we can merge it when vote has passed and 24 hours later after the release . Best, Jincheng Lei Rui 于2020年1月3日周五 下午12:22写道： Thank Justin for the quick reply. Regards, Lei Rui On 1/3/2020 11:43，Justin Mclean wrote： Hi, Do you think that when bring up a release VOTE, we need to add the PR of update the website in VOTE thread? The changes of the PR would be similar with No. You should only update the website after the IPMC vote has passes, the file put into the Apache mirrors and you waited 24 hours or so for the release to propagate. Thanks, Justin
Re: [DISCUSS] Should I change the year to 2020 in advance when releasing 0.9.1?
Thank you all for the suggestions! I have changed the year to 2020 in the NOTICE file. Hopefully, the release vote of 0.9.1 will be initiated today. Regards, Lei Rui On 12/28/2019 11:11，Xiangdong Huang wrote： +1. Look forward to the new release version. --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 jincheng sun 于2019年12月28日周六 上午7:30写道： I think 2020 would be fine ：） Christofer Dutz 于2019年12月28日 周六00:45写道： +1 From me too Chris Am 27.12.19, 17:40 schrieb "Kevin A. McGrail" : +1 to update in advance just make sure not to announce the release until 2020. On 12/27/2019 10:32 AM, Lei Rui wrote: Hi, I am the release manager of the upcoming 0.9.1 and I encounter an interesting question: As you can see, Year 2020 is just around the corner and the upcoming release vote for 0.9.1 is likely to span from 2019 to 2020. So I don't know if I should change the year in advance to 2020, or just keep it as 2019, or maybe I postpone the release vote until it is 2020? P.s., the year 2019 is now displayed in the NOTICE file like this: Apache IoTDB Copyright 2018-2019 The Apache Software Foundation. Regards, Lei Rui -- Kevin A. McGrail kmcgr...@apache.org Member, Apache Software Foundation Chair Emeritus Apache SpamAssassin Project https://www.linkedin.com/in/kmcgrail - 703.798.0171 -- Best, Jincheng - Twitter: https://twitter.com/sunjincheng121 -
[VOTE] Apache IoTDB (incubating) 0.9.1 RC1
Hi all, Apache IoTDB (Incubating) 0.9.1 has been staged under [2] and it's time to vote on accepting it for release. You can get its mainly changes from [5]. All Maven artifacts are available under [1]. If approved we will seek final release approval from the IPMC. Voting will be open for 72hr. A minimum of 3 binding +1 votes and more binding +1 than binding -1 are required to pass. Release tag: release/0.9.1 Hash for the release tag: d665690607933db9fdac057ebe250c19c9f63974 Per [3] "Before voting +1 [P]PMC members are required to download the signed source code package, compile it as provided, and test the resulting executable on their own platform, along with also verifying that the package meets the requirements of the ASF policy on releases." You can achieve the above by following [4]. [ ] +1 accept (indicate what you validated - e.g. performed the non-RM items in [4]) [ ] -1 reject (explanation required) [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1029 [2] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.1/rc1 [3] https://www.apache.org/dev/release.html#approving-a-release [4] https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release [5] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.9.1/rc1/RELEASE_NOTES.md Regards, Lei Rui
Re: [DISCUSS] Should I change the year to 2020 in advance when releasing 0.9.1?
Hi KAM, I am a little confused. Do you mean the NOTICE can't be changed to "Apache IoTDB Copyright 2018-2020" until it is 2021? Regards, Lei Rui On 1/2/2020 03:02，Kevin A. McGrail wrote： I am going to vote -1 and insist we change back to 2019. Happy New Year, KAM On Mon, Dec 30, 2019, 20:58 Lei Rui wrote: Thank you all for the suggestions! I have changed the year to 2020 in the NOTICE file. Hopefully, the release vote of 0.9.1 will be initiated today. Regards, Lei Rui On 12/28/2019 11:11，Xiangdong Huang wrote： +1. Look forward to the new release version. --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 jincheng sun 于2019年12月28日周六 上午7:30写道： I think 2020 would be fine ：） Christofer Dutz 于2019年12月28日 周六00:45写道： +1 From me too Chris Am 27.12.19, 17:40 schrieb "Kevin A. McGrail" : +1 to update in advance just make sure not to announce the release until 2020. On 12/27/2019 10:32 AM, Lei Rui wrote: Hi, I am the release manager of the upcoming 0.9.1 and I encounter an interesting question: As you can see, Year 2020 is just around the corner and the upcoming release vote for 0.9.1 is likely to span from 2019 to 2020. So I don't know if I should change the year in advance to 2020, or just keep it as 2019, or maybe I postpone the release vote until it is 2020? P.s., the year 2019 is now displayed in the NOTICE file like this: Apache IoTDB Copyright 2018-2019 The Apache Software Foundation. .... Regards, Lei Rui -- Kevin A. McGrail kmcgr...@apache.org Member, Apache Software Foundation Chair Emeritus Apache SpamAssassin Project https://www.linkedin.com/in/kmcgrail - 703.798.0171 -- Best, Jincheng - Twitter: https://twitter.com/sunjincheng121 -
Re: [DISCUSS] Should I change the year to 2020 in advance when releasing 0.9.1?
I got it :D Happy New Year!! Lei Rui On 1/2/2020 10:16，Kevin A. McGrail wrote： Justin is correct. I was trying to be silly with the Happy New Year note :-) Happy New year and my apologies for the sarcasm via email. Regards, KAM On 1/1/2020 8:55 PM, Lei Rui wrote: Hi KAM, I am a little confused. Do you mean the NOTICE can't be changed to "Apache IoTDB Copyright 2018-2020" until it is 2021? Regards, Lei Rui On 1/2/2020 03:02，Kevin A. McGrail wrote： I am going to vote -1 and insist we change back to 2019. Happy New Year, KAM On Mon, Dec 30, 2019, 20:58 Lei Rui wrote: Thank you all for the suggestions! I have changed the year to 2020 in the NOTICE file. Hopefully, the release vote of 0.9.1 will be initiated today. Regards, Lei Rui On 12/28/2019 11:11，Xiangdong Huang wrote： +1. Look forward to the new release version. --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 jincheng sun 于2019年12月28日周六 上午7:30写道： I think 2020 would be fine ：） Christofer Dutz 于2019年12月28日 周六00:45写道： +1 From me too Chris Am 27.12.19, 17:40 schrieb "Kevin A. McGrail" : +1 to update in advance just make sure not to announce the release until 2020. On 12/27/2019 10:32 AM, Lei Rui wrote: Hi, I am the release manager of the upcoming 0.9.1 and I encounter an interesting question: As you can see, Year 2020 is just around the corner and the upcoming release vote for 0.9.1 is likely to span from 2019 to 2020. So I don't know if I should change the year in advance to 2020, or just keep it as 2019, or maybe I postpone the release vote until it is 2020? P.s., the year 2019 is now displayed in the NOTICE file like this: Apache IoTDB Copyright 2018-2019 The Apache Software Foundation. .... Regards, Lei Rui -- Kevin A. McGrail kmcgr...@apache.org Member, Apache Software Foundation Chair Emeritus Apache SpamAssassin Project https://www.linkedin.com/in/kmcgrail - 703.798.0171 -- Best, Jincheng - Twitter: https://twitter.com/sunjincheng121 - -- Kevin A. McGrail kmcgr...@apache.org Member, Apache Software Foundation Chair Emeritus Apache SpamAssassin Project https://www.linkedin.com/in/kmcgrail - 703.798.0171
Re: Add a doc (in Chinese) for How to check a release
Hi,
> me too, never upload or download public key successfully
I would suggest trying this command, which works for me:
```
gpg --keyserver p80.pool.sks-keyservers.net --recv-keys 28662AC6
```
I think there is a more convenient way for users to import the managers' keys,
as described in the beginning of [1]:
> Users: gpg --import KEYS
If you use the release tool [2] borrowed from the PLC4X Project, you will find
the similar command in line 139:
> gpg --import ${BUILDTOOLS_DIR}/${DST_BASE_DIR}/KEYS
Regards,
Lei Rui
[1] https://dist.apache.org/repos/dist/dev/incubator/iotdb/KEYS
[2]
https://github.com/apache/incubator-iotdb/blob/master/tools/download_staged_release.sh
On 1/2/2020 10:33，Xiangdong Huang wrote：
OK... It works for me..
By the way, I think Justin's experience can also be indexed [1].
[1]
https://lists.apache.org/thread.html/0136167d80e60fdfb67d871bf30119c99b22340c6076fd5843d46d18%40%3Cdev.iotdb.apache.org%3E
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Boris Zhu 于2020年1月2日周四 上午10:31写道：
me too, never upload or download public key successfully
On Thu, Jan 2, 2020 at 10:29 AM Jialin Qiao wrote:
Hi,
This is because I never import the keys successfully using "gpg2
--recv-keys ", it just got stuck. I can add this option if it works
for
someone.
Xiangdong Huang 于2020年1月2日周四 上午10:25写道：
Hi Jialin,
Good job. But "how to download a new RM's PGP key" can be improved.
If you remind my document [1], you will find that using `gpg2
--recv-keys
KEY_ID` can get a new public key directly.
e.g., Lei Rui's public key is 28662AC6, so just run `gpg2 --recv-keys
28662AC6`. The id can be get from IoTDB's KEYS file.
By the way, @Lei Rui, as a new RM, you'd better show where to get the
new
KEYS file in your vote mail, i.e., [2].
[1]
https://cwiki.apache.org/confluence/display/IOTDB/How+to+certify+someone%27s+GPG+key+in+Chinese
[2] https://dist.apache.org/repos/dist/dev/incubator/iotdb/KEYS
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Jialin Qiao 于2020年1月2日周四 上午10:21写道：
Hi,
I've added a "How to Check a Release and vote" documents in Chinese
to
help
others vote for the release.
The PR is: https://github.com/apache/incubator-iotdb/pull/702
Happy new year!
--
—
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
--
—
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
Re: ????????iotdb??????????
Hi Robin, > If so, I would be glad to invite you to join the DingTalk [1]. I would also be happy if you enjoy email discussions with the bigger developer community, which is "defined as individuals actively participating in development or following the dev list". Regards, Lei Rui [1] https://qr.dingtalk.com/action/joingroup?code=v1,k1,SeAtRLHwkzHKkoqR+6RuBc8rJM8+QThTlNKgV0ylqfM=&_dt_no_comment=1=11 ??2019??12??24?? 16:52??Robin ?? iotdb?? 2000 ??. ??sql?? select count(preMeter) from root.alap.a510114.s5101141023.p5101141023100310.g2 GROUP BY(1d, [157475377, 157492657]); ??count??count1??sql ?? COUNT TIMESERIES root ?? SHOW DEVICE Fill??fill DELETE STORAGE GROUP root.ln.wf01.wt01 ??
Re: ????????iotdb??????????
Hi, ??bug[1]?? bug[1]0.8.10.9.0??0.8.2??bug ??IoTDB?? The problem you described looks like a bug [1] I encountered before. The bug [1] was found in version 0.8.1. Version 0.8.2 and 0.9.0 don't have this bug. Which version of IoTDB do you use? Regards, Lei Rui [1] https://issues.apache.org/jira/browse/IOTDB-317 ??2019??12??24?? 16:52??Robin ?? iotdb?? 2000 ??. ??sql?? select count(preMeter) from root.alap.a510114.s5101141023.p5101141023100310.g2 GROUP BY(1d, [157475377, 157492657]); ??count??count1??sql ?? COUNT TIMESERIES root ?? SHOW DEVICE Fill??fill DELETE STORAGE GROUP root.ln.wf01.wt01 ??
[DISCUSS] RELEASE_NOTES for the upcoming Apache IoTDB (incubating) 0.9.1
Sounds good. Thanks for your advice. The RELEASE_NOTES is now updated as follows: --- # Apache IoTDB (incubating) 0.9.1 ## Bug Fixes - IOTDB-317 Fix a bug that "flush + wrong aggregation query" causes the following queries to fail - IOTDB-324 Fix inaccurate statistics when writing in batch - IOTDB-327 Fix a groupBy-without-value-filter query bug caused by the wrong page skipping logic - IOTDB-331 Fix a groupBy query bug when axisOrigin-startTimeOfWindow is an integral multiple of interval - IOTDB-359 Fix a wrong-data-type bug in TsFileSketchTool - IOTDB-360 Fix bug of a deadlock in CompressionRatio - IOTDB-363 Fix link errors in Development-Contributing.md and add Development-Document.md - IOTDB-392 Fix a bug in CSV export tool - Fix apache rat header format error in some files ## Miscellaneous changes - IOTDB-321 Add definitions of time expression and LEVEL in related documents - Support pypi distribution for Python client --- Regards, Lei Rui On 12/25/2019 17:05，Xiangdong Huang wrote： Hi, How about put "- Support pypi distribution for Python Client" into "Miscellaneous changes"? As python client is supported in 0.9.0 (but users have to compile the source to get the library), PyPi is just for letting users obtain the library easier. --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Lei Rui 于2019年12月25日周三 下午4:23写道： Hi all, I am the release manager of the upcoming Apache IoTDB (incubating) 0.9.1. I have just finished a draft of RELEASE_NOTES.md and I want to invite you to help me check that. Here is the content: --- # Apache IoTDB (incubating) 0.9.1 ## New Features - Support pypi distribution for Python Client ## Bug Fixes - IOTDB-317 Fix a bug that "flush + wrong aggregation query" causes the following queries to fail - IOTDB-324 Fix inaccurate statistics when write batch - IOTDB-327 Fix a groupBy-without-value-filter query bug caused by the wrong page skipping logic - IOTDB-331 Fix a groupBy query bug when axisOrigin-startTimeOfWindow is an integral multiple of interval - IOTDB-359 Fix a wrong-data-type bug in TsFileSketchTool - IOTDB-360 Fix bug of a deadlock in CompressionRatio - IOTDB-363 Fix link errors in Development-Contributing.md and add Development-Document.md - IOTDB-392 Fix a bug in CSV export tool - Fix apache rat header format error in some files ## Miscellaneous changes - IOTDB-321 Add definitions of time expression and LEVEL in related documents ----------- Regards, Lei Rui
[DISCUSS] RELEASE_NOTES for the upcoming Apache IoTDB (incubating) 0.9.1
Hi all, I am the release manager of the upcoming Apache IoTDB (incubating) 0.9.1. I have just finished a draft of RELEASE_NOTES.md and I want to invite you to help me check that. Here is the content: --- # Apache IoTDB (incubating) 0.9.1 ## New Features - Support pypi distribution for Python Client ## Bug Fixes - IOTDB-317 Fix a bug that "flush + wrong aggregation query" causes the following queries to fail - IOTDB-324 Fix inaccurate statistics when write batch - IOTDB-327 Fix a groupBy-without-value-filter query bug caused by the wrong page skipping logic - IOTDB-331 Fix a groupBy query bug when axisOrigin-startTimeOfWindow is an integral multiple of interval - IOTDB-359 Fix a wrong-data-type bug in TsFileSketchTool - IOTDB-360 Fix bug of a deadlock in CompressionRatio - IOTDB-363 Fix link errors in Development-Contributing.md and add Development-Document.md - IOTDB-392 Fix a bug in CSV export tool - Fix apache rat header format error in some files ## Miscellaneous changes - IOTDB-321 Add definitions of time expression and LEVEL in related documents --- Regards, Lei Rui
[DISCUSS] Should I change the year to 2020 in advance when releasing 0.9.1?
Hi, I am the release manager of the upcoming 0.9.1 and I encounter an interesting question: As you can see, Year 2020 is just around the corner and the upcoming release vote for 0.9.1 is likely to span from 2019 to 2020. So I don't know if I should change the year in advance to 2020, or just keep it as 2019, or maybe I postpone the release vote until it is 2020? P.s., the year 2019 is now displayed in the NOTICE file like this: > Apache IoTDB > Copyright 2018-2019 The Apache Software Foundation. > Regards, Lei Rui
Reproduce NullPointerException in SeqTsFileRecoverTest and UnseqTsFileRecoverTest (IOTDB-159)
Hi, I encounter this problem today. Luckily I find a way to reproduce this problem. I can reproduce this problem on both linux jdk1.8 and windows jdk1.8, with master branch. The key to reproducing, is to test UnseqTsFileRecoverTest (or SeqTsFileRecoverTest) AFTER first testing an integration test under the test/java/org/apache/iotdb/db/integration directory, such as IoTDBAuthorizationIT. The complete commands, to reproduce this problem, are: ``` git clone https://github.com/apache/incubator-iotdb.git cd incubator-iotdb mvn clean install -Dmaven.test.skip=true cd server mvn test -Dtest=org.apache.iotdb.db.integration.IoTDBAuthorizationIT,org.apache.iotdb.db.writelog.recover.UnseqTsFileRecoverTest ``` The error information on linux jdk1.8 and windows jdk1.8 are copied to the end of this email. It seems that something goes wrong with the common framework of these integration tests, which can lead to the NullPointerException in the following UnseqTsFileRecoverTest (or SeqTsFileRecoverTest). But I haven't found the exact cause. I might need some help. The Error information on linux jdk1.8: - [INFO] Results: [INFO] [ERROR] Errors: [ERROR] UnseqTsFileRecoverTest.test:148 ? NullPointer [INFO] [ERROR] Tests run: 14, Failures: 0, Errors: 1, Skipped: 1 [INFO] [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 46.564 s [INFO] Finished at: 2019-12-28T23:33:48+08:00 [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.0:test (default-test) on project iotdb-server: There are test failures. [ERROR] [ERROR] Please refer to /home/ruilei/master/incubator-iotdb/server/target/surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException - The Error information on windows jdk1.8: - [INFO] Results: [INFO] [ERROR] Errors: [ERROR] org.apache.iotdb.db.writelog.recover.UnseqTsFileRecoverTest.test(org.apache.iotdb.db.writelog.recover.UnseqTsFileRecoverTest) [ERROR] Run 1: UnseqTsFileRecoverTest.test:147 ? NullPointer [ERROR] Run 2: UnseqTsFileRecoverTest.tearDown:139 ? IO Unable to delete file: temp\test.ts [INFO] [INFO] [ERROR] Tests run: 14, Failures: 0, Errors: 1, Skipped: 1 [INFO] [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 03:44 min [INFO] Finished at: 2019-12-28T23:07:32+08:00 [INFO] Final Memory: 43M/445M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.0:test (default-test) on project iotdb-server: There are test failures. [ERROR] [ERROR] Please refer to D:\yeah\incubator-iotdb\server\target\surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException - Regards, Lei Rui On 10/8/2019 12:41，xiangdong Huang (Jira) wrote： [ https://issues.apache.org/jira/browse/IOTDB-159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiangdong Huang closed IOT
Re: [VOTE] Apache IoTDB 0.11.0 RC3 release
+1
I think the doc issues can be solved next time.
1. There is a pair of Chinese single quotes used in English version "README.md":
"Insert timeseries data is a basic operation of IoTDB, you can use ‘INSERT’
command to finish this."
SUGGESTION: Use `insert` by the markdown grammar, or use English single or
double quotes.
2. The "incubat*" string occurs twice in README_ZH.md:
(1) "* 使用Docker:
dockerfile的路径是https://github.com/apache/incubat-iotdb/tree/master/docker/src/main;
(2)
"执行完成之后，可以在**distribution/target/apache-iotdb-{project.version}-incubating-bin.zip**找到编译完成的二进制版本(包括服务器和客户端)"
Actually, I use the command ( grep -r "incubat" ./ ) to find a lot more files
containing "incubat*".
I think most of them are correct, to give a few examples:
Example 1: [./RELEASE_NOTES.md] "# Apache IoTDB (incubating) 0.10.1"
Example 2: [./docs/Community/Materials.md] "2018.12.09, .., IoTDB: from
Tsinghua IoTDB to Apache IoTDB (incubating)."
Example 3: [./docker/src/main/Dockerfile-0.9.0] "wget
https://www-us.apache.org/dist/iotdb/0.9.0-incubating/apache-iotdb-0.9.0-incubating-bin.zip
\"
Some of them may need updating carefully, for example:
[./client-py/pypi/README.md] "Apache IoTDB (incubating) (Database for Internet
of Things) is an integrated ..."
--
What I checked:
✔ download staged artifacts. Check their signature and hashes.
✔ [RM] verify the reported signature is for an "apache.org" address ("gpg: Good
signature from ...")
✔ [RM] verify the reported hashes
✔ extract src bundle
✔ verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES files in the
extracted source bundle
✔ [RM] verify the staged source README, RELEASE_NOTE files correspond to those
in the extracted source bundle
❗ check the contents of LICENSE, NOTICE, README, RELEASE_NOTES
✔ UNPACK the binary artefact and check the contents of LICENSE-binary and
NOTICE-binary
✔ build from directions in README with an empty maven local repo: mvn clean
package -DskipTests -Dmaven.repo.local=../.m2
✔ ./mvnw.sh install
✔ ./mvnw.sh site:site
Regards,
Lei Rui
On 11/20/2020 14:23，Xiangdong Huang wrote：
Hi all,
This is a long-lost vote for RELEASE a new major version of IoTDB,
especially after IoTDB graduated from the incubator.
(RC1 and RC2 failed because of some critical bugs and receives -1 votes.
RC3 fixed them.)
Apache IoTDB (incubating) 0.11.0 will be the first major release version
after IoTDB graduated. This version improves the memory control for writing
data and improves the query performance.
You can get its main changes from [5].
Apache IoTDB (Incubating) 0.11.0 has been staged under [2] and it’s time to
vote
on accepting it for release. All Maven artifacts are available under [1].
Release tag: release/0.11.0
Hash for the release tag: 3dafc977766e750d868d64fccfe6a5df8f1f0e1e
Voting will be open for 72hr.
A minimum of 3 binding +1 votes and more binding +1 than binding -1
are required to pass.
[ ] +1 accept (indicate what you validated - e.g. performed the non-RM
items in [4])
[ ] -1 reject (explanation required)
Before voting +1, PMC members are required to download
the signed source code package, compile it as provided, and test
the resulting executable on their own platform, along with also
verifying that the package meets the requirements of the ASF policy
on releases.
You can achieve the above by following [4].
NOTICE: this is the first version that we remove "incubating", so there may
be some issues for the release, please pay attention and let make our
release better and better.
[1] https://repository.apache.org/content/repositories/orgapacheiotdb-1047
[2] https://dist.apache.org/repos/dist/dev/iotdb/0.11.0/rc3
[3] https://www.apache.org/dev/release.html#approving-a-release
[4]
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[5] https://dist.apache.org/repos/dist/dev/iotdb/0.11.0/rc3/RELEASE_NOTES.md
[6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Re: [VOTE] Apache IoTDB 0.11.2 RC3 release
Hi,
I found one major issue about SNAPSHOT references (❌) and some minor issues
with documents (❗).
If the SNAPSHOT references are strictly prohibited, then my vote is -1.
Otherwise, we can fix these issues in the next release.
✔ download staged artifacts. [RM] Check their signature and hashes.
✔ extract SRC bundle, verify the existence of LICENSE, NOTICE, README,
RELEASE_NOTES files in the extracted source bundle
✔ [RM] verify the staged source README, RELEASE_NOTE files correspond to those
in the extracted source bundle
❗ check the contents of LICENSE, NOTICE, README, RELEASE_NOTES
a) The NOTICE file says "Copyright 2018-2020 The Apache Software
Foundation".
Maybe it is time we change it to 2018-2021 in the next release.
b) There are still "incubat" keywords in README_ZH.md (i.e., the Chinese
version of README).
❗ UNPACK the binary artifact and check the contents of LICENSE-binary and
NOTICE-binary
The NOTICE file in the binary artifact has the same year problem as that in
the extracted source bundle.
✔ build from directions in README with an empty maven local repo: mvn clean
package -DskipTests -Dmaven.repo.local=../.m2
❌ Search for SNAPSHOT references: find . -type f -name 'pom.xml' -exec grep -l
"SNAPSHOT" {} \;
"0.11.1-SNAPSHOT" is used in the following five pom files:
(1) ./client-cpp/client-cpp-example/pom.xml
(2) ./client-cpp/pom.xml
(3) ./compile-tools/boost/pom.xml
(4) ./compile-tools/thrift/pom.xml
(5) ./compile-tools/pom.xml
✔ ./mvnw.sh install
✔ ./mvnw.sh site:site
Regards,
Lei Rui
On 1/9/2021 20:22，Xiangdong Huang wrote：
Hi all,
0.11.2 RC3 is ready, it fixes the known issues of RC1 and RC2.
Apache IoTDB 0.11.2 is a bug-fix version from 0.11.1.
You can get its main changes from [5].
Apache IoTDB 0.11.2 has been staged under [2] and it’s time to vote
on accepting it for release. All Maven artifacts are available under [1].
Voting will be open for 72hr.
A minimum of 3 binding +1 votes and more binding +1 than binding -1
are required to pass.
Release tag: release/0.11.2
Hash for the release tag: a5728115a7408268cd414130c303060c30608ea3
Before voting +1, PMC members are required to download
the signed source code package, compile it as provided, and test
the resulting executable on their own platform, along with also
verifying that the package meets the requirements of the ASF policy
on releases.[3]
You can achieve the above by following [4].
[ ] +1 accept (indicate what you validated)
[ ] -1 reject (explanation required)
Bugfix list:
* IOTDB-1049 Fix Nullpointer exception and a delete bug in Last query
* IOTDB-1060 Support full deletion for delete statement without where clause
* IOTDB-1068 Fix Time series metadata cache bug
* IOTDB-1069 restrict the flushing memtable number to avoid OOM when
mem_control is disabled
* IOTDB-1077 add insertOneDeviceRecords API in java session
* IOTDB-1087 fix compaction block flush: flush do not return until
compaction finished
* IOTDB-1106 Delete timeseries statement will incorrectly delete other
timeseries
* Github issue-2137 fix grafana value-time position bug
* Github issue-2169 GetObject returns String for all data types
* Github issue-2240 fix Sync failed: Socket is closed by peer
* Github issue-2387 The deleteData method exists in Session but not in
SessionPool.
* add thrift_max_frame_size in iotdb-engine.properties
* Fix incorrect last result after deleting all data
* Fix compaction recover block restart: IoTDB cannot restart until last
compaction recover task finished
* Fix compaction ignore modification file: delete does not work after
compaction
* print more insert error message in client
* expose enablePartition parameter into iotdb-engines.properpties
[1] https://repository.apache.org/content/repositories/orgapacheiotdb-1053
[2] https://dist.apache.org/repos/dist/dev/iotdb/0.11.2/rc3
[3] https://www.apache.org/dev/release.html#approving-a-release
[4]
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[5] https://dist.apache.org/repos/dist/dev/iotdb/0.11.2/rc3/RELEASE_NOTES.md
[6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Re: [VOTE] Apache IoTDB 0.11.2 RC4 release
+1 (binding)
What I checked:
✔ download staged artifacts. [RM] Check their signature and hashes.
✔ extract SRC bundle, verify the existence of LICENSE, NOTICE, README,
RELEASE_NOTES files in the extracted source bundle
✔ [RM] verify the staged source README, RELEASE_NOTE files correspond to those
in the extracted source bundle
✔ check the contents zzof LICENSE, NOTICE, README, RELEASE_NOTES
✔ UNPACK the binary artifact and check the contents of LICENSE-binary and
NOTICE-binary
✔ build from directions in README with an empty maven local repo: mvn clean
package -DskipTests -Dmaven.repo.local=../.m2
✔ Search for SNAPSHOT references: find . -type f -name 'pom.xml' -exec grep -l
"SNAPSHOT" {} \;
> (except for the issues in client-cpp and compile-tools, i.e., you will find
> "SNAPSHOT" word in these two folders. It will be fixed in 0.12).
✔ ./mvnw.sh install
✔ ./mvnw.sh site:site
Regards,
Lei Rui
On 1/16/2021 19:40，Xiangdong Huang wrote：
Hi all,
0.11.2 RC4 is ready, it fixes the known issues of RC1, RC2 and RC3 (except
for the issues in client-cpp and compile-tools, i.e., you will find
"SNAPSHOT" word in these two folders. It will be fixed in 0.12).
Apache IoTDB 0.11.2 is a bug-fix version from 0.11.1.
You can get its main changes from [5].
Apache IoTDB 0.11.2 has been staged under [2] and it’s time to vote
on accepting it for release. All Maven artifacts are available under [1].
Voting will be open for 72hr.
A minimum of 3 binding +1 votes and more binding +1 than binding -1
are required to pass.
Release tag: release/0.11.2
Hash for the release tag: 84364082b6565a459bb0f06218ef6046e10593c8
Before voting +1, PMC members are required to download
the signed source code package, compile it as provided, and test
the resulting executable on their own platform, along with also
verifying that the package meets the requirements of the ASF policy
on releases.[3]
You can achieve the above by following [4].
[ ] +1 accept (indicate what you validated)
[ ] -1 reject (explanation required)
Bugfix list:
* IOTDB-1049 Fix Nullpointer exception and a delete bug in Last query
* IOTDB-1060 Support full deletion for delete statement without where clause
* IOTDB-1068 Fix Time series metadata cache bug
* IOTDB-1069 restrict the flushing memtable number to avoid OOM when
mem_control is disabled
* IOTDB-1077 add insertOneDeviceRecords API in java session
* IOTDB-1087 fix compaction block flush: flush do not return until
compaction finished
* IOTDB-1106 Delete timeseries statement will incorrectly delete other
timeseries
* Github issue-2137 fix grafana value-time position bug
* Github issue-2169 GetObject returns String for all data types
* Github issue-2240 fix Sync failed: Socket is closed by peer
* Github issue-2387 The deleteData method exists in Session but not in
SessionPool.
* add thrift_max_frame_size in iotdb-engine.properties
* Fix incorrect last result after deleting all data
* Fix compaction recover block restart: IoTDB cannot restart until last
compaction recover task finished
* Fix compaction ignore modification file: delete does not work after
compaction
* print more insert error message in client
* expose enablePartition parameter into iotdb-engines.properpties
[1] https://repository.apache.org/content/repositories/orgapacheiotdb-1054
[2] https://dist.apache.org/repos/dist/dev/iotdb/0.11.2/rc4
[3] https://www.apache.org/dev/release.html#approving-a-release
[4]
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[5] https://dist.apache.org/repos/dist/dev/iotdb/0.11.2/rc4/RELEASE_NOTES.md
[6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Re: [VOTE] Apache IoTDB 0.11.2 RC4 release
Hi, The directory [1] is empty. Is it because of network delays? Regards, Lei Rui [1] https://dist.apache.org/repos/dist/dev/iotdb/0.11.2/rc4/ On 1/16/2021 19:40，Xiangdong Huang wrote： Hi all, 0.11.2 RC4 is ready, it fixes the known issues of RC1, RC2 and RC3 (except for the issues in client-cpp and compile-tools, i.e., you will find "SNAPSHOT" word in these two folders. It will be fixed in 0.12). Apache IoTDB 0.11.2 is a bug-fix version from 0.11.1. You can get its main changes from [5]. Apache IoTDB 0.11.2 has been staged under [2] and it’s time to vote on accepting it for release. All Maven artifacts are available under [1]. Voting will be open for 72hr. A minimum of 3 binding +1 votes and more binding +1 than binding -1 are required to pass. Release tag: release/0.11.2 Hash for the release tag: 84364082b6565a459bb0f06218ef6046e10593c8 Before voting +1, PMC members are required to download the signed source code package, compile it as provided, and test the resulting executable on their own platform, along with also verifying that the package meets the requirements of the ASF policy on releases.[3] You can achieve the above by following [4]. [ ] +1 accept (indicate what you validated) [ ] -1 reject (explanation required) Bugfix list: * IOTDB-1049 Fix Nullpointer exception and a delete bug in Last query * IOTDB-1060 Support full deletion for delete statement without where clause * IOTDB-1068 Fix Time series metadata cache bug * IOTDB-1069 restrict the flushing memtable number to avoid OOM when mem_control is disabled * IOTDB-1077 add insertOneDeviceRecords API in java session * IOTDB-1087 fix compaction block flush: flush do not return until compaction finished * IOTDB-1106 Delete timeseries statement will incorrectly delete other timeseries * Github issue-2137 fix grafana value-time position bug * Github issue-2169 GetObject returns String for all data types * Github issue-2240 fix Sync failed: Socket is closed by peer * Github issue-2387 The deleteData method exists in Session but not in SessionPool. * add thrift_max_frame_size in iotdb-engine.properties * Fix incorrect last result after deleting all data * Fix compaction recover block restart: IoTDB cannot restart until last compaction recover task finished * Fix compaction ignore modification file: delete does not work after compaction * print more insert error message in client * expose enablePartition parameter into iotdb-engines.properpties [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1054 [2] https://dist.apache.org/repos/dist/dev/iotdb/0.11.2/rc4 [3] https://www.apache.org/dev/release.html#approving-a-release [4] https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release [5] https://dist.apache.org/repos/dist/dev/iotdb/0.11.2/rc4/RELEASE_NOTES.md [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院
Re: [VOTE] Apache IoTDB 0.11.4 RC1 release
Besides, I found some old versions such as "0.11.1" in the "docs\apache-iotdb-0.11.4\docs\SystemDesign\TsFile\Format.md" and "zh\docs\apache-iotdb-0.11.4\docs\SystemDesign\TsFile\Format.md". On 6/11/2021 09:48，Haonan Hou wrote： Hi I vote -1. I find there are some bug-fixing PRs[1][2] merged after the RC1 released. Therefore, I think we need to conciser 0.11.4 RC2. [1] https://github.com/apache/iotdb/pull/3384 [2] https://github.com/apache/iotdb/pull/3371 Best, Haonan Hou On Jun 6, 2021, at 2:53 PM, Xiangdong Huang wrote: Hi all, 0.11.4 RC1 is ready, it fixes many bugs in 0.11.3. Apache IoTDB 0.11.4 is a bug-fix version from 0.11.3. You can get its main changes from [5]. Apache IoTDB 0.11.4 has been staged under [2] and it’s time to vote on accepting it for release. All Maven artifacts are available under [1]. Known issue: in client-cpp and compile-tools, you will find "SNAPSHOT" word in these two folders. It will be fixed in 0.12. Voting will be open for 72hr. A minimum of 3 binding +1 votes and more binding +1 than binding -1 are required to pass. Release tag: v0.11.4 Hash for the release tag: 695b9ade459237481b490659cfc90fa99c824002 Before voting +1, PMC members are required to download the signed source code package, compile it as provided, and test the resulting executable on their own platform, along with also verifying that the package meets the requirements of the ASF policy on releases.[3] You can achieve the above by following [4]. [ ] +1 accept (indicate what you validated) [ ] -1 reject (explanation required) [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1061 [2] https://dist.apache.org/repos/dist/dev/iotdb/0.11.4/rc1 [3] https://www.apache.org/dev/release.html#approving-a-release [4] https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release [5] https://dist.apache.org/repos/dist/dev/iotdb/0.11.4/rc1/RELEASE_NOTES.md [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院
Re: AW: Do we need a new Query Engine?
Hi, This discussion reminds me of an example that might bring some inspiration [1][2]: M. Symeonides, D. T., Z. Georgiou, G. Pallis and M. Dikaiakos (2019). "Query-Driven Descriptive Analytics for IoT and Edge Computing." IEEE International Conference on Cloud Engineering (IC2E): 1-11. [1] https://ieeexplore.ieee.org/stamp/stamp.jsp?tp==8790179 [2] https://github.com/UCY-LINC-LAB/edge-computing-query-model Regards, Lei Rui On 2/25/2021 00:16，Julian Feinauer wrote： Hi, sorry to be so unspecific. Prometheus has quite a nice Documentation as reference [1]. Thinks I like there are for example: Math on Timeseries: * timeseriesA +(, -, *, /) timeseriesB as a new series Reuse Aggregations as new timeseries: Avg(timeseriesA)/avg(timeseriesB) in a group by context Best Julian [1] https://prometheus.io/docs/prometheus/latest/querying/basics/ Von: Xiangdong Huang Datum: Dienstag, 23. Februar 2021 um 09:51 An: dev Betreff: Re: Do we need a new Query Engine? Hi, Welcome to more discussion about this topic. I really love there is the power of their query language. Lots of functions and the possibility to do math on timeseries Sharing some examples may be the first step to open our mind :D Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Julian Feinauer 于2021年2月23日周二 上午4:36写道： Hi folks, the title is a bit drastic but what I wanted to bring up as a discussion is what we plan to allow in our Query Language. I worked a bit with Prometheus recently and what I really love there is the power of their query language. Lots of functions and the possibility to do math on timeseries. So not only scalar and timeseries but also timeseries and timeseries. When I though about how to integrate that seamlessly into IoTDB I questioned myself if the current state of the “Query Engine” or “Execution Engine” is setup right for that or if there would be the need to “rewrite” it. My gut feeling is that it started off well and just grew over and over with many more features and branches for all the different options. What are other peoples feelings? I really would love to have some more powerful math! Julian
Re: [VOTE] Apache IoTDB 0.12.2 RC3 release
Hi,
+1 (binding).
I checked:
- download staged artifacts. [RM] Check their signature and hashes.
- extract SRC bundle, verify the existence of LICENSE, NOTICE, README,
RELEASE_NOTES files in the extracted source bundle.
- verify the staged source README, RELEASE_NOTE files correspond to those in
the extracted source bundle
- UNPACK the binary artifact and check the contents of LICENSE-binary and
NOTICE-binary
- Search for SNAPSHOT references: find . -type f -name 'pom.xml' -exec grep -l
"SNAPSHOT" {} \;
Regards,
Lei Rui
On 9/8/2021 07:48，Xiangdong Huang wrote：
Hi,
I am here again...
Apache IoTDB 0.12.2 RC3 is a bug-fix version from 0.12.1. It fixes 49 bugs
in v0.12.1, and brings a more stable compaction module.
You can get its main changes from [5].
Apache IoTDB 0.12.2 has been staged under [2] and it’s time to vote
on accepting it for release. All Maven artifacts are available under [1].
Voting will be open for 72hr.
A minimum of 3 binding +1 votes and more binding +1 than binding -1
are required to pass.
Release tag: v0.12.2
Hash for the release tag: 55303223ec31faea003efc4ae3c41625ec51b341
Before voting +1, PMC members are required to download
the signed source code package, compile it as provided, and test
the resulting executable on their own platform, along with also
verifying that the package meets the requirements of the ASF policy
on releases. [3]
You can achieve the above by following [4].
[ ] +1 accept (indicate what you validated - e.g. performed the
non-RM items in [4])
[ ] -1 reject (explanation required)
[1] https://repository.apache.org/content/repositories/orgapacheiotdb-1067
[2] https://dist.apache.org/repos/dist/dev/iotdb/0.12.2/rc3
[3] https://www.apache.org/dev/release.html#approving-a-release
[4]
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[5] https://dist.apache.org/repos/dist/dev/iotdb/0.12.2/rc3/RELEASE_NOTES.md
[6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Re: [VOTE] Apache IoTDB 0.12.5 RC1 release
+1 I checked the following steps with a cloud computer running Ubuntu 18.04 x64, Java openjdk version "11.0.13" 2021-10-19, Apache Maven 3.6.0: 1. verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES files in the extracted source bundle (Next time update NOTICE Year 2021 to 2022) [ok] 2. verify the staged source README, RELEASE_NOTE files correspond to those in the extracted source bundle [ok] 3. mvn clean package -DskipTests [ok] 4. ./mvnw.sh install [not ok] [ERROR] Errors: [ERROR] SameMeasurementsWithDifferentDataTypesTest.before:61->writeFile:121 » NullPointer But this test passes on my local computer with Windows 10, java version "1.8.0_161", Apache Maven 3.8.4. 5. ./mvnw.sh site:site [not ok] [ERROR] Failed to execute goal on project iotdb-thrift-cluster: Could not resolve dependencies for project org.apache.iotdb:iotdb-thrift-cluster:jar:0.12.5: Could not find artifact org.apache.iotdb:iotdb-thrift:jar:0.12.5 in central (https://repo.maven.apache.org/maven2) -> [Help 1] 6. SET STORAGE GROUP TO root.ln SHOW STORAGE GROUP SHOW TIMESERIES SHOW TIMESERIES root.ln.wf01.wt01.status INSERT INTO root.ln.wf01.wt01(timestamp,status) values(100,true); INSERT INTO root.ln.wf01.wt01(timestamp,status,temperature) values(200,false,20.71) SELECT status FROM root.ln.wf01.wt01 SELECT * FROM root.ln.wf01.wt01 SET time_zone=+08:00 SELECT * FROM root.ln.wf01.wt01 [ok] On 3/4/2022 20:47，Steve Yurong Su wrote： Hi all, Apache IoTDB 0.12.5 is a bug-fix version from 0.12.4. You can get its main changes from [1]. Apache IoTDB 0.12.5 has been staged under [2] and it’s time to vote on accepting it for release. All Maven artifacts are available under [3]. Release tag: v0.12.5 Hash for the release tag: 03f276e086c5c6ef2d9d3de9e01c2bada5fa0b6e Voting will be open for 72hr. A minimum of 3 binding +1 votes and more binding +1 than binding -1 are required to pass. Before voting +1, PMC members are required to download the signed source code package, compile it as provided, and test the resulting executable on their own platform, along with also verifying that the package meets the requirements of the ASF policy on releases[4]. You can achieve the above by following [5]. [ ] +1 accept (indicate what you validated - e.g. performed the non-RM items in [5]) [ ] -1 reject (explanation required) -- [1] https://dist.apache.org/repos/dist/dev/iotdb/0.12.5/rc1/RELEASE_NOTES.md [2] https://dist.apache.org/repos/dist/dev/iotdb/0.12.5/rc1 [3] https://repository.apache.org/content/repositories/orgapacheiotdb-1075/ [4] https://www.apache.org/dev/release.html#approving-a-release [5] https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS -- Thanks, Steve Yurong Su
[jira] [Created] (IOTDB-102) CLONE - IoTDB failed to start client since the required jars are not in the correct lib directory
Lei Rui created IOTDB-102:
-
Summary: CLONE - IoTDB failed to start client since the required
jars are not in the correct lib directory
Key: IOTDB-102
URL: https://issues.apache.org/jira/browse/IOTDB-102
Project: Apache IoTDB
Issue Type: Bug
Reporter: Lei Rui
*Operating system version*: Windows 10 x64
*Reproduction of the problem*
When following the "Quick Start" instruction on IoTDB website
([https://iotdb.apache.org/#/Documents/Quick%20Start]), it would result in
"Cannot find the main class 'org.apache.iotdb.cli.client.Client' error". There
is the detailed operation below:
# Let $IOTDB_HOME = /workspace/incubator-iotdb/iotdb/iotdb/
Let $IOTDB_CLI_HOME = /workspace/incubator-iotdb/iotdb-cli/cli/
#
{code:java}
> pwd
/workspace/incubator-iotdb
> mvn clean package -pl iotdb -am -Dmaven.test.skip=true{code}
#
{code:java}
> $IOTDB_HOME\bin\start-server.bat{code}
#
{code:java}
> pwd
/workspace/incubator-iotdb
> mvn clean package -pl iotdb-cli -am -Dmaven.test.skip=true
> $IOTDB_CLI_HOME\bin\start-client.bat -h -p -u {code}
which would result in:
{code:java}
Cannot find the main class 'org.apache.iotdb.cli.client.Client'{code}
*Screenshots*
Follow the "Quick Start" instruction on the website.
!5.PNG!
!6.PNG!
!7.PNG!
!8.PNG!
!2.PNG!
!1.PNG!
!3.PNG!!4.PNG!
*Reason*
The reason of causing this error is the required jars to start IoTDB client are
not in the correct lib directory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (IOTDB-64) TsFile Spark Connector
[ https://issues.apache.org/jira/browse/IOTDB-64?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Rui closed IOTDB-64. Resolution: Fixed > TsFile Spark Connector > -- > > Key: IOTDB-64 > URL: https://issues.apache.org/jira/browse/IOTDB-64 > Project: Apache IoTDB > Issue Type: New Feature > Reporter: Lei Rui >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Old realization of TsFile-Spark-Connector is not compatible with the latest > TsFile since TsFile has been redesigned. > Besides, old version loads data from TsFile into Spark table in the form of > [time, delta_object, sensor1, sensor 2, ...], which will also be changed in > this update. The updated form is like [time, delta_object1.sensor1, > delta_object1.sensor2, delta_object2.sensor_1, ...] > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IOTDB-92) The data locality principle used by Spark loses ground in the face of TsFile.
Lei Rui created IOTDB-92: Summary: The data locality principle used by Spark loses ground in the face of TsFile. Key: IOTDB-92 URL: https://issues.apache.org/jira/browse/IOTDB-92 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui In the development of TsFile-Spark-Connector, we discover the problem that the data locality principle used by Spark loses ground in the face of TsFile. We believe the problem is rooted in the storage structure design of TsFile. Our latest implementation of TsFile-Spark-Connector finds a way to guarantee the proper functionality despite the constraint. The resolvement of the data locality problem is left for future work. Below are the details. h1. 1. Spark Partition In Apache Spark, the data is stored in the form of RDDs and divided into partitions across various nodes. A partition is a logical chunk of a large distributed data set that helps parallelize distributed data processing. Spark works on data locality principle to minimize the network traffic for sending data between executors. https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-rdd-partitions.html https://techvidvan.com/tutorials/spark-partition/ h1. 2. TsFile Structure TsFile is a columnar storage file format designed for time series data, which supports efficient compression and query. Data in TsFile are organized by device-measurement hierarchy. As the figure below shows, the storage unit of a device is a chunk group and that of a measurement is a chunk. Measurements of a device are grouped together. Under this architecture, different partitions of Spark logically contain different sets of chunkgroups of TsFile. Now consider the query process of this TsFile on Spark. Supposing that we query “select * from root where d1.s6<2.5 and d2.s1>10”, a scheduled task of a partition has to deal with the whole data to get the right answer. However, this also means that it is nearly impossible to apply the data locality principle. h1. 3. Problem Now we can summarize two problems. The first problem is how to guarantee the correctness of the queried answer integrated from all the partition task without changing the current storage structure of TsFile. To solve this problem, we propose a solution by converting the space partition constraint to the time partition constraint while still requiring a single task to have access to the whole TsFile data. As shown in the figure below, the task of partition 1 is assigned the yellow marked time partition constraint; the task of partition 2 is assigned the green marked time partition constraint; the task of partition 3 is assigned empty time partition constraint because the former two tasks have completed the query. The second problem is more fundamental. That is, how we can adjust to enable some extent of data locality of TsFile when it is queried on Spark. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IOTDB-172) TsFileResource updateTime Bug
Lei Rui created IOTDB-172: - Summary: TsFileResource updateTime Bug Key: IOTDB-172 URL: https://issues.apache.org/jira/browse/IOTDB-172 Project: Apache IoTDB Issue Type: Bug Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IOTDB-211) Start command too long on windows with big classpaths in start-server.bat
Lei Rui created IOTDB-211: - Summary: Start command too long on windows with big classpaths in start-server.bat Key: IOTDB-211 URL: https://issues.apache.org/jira/browse/IOTDB-211 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IOTDB-215) Make users aware of auxiliary tools such as print-tsfile-resource-files by fulfilling docs
Lei Rui created IOTDB-215: - Summary: Make users aware of auxiliary tools such as print-tsfile-resource-files by fulfilling docs Key: IOTDB-215 URL: https://issues.apache.org/jira/browse/IOTDB-215 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IOTDB-223) TsFile skeleton viewer
Lei Rui created IOTDB-223: - Summary: TsFile skeleton viewer Key: IOTDB-223 URL: https://issues.apache.org/jira/browse/IOTDB-223 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-212) Improve the IoTDB SQL Documentation
Lei Rui created IOTDB-212: - Summary: Improve the IoTDB SQL Documentation Key: IOTDB-212 URL: https://issues.apache.org/jira/browse/IOTDB-212 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IOTDB-213) query optimization for repeated paths in SELECT clause
Lei Rui created IOTDB-213: - Summary: query optimization for repeated paths in SELECT clause Key: IOTDB-213 URL: https://issues.apache.org/jira/browse/IOTDB-213 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IOTDB-214) TSExecuteStatementResp's ignoreTimeStamp is not set in the TSServiceImpl.executeQueryStatement
Lei Rui created IOTDB-214: - Summary: TSExecuteStatementResp's ignoreTimeStamp is not set in the TSServiceImpl.executeQueryStatement Key: IOTDB-214 URL: https://issues.apache.org/jira/browse/IOTDB-214 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IOTDB-204) spark-tsfile narrow table's new way to execute query
Lei Rui created IOTDB-204: - Summary: spark-tsfile narrow table's new way to execute query Key: IOTDB-204 URL: https://issues.apache.org/jira/browse/IOTDB-204 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IOTDB-229) Inconsistent usage of Marker in TsFile
Lei Rui created IOTDB-229: - Summary: Inconsistent usage of Marker in TsFile Key: IOTDB-229 URL: https://issues.apache.org/jira/browse/IOTDB-229 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-153) LIMIT OFFSET clause is unreasonably slow when N and OffsetValue are rather small.
Lei Rui created IOTDB-153: - Summary: LIMIT OFFSET clause is unreasonably slow when N and OffsetValue are rather small. Key: IOTDB-153 URL: https://issues.apache.org/jira/browse/IOTDB-153 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (IOTDB-150) Try some other web fonts
Lei Rui created IOTDB-150: - Summary: Try some other web fonts Key: IOTDB-150 URL: https://issues.apache.org/jira/browse/IOTDB-150 Project: Apache IoTDB Issue Type: Wish Reporter: Lei Rui -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (IOTDB-244) wrong duplicated columns query result
Lei Rui created IOTDB-244:
-
Summary: wrong duplicated columns query result
Key: IOTDB-244
URL: https://issues.apache.org/jira/browse/IOTDB-244
Project: Apache IoTDB
Issue Type: Bug
Reporter: Lei Rui
I use the following sql to insert data,
{code:java}
SET STORAGE GROUP TO root.vehicle.d0
SET STORAGE GROUP TO root.vehicle.d1
CREATE TIMESERIES root.vehicle.d0.s0 WITH DATATYPE=INT32, ENCODING=RLE
CREATE TIMESERIES root.vehicle.d0.s1 WITH DATATYPE=INT64, ENCODING=RLE
CREATE TIMESERIES root.vehicle.d0.s2 WITH DATATYPE=FLOAT, ENCODING=RLE
CREATE TIMESERIES root.vehicle.d0.s3 WITH DATATYPE=TEXT, ENCODING=PLAIN
CREATE TIMESERIES root.vehicle.d0.s4 WITH DATATYPE=BOOLEAN, ENCODING=PLAIN
CREATE TIMESERIES root.vehicle.d1.s0 WITH DATATYPE=INT32, ENCODING=RLEinsert
into root.vehicle.d0(timestamp,s0) values(1,101)
insert into root.vehicle.d0(timestamp,s0) values(2,198)
insert into root.vehicle.d0(timestamp,s0) values(100,99)
insert into root.vehicle.d0(timestamp,s0) values(101,99)
insert into root.vehicle.d0(timestamp,s0) values(102,80)
insert into root.vehicle.d0(timestamp,s0) values(103,99)
insert into root.vehicle.d0(timestamp,s0) values(104,90)
insert into root.vehicle.d0(timestamp,s0) values(105,99)
insert into root.vehicle.d0(timestamp,s0) values(106,99)
insert into root.vehicle.d0(timestamp,s0) values(2,1)
insert into root.vehicle.d0(timestamp,s0) values(50,1)
insert into root.vehicle.d0(timestamp,s0) values(1000,2)insert into
root.vehicle.d0(timestamp,s1) values(1,1101)
insert into root.vehicle.d0(timestamp,s1) values(2,198)
insert into root.vehicle.d0(timestamp,s1) values(100,199)
insert into root.vehicle.d0(timestamp,s1) values(101,199)
insert into root.vehicle.d0(timestamp,s1) values(102,180)
insert into root.vehicle.d0(timestamp,s1) values(103,199)
insert into root.vehicle.d0(timestamp,s1) values(104,190)
insert into root.vehicle.d0(timestamp,s1) values(105,199)
insert into root.vehicle.d0(timestamp,s1) values(2,4)
insert into root.vehicle.d0(timestamp,s1) values(50,5)
insert into root.vehicle.d0(timestamp,s1) values(1000,5)insert into
root.vehicle.d0(timestamp,s2) values(1000,5)
insert into root.vehicle.d0(timestamp,s2) values(2,2.22)
insert into root.vehicle.d0(timestamp,s2) values(3,3.33)
insert into root.vehicle.d0(timestamp,s2) values(4,4.44)
insert into root.vehicle.d0(timestamp,s2) values(102,10.00)
insert into root.vehicle.d0(timestamp,s2) values(105,11.11)
insert into root.vehicle.d0(timestamp,s2) values(1000,1000.11)insert into
root.vehicle.d0(timestamp,s3) values(60,'a')
insert into root.vehicle.d0(timestamp,s3) values(70,'b')
insert into root.vehicle.d0(timestamp,s3) values(80,'c')
insert into root.vehicle.d0(timestamp,s3) values(101,'d')
insert into root.vehicle.d0(timestamp,s3) values(102,'f')insert into
root.vehicle.d1(timestamp,s0) values(1,999)
insert into root.vehicle.d1(timestamp,s0) values(1000,888)insert into
root.vehicle.d0(timestamp,s1) values(2000-01-01T08:00:00+08:00, 100)
insert into root.vehicle.d0(timestamp,s3) values(2000-01-01T08:00:00+08:00,
'good')insert into root.vehicle.d0(timestamp,s4) values(100, false)
insert into root.vehicle.d0(timestamp,s4) values(100, true)
{code}
and then I query as follows:
{code:java}
IoTDB> select s0,s0,s0,s1 from root.vehicle.d0
+---+--+--+--+--+
|
Time|root.vehicle.d0.s0|root.vehicle.d0.s0|root.vehicle.d0.s0|root.vehicle.d0.s1|
+---+--+--+--+--+
| 1970-01-01T08:00:00.001+08:00| 101| 101|
101| 101|
| 1970-01-01T08:00:00.002+08:00| 1| 1|
1| 1|
| 1970-01-01T08:00:00.050+08:00| 1| 1|
1| 1|
| 1970-01-01T08:00:00.100+08:00|99|99|
99|99|
| 1970-01-01T08:00:00.101+08:00|99|99|
99|99|
| 1970-01-01T08:00:00.102+08:00|80|80|
80|80|
| 1970-01-01T08:00:00.103+08:00|99|99|
99|99|
| 1970-01-01T08:00:00.104+08:00|90|90|
90|90|
| 1970-01-01T08:00:00.105+08:00|99|99|
99|99|
| 1970-01-01T08:00:00.106+08:00|99|99|
99|99|
| 1970-01-01T08:00:01.000+08:00| 2| 22
[jira] [Created] (IOTDB-301) execute "count nodes root" in client gets "Msg:3"
Lei Rui created IOTDB-301:
-
Summary: execute "count nodes root" in client gets "Msg:3"
Key: IOTDB-301
URL: https://issues.apache.org/jira/browse/IOTDB-301
Project: Apache IoTDB
Issue Type: Bug
Reporter: Lei Rui
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IOTDB-302) Bug when executing "show child paths "
Lei Rui created IOTDB-302: - Summary: Bug when executing "show child paths " Key: IOTDB-302 URL: https://issues.apache.org/jira/browse/IOTDB-302 Project: Apache IoTDB Issue Type: Bug Reporter: Lei Rui Attachments: image-2019-11-11-23-55-59-103.png !image-2019-11-11-23-55-59-103.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-305) a sql to query a group of devices separately
Lei Rui created IOTDB-305:
-
Summary: a sql to query a group of devices separately
Key: IOTDB-305
URL: https://issues.apache.org/jira/browse/IOTDB-305
Project: Apache IoTDB
Issue Type: New Feature
Reporter: Lei Rui
First of all, if Bob knows exactly what devices to query (root.sg.d1 and
root.sg.d2 in this case) , he can write sqls for every device:
{code:java}
sql1: select * from root.sg.d1 where s1=1
sql2: select * from root.sg.d2 where s1=1{code}
However, when there are many devices or devices are not specified in advance,
Bob wants to query like:
{code:java}
select * from root.sg.d1,root.sg.d2 where s1=1 (a demo
conception)
/ select * from root.sg.* where s1=1 (a demo conception)
{code}
to return the result same as the concatenation of the results of the above two
queries (i.e., sql1 and sql2) while eliminating the trouble of writing two
separate sqls.
"group by device" sql can't satisfy this demand. It is because the where
condition of "group by device" is shared by all devices. For example,
{code:java}
select * from root.sg.* where s1=1 group by device
{code}
equals
{code:java}
select * from root.sg.d1 where root.sg.d1.s1=1 and root.sg.d2.s1=1
select * from root.sg.d2 where root.sg.d1.s1=1 and root.sg.d2.s1=1
{code}
Note the "and" in the where condition. The following example further
demonstrates the effect:
{code:java}
SET STORAGE GROUP TO root.ChangSha;
CREATE TIMESERIES root.ChangSha.A.aa WITH DATATYPE=INT64, ENCODING=RLE;
CREATE TIMESERIES root.ChangSha.A.ab WITH DATATYPE=INT64, ENCODING=RLE;
CREATE TIMESERIES root.ChangSha.A.ac WITH DATATYPE=INT64, ENCODING=RLE;
CREATE TIMESERIES root.ChangSha.B.aa WITH DATATYPE=INT64, ENCODING=RLE;
CREATE TIMESERIES root.ChangSha.B.ab WITH DATATYPE=INT64, ENCODING=RLE;
CREATE TIMESERIES root.ChangSha.B.ad WITH DATATYPE=INT64, ENCODING=RLE;
insert into root.ChangSha.A(timestamp,aa,ab,ac) values(1,1,1,1);
insert into root.ChangSha.B(timestamp,aa,ab,ad) values(2,1,2,2);
select * from root.ChangSha.A, root.ChangSha.B where aa=1 group by device{code}
The above select query equals
{code:java}
select * from root.ChangSha.A where root.ChangSha.A.aa=1 and
root.ChangSha.B.aa=1
select * from root.ChangSha.B where root.ChangSha.A.aa=1 and
root.ChangSha.B.aa=1{code}
This select query will get an empty result because there is no timestamp under
which both root.ChangSha.A.aa=1 and root.ChangSha.B.aa=1.
What Bob wants is:
{code:java}
select * from root.ChangSha.A, root.ChangSha.B where aa=1 (a
demo conception)
{code}
equals
{code:java}
select * from root.ChangSha.A where root.ChangSha.A.aa=1
select * from root.ChangSha.B where root.ChangSha.B.aa=1{code}
to get the result like:
||Time||Device||aa||ab||ac||ad||
|1970-01-01T08:00:00.001+08:00|root.ChangSha.A|1|1|1|null|
|1970-01-01T08:00:00.002+08:00|root.ChangSha.B|1|2|null|2|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IOTDB-306) count query is not that fast
Lei Rui created IOTDB-306: - Summary: count query is not that fast Key: IOTDB-306 URL: https://issues.apache.org/jira/browse/IOTDB-306 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui According to my test, *q1: select count(s_10) from root.group_0.d_17 where time>=2018-09-20T00:00:00+08:00 and time<=2018-09-20T23:59:59+08:00* ||Total time cost||readTsFileMetaData||readTsDeviceMetaData||readMemChunk|| |23,998|1,367|13,591|7,592| Unit: ms *q2: select s_10 from root.group_0.d_17 where time>=2018-09-20T00:00:00+08:00 and time<=2018-09-20T23:59:59+08:00* ||Total time cost||readTsFileMetaData||readTsDeviceMetaData||readMemChunk|| |27,783|31.2+2,068|134+13,880|14.9+9,587| Unit: ms (The "+" is because the step happens in both `createNewData` and `convertQueryDataSetByFetchSize` phases.) As is shown, the total time cost of q1 is just a little bit smaller than q2. The costs of the three major steps - `readTsFileMetaData`, `readTsDeviceMetaData`, and `readMemChunk` - are very close. The reason for this consequence is that the query execution process of count query reads chunk data from disk into memory anyway and in the best cases utilizes statistics (i,e., numOfPoints) in the pageHeader instead of reading page data. However, the time cost of reading page data from ChunkBuffer (see `ChunkReader.nextBatch`) is not that large, as it is performed in memory. Therefore, the execution process of count query overlaps mostly with that of without count query. And probably other aggregate queries have the similar results. A direction of performance improvement is to avoid `readMemChunk` whenever the statistics in the ChunkMetaData can be utilized. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-335) Separate query execution of the same timeseries with different aggregate functions
Lei Rui created IOTDB-335: - Summary: Separate query execution of the same timeseries with different aggregate functions Key: IOTDB-335 URL: https://issues.apache.org/jira/browse/IOTDB-335 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui For example, select avg(s1),sum(s1) from root.sg.d0 select first_value(s1),first_time(s1) from root.sg.d0 select count(s1), sum(s1) from root.sg.d0 group by ([1,100],2ms) Probably the results of the same timeseries with different aggregate functions can be achieved in one query execution process. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-359) [0.9.0] TsFile Sketch Tool prints wrong results when the tsfile is very large
Lei Rui created IOTDB-359: - Summary: [0.9.0] TsFile Sketch Tool prints wrong results when the tsfile is very large Key: IOTDB-359 URL: https://issues.apache.org/jira/browse/IOTDB-359 Project: Apache IoTDB Issue Type: Bug Reporter: Lei Rui I used the TsFile sketch tool on a TsFile sized 2.2G, and the printed result is wrong for that the ChunkGroupMetaDatas are not printed in their address ascending order. I analyzed this bug and find that `int` is used when comparing the address of ChunkGroupMetaDatas. However, the address should be long data type. I'll fix this bug soon :) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-364) Add TsFileDataDirViewer Tool
Lei Rui created IOTDB-364: - Summary: Add TsFileDataDirViewer Tool Key: IOTDB-364 URL: https://issues.apache.org/jira/browse/IOTDB-364 Project: Apache IoTDB Issue Type: New Feature Reporter: Lei Rui To have a quick overview of the data distribution in the data directories -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (IOTDB-150) Try some other web fonts
[ https://issues.apache.org/jira/browse/IOTDB-150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Rui closed IOTDB-150. - Resolution: Won't Do > Try some other web fonts > > > Key: IOTDB-150 > URL: https://issues.apache.org/jira/browse/IOTDB-150 > Project: Apache IoTDB > Issue Type: Wish > Reporter: Lei Rui >Priority: Trivial > > My suggestion is that the website of > [IoTDB|[https://iotdb.apache.org|https://iotdb.apache.org/#/]] will look more > professional with some other web fonts. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-263) Refactor IoTDBQueryResultSet of JDBC
Lei Rui created IOTDB-263:
-
Summary: Refactor IoTDBQueryResultSet of JDBC
Key: IOTDB-263
URL: https://issues.apache.org/jira/browse/IOTDB-263
Project: Apache IoTDB
Issue Type: Improvement
Reporter: Lei Rui
Proposed by Tian Jiang, there are at least two points that can be improved of
IoTDBQueryResultSet.
The first point: Batch structure may be preferable to List (as shown
in the following code). The data structure of List looks bloated
(chinese: 臃肿的).
{code:java}
// as in nextWithoutConstraints method
TSQueryDataSet tsQueryDataSet = resp.getQueryDataSet();
List records = Utils.convertRowRecords(tsQueryDataSet);
recordItr = records.iterator();
{code}
The batch structure may incur compact memory layout and possible beneficial
batch processing strategy. So I think this point is worth trying, especially
when we have already developed a new TSQueryDataSet with batch-like structure
(pr#455).
The second point: Some inefficient interface realizations. For example, as
shown in the following code, the getDouble method first calls the getString
method and then parses the string to get the double value. However, the double
value in the field could otherwise be directly got (if the data type of this
field is double).
{code:java}
public double getDouble(String columnName) throws SQLException {
return Double.parseDouble(getValueByName(columnName));
}
private String getValueByName(String columnName) throws SQLException {
checkRecord();
if (columnName.equals(TIMESTAMP_STR)) {
return String.valueOf(record.getTimestamp());
}
int tmp = columnInfoMap.get(columnName);
int i = 0;
for (Field field : record.getFields()) {
i++;
if (i == tmp - 1) {
return field.isNull() ? null : field.getStringValue();
}
}
return null;
}
{code}
This point is indeed a true problem. Although right now only the getString
method is actually in use, we do need to improve the other getXXX methods
(e.g., getDouble, getInt, getFloat) and inform our users of these interfaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IOTDB-252) bugs in the print-tsfile-resource-files tool
Lei Rui created IOTDB-252:
-
Summary: bugs in the print-tsfile-resource-files tool
Key: IOTDB-252
URL: https://issues.apache.org/jira/browse/IOTDB-252
Project: Apache IoTDB
Issue Type: Bug
Reporter: Lei Rui
- lack execution command in the print-tsfile-resource-files.bat
- unnecessary check of the existence of ${IOTDB_HOME}/data/wal in the
print-tsfile-resource-files.sh
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IOTDB-277) Lost docs about usage of import-csv.sh and export-csv.sh
Lei Rui created IOTDB-277: - Summary: Lost docs about usage of import-csv.sh and export-csv.sh Key: IOTDB-277 URL: https://issues.apache.org/jira/browse/IOTDB-277 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui As in the readme of the branch rel/0.8([https://github.com/apache/incubator-iotdb/tree/rel/0.8]), there are docs about usage of import-csv.sh and export-csv.sh. But now I can't find them in the docs of the master branch. Are they missing? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-331) [0.9.0]A groupBy query bug when axisOrigin-startTimeOfWindow is an integral multiple of interval
Lei Rui created IOTDB-331:
-
Summary: [0.9.0]A groupBy query bug when
axisOrigin-startTimeOfWindow is an integral multiple of interval
Key: IOTDB-331
URL: https://issues.apache.org/jira/browse/IOTDB-331
Project: Apache IoTDB
Issue Type: Bug
Reporter: Lei Rui
{code:java}
SET STORAGE GROUP TO root.ln.wf01.wt01
CREATE TIMESERIES root.ln.wf01.wt01.status WITH DATATYPE=BOOLEAN, ENCODING=PLAIN
CREATE TIMESERIES root.ln.wf01.wt01.temperature WITH DATATYPE=DOUBLE,
ENCODING=PLAIN
CREATE TIMESERIES root.ln.wf01.wt01.hardware WITH DATATYPE=INT32, ENCODING=PLAIN
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(1,
1.1, false, 11)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(2,
2.2, true, 22)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(3,
3.3, false, 33)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(4,
4.4, false, 44)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(5,
5.5, false, 55)
flush
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(100, 100.1, false, 110)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(150, 200.2, true, 220)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(200, 300.3, false, 330)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(250, 400.4, false, 440)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(300, 500.5, false, 550)
flush
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(10, 10.1, false, 110)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(20, 20.2, true, 220)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(30, 30.3, false, 330)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(40, 40.4, false, 440)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(50, 50.5, false, 550)
flush
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(500, 100.1, false, 110)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(510, 200.2, true, 220)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(520, 300.3, false, 330)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(530, 400.4, false, 440)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(540, 500.5, false, 550)
flush
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(580, 100.1, false, 110)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(590, 200.2, true, 220)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(600, 300.3, false, 330)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(610, 400.4, false, 440)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(620, 500.5, false, 550)
select count(temperature), sum(temperature), avg(temperature) from
root.ln.wf01.wt01 where temperature > 3 GROUP BY (20ms, 25,[5,30], [35,37],
[50, 160], [310, 314])
{code}
Note that in the above groupBy query, 25-5=1*20.
The result of the above query is:
{code:java}
5,0,0.0,null // This is not right
5,3,35.8,11.932
25,1,30.3,30.3
50,1,50.5,50.5
65,0,0.0,null
85,1,100.1,100.1
105,0,0.0,null
125,0,0.0,null
145,1,200.2,200.2
310,0,0.0,null
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IOTDB-332) Bugs when using Chinese character in the timeseries path
Lei Rui created IOTDB-332:
-
Summary: Bugs when using Chinese character in the timeseries path
Key: IOTDB-332
URL: https://issues.apache.org/jira/browse/IOTDB-332
Project: Apache IoTDB
Issue Type: Bug
Reporter: Lei Rui
Note: At the beginning of every case, the data directory is deleted and the
server is restarted.
Case 1: insert into root."哈哈".b(timestamp,c) values(1,2)
{code:java}
IoTDB> insert into root."哈哈".b(timestamp,c) values(1,2)
Msg: Fail to execute insert into root."哈哈".b(timestamp,c) values(1,2) after
reconnecting. please check server status
{code}
Case 2: insert into root.a.b(timestamp,"哈哈") values(1,2)
{code:java}
IoTDB> insert into root.a.b(timestamp,"哈哈") values(1,2)
It costs 0.240sIoTDB> select * from root
+---+---+
| Time|root.a.b.哈哈|
+---+---+
| 1970-01-01T08:00:00.001+08:00| 2|
+---+---+
Total line number = 1
It costs 0.145s
IoTDB> flush
It costs 0.441sIoTDB> select * from root
.
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0]. Actual:26
It costs 1.587
{code}
Case 3: insert into root.a.b(timestamp,"哈哈") values(1,"哈哈")
{code:java}
IoTDB> insert into root.a.b(timestamp,"哈哈") values(1,"哈哈")
It costs 0.281s
IoTDB> select * from root
+---+---+
| Time|root.a.b.哈哈|
+---+---+
| 1970-01-01T08:00:00.001+08:00| 哈哈|
+---+---+
Total line number = 1
It costs 0.566s
IoTDB> flush
It costs 0.388s
IoTDB> select * from root
Msg: Cannot fetch result from server, because of network connection: {}
It costs 0.080s{code}
Case 4: insert into root.哈哈.b(timestamp,c) values(1,100)
{code:java}
IoTDB> insert into root.哈哈.b(timestamp,c) values(1,100)
Msg: Statement format is not right: Parsing error, statement [insert into
root.哈哈.b(timestamp,c) values(1,100)] failed when parsing AST tree to generate
logical operator. Detailed information: [line 1:17 character '哈' not supported
here. (Note that time format should be something like 1) number: eg.123456 2)
function: eg.now() 3) datatime: eg.-MM-dd HH:mm:ss, please check whether
inputting correct time format or referring to sql document)
line 1:18 character '哈' not supported here. (Note that time format should be
something like 1) number: eg.123456 2) function: eg.now() 3) datatime:
eg.-MM-dd HH:mm:ss, please check whether inputting correct time format or
referring to sql document)]
It costs 0.117s
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IOTDB-321) Lack the definition of time expression in 4-SQL Reference.md
Lei Rui created IOTDB-321: - Summary: Lack the definition of time expression in 4-SQL Reference.md Key: IOTDB-321 URL: https://issues.apache.org/jira/browse/IOTDB-321 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui In the "4-SQL Reference.md/Select Record Statement", the definition of TimeExpr is not updated to accomodate the time expression feature developed in [https://github.com/apache/incubator-iotdb/pull/462|https://github.com/apache/incubator-iotdb/pull/462/files]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-316) The "mean" function in 4-SQL Reference.md should be "avg" for latest version
Lei Rui created IOTDB-316:
-
Summary: The "mean" function in 4-SQL Reference.md should be "avg"
for latest version
Key: IOTDB-316
URL: https://issues.apache.org/jira/browse/IOTDB-316
Project: Apache IoTDB
Issue Type: Bug
Reporter: Lei Rui
In version 0.8.1, the function name is "mean“.
In version 0.9.0 and master, the function name has changed to "avg", while the
doc is not updated yet.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IOTDB-315) Can't start iotdb on windows
Lei Rui created IOTDB-315: - Summary: Can't start iotdb on windows Key: IOTDB-315 URL: https://issues.apache.org/jira/browse/IOTDB-315 Project: Apache IoTDB Issue Type: Bug Reporter: Lei Rui D:\new\incubator-iotdb\server\target\iotdb-server-0.9.0-SNAPSHOT\sbin>.\start-server.bat Starting IoTDB If you want to change this configuration, please check conf/iotdb-env.sh(Unix or OS X, if you use Windows, check conf/iotdb-env.bat). 此时不应有 (。 I use master branch. commit id=def4daf2addef0062fe04f698a825301ffa343bc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (IOTDB-237) client fetches all data in spite of maxPrintRowCount
[ https://issues.apache.org/jira/browse/IOTDB-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Rui closed IOTDB-237. - Resolution: Duplicate > client fetches all data in spite of maxPrintRowCount > > > Key: IOTDB-237 > URL: https://issues.apache.org/jira/browse/IOTDB-237 > Project: Apache IoTDB > Issue Type: Improvement > Reporter: Lei Rui >Priority: Major > > In line 202-208 of AbstractClient.java, the client fetches all data from the > server, showing maxPrintRowCount rows. > In line 223, the client shows the total row count of the whole queried result. > Can the client only fetch data within the maxPrintRowCount constraint and > still be able to know the total result count? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-317) [0.8.1] flush data, execute a wrong aggregation query and the server fails
Lei Rui created IOTDB-317: - Summary: [0.8.1] flush data, execute a wrong aggregation query and the server fails Key: IOTDB-317 URL: https://issues.apache.org/jira/browse/IOTDB-317 Project: Apache IoTDB Issue Type: Bug Reporter: Lei Rui *tag release/0.8.1* I start a new server and run the following commands in a client: ``` SET STORAGE GROUP TO root.ln.wf01.wt01; CREATE TIMESERIES root.ln.wf01.wt01.status WITH DATATYPE=BOOLEAN, ENCODING=PLAIN CREATE TIMESERIES root.ln.wf01.wt01.temperature WITH DATATYPE=FLOAT, ENCODING=PLAIN CREATE TIMESERIES root.ln.wf01.wt01.hardware WITH DATATYPE=INT32, ENCODING=PLAIN INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(1, 1.1, false, 11) INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(2, 2.2, true, 22) INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(3, 3.3, false, 33 ) INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(4, 4.4, false, 44) INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(5, 5.5, false, 55) INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(6, 1.1, false, 11) INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(7, 2.2, true, 22) INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(8, 3.3, false, 33 ) INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(9, 4.4, false, 44) INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(10, 5.5, false, 55) select mean(temperature) from root.ln.wf01.wt01 *select abc(temperature) from root.ln.wf01.wt01* // Msg: Cannot get column abc(root.ln.wf01.wt01.temperature) data type // It costs 0.014s select mean(temperature) from root.ln.wf01.wt01 select count(temperature) from root.ln.wf01.wt01 *flush* select count(temperature) from root.ln.wf01.wt01 // Msg: null // It costs 0.011s select * from root // Msg: java.nio.channels.ClosedByInterruptException // It costs 0.015s ``` As can be seen, after executing a wrong aggregation query "select abc(temperature) from root.ln.wf01.wt01" and a flush command (you can also flush before executing this wrong aggregation query), the following two queries fail. I then close the client and restart a new client, queries still fail. I close the server and restart, queries work finally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (IOTDB-302) Bug when executing "show child paths "
[ https://issues.apache.org/jira/browse/IOTDB-302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Rui closed IOTDB-302. - Resolution: Not A Problem > Bug when executing "show child paths " > --- > > Key: IOTDB-302 > URL: https://issues.apache.org/jira/browse/IOTDB-302 > Project: Apache IoTDB > Issue Type: Bug >Reporter: Lei Rui >Priority: Major > Attachments: image-2019-11-11-23-55-59-103.png > > > !image-2019-11-11-23-55-59-103.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-237) client fetches all data in spite of maxPrintRowCount
Lei Rui created IOTDB-237: - Summary: client fetches all data in spite of maxPrintRowCount Key: IOTDB-237 URL: https://issues.apache.org/jira/browse/IOTDB-237 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-346) StorageGroupProcessor.sequenceFileList is ordered by fileName rather than dataTime
Lei Rui created IOTDB-346:
-
Summary: StorageGroupProcessor.sequenceFileList is ordered by
fileName rather than dataTime
Key: IOTDB-346
URL: https://issues.apache.org/jira/browse/IOTDB-346
Project: Apache IoTDB
Issue Type: Bug
Reporter: Lei Rui
`StorageGroupProcessor.sequenceFileList` is ordered by fileName rather than by
time of data, as reflected in the `StorageGroupProcessor.getAllFiles` method
code:
{code:java}
tsFiles.sort(this::compareFileName);
{code}
I use the following examples to expose the bug when the order of fileName is
inconsistent with that of dataTime.
First, for preparation, I created three tsfiles using the following sql:
{code:java}
SET STORAGE GROUP TO root.ln.wf01.wt01
CREATE TIMESERIES root.ln.wf01.wt01.status WITH DATATYPE=BOOLEAN, ENCODING=PLAIN
CREATE TIMESERIES root.ln.wf01.wt01.temperature WITH DATATYPE=DOUBLE,
ENCODING=PLAIN
CREATE TIMESERIES root.ln.wf01.wt01.hardware WITH DATATYPE=INT32, ENCODING=PLAIN
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(1,
1.1, false, 11)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(2,
2.2, true, 22)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(3,
3.3, false, 33)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(4,
4.4, false, 44)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware) values(5,
5.5, false, 55)
flush
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(100, 100.1, false, 110)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(150, 200.2, true, 220)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(200, 300.3, false, 330)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(250, 400.4, false, 440)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(300, 500.5, false, 550)
flush
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(10, 10.1, false, 110)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(20, 20.2, true, 220)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(30, 30.3, false, 330)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(40, 40.4, false, 440)
INSERT INTO root.ln.wf01.wt01(timestamp,temperature,status, hardware)
values(50, 50.5, false, 550)
flush
{code}
The tsfiles created are organized in the following directory structure:
{code:java}
|data
|--sequence
|root.ln.wf01.wt01
|--1575813520203-101-0.tsfile
|--1575813520203-101-0.tsfile.resource
|--1575813520669-103-0.tsfile
|--1575813520669-103-0.tsfile.resource
|--unsequence
|root.ln.wf01.wt01
|--1575813521063-105-0.tsfile
|--1575813521063-105-0.tsfile.resource
{code}
||File Name||Data Time||
|(a) 1575813520203-101-0.tsfile|1-5|
|(c) 1575813521063-105-0.tsfile|10-50|
|(b) 1575813520669-103-0.tsfile|100-300|
Note how the order of fileName is inconsistent with that of dataTime.
By the way, if you look into the code, you will know how the file name is
generated:
{code:java}
System.currentTimeMillis() + IoTDBConstant.TSFILE_NAME_SEPARATOR +
versionController.nextVersion() + IoTDBConstant.TSFILE_NAME_SEPARATOR + "0" +
TSFILE_SUFFIX
{code}
Then, I loaded the three tsfiles into another brand new IoTDB. I did two
experiments with different loading orders each.
In the first experiment, the tsfiles were loaded in their data time order. That
is,
{code:java}
IoTDB> load 1575813520203-101-0.tsfile // tsfile (a), with data time 1-5
IoTDB> load 1575813521063-105-0.tsfile // tsfile (c), with data time 10-50
IoTDB> load 1575813520669-103-0.tsfile // tsfile (b), with data time
100-300{code}
After loading successfully, I did the following query in the same client window
and got the wrong result:
{code:java}
IoTDB> select * from root
+---+-+-+-+
| Time|root.ln.wf01.wt01.temperature|
root.ln.wf01.wt01.status| root.ln.wf01.wt01.hardware|
+---+-+-+-+
| 1970-01-01T08:00:00.001+08:00| 1.1|
false| 11|
| 1970-01-01T08:00:00.002+08:00| 2.2|
true| 22|
| 1970-01-01T08:00:00.003+08:00| 3.3|
false| 33|
| 1970-01-01T08:00:00.004+08:00| 4.4|
false| 44|
|
[jira] [Created] (IOTDB-416) [0.9.1] ASF header format issues
Lei Rui created IOTDB-416:
-
Summary: [0.9.1] ASF header format issues
Key: IOTDB-416
URL: https://issues.apache.org/jira/browse/IOTDB-416
Project: Apache IoTDB
Issue Type: Improvement
Reporter: Lei Rui
see
[https://lists.apache.org/thread.html/209371ab5158fc61f2342115c1eef199f78555059dfb98bfdad70d72%40%3Cdev.iotdb.apache.org%3E]
Hi,
using `find **/src -type f -exec grep -i "copyright" {} \; -print | sort
-u`, many apache header format issues can be found..
e.g.,
1. showTTLPlan.java
2. all files for QueryMetricUI function...
3. All scala files are using /** so the blank space is two chars while it
is one char in all java files.
4. there are two blank spaces between "#" and content in all Python files
while it is one blank space in all sh files.
I do not vote -1 but at least we need to fix these issues in next release.
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IOTDB-417) [0.9.0] With German Locale it does NOT work as one test fails due to decimal comma vs decimal point
Lei Rui created IOTDB-417: - Summary: [0.9.0] With German Locale it does NOT work as one test fails due to decimal comma vs decimal point Key: IOTDB-417 URL: https://issues.apache.org/jira/browse/IOTDB-417 Project: Apache IoTDB Issue Type: Bug Affects Versions: 0.9.1 Reporter: Lei Rui Fix For: 0.10.0-SNAPSHOT Credit to Julian Feinauer. See [https://lists.apache.org/thread.html/rfef7f0035f28c3ee81ce32135e46e53774ce19b4af0bc06db426d4e3%40%3Cdev.iotdb.apache.org%3E] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-419) Rename first/last aggregate query to first_value/last_value
Lei Rui created IOTDB-419: - Summary: Rename first/last aggregate query to first_value/last_value Key: IOTDB-419 URL: https://issues.apache.org/jira/browse/IOTDB-419 Project: Apache IoTDB Issue Type: Improvement Reporter: Lei Rui -- This message was sent by Atlassian Jira (v8.3.4#803005)
