[GitHub] [incubator-iotdb] JackieTien97 opened a new pull request #452: change hadoop package name

2019-10-16 Thread GitBox
JackieTien97 opened a new pull request #452: change hadoop package name
URL: https://github.com/apache/incubator-iotdb/pull/452
 
 
   change 'org.apache.iotdb.tsfile.hadoop' to 'org.apahce.iotdb.hadoop.tsfile'


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Genius-pig commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-15 Thread GitBox
Genius-pig commented on a change in pull request #440: reconstruct antlrv3 
grammar to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r335009423
 
 

 ##
 File path: server/src/main/antlr3/org/apache/iotdb/db/sql/parse/TqlLexer.g
 ##
 @@ -0,0 +1,539 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+lexer grammar TqlLexer;
+
+@header {
+package org.apache.iotdb.db.sql.parse;
+}
+
+//*** key words *
+K_SELECT
+: S E L E C T
+;
+
+K_INTO
+: I N T O
+;
+
+K_ROOT
+: R O O T
+;
+
+K_FROM
+: F R O M
+;
+
+K_WHERE
+: W H E R E
+;
+
+K_LIMIT
+: L I M I T
+;
+
+K_OFFSET
+: O F F S E T
+;
+
+K_SLIMIT
+: S L I M I T
+;
+
+K_SOFFSET
+: S O F F S E T
+;
+
+K_NOW
+: N O W
+;
+
+K_GROUP
+: G R O U P
+;
+
+K_BY
+: B Y
+;
+
+K_FILL
+: F I L L
+;
+
+K_PREVIOUS
+: P R E V I O U S
+;
+
+K_LINEAR
+: L I N E A R
+;
+
+K_INT32
+: I N T '3' '2'
+;
+
+K_INT64
+: I N T '6' '4'
+;
+
+K_FLOAT
+: F L O A T
+;
+
+K_DOUBLE
+: D O U B L E
+;
+
+K_BOOLEAN
+: B O O L E A N
+;
+
+K_TEXT
+: T E X T
+;
+
+K_INSERT
+: I N S E R T
+;
+
+K_VALUES
+: V A L U E S
+;
+
+K_TIMESTAMP
+: T I M E S T A M P
+;
+
+K_UPDATE
+: U P D A T E
+;
+
+K_SET
+: S E T
+;
+
+K_DELETE
+: D E L E T E
+;
+
+K_CREATE
+: C R E A T E
+;
+
+K_TIMESERIES
+: T I M E S E R I E S
+;
+
+K_WITH
+: W I T H
+;
+
+K_DATATYPE
+: D A T A T Y P E
+;
+
+K_ENCODING
+: E N C O D I N G
+;
+
+K_COMPRESSOR
+: C O M P R E S S O R
+;
+
+K_STORAGE
+: S T O R A G E
+;
+
+K_TO
+: T O
+;
+
+K_PROPERTY
+: P R O P E R T Y
+;
+
+K_LABEL
+: L A B E L
+;
+
+
+K_LINK
+: L I N K
+;
+
+K_UNLINK
+: U N L I N K
+;
+
+K_SHOW
+: S H O W
+;
+
+K_METADATA
+: M E T A D A T A
+;
+
+K_DESCRIBE
+: D E S C R I B E
+;
+
+K_INDEX
+: I N D E X
+;
+
+K_ON
+: O N
+;
+
+K_USING
+: U S I N G
+;
+
+K_DROP
+: D R O P
+;
+
+K_MERGE
+: M E R G E
+;
+
+K_LIST
+: L I S T
+;
+
+K_USER
+: U S E R
+;
+
+K_PRIVILEGES
+ : P R I V I L E G E S
+ ;
+
+K_ROLE
+: R O L E
+;
+
+K_ALL
+: A L L
+;
+
+K_OF
+: O F
+;
+
+K_QUIT
+: Q U I T
+;
+
+K_ALTER
+: A L T E R
+;
+
+K_PASSWORD
+: P A S S W O R D
+;
+
+K_GRANT
+: G R A N T
+;
+
+K_REVOKE
+: R E V O K E
+;
+
+K_PATH
+: P A T H
+;
+
+K_LOAD
+: L O A D
+;
+
+K_WATERMARK_EMBEDDING
+: W A T E R M A R K '_' E M B E D D I N G
+;
+
+//** logical operator***
+OPERATOR_AND
+: A N D
+| '&'
+| '&&'
+;
+
+OPERATOR_OR
+: O R
+| '|'
+| '||'
+;
+
+OPERATOR_NOT
+: N O T | '!'
+;
+
+
+// data type ***
+K_PLAIN
+   : P L A I N
+   ;
+K_PLAIN_DICTIONARY
+   : P L A I N '_' D I C T I O N A R Y
+   ;
+K_RLE
+   : R L E
+   ;
+K_DIFF
+   : D I F F
+   ;
+K_TS_2DIFF
+   : T S '_' '2' D I F F
+   ;
+K_BITMAP
+   : B I T M A P
+   ;
+K_GORILLA
+   : G O R I L L A
+   ;
+K_REGULAR
+   : R E G U L A R
+   ;
+
+K_ADD
+: A D D
+;
+// *** comparison ***
+OPERATOR_GT
+: '>'
+;
+
+OPERATOR_GTE
+: '>='
+;
+
+OPERATOR_LT
+: '<'
+;
+
+OPERATOR_LTE
+: '<='
+;
+
+OPERATOR_EQ
+: '=' | '=='
+;
+
+OPERATOR_NEQ
+: '!=' | '<>'
+;
+
+// operator ***
+STAR
+: '*'
+;
+
+MINUS
+: '-'
+;
+
+PLUS
+: '+'
+;
+
+DIVIDE
+: '/'
+;
+
+// symbol ***
+SEMI
+: ';'
+;
+
+DOT
+: '.'
+;
+
+COMMA
+: ','
+;
+
+LR_BRACKET
+: '('
+;
+
+RR_BRACKET
+: ')'
+;
+
+LS_BRACKET
+: '['
+;
+
+RS_BRACKET
+: ']'
+;
+
+STRING_LITERAL
+   : DQUOTA_STRING
+   | SQUOTA_STRING
+   ;
+
+fragment DQUOTA_STRING
+   : '"' ( '\\'. | '""' | ~('"'| '\\') )* '"'
+   ;
+
+fragment SQUOTA_STRING
+   : 

[GitHub] [incubator-iotdb] jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-15 Thread GitBox
jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 
grammar to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334965163
 
 

 ##
 File path: server/src/main/antlr3/org/apache/iotdb/db/sql/parse/TqlLexer.g
 ##
 @@ -0,0 +1,539 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+lexer grammar TqlLexer;
+
+@header {
+package org.apache.iotdb.db.sql.parse;
+}
+
+//*** key words *
+K_SELECT
+: S E L E C T
+;
+
+K_INTO
+: I N T O
+;
+
+K_ROOT
+: R O O T
+;
+
+K_FROM
+: F R O M
+;
+
+K_WHERE
+: W H E R E
+;
+
+K_LIMIT
+: L I M I T
+;
+
+K_OFFSET
+: O F F S E T
+;
+
+K_SLIMIT
+: S L I M I T
+;
+
+K_SOFFSET
+: S O F F S E T
+;
+
+K_NOW
+: N O W
+;
+
+K_GROUP
+: G R O U P
+;
+
+K_BY
+: B Y
+;
+
+K_FILL
+: F I L L
+;
+
+K_PREVIOUS
+: P R E V I O U S
+;
+
+K_LINEAR
+: L I N E A R
+;
+
+K_INT32
+: I N T '3' '2'
+;
+
+K_INT64
+: I N T '6' '4'
+;
+
+K_FLOAT
+: F L O A T
+;
+
+K_DOUBLE
+: D O U B L E
+;
+
+K_BOOLEAN
+: B O O L E A N
+;
+
+K_TEXT
+: T E X T
+;
+
+K_INSERT
+: I N S E R T
+;
+
+K_VALUES
+: V A L U E S
+;
+
+K_TIMESTAMP
+: T I M E S T A M P
+;
+
+K_UPDATE
+: U P D A T E
+;
+
+K_SET
+: S E T
+;
+
+K_DELETE
+: D E L E T E
+;
+
+K_CREATE
+: C R E A T E
+;
+
+K_TIMESERIES
+: T I M E S E R I E S
+;
+
+K_WITH
+: W I T H
+;
+
+K_DATATYPE
+: D A T A T Y P E
+;
+
+K_ENCODING
+: E N C O D I N G
+;
+
+K_COMPRESSOR
+: C O M P R E S S O R
+;
+
+K_STORAGE
+: S T O R A G E
+;
+
+K_TO
+: T O
+;
+
+K_PROPERTY
+: P R O P E R T Y
+;
+
+K_LABEL
+: L A B E L
+;
+
+
+K_LINK
+: L I N K
+;
+
+K_UNLINK
+: U N L I N K
+;
+
+K_SHOW
+: S H O W
+;
+
+K_METADATA
+: M E T A D A T A
+;
+
+K_DESCRIBE
+: D E S C R I B E
+;
+
+K_INDEX
+: I N D E X
+;
+
+K_ON
+: O N
+;
+
+K_USING
+: U S I N G
+;
+
+K_DROP
+: D R O P
+;
+
+K_MERGE
+: M E R G E
+;
+
+K_LIST
+: L I S T
+;
+
+K_USER
+: U S E R
+;
+
+K_PRIVILEGES
+ : P R I V I L E G E S
+ ;
+
+K_ROLE
+: R O L E
+;
+
+K_ALL
+: A L L
+;
+
+K_OF
+: O F
+;
+
+K_QUIT
+: Q U I T
+;
+
+K_ALTER
+: A L T E R
+;
+
+K_PASSWORD
+: P A S S W O R D
+;
+
+K_GRANT
+: G R A N T
+;
+
+K_REVOKE
+: R E V O K E
+;
+
+K_PATH
+: P A T H
+;
+
+K_LOAD
+: L O A D
+;
+
+K_WATERMARK_EMBEDDING
+: W A T E R M A R K '_' E M B E D D I N G
+;
+
+//** logical operator***
+OPERATOR_AND
+: A N D
+| '&'
+| '&&'
+;
+
+OPERATOR_OR
+: O R
+| '|'
+| '||'
+;
+
+OPERATOR_NOT
+: N O T | '!'
+;
+
+
+// data type ***
+K_PLAIN
+   : P L A I N
+   ;
+K_PLAIN_DICTIONARY
+   : P L A I N '_' D I C T I O N A R Y
+   ;
+K_RLE
+   : R L E
+   ;
+K_DIFF
+   : D I F F
+   ;
+K_TS_2DIFF
+   : T S '_' '2' D I F F
+   ;
+K_BITMAP
+   : B I T M A P
+   ;
+K_GORILLA
+   : G O R I L L A
+   ;
+K_REGULAR
+   : R E G U L A R
+   ;
+
+K_ADD
+: A D D
+;
+// *** comparison ***
+OPERATOR_GT
+: '>'
+;
+
+OPERATOR_GTE
+: '>='
+;
+
+OPERATOR_LT
+: '<'
+;
+
+OPERATOR_LTE
+: '<='
+;
+
+OPERATOR_EQ
+: '=' | '=='
+;
+
+OPERATOR_NEQ
+: '!=' | '<>'
+;
+
+// operator ***
+STAR
+: '*'
+;
+
+MINUS
+: '-'
+;
+
+PLUS
+: '+'
+;
+
+DIVIDE
+: '/'
+;
+
+// symbol ***
+SEMI
+: ';'
+;
+
+DOT
+: '.'
+;
+
+COMMA
+: ','
+;
+
+LR_BRACKET
+: '('
+;
+
+RR_BRACKET
+: ')'
+;
+
+LS_BRACKET
+: '['
+;
+
+RS_BRACKET
+: ']'
+;
+
+STRING_LITERAL
+   : DQUOTA_STRING
+   | SQUOTA_STRING
+   ;
+
+fragment DQUOTA_STRING
+   : '"' ( '\\'. | '""' | ~('"'| '\\') )* '"'
+   ;
+
+fragment SQUOTA_STRING
+   : 

[GitHub] [incubator-iotdb] SolomonAnn commented on a change in pull request #430: [IOTDB-193]Create schema automatically

2019-10-15 Thread GitBox
SolomonAnn commented on a change in pull request #430: [IOTDB-193]Create schema 
automatically
URL: https://github.com/apache/incubator-iotdb/pull/430#discussion_r334953104
 
 

 ##
 File path: server/src/main/java/org/apache/iotdb/db/utils/DeviceIdCache.java
 ##
 @@ -0,0 +1,53 @@
+package org.apache.iotdb.db.utils;
+
+import org.apache.iotdb.db.exception.PathErrorException;
+import org.apache.iotdb.db.exception.StorageGroupException;
+import org.apache.iotdb.tsfile.exception.cache.CacheException;
+
+import java.util.List;
+import java.util.concurrent.CopyOnWriteArrayList;
+
+public abstract class DeviceIdCache {
 
 Review comment:
   I'm confused that whether I should use `mNodeCache `directly because its key 
is just `deviceId`. However, if I choose to use `mNodeCache`, I will have 
trouble with handling exceptions since I need `PathErrorException `and 
`StorageGroupException` to distinguish two different cases while `mNodeCache` 
throws `CacheException` in any case. Do you have any suggestions?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on issue #429: [IOTDB-205]Support storage-group-level data ttl

2019-10-15 Thread GitBox
LeiRui commented on issue #429: [IOTDB-205]Support storage-group-level data ttl
URL: https://github.com/apache/incubator-iotdb/pull/429#issuecomment-542204822
 
 
   Hi, 
   This is a review on functionality.
   I find the operation `unset TTL to ` REALLY requires a 
seasoned user who knows exactly what he/she is doing. And I totally agree with 
what you have noted in the guidance:
   > Notice: When you set TTL to some storage groups, data out of the TTL will 
be made invisible
   immediately, but because the data files may contain both out-dated and 
living data or the data files may be being used by queries, the physical 
removal of data is stale. If you increase or unset TTL
   just after setting it previously, some previously invisible data may be seen 
again, but the
   physically removed one is lost forever. **In other word, different from 
delete statement, the
   atomicity of data deletion is not guaranteed for efficiency concerns.** **So 
we recommend that you do not change the TTL once it is set or at least do not 
reset it frequently, unless you are determined to suffer the 
unpredictability.** 
   
   I want to draw attention to another point that can be easily misunderstood 
when using the operation `unset TTL to `. That is, the latest 
insertion time of a storage group, which is the split point used to determine 
whether the coming data point is sequential or unsequential, will not be 
“recycled" even if the data before `now()-TTL` are really deleted.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] qiaojialin closed pull request #447: [IOTDB-250] exclude .mvn/wrapper/maven-wrapper.jar

2019-10-15 Thread GitBox
qiaojialin closed pull request #447: [IOTDB-250] exclude 
.mvn/wrapper/maven-wrapper.jar
URL: https://github.com/apache/incubator-iotdb/pull/447
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Ring-k commented on issue #440: reconstruct antlrv3 grammar to improve performance

2019-10-15 Thread GitBox
Ring-k commented on issue #440: reconstruct antlrv3 grammar to improve 
performance
URL: https://github.com/apache/incubator-iotdb/pull/440#issuecomment-542160163
 
 
   > > No "=>" operators anymore.
   > 
   > "=>" operator means always execute predicate. Removing all of them is a 
real improvement? Syntactic predicates were used to work around a prediction 
weakness in ANTLR 3.
   
   For each syntactic predicate, ANTLR defines a special method that returns 
true or false depending on whether the predicate’s grammar fragment matches the 
next input symbols. So if syntactic predicate is frequently used, the method 
will be frequently called. So, in my opinion, a simpler design leads to better 
performance.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] HTHou commented on issue #450: modify documents

2019-10-15 Thread GitBox
HTHou commented on issue #450: modify documents
URL: https://github.com/apache/incubator-iotdb/pull/450#issuecomment-542115088
 
 
   > * about organization logic
   >   
   >   * I find that the TsFile docs are splitted into `6-API` and `7-System 
Design`. My questions are:
   > 
   > * Why TsFile's installation and usage are classified as API?
   > * Is the chapter title `System Design` too big when there is only a 
doc about TsFile's hierarchy design in this chapter?
   > * Why splitting TsFile docs?
   > * about titles and outlines
   >   
   >   * Some articles under the directory `4-Operation Manual` have the stale 
chapter title `Chapter 3: Operation Manual`.
   >   * It seems all articles under the directory `6-API` lack their article 
titles.
   >   * Some files have outlines while some others don't. Best every file has 
its chapter title (if there is), article title and outline.
   > * about links
   >   
   >   * I find in QuickStart.md, some links link to the website 0.8.0 version 
docs. Shouldn't they link to master docs since the master docs are latest?
   > * about Chinese docs
   >   
   >   * Since they are written in Chinese, I wonder whether the file names and 
directory names can be in Chinese too.
   
   * for organization logic 
* The chapter 6 should be related to operation, the name `API` may not good 
enough. Any advice?
* In chapter `System Design`, there is only one section now. There will be 
more documents in this chapter in the future. 
* The reason why I split TsFile docs is the TsFile hierarchy is more 
related to the introduction of design, and other docs are focus on user's 
operation. 
   * about titles and outlines
* The issues you listed are some mistakes. They should be fixed. 
   * about links
* It is a mistake too. I didn't check `QuickStart.md` when I modify the 
docs. Thank you.
   * about Chinese docs
* If we change the file names and directory names, we have to change our 
website too, since our Chinese docs website is using same directory and file 
names. 
  


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] samperson1997 commented on a change in pull request #417: [IOTDB-234] Refactor TsFile storage on HDFS

2019-10-15 Thread GitBox
samperson1997 commented on a change in pull request #417: [IOTDB-234] Refactor 
TsFile storage on HDFS
URL: https://github.com/apache/incubator-iotdb/pull/417#discussion_r334832198
 
 

 ##
 File path: 
tsfile/src/main/java/org/apache/iotdb/tsfile/fileSystem/FileInputFactory.java
 ##
 @@ -19,33 +19,39 @@
 
 package org.apache.iotdb.tsfile.fileSystem;
 
+import java.io.IOException;
+import java.lang.reflect.InvocationTargetException;
+import java.nio.file.Paths;
 import org.apache.iotdb.tsfile.common.conf.TSFileDescriptor;
 import org.apache.iotdb.tsfile.read.reader.DefaultTsFileInput;
 import org.apache.iotdb.tsfile.read.reader.TsFileInput;
-import org.apache.iotdb.tsfile.write.writer.TsFileIOWriter;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
-import java.io.IOException;
-import java.nio.file.Paths;
-
 public enum FileInputFactory {
 
 Review comment:
   Should I separate these different factories into three packages? like 
`fsFactory`, `fileInputFactory` and `fileOutputFactory`


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #417: [IOTDB-234] Refactor TsFile storage on HDFS

2019-10-15 Thread GitBox
jt2594838 commented on a change in pull request #417: [IOTDB-234] Refactor 
TsFile storage on HDFS
URL: https://github.com/apache/incubator-iotdb/pull/417#discussion_r334831041
 
 

 ##
 File path: 
tsfile/src/main/java/org/apache/iotdb/tsfile/fileSystem/FileInputFactory.java
 ##
 @@ -19,33 +19,39 @@
 
 package org.apache.iotdb.tsfile.fileSystem;
 
+import java.io.IOException;
+import java.lang.reflect.InvocationTargetException;
+import java.nio.file.Paths;
 import org.apache.iotdb.tsfile.common.conf.TSFileDescriptor;
 import org.apache.iotdb.tsfile.read.reader.DefaultTsFileInput;
 import org.apache.iotdb.tsfile.read.reader.TsFileInput;
-import org.apache.iotdb.tsfile.write.writer.TsFileIOWriter;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
-import java.io.IOException;
-import java.nio.file.Paths;
-
 public enum FileInputFactory {
 
 Review comment:
   Better to perform your refactoring on these two factories, too.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-15 Thread GitBox
Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar 
to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334828519
 
 

 ##
 File path: server/src/main/antlr3/org/apache/iotdb/db/sql/parse/TqlLexer.g
 ##
 @@ -0,0 +1,539 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+lexer grammar TqlLexer;
+
+@header {
+package org.apache.iotdb.db.sql.parse;
+}
+
+//*** key words *
+K_SELECT
+: S E L E C T
+;
+
+K_INTO
+: I N T O
+;
+
+K_ROOT
+: R O O T
+;
+
+K_FROM
+: F R O M
+;
+
+K_WHERE
+: W H E R E
+;
+
+K_LIMIT
+: L I M I T
+;
+
+K_OFFSET
+: O F F S E T
+;
+
+K_SLIMIT
+: S L I M I T
+;
+
+K_SOFFSET
+: S O F F S E T
+;
+
+K_NOW
+: N O W
+;
+
+K_GROUP
+: G R O U P
+;
+
+K_BY
+: B Y
+;
+
+K_FILL
+: F I L L
+;
+
+K_PREVIOUS
+: P R E V I O U S
+;
+
+K_LINEAR
+: L I N E A R
+;
+
+K_INT32
+: I N T '3' '2'
+;
+
+K_INT64
+: I N T '6' '4'
+;
+
+K_FLOAT
+: F L O A T
+;
+
+K_DOUBLE
+: D O U B L E
+;
+
+K_BOOLEAN
+: B O O L E A N
+;
+
+K_TEXT
+: T E X T
+;
+
+K_INSERT
+: I N S E R T
+;
+
+K_VALUES
+: V A L U E S
+;
+
+K_TIMESTAMP
+: T I M E S T A M P
+;
+
+K_UPDATE
+: U P D A T E
+;
+
+K_SET
+: S E T
+;
+
+K_DELETE
+: D E L E T E
+;
+
+K_CREATE
+: C R E A T E
+;
+
+K_TIMESERIES
+: T I M E S E R I E S
+;
+
+K_WITH
+: W I T H
+;
+
+K_DATATYPE
+: D A T A T Y P E
+;
+
+K_ENCODING
+: E N C O D I N G
+;
+
+K_COMPRESSOR
+: C O M P R E S S O R
+;
+
+K_STORAGE
+: S T O R A G E
+;
+
+K_TO
+: T O
+;
+
+K_PROPERTY
+: P R O P E R T Y
+;
+
+K_LABEL
+: L A B E L
+;
+
+
+K_LINK
+: L I N K
+;
+
+K_UNLINK
+: U N L I N K
+;
+
+K_SHOW
+: S H O W
+;
+
+K_METADATA
+: M E T A D A T A
+;
+
+K_DESCRIBE
+: D E S C R I B E
+;
+
+K_INDEX
+: I N D E X
+;
+
+K_ON
+: O N
+;
+
+K_USING
+: U S I N G
+;
+
+K_DROP
+: D R O P
+;
+
+K_MERGE
+: M E R G E
+;
+
+K_LIST
+: L I S T
+;
+
+K_USER
+: U S E R
+;
+
+K_PRIVILEGES
+ : P R I V I L E G E S
+ ;
+
+K_ROLE
+: R O L E
+;
+
+K_ALL
+: A L L
+;
+
+K_OF
+: O F
+;
+
+K_QUIT
+: Q U I T
+;
+
+K_ALTER
+: A L T E R
+;
+
+K_PASSWORD
+: P A S S W O R D
+;
+
+K_GRANT
+: G R A N T
+;
+
+K_REVOKE
+: R E V O K E
+;
+
+K_PATH
+: P A T H
+;
+
+K_LOAD
+: L O A D
+;
+
+K_WATERMARK_EMBEDDING
+: W A T E R M A R K '_' E M B E D D I N G
+;
+
+//** logical operator***
+OPERATOR_AND
+: A N D
+| '&'
+| '&&'
+;
+
+OPERATOR_OR
+: O R
+| '|'
+| '||'
+;
+
+OPERATOR_NOT
+: N O T | '!'
+;
+
+
+// data type ***
+K_PLAIN
+   : P L A I N
+   ;
+K_PLAIN_DICTIONARY
+   : P L A I N '_' D I C T I O N A R Y
+   ;
+K_RLE
+   : R L E
+   ;
+K_DIFF
+   : D I F F
+   ;
+K_TS_2DIFF
+   : T S '_' '2' D I F F
+   ;
+K_BITMAP
+   : B I T M A P
+   ;
+K_GORILLA
+   : G O R I L L A
+   ;
+K_REGULAR
+   : R E G U L A R
+   ;
+
+K_ADD
+: A D D
+;
+// *** comparison ***
+OPERATOR_GT
+: '>'
+;
+
+OPERATOR_GTE
+: '>='
+;
+
+OPERATOR_LT
+: '<'
+;
+
+OPERATOR_LTE
+: '<='
+;
+
+OPERATOR_EQ
+: '=' | '=='
+;
+
+OPERATOR_NEQ
+: '!=' | '<>'
+;
+
+// operator ***
+STAR
+: '*'
+;
+
+MINUS
+: '-'
+;
+
+PLUS
+: '+'
+;
+
+DIVIDE
+: '/'
+;
+
+// symbol ***
+SEMI
+: ';'
+;
+
+DOT
+: '.'
+;
+
+COMMA
+: ','
+;
+
+LR_BRACKET
+: '('
+;
+
+RR_BRACKET
+: ')'
+;
+
+LS_BRACKET
+: '['
+;
+
+RS_BRACKET
+: ']'
+;
+
+STRING_LITERAL
+   : DQUOTA_STRING
+   | SQUOTA_STRING
+   ;
+
+fragment DQUOTA_STRING
+   : '"' ( '\\'. | '""' | ~('"'| '\\') )* '"'
+   ;
+
+fragment SQUOTA_STRING
+   : '\'' 

[GitHub] [incubator-iotdb] Genius-pig commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-15 Thread GitBox
Genius-pig commented on a change in pull request #440: reconstruct antlrv3 
grammar to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334822659
 
 

 ##
 File path: server/src/main/antlr3/org/apache/iotdb/db/sql/parse/TqlLexer.g
 ##
 @@ -0,0 +1,539 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+lexer grammar TqlLexer;
+
+@header {
+package org.apache.iotdb.db.sql.parse;
+}
+
+//*** key words *
+K_SELECT
+: S E L E C T
+;
+
+K_INTO
+: I N T O
+;
+
+K_ROOT
+: R O O T
+;
+
+K_FROM
+: F R O M
+;
+
+K_WHERE
+: W H E R E
+;
+
+K_LIMIT
+: L I M I T
+;
+
+K_OFFSET
+: O F F S E T
+;
+
+K_SLIMIT
+: S L I M I T
+;
+
+K_SOFFSET
+: S O F F S E T
+;
+
+K_NOW
+: N O W
+;
+
+K_GROUP
+: G R O U P
+;
+
+K_BY
+: B Y
+;
+
+K_FILL
+: F I L L
+;
+
+K_PREVIOUS
+: P R E V I O U S
+;
+
+K_LINEAR
+: L I N E A R
+;
+
+K_INT32
+: I N T '3' '2'
+;
+
+K_INT64
+: I N T '6' '4'
+;
+
+K_FLOAT
+: F L O A T
+;
+
+K_DOUBLE
+: D O U B L E
+;
+
+K_BOOLEAN
+: B O O L E A N
+;
+
+K_TEXT
+: T E X T
+;
+
+K_INSERT
+: I N S E R T
+;
+
+K_VALUES
+: V A L U E S
+;
+
+K_TIMESTAMP
+: T I M E S T A M P
+;
+
+K_UPDATE
+: U P D A T E
+;
+
+K_SET
+: S E T
+;
+
+K_DELETE
+: D E L E T E
+;
+
+K_CREATE
+: C R E A T E
+;
+
+K_TIMESERIES
+: T I M E S E R I E S
+;
+
+K_WITH
+: W I T H
+;
+
+K_DATATYPE
+: D A T A T Y P E
+;
+
+K_ENCODING
+: E N C O D I N G
+;
+
+K_COMPRESSOR
+: C O M P R E S S O R
+;
+
+K_STORAGE
+: S T O R A G E
+;
+
+K_TO
+: T O
+;
+
+K_PROPERTY
+: P R O P E R T Y
+;
+
+K_LABEL
+: L A B E L
+;
+
+
+K_LINK
+: L I N K
+;
+
+K_UNLINK
+: U N L I N K
+;
+
+K_SHOW
+: S H O W
+;
+
+K_METADATA
+: M E T A D A T A
+;
+
+K_DESCRIBE
+: D E S C R I B E
+;
+
+K_INDEX
+: I N D E X
+;
+
+K_ON
+: O N
+;
+
+K_USING
+: U S I N G
+;
+
+K_DROP
+: D R O P
+;
+
+K_MERGE
+: M E R G E
+;
+
+K_LIST
+: L I S T
+;
+
+K_USER
+: U S E R
+;
+
+K_PRIVILEGES
+ : P R I V I L E G E S
+ ;
+
+K_ROLE
+: R O L E
+;
+
+K_ALL
+: A L L
+;
+
+K_OF
+: O F
+;
+
+K_QUIT
+: Q U I T
+;
+
+K_ALTER
+: A L T E R
+;
+
+K_PASSWORD
+: P A S S W O R D
+;
+
+K_GRANT
+: G R A N T
+;
+
+K_REVOKE
+: R E V O K E
+;
+
+K_PATH
+: P A T H
+;
+
+K_LOAD
+: L O A D
+;
+
+K_WATERMARK_EMBEDDING
+: W A T E R M A R K '_' E M B E D D I N G
+;
+
+//** logical operator***
+OPERATOR_AND
+: A N D
+| '&'
+| '&&'
+;
+
+OPERATOR_OR
+: O R
+| '|'
+| '||'
+;
+
+OPERATOR_NOT
+: N O T | '!'
+;
+
+
+// data type ***
+K_PLAIN
+   : P L A I N
+   ;
+K_PLAIN_DICTIONARY
+   : P L A I N '_' D I C T I O N A R Y
+   ;
+K_RLE
+   : R L E
+   ;
+K_DIFF
+   : D I F F
+   ;
+K_TS_2DIFF
+   : T S '_' '2' D I F F
+   ;
+K_BITMAP
+   : B I T M A P
+   ;
+K_GORILLA
+   : G O R I L L A
+   ;
+K_REGULAR
+   : R E G U L A R
+   ;
+
+K_ADD
+: A D D
+;
+// *** comparison ***
+OPERATOR_GT
+: '>'
+;
+
+OPERATOR_GTE
+: '>='
+;
+
+OPERATOR_LT
+: '<'
+;
+
+OPERATOR_LTE
+: '<='
+;
+
+OPERATOR_EQ
+: '=' | '=='
+;
+
+OPERATOR_NEQ
+: '!=' | '<>'
+;
+
+// operator ***
+STAR
+: '*'
+;
+
+MINUS
+: '-'
+;
+
+PLUS
+: '+'
+;
+
+DIVIDE
+: '/'
+;
+
+// symbol ***
+SEMI
+: ';'
+;
+
+DOT
+: '.'
+;
+
+COMMA
+: ','
+;
+
+LR_BRACKET
+: '('
+;
+
+RR_BRACKET
+: ')'
+;
+
+LS_BRACKET
+: '['
+;
+
+RS_BRACKET
+: ']'
+;
+
+STRING_LITERAL
+   : DQUOTA_STRING
+   | SQUOTA_STRING
+   ;
+
+fragment DQUOTA_STRING
+   : '"' ( '\\'. | '""' | ~('"'| '\\') )* '"'
+   ;
+
+fragment SQUOTA_STRING
+   : 

[GitHub] [incubator-iotdb] Genius-pig commented on issue #440: reconstruct antlrv3 grammar to improve performance

2019-10-15 Thread GitBox
Genius-pig commented on issue #440: reconstruct antlrv3 grammar to improve 
performance
URL: https://github.com/apache/incubator-iotdb/pull/440#issuecomment-542104524
 
 
   >  No "=>" operators anymore.
   "=>" operator means always execute predicate. Removing all of them is a real 
improvement? Syntactic predicates were used to work around a prediction 
weakness in ANTLR 3. 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Genius-pig edited a comment on issue #440: reconstruct antlrv3 grammar to improve performance

2019-10-15 Thread GitBox
Genius-pig edited a comment on issue #440: reconstruct antlrv3 grammar to 
improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#issuecomment-542104524
 
 
   >  No "=>" operators anymore.
   
   "=>" operator means always execute predicate. Removing all of them is a real 
improvement? Syntactic predicates were used to work around a prediction 
weakness in ANTLR 3. 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #417: [IOTDB-234] Refactor TsFile storage on HDFS

2019-10-15 Thread GitBox
jt2594838 commented on a change in pull request #417: [IOTDB-234] Refactor 
TsFile storage on HDFS
URL: https://github.com/apache/incubator-iotdb/pull/417#discussion_r334809838
 
 

 ##
 File path: 
tsfile/src/main/java/org/apache/iotdb/tsfile/fileSystem/HDFSFactory.java
 ##
 @@ -0,0 +1,178 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.tsfile.fileSystem;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.lang.reflect.Constructor;
+import java.lang.reflect.InvocationTargetException;
+import java.net.URI;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class HDFSFactory implements FSFactory {
+
+  private static final Logger logger = 
LoggerFactory.getLogger(HDFSFactory.class);
+  private static Class clazz;
+  private static Constructor constructorWithPathname;
+  private static Constructor constructorWithParentStringAndChild;
+  private static Constructor constructorWithParentFileAndChild;
+  private static Constructor constructorWithUri;
+
+  static {
+try {
+  clazz = Class.forName("org.apache.iotdb.tsfile.fileSystem.HDFSFile");
+  constructorWithPathname = clazz.getConstructor(String.class);
+  constructorWithParentStringAndChild = clazz.getConstructor(String.class, 
String.class);
+  constructorWithParentFileAndChild = clazz.getConstructor(File.class, 
String.class);
+  constructorWithUri = clazz.getConstructor(URI.class);
+} catch (ClassNotFoundException | NoSuchMethodException e) {
+  logger.error(
+  "Failed to get Hadoop file system. Please check your dependency of 
Hadoop module.", e);
+}
+  }
+
+  public File getFile(String pathname) {
+try {
+  return (File) constructorWithPathname.newInstance(pathname);
+} catch (InstantiationException | InvocationTargetException | 
IllegalAccessException e) {
+  logger.error(
+  "Failed to get file: {}. Please check your dependency of Hadoop 
module.", pathname, e);
+  return null;
+}
+  }
+
+  public File getFile(String parent, String child) {
+try {
+  return (File) constructorWithParentStringAndChild.newInstance(parent, 
child);
+} catch (InstantiationException | InvocationTargetException | 
IllegalAccessException e) {
+  logger.error(
+  "Failed to get file: {}" + File.separator
+  + "{}. Please check your dependency of Hadoop module.", parent, 
child, e);
+  return null;
+}
+  }
+
+  public File getFile(File parent, String child) {
+try {
+  return (File) constructorWithParentFileAndChild.newInstance(parent, 
child);
+} catch (InstantiationException | InvocationTargetException | 
IllegalAccessException e) {
+  logger.error(
+  "Failed to get file: {}" + File.separator
+  + "{}. Please check your dependency of Hadoop module.", 
parent.getAbsolutePath(),
+  child, e);
+  return null;
+}
+  }
+
+  public File getFile(URI uri) {
+try {
+  return (File) constructorWithUri.newInstance(uri);
+} catch (InstantiationException | InvocationTargetException | 
IllegalAccessException e) {
+  logger.error(
+  "Failed to get file: {}. Please check your dependency of Hadoop 
module.",
+  uri.toString(), e);
+  return null;
+}
+  }
+
+  public BufferedReader getBufferedReader(String filePath) {
+try {
+  return (BufferedReader) clazz.getMethod("getBufferedReader", 
String.class)
+  .invoke(clazz.newInstance(), filePath);
 
 Review comment:
   I think you can save the getter Methods as you have saved the Constructors.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui opened a new pull request #451: [IOTDB-252]add/fix shell and bat for TsFileSketchTool/TsFileResourcePrinter

2019-10-15 Thread GitBox
LeiRui opened a new pull request #451: [IOTDB-252]add/fix shell and bat for 
TsFileSketchTool/TsFileResourcePrinter
URL: https://github.com/apache/incubator-iotdb/pull/451
 
 
   Usage Examples:
   - ./print-tsfile-resource-files.sh /home/rl/fileDir
   ```
   -
   Starting Printing the TsFileResources
   -
   15:44:46.640 [main] WARN org.apache.iotdb.db.conf.IoTDBDescriptor - Cannot 
find IOTDB_HOME or IOTDB_CONF environment variable when loading config file 
iotdb-engine.properties, use default configuration
   15:44:46.655 [main] WARN 
org.apache.iotdb.tsfile.common.conf.TSFileDescriptor - Failed to find config 
file tsfile-format.properties at classpath, use default configuration
   analyzing /home/rl/tmp/1571064257513-101.tsfile ...
   device root.vehicle.d0, start time 3000 
(1970-01-01T08:00:03+08:00[Asia/Shanghai]), end time 100999 
(1970-01-01T08:01:40.999+08:00[Asia/Shanghai])
   analyzing the resource file finished.
   ```
   - print-tsfile-resource-files.bat /home/rl/fileDir is similar
   
   - print-tsfile-sketch.sh /home/rl/fileDir/1571064257513-101.tsfile
   ```
   -
   Starting Printing the TsFile Sketch
   -
   filename:/home/rl/tmp/1571064257513-101.tsfile
   outFile:TsFile_sketch_view.txt
   15:47:24.881 [main] WARN 
org.apache.iotdb.tsfile.common.conf.TSFileDescriptor - Failed to find config 
file tsfile-format.properties at classpath, use default configuration
    TsFile Sketch 

   file path: /home/rl/tmp/1571064257513-101.tsfile
   file length: 187349
   
   POSITION|CONTENT
    ---
  0|[magic head] TsFilev0.8.0
   >[Chunk Group] of root.vehicle.d0 begins at pos 
12, ends at pos 186469, version:102, num of Chunks:6
 12|[Chunk] of s3, numOfPoints:10600, time 
range:[3000,13599], tsDataType:TEXT, 

TsDigest:[min_value:A,max_value:E,first_value:A,last_value:E,sum_value:0.0]
   |[marker] 1
   |[ChunkHeader]
   |num of pages: 11
  55718|[Chunk] of s4, numOfPoints:10600, time 
range:[3000,13599], tsDataType:BOOLEAN, 

TsDigest:[min_value:false,max_value:true,first_value:true,last_value:false,sum_value:0.0]
   |[marker] 1
   |[ChunkHeader]
   |num of pages: 11
  68848|[Chunk] of s5, numOfPoints:10600, time 
range:[3000,13599], tsDataType:DOUBLE, 

TsDigest:[min_value:3000.0,max_value:13599.0,first_value:3000.0,last_value:13599.0,sum_value:8.79747E7]
   |[marker] 1
   |[ChunkHeader]
   |num of pages: 11
  98474|[Chunk] of s0, numOfPoints:21900, time 
range:[3000,100999], tsDataType:INT32, 

TsDigest:[min_value:0,max_value:99,first_value:0,last_value:19,sum_value:889750.0]
   |[marker] 1
   |[ChunkHeader]
   |num of pages: 22
 123369|[Chunk] of s1, numOfPoints:21900, time 
range:[3000,100999], tsDataType:INT64, 

TsDigest:[min_value:0,max_value:39,first_value:8,last_value:19,sum_value:300386.0]
   |[marker] 1
   |[ChunkHeader]
   |num of pages: 22
 144741|[Chunk] of s2, numOfPoints:21900, time 
range:[3000,100999], tsDataType:FLOAT, 

TsDigest:[min_value:0.0,max_value:122.0,first_value:8.0,last_value:52.0,sum_value:778581.0]
   |[marker] 1
   |[ChunkHeader]
   |num of pages: 22
   |[marker] 0
   |[Chunk Group Footer]
   <[Chunk Group] of root.vehicle.d0 ends
 186469|[marker] 2
 186470|[TsDeviceMetadata] of root.vehicle.d0, 
startTime:3000, endTime:100999
   |num of ChunkGroupMetaData: 1
 187133|[TsFileMetaData]
   |num of TsDeviceMetadataIndex: 1
   |num of measurementSchema: 6
 187333|[TsFileMetaDataSize] 200
 187337|[magic tail] TsFilev0.8.0
 187349|

[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #447: [IOTDB-250] exclude .mvn/wrapper/maven-wrapper.jar

2019-10-15 Thread GitBox
qiaojialin commented on a change in pull request #447: [IOTDB-250] exclude 
.mvn/wrapper/maven-wrapper.jar
URL: https://github.com/apache/incubator-iotdb/pull/447#discussion_r334776744
 
 

 ##
 File path: .gitignore
 ##
 @@ -79,7 +82,6 @@ grafana/data/
 
 grafana/data/test.csv
 **/lib/
-*.jar
 
 Review comment:
   it is duplicated...


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] samperson1997 commented on issue #417: [IOTDB-234] Refactor TsFile storage on HDFS

2019-10-14 Thread GitBox
samperson1997 commented on issue #417: [IOTDB-234] Refactor TsFile storage on 
HDFS
URL: https://github.com/apache/incubator-iotdb/pull/417#issuecomment-542042012
 
 
   > If you insist on this design, at least you should save the constructor.
   > And this test will show you that this is still slower compared with a 
direct call to new().
   > 
[ReflectionTest.txt](https://github.com/apache/incubator-iotdb/files/3722994/ReflectionTest.txt)
   
   Thanks for the suggested design  @jt2594838 ! I have accepted it by using 
two factories (`LocalFSFactory` and `HDFSFactory`) with `FSFactoryProducer` to 
produce either one of them according to user config. I think this design is so 
much better for being less heavy and slow and being more scalable for future 
design. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-14 Thread GitBox
Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar 
to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334737366
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/LogicalGenerator.java
 ##
 @@ -1,14 +1,14 @@
-/*
+/**
 
 Review comment:
   I replaced /** with /*. Thanks for your suggestion.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-14 Thread GitBox
Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar 
to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334737388
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/constant/TqlParserConstant.java
 ##
 @@ -1,4 +1,4 @@
-/*
+/**
 
 Review comment:
   I replaced /** with /*. Thanks for your suggestion.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-14 Thread GitBox
Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar 
to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334737286
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/LogicalGenerator.java
 ##
 @@ -6,9 +6,9 @@
  * to you under the Apache License, Version 2.0 (the
  * "License"); you may not use this file except in compliance
  * with the License.  You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
+ * 
 
 Review comment:
   I removed the ""s. Thanks for your suggestion.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-14 Thread GitBox
Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar 
to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334736715
 
 

 ##
 File path: 
server/src/test/java/org/apache/iotdb/db/qp/plan/LogicalPlanSmallTest.java
 ##
 @@ -197,4 +197,21 @@ public void testLimit2()
 // expected to throw LogicalOperatorException: LIMIT : N must be a 
positive integer and can not be zero.
   }
 
+  @Test
+  public void testDeleteStorageGroup()
+  throws QueryProcessorException, ArgsErrorException, 
MetadataErrorException {
+String sqlStr = "delete storage group root.vehicle.d1";
+AstNode astTree;
+try {
+  astTree = ParseGenerator.generateAST(sqlStr); // parse string to ASTTree
+} catch (ParseException e) {
+  // e.printStackTrace();
+  throw new IllegalASTFormatException(
+  "parsing error,statement: " + sqlStr + " .message:" + 
e.getMessage());
+}
+AstNode astNode = ParseUtils.findRootNonNullToken(astTree);
+RootOperator operator = generator.getLogicalPlan(astNode);
+// expected to throw LogicalOperatorException: LIMIT : N must be a 
positive integer and can not be zero.
 
 Review comment:
   I added some expectations. Thanks for your suggestion.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-14 Thread GitBox
Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar 
to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334732471
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/LogicalGenerator.java
 ##
 @@ -829,42 +819,56 @@ private long parseTimeUnit(AstNode node) throws 
LogicalOperatorException {
 return timeInterval;
   }
 
+//  private void print(AstNode node, AstNode parent, int level) {
+//if(parent == null) System.out.println("["+level+++"]\t"+ "root\t" + 
node.getText());
+//else System.out.println("["+level+++"]\t"+ parent.getText()+"\t" + 
node.getText());
+//for(Object o : node.getChildren()){
+//  print((AstNode) o, node, level);
+//}
+//  }
+
   private Pair parseLeafNode(AstNode node) throws 
LogicalOperatorException {
 if (node.getChildCount() != 2) {
   throw new LogicalOperatorException(
-  "error format in SQL statement, please check whether SQL statement 
is correct.");
+  "error format in SQL statement, please check whether SQL 
statement is correct.");
 }
 AstNode col = node.getChild(0);
-if (col.getType() != TSParser.TOK_PATH) {
+if (col.getType() != TqlParser.TOK_PATH) {
   throw new LogicalOperatorException(
-  "error format in SQL statement, please check whether SQL statement 
is correct.");
+  "error format in SQL statement, please check whether SQL 
statement is correct.");
 }
 Path seriesPath = parsePath(col);
 AstNode rightKey = node.getChild(1);
 String seriesValue;
-if (rightKey.getType() == TSParser.TOK_PATH) {
-  seriesValue = parsePath(rightKey).getFullPath();
-} else if (rightKey.getType() == TSParser.TOK_DATETIME) {
+if(rightKey.getChild(0).getType() == TqlParser.TOK_DATETIME){
   if (!seriesPath.equals(SQLConstant.RESERVED_TIME)) {
 throw new LogicalOperatorException("Date can only be used to time");
   }
-  seriesValue = parseTokenTime(rightKey);
-} else if (rightKey.getType() == TSParser.TOK_FLOAT_COMB) {
-  seriesValue = parseTokens(rightKey);
-} else {
-  seriesValue = rightKey.getText();
+  seriesValue = parseTokenTime(rightKey.getChild(0));
+}else{
+  seriesValue = cascadeChildrenText(rightKey);
 }
+//if (rightKey.getType() == TqlParser.TOK_PATH) {
 
 Review comment:
   The useless codes are removed. Thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] HTHou opened a new pull request #450: modify documents

2019-10-14 Thread GitBox
HTHou opened a new pull request #450: modify documents
URL: https://github.com/apache/incubator-iotdb/pull/450
 
 
   Changed the structure of the User Guide. 
   Fixed some link errors.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-14 Thread GitBox
jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 
grammar to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334729689
 
 

 ##
 File path: 
server/src/test/java/org/apache/iotdb/db/qp/plan/LogicalPlanSmallTest.java
 ##
 @@ -197,4 +197,21 @@ public void testLimit2()
 // expected to throw LogicalOperatorException: LIMIT : N must be a 
positive integer and can not be zero.
   }
 
+  @Test
+  public void testDeleteStorageGroup()
+  throws QueryProcessorException, ArgsErrorException, 
MetadataErrorException {
+String sqlStr = "delete storage group root.vehicle.d1";
+AstNode astTree;
+try {
+  astTree = ParseGenerator.generateAST(sqlStr); // parse string to ASTTree
+} catch (ParseException e) {
+  // e.printStackTrace();
+  throw new IllegalASTFormatException(
+  "parsing error,statement: " + sqlStr + " .message:" + 
e.getMessage());
+}
+AstNode astNode = ParseUtils.findRootNonNullToken(astTree);
+RootOperator operator = generator.getLogicalPlan(astNode);
+// expected to throw LogicalOperatorException: LIMIT : N must be a 
positive integer and can not be zero.
 
 Review comment:
   Hi, this Test has not expectation?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-14 Thread GitBox
jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 
grammar to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334729096
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/LogicalGenerator.java
 ##
 @@ -829,42 +819,56 @@ private long parseTimeUnit(AstNode node) throws 
LogicalOperatorException {
 return timeInterval;
   }
 
+//  private void print(AstNode node, AstNode parent, int level) {
+//if(parent == null) System.out.println("["+level+++"]\t"+ "root\t" + 
node.getText());
+//else System.out.println("["+level+++"]\t"+ parent.getText()+"\t" + 
node.getText());
+//for(Object o : node.getChildren()){
+//  print((AstNode) o, node, level);
+//}
+//  }
+
   private Pair parseLeafNode(AstNode node) throws 
LogicalOperatorException {
 if (node.getChildCount() != 2) {
   throw new LogicalOperatorException(
-  "error format in SQL statement, please check whether SQL statement 
is correct.");
+  "error format in SQL statement, please check whether SQL 
statement is correct.");
 }
 AstNode col = node.getChild(0);
-if (col.getType() != TSParser.TOK_PATH) {
+if (col.getType() != TqlParser.TOK_PATH) {
   throw new LogicalOperatorException(
-  "error format in SQL statement, please check whether SQL statement 
is correct.");
+  "error format in SQL statement, please check whether SQL 
statement is correct.");
 }
 Path seriesPath = parsePath(col);
 AstNode rightKey = node.getChild(1);
 String seriesValue;
-if (rightKey.getType() == TSParser.TOK_PATH) {
-  seriesValue = parsePath(rightKey).getFullPath();
-} else if (rightKey.getType() == TSParser.TOK_DATETIME) {
+if(rightKey.getChild(0).getType() == TqlParser.TOK_DATETIME){
   if (!seriesPath.equals(SQLConstant.RESERVED_TIME)) {
 throw new LogicalOperatorException("Date can only be used to time");
   }
-  seriesValue = parseTokenTime(rightKey);
-} else if (rightKey.getType() == TSParser.TOK_FLOAT_COMB) {
-  seriesValue = parseTokens(rightKey);
-} else {
-  seriesValue = rightKey.getText();
+  seriesValue = parseTokenTime(rightKey.getChild(0));
+}else{
+  seriesValue = cascadeChildrenText(rightKey);
 }
+//if (rightKey.getType() == TqlParser.TOK_PATH) {
 
 Review comment:
   hm... useless codes?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-14 Thread GitBox
jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 
grammar to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334728810
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/LogicalGenerator.java
 ##
 @@ -6,9 +6,9 @@
  * to you under the Apache License, Version 2.0 (the
  * "License"); you may not use this file except in compliance
  * with the License.  You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
+ * 
 
 Review comment:
   same problem


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 commented on issue #339: [IOTDB-158] add metrics service

2019-10-14 Thread GitBox
jixuan1989 commented on issue #339: [IOTDB-158] add metrics service
URL: https://github.com/apache/incubator-iotdb/pull/339#issuecomment-542006976
 
 
   One more thing, if there is a markdown document for introducing the tool, it 
could be better.  You can put the markdown file under: 
`docs/Documentation/UserGuide/9-System Tools` (For Chinese version, put it 
under `docs/Documentation-CHN/UserGuide/9-System Tools` 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 commented on issue #339: [IOTDB-158] add metrics service

2019-10-14 Thread GitBox
jixuan1989 commented on issue #339: [IOTDB-158] add metrics service
URL: https://github.com/apache/incubator-iotdb/pull/339#issuecomment-542006014
 
 
   > Usually where we need something like this we download the resource as part 
of the build and take the parts we need and then reference that file. By 
actually adding it to the repo, I guess you have to mention it in the LICENSE 
or NOTICE files.
   
   Declare it in the LICENSE and NOTICE is a choice. You can find a better way 
in the following discussion:
   
   
https://lists.apache.org/thread.html/29be94b4fa6d201445e736946d1e04f29050349ab5ee0a72717e8d84@%3Cdev.iotdb.apache.org%3E
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-14 Thread GitBox
jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 
grammar to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334715524
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/LogicalGenerator.java
 ##
 @@ -1,14 +1,14 @@
-/*
+/**
 
 Review comment:
   use /*
   and seems no "" marker


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-14 Thread GitBox
jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 
grammar to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334715385
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/constant/TqlParserConstant.java
 ##
 @@ -1,4 +1,4 @@
-/*
+/**
 
 Review comment:
   use /*  rather than /**


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 commented on issue #445: generate cpp, go, and python thrift files under service-rpc

2019-10-14 Thread GitBox
jixuan1989 commented on issue #445: generate cpp, go, and python thrift files 
under service-rpc
URL: https://github.com/apache/incubator-iotdb/pull/445#issuecomment-541988772
 
 
   > This is fine.
   > But programmers of other languages do not usually have maven installed, so 
I doubt whether this is that useful.
   > And I would appreciate it if you add a parameter like `mvn 
generate-sources py` to let the user select what they want to build instead of 
building all at once.
   
   Actually, for users who do not install maven, they can use `./mvnw.sh 
generate-sources` ... 
   allowing claiming which language codes users want is good, but I think it is 
ok now because generating codes is very fast..
   
   What interests me is that  @chrisdutz  said "this is just generating codes, 
rather compile them as a module"... 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on issue #440: reconstruct antlrv3 grammar to improve performance

2019-10-14 Thread GitBox
LeiRui commented on issue #440: reconstruct antlrv3 grammar to improve 
performance
URL: https://github.com/apache/incubator-iotdb/pull/440#issuecomment-541576488
 
 
   > "+" is not allowed to identify a positive value
   
   What about updating docs where necessary?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on issue #445: generate cpp, go, and python thrift files under service-rpc

2019-10-14 Thread GitBox
jt2594838 commented on issue #445: generate cpp, go, and python thrift files 
under service-rpc
URL: https://github.com/apache/incubator-iotdb/pull/445#issuecomment-541538420
 
 
   This is fine.
   But programmers of other languages do not usually have maven installed, so I 
doubt whether this is that useful.
   And I would appreciate it if you add a parameter like `mvn generate-sources 
py` to let the user select what they want to build instead of building all at 
once.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #444: Add python client example in user guide

2019-10-14 Thread GitBox
jt2594838 commented on a change in pull request #444: Add python client example 
in user guide
URL: https://github.com/apache/incubator-iotdb/pull/444#discussion_r334359259
 
 

 ##
 File path: docs/Documentation-CHN/OtherMaterial-Examples.md
 ##
 @@ -81,3 +81,172 @@ public class IotdbHelloWorld {
 
 
 ```
+# 使用Python进行连接
+## 介绍
+以下例子展示了如何使用Thrift编译出Python的库,并调用该库连接IoTDB并进行一些基本操作。在Windows和Linux上的
+使用大同小异,但请注意二者在路径等方面的细微区别。
+
+## Prerequisites
+建议使用python3.7或更高级的版本。
+
+您需要预先安装Thrift才能将我们的Thrift源文件编译为Python的库代码。下面是Thrift的官方安装教程,最终,您需要
+得到一个Thrift的二进制可执行文件。
+```
+http://thrift.apache.org/docs/install/
+```
+
+## Compile the thrift library
+如果您已经将Thrift的可执行文件放到了您的PATH环境变量的目录下,您可以直接运行脚本`client-py/compile.sh`或
+`client-py/compile.bat`,否则您需要修改脚本中的变量`THRIFT_EXE`来指向您的Thrift可执行文件。这会在脚本文件
+的同一目录下生成`target`文件夹,该文件夹即包含了IoTDB的Python版本的Thrift库,当您将该文件夹加入到您的`PYTHONPATH`
+环境变量里后,您就可以在自己的代码中引用该库。请注意,上述脚本通过相对路径寻找我们的Thrift的源文件,如果您改变
+脚本所在位置,脚本可能失效。
+
+或者,如果您对Thrift的基本用法有所了解,您就不需要下载整个项目,您可以仅下载我们的Thrift源文件`service-rpc\src\main\thrift\rpc.thrift`,
+并使用命令`thrift -gen py -out ./target rpc.thrift`来编译Thrift库。
+
+## Example
+下面是一段使用生成的Thrift库连接IoTDB的示例代码,请在编写您自己的代码之前仔细地阅读。
+```python
+
+import sys, struct
+sys.path.append("../target")
+
+from thrift.protocol import TBinaryProtocol
+from thrift.transport import TSocket, TTransport
+
+from rpc.TSIService import Client, TSCreateTimeseriesReq, TSInsertionReq, \
+TSBatchInsertionReq, TSExecuteStatementReq, \
+TS_SessionHandle, TSHandleIdentifier, TSOpenSessionReq, TSQueryDataSet, \
+TSFetchResultsReq, TSCloseOperationReq, \
+TSCloseSessionReq, TSProtocolVersion
+
+TSDataType = {
+'BOOLEAN' : 0,
+'INT32' : 1,
+'INT64' : 2,
+'FLOAT' : 3,
+'DOUBLE' : 4,
+'TEXT' : 5
+}
+
+TSEncoding = {
+'PLAIN' : 0,
+'PLAIN_DICTIONARY' : 1,
+'RLE' : 2,
+'DIFF' : 3,
+'TS_2DIFF' : 4,
+'BITMAP' : 5,
+'GORILLA' : 6,
+'REGULAR' : 7
+}
+
+Compressor = {
+'UNCOMPRESSED' : 0,
+'SNAPPY' : 1,
+'GZIP' : 2,
+'LZO' : 3,
+'SDT' : 4,
+'PAA' : 5,
+'PLA' : 6
+}
+
+
+if __name__ == '__main__':
+ip = "localhost"
+port = "6667"
+username = 'root'
+password = 'root'
+# Make socket
+transport = TSocket.TSocket(ip, port)
+
+# Buffering is critical. Raw sockets are very slow
 
 Review comment:
   Other example codes also use English comments and I do not think this is a 
big issue.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on issue #444: Add python client example in user guide

2019-10-14 Thread GitBox
jt2594838 commented on issue #444: Add python client example in user guide
URL: https://github.com/apache/incubator-iotdb/pull/444#issuecomment-541537156
 
 
   > considering #445 ?
   
   I suggest keeping these scripts as you can go without maven.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #430: [IOTDB-193]Create schema automatically

2019-10-14 Thread GitBox
jt2594838 commented on a change in pull request #430: [IOTDB-193]Create schema 
automatically
URL: https://github.com/apache/incubator-iotdb/pull/430#discussion_r334353560
 
 

 ##
 File path: server/src/main/java/org/apache/iotdb/db/utils/DeviceIdCache.java
 ##
 @@ -0,0 +1,53 @@
+package org.apache.iotdb.db.utils;
+
+import org.apache.iotdb.db.exception.PathErrorException;
+import org.apache.iotdb.db.exception.StorageGroupException;
+import org.apache.iotdb.tsfile.exception.cache.CacheException;
+
+import java.util.List;
+import java.util.concurrent.CopyOnWriteArrayList;
+
+public abstract class DeviceIdCache {
 
 Review comment:
   The number of devices in a system is rather limited and holding all in 
memory is not a burden, so I do not suggest making it an LRU cache for 
meaningless overhead. 
   Nevertheless, I seriously doubt the performance of using a List, as the 
`contains()` method can be extremely slow even when there are only thousands of 
devices (not to mention you are using a COWList). I strongly recommend using a 
HashSet.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #430: [IOTDB-193]Create schema automatically

2019-10-14 Thread GitBox
jt2594838 commented on a change in pull request #430: [IOTDB-193]Create schema 
automatically
URL: https://github.com/apache/incubator-iotdb/pull/430#discussion_r334354740
 
 

 ##
 File path: 
server/src/test/java/org/apache/iotdb/db/integration/IoTDBAutoCreateSchemaIT.java
 ##
 @@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.integration;
+
+import org.apache.iotdb.db.conf.IoTDBConfig;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
+import org.apache.iotdb.db.service.IoTDB;
+import org.apache.iotdb.db.utils.EnvironmentUtils;
+import org.apache.iotdb.jdbc.Config;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.sql.Connection;
+import java.sql.DatabaseMetaData;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.ResultSetMetaData;
+import java.sql.Statement;
+
+/**
+ * Notice that, all test begins with "IoTDB" is integration test. All test 
which will start the IoTDB server should be
+ * defined as integration test.
+ */
+public class IoTDBAutoCreateSchemaIT {
+
+  private static IoTDB daemon;
+
+  private static IoTDBConfig conf = IoTDBDescriptor.getInstance().getConfig();
+
+  @Before
+  public void setUp() throws Exception {
+EnvironmentUtils.closeStatMonitor();
+daemon = IoTDB.getInstance();
+daemon.active();
+EnvironmentUtils.envSetUp();
+conf.setAutoCreateSchemaEnabled(true);
+  }
+
+  @After
+  public void tearDown() throws Exception {
+daemon.stop();
+EnvironmentUtils.cleanEnv();
+  }
+
+  /**
+   * create timeseries without setting storage group
+   */
+  @Test
+  public void createTimeseriesTest() throws ClassNotFoundException {
+String[] sqls = {"CREATE TIMESERIES root.sg0.d1.s2 WITH 
DATATYPE=INT32,ENCODING=RLE",
+"INSERT INTO root.sg0.d1(timestamp,s2) values(1,123)",
+};
+executeSQL(sqls);
+  }
+
+  /**
+   * insert data when storage group has been set but timeseries hasn't been 
created
+   */
+  @Test
+  public void insertTest1() throws ClassNotFoundException {
+String[] sqls = {"SET STORAGE GROUP TO root.sg0",
+"INSERT INTO root.sg0.d1(timestamp,s2) values(1,123.123)",
+"INSERT INTO root.sg0.d1(timestamp,s3) values(1,\"abc\")",
+};
+executeSQL(sqls);
+  }
+
+  /**
+   * insert data when storage group hasn't been set and timeseries hasn't been 
created
+   */
+  @Test
+  public void insertTest2() throws ClassNotFoundException {
+String[] sqls = {"INSERT INTO root.sg0.d1(timestamp,s3) values(1,\"abc\")",
+"INSERT INTO root.sg0.d2(timestamp,s2) values(1,123.123)",
+"INSERT INTO root.sg0.d3(timestamp,s4) values(1,123456)",
+};
+executeSQL(sqls);
+  }
 
 Review comment:
   Add a test where you insert a non-exist time series with different data 
types.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #430: [IOTDB-193]Create schema automatically

2019-10-14 Thread GitBox
jt2594838 commented on a change in pull request #430: [IOTDB-193]Create schema 
automatically
URL: https://github.com/apache/incubator-iotdb/pull/430#discussion_r334356054
 
 

 ##
 File path: 
server/src/test/java/org/apache/iotdb/db/utils/TypeInferenceUtilsTest.java
 ##
 @@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.utils;
+
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.junit.Test;
+
+import static org.junit.Assert.assertEquals;
+
+public class TypeInferenceUtilsTest {
+
+  @Test
+  public void isNumberTest() {
+String[] values = {"123", "123.123", "-123.123", "+123.123", ".123", 
String.valueOf(Integer.MAX_VALUE),
+String.valueOf(Integer.MIN_VALUE), "abc", "123.", "123a", ".", 
"123.1.23"};
+boolean[] results = {true, true, true, true, true, true,
+true, false, false, false, false, false};
 
 Review comment:
   Add coverage of scientific notation.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #430: [IOTDB-193]Create schema automatically

2019-10-14 Thread GitBox
jt2594838 commented on a change in pull request #430: [IOTDB-193]Create schema 
automatically
URL: https://github.com/apache/incubator-iotdb/pull/430#discussion_r334353987
 
 

 ##
 File path: 
server/src/test/java/org/apache/iotdb/db/integration/IoTDBAutoCreateSchemaIT.java
 ##
 @@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.integration;
+
+import org.apache.iotdb.db.conf.IoTDBConfig;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
+import org.apache.iotdb.db.service.IoTDB;
+import org.apache.iotdb.db.utils.EnvironmentUtils;
+import org.apache.iotdb.jdbc.Config;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.sql.Connection;
+import java.sql.DatabaseMetaData;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.ResultSetMetaData;
+import java.sql.Statement;
+
+/**
+ * Notice that, all test begins with "IoTDB" is integration test. All test 
which will start the IoTDB server should be
+ * defined as integration test.
+ */
+public class IoTDBAutoCreateSchemaIT {
+
+  private static IoTDB daemon;
+
+  private static IoTDBConfig conf = IoTDBDescriptor.getInstance().getConfig();
+
+  @Before
+  public void setUp() throws Exception {
+EnvironmentUtils.closeStatMonitor();
+daemon = IoTDB.getInstance();
+daemon.active();
+EnvironmentUtils.envSetUp();
+conf.setAutoCreateSchemaEnabled(true);
+  }
+
+  @After
+  public void tearDown() throws Exception {
+daemon.stop();
+EnvironmentUtils.cleanEnv();
 
 Review comment:
   Recover in `tearDown()` what configs you have modified in `setUp()`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] fanhualta commented on a change in pull request #430: [IOTDB-193]Create schema automatically

2019-10-14 Thread GitBox
fanhualta commented on a change in pull request #430: [IOTDB-193]Create schema 
automatically
URL: https://github.com/apache/incubator-iotdb/pull/430#discussion_r33438
 
 

 ##
 File path: server/src/main/java/org/apache/iotdb/db/utils/DeviceIdCache.java
 ##
 @@ -0,0 +1,53 @@
+package org.apache.iotdb.db.utils;
+
+import org.apache.iotdb.db.exception.PathErrorException;
+import org.apache.iotdb.db.exception.StorageGroupException;
+import org.apache.iotdb.tsfile.exception.cache.CacheException;
+
+import java.util.List;
+import java.util.concurrent.CopyOnWriteArrayList;
+
+public abstract class DeviceIdCache {
+
+  private int cacheSize;
+  private List cache;
+
+  public DeviceIdCache(int cacheSize) {
+this.cacheSize = cacheSize;
+this.cache= new CopyOnWriteArrayList<>();
+  }
+
+  public void get(String deviceId) throws CacheException, 
StorageGroupException, PathErrorException {
+if (!cache.contains(deviceId)) {
+  randomRemoveObjectIfCacheIsFull();
+  loadObject(deviceId);
+  cache.add(deviceId);
+}
+  }
+
+  private void randomRemoveObjectIfCacheIsFull() throws CacheException {
+if (cache.size() == this.cacheSize) {
+  removeFirstObject();
+}
+  }
+
+  private void removeFirstObject() throws CacheException {
+if (cache.size() == 0) {
+  return;
+}
+String deviceId = cache.iterator().next();
+cache.remove(deviceId);
 
 Review comment:
   I think it's a little bit complicated. How about
   ```suggestion
   if (cache.size() != 0) {
 cache.remove(0);
   }
   ```
   What's more, I think it's better not to split `RemoveObjectIfCacheIsFull` 
into two methods.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] fanhualta commented on a change in pull request #430: [IOTDB-193]Create schema automatically

2019-10-14 Thread GitBox
fanhualta commented on a change in pull request #430: [IOTDB-193]Create schema 
automatically
URL: https://github.com/apache/incubator-iotdb/pull/430#discussion_r334342259
 
 

 ##
 File path: server/src/main/java/org/apache/iotdb/db/utils/DeviceIdCache.java
 ##
 @@ -0,0 +1,53 @@
+package org.apache.iotdb.db.utils;
+
+import org.apache.iotdb.db.exception.PathErrorException;
+import org.apache.iotdb.db.exception.StorageGroupException;
+import org.apache.iotdb.tsfile.exception.cache.CacheException;
+
+import java.util.List;
+import java.util.concurrent.CopyOnWriteArrayList;
+
+public abstract class DeviceIdCache {
+
+  private int cacheSize;
+  private List cache;
+
+  public DeviceIdCache(int cacheSize) {
+this.cacheSize = cacheSize;
+this.cache= new CopyOnWriteArrayList<>();
+  }
+
+  public void get(String deviceId) throws CacheException, 
StorageGroupException, PathErrorException {
+if (!cache.contains(deviceId)) {
+  randomRemoveObjectIfCacheIsFull();
+  loadObject(deviceId);
+  cache.add(deviceId);
+}
+  }
+
+  private void randomRemoveObjectIfCacheIsFull() throws CacheException {
+if (cache.size() == this.cacheSize) {
+  removeFirstObject();
 
 Review comment:
   I don't think it's meaningful tp remove the first object with the name 
"random".
   It's better to rename the method to `removeObjectIfCacheIsFull()`


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] fanhualta commented on a change in pull request #430: [IOTDB-193]Create schema automatically

2019-10-14 Thread GitBox
fanhualta commented on a change in pull request #430: [IOTDB-193]Create schema 
automatically
URL: https://github.com/apache/incubator-iotdb/pull/430#discussion_r334341654
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/executor/QueryProcessExecutor.java
 ##
 @@ -661,4 +734,63 @@ private AuthDataSet executeListUserPrivileges(IAuthorizer 
authorizer, String use
 }
 return dataSet;
   }
+
+  /**
+   * Set storage group when creating schema automatically is enable
+   */
+  private void setStorageGroupToMTreeByAutoLevel(String deviceId) throws 
PathErrorException, MetadataErrorException {
+IoTDBConfig conf = IoTDBDescriptor.getInstance().getConfig();
+String storageGroupName = 
mManager.getStorageGroupNameByAutoLevel(deviceId, 
conf.getDefaultStorageGroupLevel());
+mManager.setStorageGroupToMTree(storageGroupName);
+  }
+
+  private void addPathListToMTree(String deviceId, String[] measurementList, 
Object[] values)
+  throws MetadataErrorException, PathErrorException, 
StorageEngineException {
+for (int i = 0; i < measurementList.length; i++) {
+  addPathToMTree(deviceId, measurementList[i], values[i]);
+}
+  }
+
+  private void addPathToMTree(String deviceId, String measurementId, Object 
value)
+  throws PathErrorException, MetadataErrorException, 
StorageEngineException {
+String fullPath = deviceId + "." + measurementId;
 
 Review comment:
   ```suggestion
   String fullPath = deviceId +  IoTDBConstant.PATH_SEPARATOR + 
measurementId;
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] fanhualta commented on a change in pull request #430: [IOTDB-193]Create schema automatically

2019-10-14 Thread GitBox
fanhualta commented on a change in pull request #430: [IOTDB-193]Create schema 
automatically
URL: https://github.com/apache/incubator-iotdb/pull/430#discussion_r334341958
 
 

 ##
 File path: server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
 ##
 @@ -48,6 +51,8 @@
 public class MManager {
 
   private static final Logger logger = LoggerFactory.getLogger(MManager.class);
+  private static final String PATH_SEPARATOR = ".";
 
 Review comment:
   It‘s duplicated with `IoTDBConstant.PATH_SEPARATOR` 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 opened a new pull request #449: Tooling for release

2019-10-13 Thread GitBox
jixuan1989 opened a new pull request #449: Tooling for release
URL: https://github.com/apache/incubator-iotdb/pull/449
 
 
   just merging LeiRui's modification.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 merged pull request #441: fix release tools

2019-10-13 Thread GitBox
jixuan1989 merged pull request #441: fix release tools
URL: https://github.com/apache/incubator-iotdb/pull/441
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 commented on a change in pull request #447: [IOTDB-250] exclude .mvn/wrapper/maven-wrapper.jar

2019-10-13 Thread GitBox
jixuan1989 commented on a change in pull request #447: [IOTDB-250] exclude 
.mvn/wrapper/maven-wrapper.jar
URL: https://github.com/apache/incubator-iotdb/pull/447#discussion_r334327377
 
 

 ##
 File path: .gitignore
 ##
 @@ -79,7 +82,6 @@ grafana/data/
 
 grafana/data/test.csv
 **/lib/
-*.jar
 
 Review comment:
   why you remove *.jar... 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 merged pull request #448: Cherry pick from rel0.8

2019-10-13 Thread GitBox
jixuan1989 merged pull request #448: Cherry pick from rel0.8
URL: https://github.com/apache/incubator-iotdb/pull/448
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #443: [IOTDB-240] fix unknown time series in where clause

2019-10-13 Thread GitBox
jt2594838 commented on a change in pull request #443: [IOTDB-240] fix unknown 
time series in where clause
URL: https://github.com/apache/incubator-iotdb/pull/443#discussion_r334327289
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/optimizer/ConcatPathOptimizer.java
 ##
 @@ -343,7 +346,7 @@ private void removeStarsInPath(List paths, 
List afterConcatAggrega
   try {
 List actualPaths = 
executor.getAllPaths(paths.get(i).getFullPath());
 if(actualPaths.isEmpty()){
-  throw new LogicalOptimizeException("Path: \"" + paths.get(i) + "\" 
not corresponding any known time series");
+  throw new LogicalOptimizeException("Path: \"" + paths.get(i) + "\" 
doesn't corresponding to any known time series");
 
 Review comment:
   not "ing"


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on issue #417: [IOTDB-234] Refactor TsFile storage on HDFS

2019-10-13 Thread GitBox
jt2594838 commented on issue #417: [IOTDB-234] Refactor TsFile storage on HDFS
URL: https://github.com/apache/incubator-iotdb/pull/417#issuecomment-541502505
 
 
   If you insist on this design, at least you should save the constructor.
   And this test will show you that this is still slower compared with a direct 
call to new().
   
[ReflectionTest.txt](https://github.com/apache/incubator-iotdb/files/3722994/ReflectionTest.txt)
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Genius-pig edited a comment on issue #409: [IOTDB-165][TsFile] Delete a current version and add a number version and an exception.

2019-10-13 Thread GitBox
Genius-pig edited a comment on issue #409: [IOTDB-165][TsFile] Delete a current 
version and add a number version and an exception.
URL: https://github.com/apache/incubator-iotdb/pull/409#issuecomment-541491965
 
 
   > Hi, how about the new progress of this PR
   
   @jt2594838 He wanted me to delete some `position()` methods, do you have any 
suggestions?@jixuan1989


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Genius-pig commented on issue #409: [IOTDB-165][TsFile] Delete a current version and add a number version and an exception.

2019-10-13 Thread GitBox
Genius-pig commented on issue #409: [IOTDB-165][TsFile] Delete a current 
version and add a number version and an exception.
URL: https://github.com/apache/incubator-iotdb/pull/409#issuecomment-541491965
 
 
   > Hi, how about the new progress of this PR
   
   @jt2594838 He wants me to delete some `position()` method, do you have any 
suggestions?@jixuan1989


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 commented on issue #409: [IOTDB-165][TsFile] Delete a current version and add a number version and an exception.

2019-10-13 Thread GitBox
jixuan1989 commented on issue #409: [IOTDB-165][TsFile] Delete a current 
version and add a number version and an exception.
URL: https://github.com/apache/incubator-iotdb/pull/409#issuecomment-541491033
 
 
   Hi, how about the new progress of this PR


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 opened a new pull request #448: Cherry pick from rel0.8

2019-10-13 Thread GitBox
jixuan1989 opened a new pull request #448: Cherry pick from rel0.8
URL: https://github.com/apache/incubator-iotdb/pull/448
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Ring-k commented on issue #440: reconstruct antlrv3 grammar to improve performance

2019-10-12 Thread GitBox
Ring-k commented on issue #440: reconstruct antlrv3 grammar to improve 
performance
URL: https://github.com/apache/incubator-iotdb/pull/440#issuecomment-541295300
 
 
   The core idea of this reconstruction is
   1. The statement is now grouped as ddlStatement, dmlStatement, and 
administrationStatement, which is more consistent with the conventional sql 
design.
   2. I reorganized the format of constant. In previous version, we had 
negetive integer, positive integer, usigned integer, etc. To aviod problems of 
longest match, we need use "=>" operator. This operator was widely used in 
previous version. I simplified the grammer definitions by just defining the 
integer and real numbers, regardless of the sign. No "=>" operators anymore. 
Althought, some acceptable constraints are introduced, like "+" is not allowed 
to identify a positive value. I think because of less checkings during parsing, 
the performance is improved, especially in parsing full-digit paths.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Ring-k removed a comment on issue #440: reconstruct antlrv3 grammar to improve performance

2019-10-12 Thread GitBox
Ring-k removed a comment on issue #440: reconstruct antlrv3 grammar to improve 
performance
URL: https://github.com/apache/incubator-iotdb/pull/440#issuecomment-541295084
 
 
   The core idea of this reconstruction is
   1. The statement is now grouped as ddlStatement, dmlStatement, and 
administrationStatement, which is more consistent with the conventional sql 
design.
   2. I reorganize the format of constant. In previous version, we have 
negetive integer, positive integer, usigned integer, etc. To aviod problems of 
longest match, we need to use "=>" operator. This operator is widely used in 
previous version. I simplified the grammer definition by just defining the 
integer and real numbers, regardless of the sign. No "=>" operator anymore. 
Althought, some acceptable constraints are introduced, like "+" is not allowed 
to identify a positive value. I think because of less checkings during parsing, 
the performance is improved, especially in parsing full-digit paths.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Ring-k commented on issue #440: reconstruct antlrv3 grammar to improve performance

2019-10-12 Thread GitBox
Ring-k commented on issue #440: reconstruct antlrv3 grammar to improve 
performance
URL: https://github.com/apache/incubator-iotdb/pull/440#issuecomment-541295084
 
 
   The core idea of this reconstruction is
   1. The statement is now grouped as ddlStatement, dmlStatement, and 
administrationStatement, which is more consistent with the conventional sql 
design.
   2. I reorganize the format of constant. In previous version, we have 
negetive integer, positive integer, usigned integer, etc. To aviod problems of 
longest match, we need to use "=>" operator. This operator is widely used in 
previous version. I simplified the grammer definition by just defining the 
integer and real numbers, regardless of the sign. No "=>" operator anymore. 
Althought, some acceptable constraints are introduced, like "+" is not allowed 
to identify a positive value. I think because of less checkings during parsing, 
the performance is improved, especially in parsing full-digit paths.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-12 Thread GitBox
Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar 
to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334224410
 
 

 ##
 File path: server/src/main/antlr3/org/apache/iotdb/db/sql/parse/TqlParser.g
 ##
 @@ -0,0 +1,726 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+parser grammar TqlParser;
+
+options {
+tokenVocab=TqlLexer;
+output=AST;
+ASTLabelType=CommonTree;
+backtrack=false;
+k=3;
+}
+
+tokens{
+TOK_CREATE;
+TOK_TIMESERIES;
+TOK_PATH;
+TOK_WITH;
+TOK_DATATYPE;
+TOK_ENCODING;
+TOK_COMPRESSOR;
+TOK_PROPERTY;
+TOK_QUERY;
+TOK_AGGREGATE;
+TOK_SELECT;
+TOK_FROM;
+TOK_ROOT;
+TOK_WHERE;
+TOK_AND;
+TOK_OR;
+TOK_NOT;
+TOK_GT;
+TOK_GTE;
+TOK_LT;
+TOK_LTE;
+TOK_EQ;
+TOK_NEQ;
+TOK_DATETIME;
+TOK_INSERT;
+TOK_INSERT_COLUMNS;
+TOK_TIME;
+TOK_INSERT_VALUES;
+TOK_UPDATE;
+TOK_SET;
+TOK_DELETE;
+TOK_LABEL;
+TOK_ADD;
+TOK_LINK;
+TOK_UNLINK;
+TOK_SHOW_METADATA;
+TOK_INDEX;
+TOK_FUNCTION;
+TOK_INDEX_KV;
+TOK_DESCRIBE;
+TOK_DROP;
+TOK_MERGE;
+TOK_LIST;
+TOK_PRIVILEGES;
+TOK_ALL;
+TOK_USER;
+TOK_ROLE;
+TOK_QUIT;
+TOK_PASSWORD;
+TOK_ALTER;
+TOK_ALTER_PSWD;
+TOK_GRANT;
+TOK_REVOKE;
+TOK_SLIMIT;
+TOK_LIMIT;
+TOK_SOFFSET;
+TOK_OFFSET;
+TOK_GROUPBY;
+TOK_TIMEUNIT;
+TOK_TIMEORIGIN;
+TOK_TIMEINTERVAL;
+TOK_FILL;
+TOK_TYPE;
+TOK_PREVIOUS;
+TOK_LINEAR;
+TOK_LOAD;
+TOK_GRANT_WATERMARK_EMBEDDING;
+TOK_REVOKE_WATERMARK_EMBEDDING;
+TOK_STORAGEGROUP;
+TOK_VALUE;
+TOK_CONSTANT;
+TOK_TIMEINTERVALPAIR;
+TOK_PROPERTY_VALUE;
+}
+
+@header{
+package org.apache.iotdb.db.sql.parse;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+}
+
+@members {
+ArrayList errors = new ArrayList();
+Stack messages = new Stack();
+private static HashMap tokenNameMap;
+static {
+tokenNameMap = new HashMap();
+tokenNameMap.put("K_AND", "AND");
+tokenNameMap.put("K_OR", "OR");
+tokenNameMap.put("K_NOT", "NOT");
+tokenNameMap.put("K_LIKE", "LIKE");
+tokenNameMap.put("K_BY", "BY");
+tokenNameMap.put("K_GROUP", "GROUP");
+   tokenNameMap.put("K_FILL", "FILL");
+   tokenNameMap.put("K_LINEAR", "LINEAR");
+   tokenNameMap.put("K_PREVIOUS", "PREVIOUS");
+   tokenNameMap.put("K_WHERE", "WHERE");
+   tokenNameMap.put("K_FROM", "FROM");
+   tokenNameMap.put("K_SELECT", "SELECT");
+   tokenNameMap.put("K_INSERT", "INSERT");
+   tokenNameMap.put("K_LIMIT","LIMIT");
+   tokenNameMap.put("K_OFFSET","OFFSET");
+   tokenNameMap.put("K_SLIMIT","SLIMIT");
+   tokenNameMap.put("K_SOFFSET","SOFFSET");
+   tokenNameMap.put("K_ON", "ON");
+   tokenNameMap.put("K_ROOT", "ROOT");
+   tokenNameMap.put("K_SHOW", "SHOW");
+   tokenNameMap.put("K_CLUSTER", "CLUSTER");
+   tokenNameMap.put("K_LOAD", "LOAD");
+   tokenNameMap.put("K_NULL", "NULL");
+   tokenNameMap.put("K_CREATE", "CREATE");
+   tokenNameMap.put("K_DESCRIBE", "DESCRIBE");
+   tokenNameMap.put("K_TO", "TO");
+   tokenNameMap.put("K_ON", "ON");
+   tokenNameMap.put("K_USING", "USING");
+   tokenNameMap.put("K_DATETIME", "DATETIME");
+   tokenNameMap.put("K_TIMESTAMP", "TIMESTAMP");
+   tokenNameMap.put("K_TIME", "TIME");
+   tokenNameMap.put("K_AGGREGATION", "CLUSTERED");
+   tokenNameMap.put("K_INTO", "INTO");
+   tokenNameMap.put("K_ROW", "ROW");
+   tokenNameMap.put("K_STORED", "STORED");
+   tokenNameMap.put("K_OF", "OF");
+   tokenNameMap.put("K_ADD", "ADD");
+   tokenNameMap.put("K_FUNCTION", "FUNCTION");
+   tokenNameMap.put("K_WITH", "WITH");
+   tokenNameMap.put("K_SET", "SET");
+   tokenNameMap.put("K_UPDATE", "UPDATE");
+   tokenNameMap.put("K_VALUES", "VALUES");
+   tokenNameMap.put("K_KEY", "KEY");
+   tokenNameMap.put("K_ENABLE", "ENABLE");
+   tokenNameMap.put("K_DISABLE", "DISABLE");
+   tokenNameMap.put("K_ALL", "ALL");
+   

[GitHub] [incubator-iotdb] Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance

2019-10-12 Thread GitBox
Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar 
to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334224374
 
 

 ##
 File path: server/src/main/antlr3/org/apache/iotdb/db/sql/parse/TqlParser.g
 ##
 @@ -0,0 +1,726 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+parser grammar TqlParser;
+
+options {
+tokenVocab=TqlLexer;
+output=AST;
+ASTLabelType=CommonTree;
+backtrack=false;
+k=3;
+}
+
+tokens{
+TOK_CREATE;
+TOK_TIMESERIES;
+TOK_PATH;
+TOK_WITH;
+TOK_DATATYPE;
+TOK_ENCODING;
+TOK_COMPRESSOR;
+TOK_PROPERTY;
+TOK_QUERY;
+TOK_AGGREGATE;
+TOK_SELECT;
+TOK_FROM;
+TOK_ROOT;
+TOK_WHERE;
+TOK_AND;
+TOK_OR;
+TOK_NOT;
+TOK_GT;
+TOK_GTE;
+TOK_LT;
+TOK_LTE;
+TOK_EQ;
+TOK_NEQ;
+TOK_DATETIME;
+TOK_INSERT;
+TOK_INSERT_COLUMNS;
+TOK_TIME;
+TOK_INSERT_VALUES;
+TOK_UPDATE;
+TOK_SET;
+TOK_DELETE;
+TOK_LABEL;
+TOK_ADD;
+TOK_LINK;
+TOK_UNLINK;
+TOK_SHOW_METADATA;
+TOK_INDEX;
+TOK_FUNCTION;
+TOK_INDEX_KV;
+TOK_DESCRIBE;
+TOK_DROP;
+TOK_MERGE;
+TOK_LIST;
+TOK_PRIVILEGES;
+TOK_ALL;
+TOK_USER;
+TOK_ROLE;
+TOK_QUIT;
+TOK_PASSWORD;
+TOK_ALTER;
+TOK_ALTER_PSWD;
+TOK_GRANT;
+TOK_REVOKE;
+TOK_SLIMIT;
+TOK_LIMIT;
+TOK_SOFFSET;
+TOK_OFFSET;
+TOK_GROUPBY;
+TOK_TIMEUNIT;
+TOK_TIMEORIGIN;
+TOK_TIMEINTERVAL;
+TOK_FILL;
+TOK_TYPE;
+TOK_PREVIOUS;
+TOK_LINEAR;
+TOK_LOAD;
+TOK_GRANT_WATERMARK_EMBEDDING;
+TOK_REVOKE_WATERMARK_EMBEDDING;
+TOK_STORAGEGROUP;
+TOK_VALUE;
+TOK_CONSTANT;
+TOK_TIMEINTERVALPAIR;
+TOK_PROPERTY_VALUE;
+}
+
+@header{
+package org.apache.iotdb.db.sql.parse;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+}
+
+@members {
+ArrayList errors = new ArrayList();
+Stack messages = new Stack();
+private static HashMap tokenNameMap;
+static {
+tokenNameMap = new HashMap();
+tokenNameMap.put("K_AND", "AND");
+tokenNameMap.put("K_OR", "OR");
+tokenNameMap.put("K_NOT", "NOT");
+tokenNameMap.put("K_LIKE", "LIKE");
+tokenNameMap.put("K_BY", "BY");
+tokenNameMap.put("K_GROUP", "GROUP");
+   tokenNameMap.put("K_FILL", "FILL");
+   tokenNameMap.put("K_LINEAR", "LINEAR");
+   tokenNameMap.put("K_PREVIOUS", "PREVIOUS");
+   tokenNameMap.put("K_WHERE", "WHERE");
+   tokenNameMap.put("K_FROM", "FROM");
+   tokenNameMap.put("K_SELECT", "SELECT");
+   tokenNameMap.put("K_INSERT", "INSERT");
+   tokenNameMap.put("K_LIMIT","LIMIT");
+   tokenNameMap.put("K_OFFSET","OFFSET");
+   tokenNameMap.put("K_SLIMIT","SLIMIT");
+   tokenNameMap.put("K_SOFFSET","SOFFSET");
+   tokenNameMap.put("K_ON", "ON");
+   tokenNameMap.put("K_ROOT", "ROOT");
+   tokenNameMap.put("K_SHOW", "SHOW");
+   tokenNameMap.put("K_CLUSTER", "CLUSTER");
+   tokenNameMap.put("K_LOAD", "LOAD");
+   tokenNameMap.put("K_NULL", "NULL");
+   tokenNameMap.put("K_CREATE", "CREATE");
+   tokenNameMap.put("K_DESCRIBE", "DESCRIBE");
+   tokenNameMap.put("K_TO", "TO");
+   tokenNameMap.put("K_ON", "ON");
+   tokenNameMap.put("K_USING", "USING");
+   tokenNameMap.put("K_DATETIME", "DATETIME");
+   tokenNameMap.put("K_TIMESTAMP", "TIMESTAMP");
+   tokenNameMap.put("K_TIME", "TIME");
+   tokenNameMap.put("K_AGGREGATION", "CLUSTERED");
+   tokenNameMap.put("K_INTO", "INTO");
+   tokenNameMap.put("K_ROW", "ROW");
+   tokenNameMap.put("K_STORED", "STORED");
+   tokenNameMap.put("K_OF", "OF");
+   tokenNameMap.put("K_ADD", "ADD");
+   tokenNameMap.put("K_FUNCTION", "FUNCTION");
+   tokenNameMap.put("K_WITH", "WITH");
+   tokenNameMap.put("K_SET", "SET");
+   tokenNameMap.put("K_UPDATE", "UPDATE");
+   tokenNameMap.put("K_VALUES", "VALUES");
+   tokenNameMap.put("K_KEY", "KEY");
+   tokenNameMap.put("K_ENABLE", "ENABLE");
+   tokenNameMap.put("K_DISABLE", "DISABLE");
+   tokenNameMap.put("K_ALL", "ALL");
+   

[GitHub] [incubator-iotdb] jixuan1989 commented on issue #447: [IOTDB-250] exclude .mvn/wrapper/maven-wrapper.jar

2019-10-12 Thread GitBox
jixuan1989 commented on issue #447: [IOTDB-250] exclude 
.mvn/wrapper/maven-wrapper.jar
URL: https://github.com/apache/incubator-iotdb/pull/447#issuecomment-541291438
 
 
   > Not sure this fixes anything
   
   I think this is just for avoiding someone upload the file to the repo... No 
help for futures' release. RM must check their local folder carefully.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on issue #441: fix release tools

2019-10-11 Thread GitBox
LeiRui commented on issue #441: fix release tools
URL: https://github.com/apache/incubator-iotdb/pull/441#issuecomment-541278398
 
 
   > I would appreciate it if you give a detailed description of the problems 
(better with screenshots) and its root cause, the best if you issue it.
   
   I propose this pull request to the branch `tooling-for-release` so I think I 
would be fine if I don't issue it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on issue #441: fix release tools

2019-10-11 Thread GitBox
LeiRui commented on issue #441: fix release tools
URL: https://github.com/apache/incubator-iotdb/pull/441#issuecomment-541278127
 
 
   > I would appreciate it if you give a detailed description of the problems 
(better with screenshots) and its root cause, the best if you issue it.
   
   As I have mentioned, I fix some problems in this tool to make it functions:
   - fatal bug: flip the returned result of checkVerNum and checkRcNum
   - adaptation bug: fix some link paths


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on a change in pull request #443: [IOTDB-240] fix unknown time series in where clause

2019-10-11 Thread GitBox
LeiRui commented on a change in pull request #443: [IOTDB-240] fix unknown time 
series in where clause
URL: https://github.com/apache/incubator-iotdb/pull/443#discussion_r334215980
 
 

 ##
 File path: server/src/main/java/org/apache/iotdb/db/metadata/MTree.java
 ##
 @@ -512,8 +512,7 @@ private MNode checkPath(String path) throws 
PathErrorException {
 MNode cur = getRoot();
 for (int i = 1; i < nodes.length; i++) {
   if (!cur.hasChild(nodes[i])) {
-throw new PathErrorException(
-String.format(NO_CHILD_ERROR,cur.getName(),nodes[i]));
+throw new PathErrorException("Path: \"" + path + "\" doesn't 
corresponding to any known time series");
 
 Review comment:
   grammar: doesn't correspond to 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on a change in pull request #443: [IOTDB-240] fix unknown time series in where clause

2019-10-11 Thread GitBox
LeiRui commented on a change in pull request #443: [IOTDB-240] fix unknown time 
series in where clause
URL: https://github.com/apache/incubator-iotdb/pull/443#discussion_r334217063
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/optimizer/ConcatPathOptimizer.java
 ##
 @@ -320,6 +320,9 @@ private FilterOperator 
constructTwoForkFilterTreeWithAnd(List noStarPaths,
   for (Path path : paths) {
 List all;
 all = executor.getAllPaths(path.getFullPath());
+if(all.isEmpty()){
+  throw new LogicalOptimizeException("Path: \"" + path + "\" doesn't 
corresponding to any known time series");
 
 Review comment:
   Grammar: doesn't correspond to 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on a change in pull request #443: [IOTDB-240] fix unknown time series in where clause

2019-10-11 Thread GitBox
LeiRui commented on a change in pull request #443: [IOTDB-240] fix unknown time 
series in where clause
URL: https://github.com/apache/incubator-iotdb/pull/443#discussion_r334217520
 
 

 ##
 File path: 
server/src/test/java/org/apache/iotdb/db/integration/IoTDBMultiSeriesIT.java
 ##
 @@ -334,4 +334,32 @@ public void crossSeriesReadUpdateTest() throws 
ClassNotFoundException, SQLExcept
   fail(e.getMessage());
 }
   }
+
+  @Test
+  public void selectUnknownTimeSeries() throws ClassNotFoundException {
+Class.forName(Config.JDBC_DRIVER_NAME);
+
+try (Connection connection = DriverManager
+.getConnection(Config.IOTDB_URL_PREFIX + "127.0.0.1:6667/", "root", 
"root");
+Statement statement = connection.createStatement()) {
+  statement.execute("select s10 from root.vehicle.d0");
+  fail("not throw exception when select unknown time series");
+} catch (SQLException e) {
+  assertEquals("Execute statement error: Path: \"root.vehicle.d0.s10\" 
doesn't corresponding to any known time series", e.getMessage());
+}
+  }
+
+  @Test
+  public void selectWhereUnknownTimeSeries() throws ClassNotFoundException {
+Class.forName(Config.JDBC_DRIVER_NAME);
+
+try (Connection connection = DriverManager
+.getConnection(Config.IOTDB_URL_PREFIX + "127.0.0.1:6667/", "root", 
"root");
+Statement statement = connection.createStatement()) {
+  statement.execute("select s1 from root.vehicle.d0 where s0 < 111 and s10 
< 111");
 
 Review comment:
   Better add a test for the case where filter paths start with ROOT. 
Correspond to the error exception you add in the class `MTree`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on a change in pull request #443: [IOTDB-240] fix unknown time series in where clause

2019-10-11 Thread GitBox
LeiRui commented on a change in pull request #443: [IOTDB-240] fix unknown time 
series in where clause
URL: https://github.com/apache/incubator-iotdb/pull/443#discussion_r334215980
 
 

 ##
 File path: server/src/main/java/org/apache/iotdb/db/metadata/MTree.java
 ##
 @@ -512,8 +512,7 @@ private MNode checkPath(String path) throws 
PathErrorException {
 MNode cur = getRoot();
 for (int i = 1; i < nodes.length; i++) {
   if (!cur.hasChild(nodes[i])) {
-throw new PathErrorException(
-String.format(NO_CHILD_ERROR,cur.getName(),nodes[i]));
+throw new PathErrorException("Path: \"" + path + "\" doesn't 
corresponding to any known time series");
 
 Review comment:
   grammar: doesn't correspond to 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] justinmclean commented on issue #447: [IOTDB-250] exclude .mvn/wrapper/maven-wrapper.jar

2019-10-11 Thread GitBox
justinmclean commented on issue #447: [IOTDB-250] exclude 
.mvn/wrapper/maven-wrapper.jar
URL: https://github.com/apache/incubator-iotdb/pull/447#issuecomment-541276184
 
 
   Not sure this fixes anything


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on a change in pull request #433: [IOTDB-203]add "group by device" function for narrow table display

2019-10-11 Thread GitBox
LeiRui commented on a change in pull request #433: [IOTDB-203]add "group by 
device" function for narrow table display
URL: https://github.com/apache/incubator-iotdb/pull/433#discussion_r334216535
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/PhysicalGenerator.java
 ##
 @@ -260,75 +269,122 @@ private PhysicalPlan transformQuery(QueryOperator 
queryOperator)
   queryPlan = new QueryPlan();
 }
 
-List paths = queryOperator.getSelectedPaths();
-queryPlan.setPaths(paths);
-queryPlan.checkPaths(executor);
+if (!queryOperator.isGroupByDevice()) {
+  List paths = queryOperator.getSelectedPaths();
+  queryPlan.setPaths(paths);
 
-if (queryOperator.isGroupByDevice()) {
-  queryPlan.setGroupByDevice(true);
-  // group paths and aggregations by devices
-  List aggregations = 
queryOperator.getSelectOperator().getAggregations();
-  TreeMap> pathsGroupByDevice = new TreeMap<>();
-  TreeMap> aggregationsGroupByDevice = new 
TreeMap<>();
-  TreeMap> sensorColumnsGroupByDevice = new 
TreeMap<>();
-  for (int i = 0; i < paths.size(); i++) {
-Path path = paths.get(i);
-String device = path.getDevice();
-if (!pathsGroupByDevice.containsKey(device)) {
-  pathsGroupByDevice.put(device, new ArrayList<>());
-  aggregationsGroupByDevice.put(device, new ArrayList<>());
-  sensorColumnsGroupByDevice.put(device, new ArrayList<>());
-}
-pathsGroupByDevice.get(device).add(path);
-if (aggregations != null && !aggregations.isEmpty()) {
-  aggregationsGroupByDevice.get(device).add(aggregations.get(i));
-  sensorColumnsGroupByDevice.get(device)
-  .add(aggregations.get(i) + "(" + paths.get(i).getMeasurement() + 
")");
-} else {
-  
sensorColumnsGroupByDevice.get(device).add(paths.get(i).getMeasurement());
+} else {
+  // below is the core realization of GROUP_BY_DEVICE sql
+  List prefixPaths = 
queryOperator.getFromOperator().getPrefixPaths();
+  List suffixPaths = 
queryOperator.getSelectOperator().getSuffixPaths();
+  List originAggregations = 
queryOperator.getSelectOperator().getAggregations();
+
+  List measurementColumnList = new ArrayList<>();
+  Map> measurementColumnsGroupByDevice = new 
LinkedHashMap<>();
+  Map dataTypeConsistencyChecker = new HashMap<>();
+  Set allSelectPaths = new HashSet<>();
+
+  for (int i = 0; i < suffixPaths.size(); i++) { // per suffix
+Path suffixPath = suffixPaths.get(i);
+Set deviceSetOfGivenSuffix = new HashSet<>();
+Set measurementSetOfGivenSuffix = new TreeSet<>();
+for (Path prefixPath : prefixPaths) { // per prefix
+  if (!prefixPath.startWith(SQLConstant.ROOT)) {
+throw new QueryProcessorException("illegal from clause : " + 
prefixPath.getFullPath());
+  }
 
 Review comment:
   removed two places


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on a change in pull request #433: [IOTDB-203]add "group by device" function for narrow table display

2019-10-11 Thread GitBox
LeiRui commented on a change in pull request #433: [IOTDB-203]add "group by 
device" function for narrow table display
URL: https://github.com/apache/incubator-iotdb/pull/433#discussion_r334216535
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/PhysicalGenerator.java
 ##
 @@ -260,75 +269,122 @@ private PhysicalPlan transformQuery(QueryOperator 
queryOperator)
   queryPlan = new QueryPlan();
 }
 
-List paths = queryOperator.getSelectedPaths();
-queryPlan.setPaths(paths);
-queryPlan.checkPaths(executor);
+if (!queryOperator.isGroupByDevice()) {
+  List paths = queryOperator.getSelectedPaths();
+  queryPlan.setPaths(paths);
 
-if (queryOperator.isGroupByDevice()) {
-  queryPlan.setGroupByDevice(true);
-  // group paths and aggregations by devices
-  List aggregations = 
queryOperator.getSelectOperator().getAggregations();
-  TreeMap> pathsGroupByDevice = new TreeMap<>();
-  TreeMap> aggregationsGroupByDevice = new 
TreeMap<>();
-  TreeMap> sensorColumnsGroupByDevice = new 
TreeMap<>();
-  for (int i = 0; i < paths.size(); i++) {
-Path path = paths.get(i);
-String device = path.getDevice();
-if (!pathsGroupByDevice.containsKey(device)) {
-  pathsGroupByDevice.put(device, new ArrayList<>());
-  aggregationsGroupByDevice.put(device, new ArrayList<>());
-  sensorColumnsGroupByDevice.put(device, new ArrayList<>());
-}
-pathsGroupByDevice.get(device).add(path);
-if (aggregations != null && !aggregations.isEmpty()) {
-  aggregationsGroupByDevice.get(device).add(aggregations.get(i));
-  sensorColumnsGroupByDevice.get(device)
-  .add(aggregations.get(i) + "(" + paths.get(i).getMeasurement() + 
")");
-} else {
-  
sensorColumnsGroupByDevice.get(device).add(paths.get(i).getMeasurement());
+} else {
+  // below is the core realization of GROUP_BY_DEVICE sql
+  List prefixPaths = 
queryOperator.getFromOperator().getPrefixPaths();
+  List suffixPaths = 
queryOperator.getSelectOperator().getSuffixPaths();
+  List originAggregations = 
queryOperator.getSelectOperator().getAggregations();
+
+  List measurementColumnList = new ArrayList<>();
+  Map> measurementColumnsGroupByDevice = new 
LinkedHashMap<>();
+  Map dataTypeConsistencyChecker = new HashMap<>();
+  Set allSelectPaths = new HashSet<>();
+
+  for (int i = 0; i < suffixPaths.size(); i++) { // per suffix
+Path suffixPath = suffixPaths.get(i);
+Set deviceSetOfGivenSuffix = new HashSet<>();
+Set measurementSetOfGivenSuffix = new TreeSet<>();
+for (Path prefixPath : prefixPaths) { // per prefix
+  if (!prefixPath.startWith(SQLConstant.ROOT)) {
+throw new QueryProcessorException("illegal from clause : " + 
prefixPath.getFullPath());
+  }
 
 Review comment:
   removed


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on a change in pull request #433: [IOTDB-203]add "group by device" function for narrow table display

2019-10-11 Thread GitBox
LeiRui commented on a change in pull request #433: [IOTDB-203]add "group by 
device" function for narrow table display
URL: https://github.com/apache/incubator-iotdb/pull/433#discussion_r334216526
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/PhysicalGenerator.java
 ##
 @@ -260,75 +269,122 @@ private PhysicalPlan transformQuery(QueryOperator 
queryOperator)
   queryPlan = new QueryPlan();
 }
 
-List paths = queryOperator.getSelectedPaths();
-queryPlan.setPaths(paths);
-queryPlan.checkPaths(executor);
+if (!queryOperator.isGroupByDevice()) {
+  List paths = queryOperator.getSelectedPaths();
+  queryPlan.setPaths(paths);
 
-if (queryOperator.isGroupByDevice()) {
-  queryPlan.setGroupByDevice(true);
-  // group paths and aggregations by devices
-  List aggregations = 
queryOperator.getSelectOperator().getAggregations();
-  TreeMap> pathsGroupByDevice = new TreeMap<>();
-  TreeMap> aggregationsGroupByDevice = new 
TreeMap<>();
-  TreeMap> sensorColumnsGroupByDevice = new 
TreeMap<>();
-  for (int i = 0; i < paths.size(); i++) {
-Path path = paths.get(i);
-String device = path.getDevice();
-if (!pathsGroupByDevice.containsKey(device)) {
-  pathsGroupByDevice.put(device, new ArrayList<>());
-  aggregationsGroupByDevice.put(device, new ArrayList<>());
-  sensorColumnsGroupByDevice.put(device, new ArrayList<>());
-}
-pathsGroupByDevice.get(device).add(path);
-if (aggregations != null && !aggregations.isEmpty()) {
-  aggregationsGroupByDevice.get(device).add(aggregations.get(i));
-  sensorColumnsGroupByDevice.get(device)
-  .add(aggregations.get(i) + "(" + paths.get(i).getMeasurement() + 
")");
-} else {
-  
sensorColumnsGroupByDevice.get(device).add(paths.get(i).getMeasurement());
+} else {
+  // below is the core realization of GROUP_BY_DEVICE sql
+  List prefixPaths = 
queryOperator.getFromOperator().getPrefixPaths();
+  List suffixPaths = 
queryOperator.getSelectOperator().getSuffixPaths();
+  List originAggregations = 
queryOperator.getSelectOperator().getAggregations();
+
+  List measurementColumnList = new ArrayList<>();
+  Map> measurementColumnsGroupByDevice = new 
LinkedHashMap<>();
+  Map dataTypeConsistencyChecker = new HashMap<>();
+  Set allSelectPaths = new HashSet<>();
+
+  for (int i = 0; i < suffixPaths.size(); i++) { // per suffix
+Path suffixPath = suffixPaths.get(i);
+Set deviceSetOfGivenSuffix = new HashSet<>();
+Set measurementSetOfGivenSuffix = new TreeSet<>();
+for (Path prefixPath : prefixPaths) { // per prefix
+  if (!prefixPath.startWith(SQLConstant.ROOT)) {
+throw new QueryProcessorException("illegal from clause : " + 
prefixPath.getFullPath());
+  }
+  Path fullPath = Path.addPrefixPath(suffixPath, prefixPath);
+  Set tmpDeviceSet = new HashSet<>();
+  try {
+List actualPaths = 
executor.getAllPaths(fullPath.getFullPath());
+for (String pathStr : actualPaths) {
+  Path path = new Path(pathStr);
+  String device = path.getDevice();
+  // update tmpDeviceSet for a full path
+  tmpDeviceSet.add(device);
+
+  // ignore the duplicate prefix device for a given suffix path
+  // e.g. select s0 from root.vehicle.d0, root.vehicle.d0 group by 
device,
+  // for the given suffix path "s0", the second prefix device 
"root.vehicle.d0" is
+  // duplicated and should be neglected.
+  if (deviceSetOfGivenSuffix.contains(device)) {
+continue;
+  }
+
+  // get pathForDataType and measurementColumn
+  String measurement = path.getMeasurement();
+  String pathForDataType;
+  String measurementColumn;
+  if (originAggregations != null && !originAggregations.isEmpty()) 
{
+pathForDataType = originAggregations.get(i) + "(" + 
path.getFullPath() + ")";
+measurementColumn = originAggregations.get(i) + "(" + 
measurement + ")";
+  } else {
+pathForDataType = path.getFullPath();
+measurementColumn = measurement;
+  }
+  // check the consistency of data types
+  // a example of inconsistency: select s0 from root.sg1.d1, 
root.sg2.d3 group by device,
+  // while root.sg1.d1.s0 is INT32 and root.sg2.d3.s0 is FLOAT.
+  TSDataType dataType = 
TSServiceImpl.getSeriesType(pathForDataType);
+  if (dataTypeConsistencyChecker.containsKey(measurementColumn)) {
+if 
(!dataType.equals(dataTypeConsistencyChecker.get(measurementColumn))) {
+  throw new QueryProcessorException(
+  

[GitHub] [incubator-iotdb] qiaojialin opened a new pull request #447: [IOTDB-250] exclude .mvn/wrapper/maven-wrapper.jar

2019-10-11 Thread GitBox
qiaojialin opened a new pull request #447: [IOTDB-250] exclude 
.mvn/wrapper/maven-wrapper.jar
URL: https://github.com/apache/incubator-iotdb/pull/447
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] qiaojialin closed pull request #446: exclude .mvn/wrapper/maven-wrapper.jar

2019-10-11 Thread GitBox
qiaojialin closed pull request #446: exclude .mvn/wrapper/maven-wrapper.jar
URL: https://github.com/apache/incubator-iotdb/pull/446
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] qiaojialin opened a new pull request #446: exclude .mvn/wrapper/maven-wrapper.jar

2019-10-11 Thread GitBox
qiaojialin opened a new pull request #446: exclude 
.mvn/wrapper/maven-wrapper.jar
URL: https://github.com/apache/incubator-iotdb/pull/446
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui edited a comment on issue #433: [IOTDB-203]add "group by device" function for narrow table display

2019-10-11 Thread GitBox
LeiRui edited a comment on issue #433: [IOTDB-203]add "group by device" 
function for narrow table display
URL: https://github.com/apache/incubator-iotdb/pull/433#issuecomment-538706069
 
 
   For the development document please see discussion in the email thread named 
"Re: A new result set format".


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on issue #434: [IOTDB-244]fix bug when querying with duplicated columns

2019-10-11 Thread GitBox
LeiRui commented on issue #434: [IOTDB-244]fix bug when querying with 
duplicated columns
URL: https://github.com/apache/incubator-iotdb/pull/434#issuecomment-541269319
 
 
   In the original code, the client side IoTDBQueryResultSet assumes that the 
server returns deduplicated query results. However, in the original code, the 
sever returns query results without the step of deduplication. That's the cause 
of the bug IOTDB-244.
   So, I fix this bug by adding the deduplication step in the server side and 
uses the same deduplication or map strategy as the client.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui removed a comment on issue #434: [IOTDB-244]fix bug when querying with duplicated columns

2019-10-11 Thread GitBox
LeiRui removed a comment on issue #434: [IOTDB-244]fix bug when querying with 
duplicated columns
URL: https://github.com/apache/incubator-iotdb/pull/434#issuecomment-541269319
 
 
   In the original code, the client side IoTDBQueryResultSet assumes that the 
server returns deduplicated query results. However, in the original code, the 
sever returns query results without the step of deduplication. That's the cause 
of the bug IOTDB-244.
   So, I fix this bug by adding the deduplication step in the server side and 
uses the same deduplication or map strategy as the client.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #444: Add python client example in user guide

2019-10-11 Thread GitBox
qiaojialin commented on a change in pull request #444: Add python client 
example in user guide
URL: https://github.com/apache/incubator-iotdb/pull/444#discussion_r333896921
 
 

 ##
 File path: docs/Documentation-CHN/OtherMaterial-Examples.md
 ##
 @@ -81,3 +81,172 @@ public class IotdbHelloWorld {
 
 
 ```
+# 使用Python进行连接
+## 介绍
+以下例子展示了如何使用Thrift编译出Python的库,并调用该库连接IoTDB并进行一些基本操作。在Windows和Linux上的
+使用大同小异,但请注意二者在路径等方面的细微区别。
+
+## Prerequisites
+建议使用python3.7或更高级的版本。
+
+您需要预先安装Thrift才能将我们的Thrift源文件编译为Python的库代码。下面是Thrift的官方安装教程,最终,您需要
+得到一个Thrift的二进制可执行文件。
+```
+http://thrift.apache.org/docs/install/
+```
+
+## Compile the thrift library
+如果您已经将Thrift的可执行文件放到了您的PATH环境变量的目录下,您可以直接运行脚本`client-py/compile.sh`或
+`client-py/compile.bat`,否则您需要修改脚本中的变量`THRIFT_EXE`来指向您的Thrift可执行文件。这会在脚本文件
+的同一目录下生成`target`文件夹,该文件夹即包含了IoTDB的Python版本的Thrift库,当您将该文件夹加入到您的`PYTHONPATH`
+环境变量里后,您就可以在自己的代码中引用该库。请注意,上述脚本通过相对路径寻找我们的Thrift的源文件,如果您改变
+脚本所在位置,脚本可能失效。
+
+或者,如果您对Thrift的基本用法有所了解,您就不需要下载整个项目,您可以仅下载我们的Thrift源文件`service-rpc\src\main\thrift\rpc.thrift`,
+并使用命令`thrift -gen py -out ./target rpc.thrift`来编译Thrift库。
+
+## Example
+下面是一段使用生成的Thrift库连接IoTDB的示例代码,请在编写您自己的代码之前仔细地阅读。
+```python
+
+import sys, struct
+sys.path.append("../target")
+
+from thrift.protocol import TBinaryProtocol
+from thrift.transport import TSocket, TTransport
+
+from rpc.TSIService import Client, TSCreateTimeseriesReq, TSInsertionReq, \
+TSBatchInsertionReq, TSExecuteStatementReq, \
+TS_SessionHandle, TSHandleIdentifier, TSOpenSessionReq, TSQueryDataSet, \
+TSFetchResultsReq, TSCloseOperationReq, \
+TSCloseSessionReq, TSProtocolVersion
+
+TSDataType = {
+'BOOLEAN' : 0,
+'INT32' : 1,
+'INT64' : 2,
+'FLOAT' : 3,
+'DOUBLE' : 4,
+'TEXT' : 5
+}
+
+TSEncoding = {
+'PLAIN' : 0,
+'PLAIN_DICTIONARY' : 1,
+'RLE' : 2,
+'DIFF' : 3,
+'TS_2DIFF' : 4,
+'BITMAP' : 5,
+'GORILLA' : 6,
+'REGULAR' : 7
+}
+
+Compressor = {
+'UNCOMPRESSED' : 0,
+'SNAPPY' : 1,
+'GZIP' : 2,
+'LZO' : 3,
+'SDT' : 4,
+'PAA' : 5,
+'PLA' : 6
+}
+
+
+if __name__ == '__main__':
+ip = "localhost"
+port = "6667"
+username = 'root'
+password = 'root'
+# Make socket
+transport = TSocket.TSocket(ip, port)
+
+# Buffering is critical. Raw sockets are very slow
 
 Review comment:
   in Chinese?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] samperson1997 commented on issue #417: [IOTDB-234] Refactor TsFile storage on HDFS

2019-10-11 Thread GitBox
samperson1997 commented on issue #417: [IOTDB-234] Refactor TsFile storage on 
HDFS
URL: https://github.com/apache/incubator-iotdb/pull/417#issuecomment-540955410
 
 
   > I think using reflection every time is too heavy and slow when all you 
want to do is to get a new Object. To put it simply, reflections should be used 
as less frequently as possible.
   > 
   > Here is my suggestion: use two factories (the LocalFactory and the 
HDFSFactory) instead of one, and initialize one of them using reflection 
according to the configuration at the system start-up. Thus the occurrence of 
reflections can be reduced to only 1.
   
   Hi, many thanks for your useful suggestion!
   
   In the new commit, I tried to reduce the occurrence of reflections to only 1 
by initializing `private static Class clazz` when constructing 
`TSFileFactory`. I think this also reached your main purpose, so I didn't use 
two factories... Do you think it is acceptable? > <


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jixuan1989 opened a new pull request #445: generate cpp, go, and python thrift files under service-rpc

2019-10-11 Thread GitBox
jixuan1989 opened a new pull request #445: generate cpp, go, and python thrift 
files under service-rpc
URL: https://github.com/apache/incubator-iotdb/pull/445
 
 
   after run `mvn generate-sources`, you can find cpp, python, go apis under 
service-rpc/target/generated-source-{language}.
   
   So, you just need to supply how to use these APIs in other PRs.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb-website] jixuan1989 merged pull request #38: Bump webpack-bundle-analyzer from 2.13.1 to 3.5.2

2019-10-10 Thread GitBox
jixuan1989 merged pull request #38: Bump webpack-bundle-analyzer from 2.13.1 to 
3.5.2
URL: https://github.com/apache/incubator-iotdb-website/pull/38
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb-website] jixuan1989 merged pull request #37: Update doc source to tag release/0.8.0 instead of branch rel/0.8

2019-10-10 Thread GitBox
jixuan1989 merged pull request #37: Update doc source to tag release/0.8.0 
instead of branch rel/0.8
URL: https://github.com/apache/incubator-iotdb-website/pull/37
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #443: Iotdb 240 where fix

2019-10-10 Thread GitBox
jt2594838 commented on a change in pull request #443: Iotdb 240 where fix
URL: https://github.com/apache/incubator-iotdb/pull/443#discussion_r333815401
 
 

 ##
 File path: 
server/src/test/java/org/apache/iotdb/db/integration/IoTDBMultiSeriesIT.java
 ##
 @@ -334,4 +334,34 @@ public void crossSeriesReadUpdateTest() throws 
ClassNotFoundException, SQLExcept
   fail(e.getMessage());
 }
   }
+
+  @Test
+  public void selectUnknownTimeSeries() throws ClassNotFoundException {
+Class.forName(Config.JDBC_DRIVER_NAME);
+
+try (Connection connection = DriverManager
+.getConnection(Config.IOTDB_URL_PREFIX + "127.0.0.1:6667/", "root", 
"root");
+Statement statement = connection.createStatement()) {
+  statement.execute("select s10 from root.vehicle.d0");
+  fail("not throw exception when select unknown time series");
+} catch (SQLException e) {
+  assertEquals("Execute statement error: Path: \"root.vehicle.d0.s10\" not 
corresponding any known time series", e.getMessage());
+  e.printStackTrace();
 
 Review comment:
   Remove the stack print.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 edited a comment on issue #443: Iotdb 240 where fix

2019-10-10 Thread GitBox
jt2594838 edited a comment on issue #443: Iotdb 240 where fix
URL: https://github.com/apache/incubator-iotdb/pull/443#issuecomment-540886981
 
 
   Pay attention to the format of the title and make it more understandable by 
adding a few words.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on issue #443: Iotdb 240 where fix

2019-10-10 Thread GitBox
jt2594838 commented on issue #443: Iotdb 240 where fix
URL: https://github.com/apache/incubator-iotdb/pull/443#issuecomment-540886981
 
 
   Pay attention to the format of the title.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #415: [IOTDB-221] Add C++ client

2019-10-10 Thread GitBox
jt2594838 commented on a change in pull request #415: [IOTDB-221] Add C++ client
URL: https://github.com/apache/incubator-iotdb/pull/415#discussion_r333800032
 
 

 ##
 File path: client-cpp/readme
 ##
 @@ -0,0 +1,14 @@
+Introduction:
+This module provide an example of how to connect to IoTDB with c++. You 
need to write your own program by simulating src/SessionExample.cpp
 
 Review comment:
   provide -> provides
   I do not think the phrase “by simulating” quite fit here, maybe "referring 
to" works better.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #415: [IOTDB-221] Add C++ client

2019-10-10 Thread GitBox
jt2594838 commented on a change in pull request #415: [IOTDB-221] Add C++ client
URL: https://github.com/apache/incubator-iotdb/pull/415#discussion_r333800277
 
 

 ##
 File path: client-cpp/readme
 ##
 @@ -0,0 +1,14 @@
+Introduction:
+This module provide an example of how to connect to IoTDB with c++. You 
need to write your own program by simulating src/SessionExample.cpp
+
+Requirement:
+Using this module requires you to install thrift(0.11.0 or later). Below 
is the official tutorial of installation:
+http://thrift.apache.org/docs/install/
+
+Compile:
+You need to mimic src/SessionExample.cpp to write a program to use the 
interface you need. After you've written it, run compile.sh to generate the 
executable.
+Use the following command to generate an executable file:
+./compile.sh executable_file_name cpp_program_file_name
+e.g.
+./compile.sh Example src/SessionExample.cpp
 
 Review comment:
   Please also copy this into the user guide.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #415: [IOTDB-221] Add C++ client

2019-10-10 Thread GitBox
jt2594838 commented on a change in pull request #415: [IOTDB-221] Add C++ client
URL: https://github.com/apache/incubator-iotdb/pull/415#discussion_r333810482
 
 

 ##
 File path: client-cpp/src/IOTDBSession.cpp
 ##
 @@ -0,0 +1,236 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+ 
+#include "IOTDBSession.h"
+
+using namespace std;
+using ::apache::thrift::protocol::TBinaryProtocol;
+using ::apache::thrift::protocol::TCompactProtocol;
+using ::apache::thrift::transport::TSocket;
+using ::apache::thrift::transport::TTransportException;
+using ::apache::thrift::TException;
+
+
+
+void verifySuccess(TSStatus status)
+{
+if (status.statusType.code != 200) 
+{
+throw IoTDBSessionException(status.statusType.message);
+}
+}
+
+
+
+void Session::open()
+{
+open(false, 0);
+}
+
+
+
+void Session::open(bool enableRPCCompression, int connectionTimeoutInMs)
+{
+if (!isClosed) 
+{
+return;
+}
+shared_ptr tmp(new TSocket(host, port));
+transport = tmp;   
+transport->setConnTimeout(connectionTimeoutInMs);
+if (!transport->isOpen()) 
+{
+try 
+{
+transport->open();
+} 
+catch (TTransportException e) 
+{
+throw IoTDBSessionException(e.what());
+}
+}
+if (enableRPCCompression) 
+{
+shared_ptr tmp2(new TCompactProtocol(transport));
+client = new TSIServiceClient(tmp2);
+}
+else 
+{
+shared_ptr tmp2(new TBinaryProtocol(transport));
+client = new TSIServiceClient(tmp2);
+}
+TSOpenSessionReq* openReq = new TSOpenSessionReq();
+
openReq->__set_client_protocol(TSProtocolVersion::IOTDB_SERVICE_PROTOCOL_V1);
+openReq->__set_username(username);
+openReq->__set_password(password);
+try 
+{
+TSOpenSessionResp* openResp = new TSOpenSessionResp();
+client->openSession(*openResp,*openReq);
+verifySuccess(openResp->status);
+if (TSProtocolVersion::IOTDB_SERVICE_PROTOCOL_V1 != 
openResp->serverProtocolVersion) 
+{
+char buf[111];
+sprintf(buf,"Protocol not supported");
+throw IoTDBSessionException(buf);
+}
+sessionHandle = &(openResp->sessionHandle);
+} 
+catch (exception e) 
+{
+transport->close();
+char buf[111];
+sprintf(buf,"Can not open session to %s:%d with user: 
%s.",host.c_str(), port, username.c_str());
+throw IoTDBSessionException(buf);
+}
+isClosed = false;
+}
+
+
+
+void Session::close()
+{
+if (isClosed) 
+{
+return;
+}
+TSCloseSessionReq* req = new TSCloseSessionReq();
+req->__set_sessionHandle(*sessionHandle);
+try 
+{
+TSStatus *resp = new TSStatus();
+client->closeSession(*resp,*req);
+} 
+catch (TException e) 
+{
+throw IoTDBSessionException("Error occurs when closing session at 
server. Maybe server is down.");
 
 Review comment:
   Better to preserve the error message in e.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #415: [IOTDB-221] Add C++ client

2019-10-10 Thread GitBox
jt2594838 commented on a change in pull request #415: [IOTDB-221] Add C++ client
URL: https://github.com/apache/incubator-iotdb/pull/415#discussion_r333807804
 
 

 ##
 File path: client-cpp/src/IOTDBSession.cpp
 ##
 @@ -0,0 +1,236 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+ 
+#include "IOTDBSession.h"
+
+using namespace std;
+using ::apache::thrift::protocol::TBinaryProtocol;
+using ::apache::thrift::protocol::TCompactProtocol;
+using ::apache::thrift::transport::TSocket;
+using ::apache::thrift::transport::TTransportException;
+using ::apache::thrift::TException;
+
+
+
+void verifySuccess(TSStatus status)
+{
+if (status.statusType.code != 200) 
+{
+throw IoTDBSessionException(status.statusType.message);
+}
+}
+
+
+
+void Session::open()
+{
+open(false, 0);
+}
+
+
+
+void Session::open(bool enableRPCCompression, int connectionTimeoutInMs)
+{
+if (!isClosed) 
+{
+return;
+}
+shared_ptr tmp(new TSocket(host, port));
+transport = tmp;   
+transport->setConnTimeout(connectionTimeoutInMs);
+if (!transport->isOpen()) 
+{
+try 
+{
+transport->open();
+} 
+catch (TTransportException e) 
+{
+throw IoTDBSessionException(e.what());
+}
+}
+if (enableRPCCompression) 
+{
+shared_ptr tmp2(new TCompactProtocol(transport));
+client = new TSIServiceClient(tmp2);
+}
+else 
+{
+shared_ptr tmp2(new TBinaryProtocol(transport));
+client = new TSIServiceClient(tmp2);
+}
+TSOpenSessionReq* openReq = new TSOpenSessionReq();
+
openReq->__set_client_protocol(TSProtocolVersion::IOTDB_SERVICE_PROTOCOL_V1);
+openReq->__set_username(username);
+openReq->__set_password(password);
+try 
+{
+TSOpenSessionResp* openResp = new TSOpenSessionResp();
+client->openSession(*openResp,*openReq);
+verifySuccess(openResp->status);
+if (TSProtocolVersion::IOTDB_SERVICE_PROTOCOL_V1 != 
openResp->serverProtocolVersion) 
+{
+char buf[111];
+sprintf(buf,"Protocol not supported");
+throw IoTDBSessionException(buf);
+}
+sessionHandle = &(openResp->sessionHandle);
+} 
 
 Review comment:
   openReq and openResp are not deleted. Is that okay?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 merged pull request #442: [IOTDB-249]enable lowercase in create_timeseries sql

2019-10-10 Thread GitBox
jt2594838 merged pull request #442: [IOTDB-249]enable lowercase in 
create_timeseries sql
URL: https://github.com/apache/incubator-iotdb/pull/442
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 opened a new pull request #444: Add python client example in user guide

2019-10-10 Thread GitBox
jt2594838 opened a new pull request #444: Add python client example in user 
guide
URL: https://github.com/apache/incubator-iotdb/pull/444
 
 
   Refine the python client example readme and also put it into the user guide.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] Genius-pig commented on a change in pull request #409: [IOTDB-165][TsFile] Delete a current version and add a number version and an exception.

2019-10-09 Thread GitBox
Genius-pig commented on a change in pull request #409: [IOTDB-165][TsFile] 
Delete a current version and add a number version and an exception.
URL: https://github.com/apache/incubator-iotdb/pull/409#discussion_r38511
 
 

 ##
 File path: 
tsfile/src/main/java/org/apache/iotdb/tsfile/read/TsFileSequenceReader.java
 ##
 @@ -167,8 +167,8 @@ public void loadMetadataSize() throws IOException {
 fileMetadataSize = ReadWriteIOUtils.readInt(metadataSize);
 fileMetadataPos =
 tsFileInput.size() - TSFileConfig.MAGIC_STRING.length() - 
Integer.BYTES - fileMetadataSize;
-// skip the magic header
-tsFileInput.position(TSFileConfig.MAGIC_STRING.length());
+// skip the magic header and version number
+tsFileInput.position(TSFileConfig.MAGIC_STRING.length() + 
TSFileConfig.VERSION_NUMBER.length());
 
 Review comment:
   @jixuan1989 do you have some suggestions about that?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] SilverNarcissus opened a new pull request #443: Iotdb 240 where fix

2019-10-09 Thread GitBox
SilverNarcissus opened a new pull request #443: Iotdb 240 where fix
URL: https://github.com/apache/incubator-iotdb/pull/443
 
 
   fix unknown time series in where clause and add test


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 edited a comment on issue #441: fix release tools

2019-10-09 Thread GitBox
jt2594838 edited a comment on issue #441: fix release tools
URL: https://github.com/apache/incubator-iotdb/pull/441#issuecomment-540303177
 
 
   I would appreciate it if you give a detailed description of the problems 
(better with screenshots) and its root cause, the best if you issue it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on issue #441: fix release tools

2019-10-09 Thread GitBox
jt2594838 commented on issue #441: fix release tools
URL: https://github.com/apache/incubator-iotdb/pull/441#issuecomment-540303177
 
 
   I would appreciate it if you give a detailed description of the problems 
(better with screenshots) and its root, the best if you issue it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on issue #435: [IOTDB-174] Fix querying timeseries bug

2019-10-09 Thread GitBox
LeiRui commented on issue #435: [IOTDB-174] Fix querying timeseries bug
URL: https://github.com/apache/incubator-iotdb/pull/435#issuecomment-540294979
 
 
   One more suggestion: the header of this pr is a little bit general. Make it 
more specific would be better.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on a change in pull request #433: [IOTDB-203]add "group by device" function for narrow table display

2019-10-09 Thread GitBox
LeiRui commented on a change in pull request #433: [IOTDB-203]add "group by 
device" function for narrow table display
URL: https://github.com/apache/incubator-iotdb/pull/433#discussion_r02587
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/PhysicalGenerator.java
 ##
 @@ -260,75 +269,122 @@ private PhysicalPlan transformQuery(QueryOperator 
queryOperator)
   queryPlan = new QueryPlan();
 }
 
-List paths = queryOperator.getSelectedPaths();
-queryPlan.setPaths(paths);
-queryPlan.checkPaths(executor);
+if (!queryOperator.isGroupByDevice()) {
+  List paths = queryOperator.getSelectedPaths();
+  queryPlan.setPaths(paths);
 
-if (queryOperator.isGroupByDevice()) {
-  queryPlan.setGroupByDevice(true);
-  // group paths and aggregations by devices
-  List aggregations = 
queryOperator.getSelectOperator().getAggregations();
-  TreeMap> pathsGroupByDevice = new TreeMap<>();
-  TreeMap> aggregationsGroupByDevice = new 
TreeMap<>();
-  TreeMap> sensorColumnsGroupByDevice = new 
TreeMap<>();
-  for (int i = 0; i < paths.size(); i++) {
-Path path = paths.get(i);
-String device = path.getDevice();
-if (!pathsGroupByDevice.containsKey(device)) {
-  pathsGroupByDevice.put(device, new ArrayList<>());
-  aggregationsGroupByDevice.put(device, new ArrayList<>());
-  sensorColumnsGroupByDevice.put(device, new ArrayList<>());
-}
-pathsGroupByDevice.get(device).add(path);
-if (aggregations != null && !aggregations.isEmpty()) {
-  aggregationsGroupByDevice.get(device).add(aggregations.get(i));
-  sensorColumnsGroupByDevice.get(device)
-  .add(aggregations.get(i) + "(" + paths.get(i).getMeasurement() + 
")");
-} else {
-  
sensorColumnsGroupByDevice.get(device).add(paths.get(i).getMeasurement());
+} else {
+  // below is the core realization of GROUP_BY_DEVICE sql
+  List prefixPaths = 
queryOperator.getFromOperator().getPrefixPaths();
+  List suffixPaths = 
queryOperator.getSelectOperator().getSuffixPaths();
+  List originAggregations = 
queryOperator.getSelectOperator().getAggregations();
+
+  List measurementColumnList = new ArrayList<>();
+  Map> measurementColumnsGroupByDevice = new 
LinkedHashMap<>();
+  Map dataTypeConsistencyChecker = new HashMap<>();
+  Set allSelectPaths = new HashSet<>();
+
+  for (int i = 0; i < suffixPaths.size(); i++) { // per suffix
+Path suffixPath = suffixPaths.get(i);
+Set deviceSetOfGivenSuffix = new HashSet<>();
+Set measurementSetOfGivenSuffix = new TreeSet<>();
+for (Path prefixPath : prefixPaths) { // per prefix
+  if (!prefixPath.startWith(SQLConstant.ROOT)) {
+throw new QueryProcessorException("illegal from clause : " + 
prefixPath.getFullPath());
+  }
+  Path fullPath = Path.addPrefixPath(suffixPath, prefixPath);
+  Set tmpDeviceSet = new HashSet<>();
+  try {
+List actualPaths = 
executor.getAllPaths(fullPath.getFullPath());
+for (String pathStr : actualPaths) {
+  Path path = new Path(pathStr);
+  String device = path.getDevice();
+  // update tmpDeviceSet for a full path
+  tmpDeviceSet.add(device);
+
+  // ignore the duplicate prefix device for a given suffix path
+  // e.g. select s0 from root.vehicle.d0, root.vehicle.d0 group by 
device,
+  // for the given suffix path "s0", the second prefix device 
"root.vehicle.d0" is
+  // duplicated and should be neglected.
+  if (deviceSetOfGivenSuffix.contains(device)) {
+continue;
+  }
+
+  // get pathForDataType and measurementColumn
+  String measurement = path.getMeasurement();
+  String pathForDataType;
+  String measurementColumn;
+  if (originAggregations != null && !originAggregations.isEmpty()) 
{
+pathForDataType = originAggregations.get(i) + "(" + 
path.getFullPath() + ")";
+measurementColumn = originAggregations.get(i) + "(" + 
measurement + ")";
+  } else {
+pathForDataType = path.getFullPath();
+measurementColumn = measurement;
+  }
+  // check the consistency of data types
+  // a example of inconsistency: select s0 from root.sg1.d1, 
root.sg2.d3 group by device,
+  // while root.sg1.d1.s0 is INT32 and root.sg2.d3.s0 is FLOAT.
+  TSDataType dataType = 
TSServiceImpl.getSeriesType(pathForDataType);
+  if (dataTypeConsistencyChecker.containsKey(measurementColumn)) {
+if 
(!dataType.equals(dataTypeConsistencyChecker.get(measurementColumn))) {
+  throw new QueryProcessorException(
+  

[GitHub] [incubator-iotdb] LeiRui commented on a change in pull request #433: [IOTDB-203]add "group by device" function for narrow table display

2019-10-09 Thread GitBox
LeiRui commented on a change in pull request #433: [IOTDB-203]add "group by 
device" function for narrow table display
URL: https://github.com/apache/incubator-iotdb/pull/433#discussion_r01770
 
 

 ##
 File path: docs/Documentation/UserGuide/5-IoTDB SQL Documentation/1-IoTDB 
Query Statement.md
 ##
 @@ -245,20 +245,77 @@ Note:  can not use  but not 
.
 ```
 GroupbyDeviceClause : GROUP BY DEVICE
 
-Note: 
-- Both uppercase and lowercase are ok.
-- GroupbyDeviceClause can only be used in the end of a query statement.
-
-Eg: 
-- select * from root.vehicle group by device
-- select s0,s0,s1 from root.vehicle.* group by device
-- select s0,s1 from root.vehicle.* limit 10 offset 1 group by device
-- select * from root.vehicle slimit 10 soffset 2 group by device
-- select * from root.vehicle where time>10 group by device
-- select * from root.vehicle where root.vehicle.d0.s0>0 group by device
-- select count(*) from root.vehicle group by device
-- select sum(*) from root.vehicle GROUP BY (20ms,0,[2,50]) group by device
-- select * from root.vehicle where time = 3 Fill(int32[previous, 5ms]) group 
by device
+Rules:  
+1. Both uppercase and lowercase are ok.  
+Correct example: select * from root.sg1 group by device  
+Correct example: select * from root.sg1 GROUP BY DEVICE  
+
+2. GroupbyDeviceClause can only be used at the end of a query statement.  
+Correct example: select * from root.sg1 where time > 10 group by device  
+Wrong example: select * from root.sg1 group by device where time > 10  
+
+3. The paths of the SELECT clause can only be single level. In other words, 
the paths of the SELECT clause can only be measurements or STAR, without DOT.
+Correct example: select s0,s1 from root.sg1.* group by device  
+Correct example: select s0,s1 from root.sg1.d0, root.sg1.d1 group by device  
+Correct example: select * from root.sg1.* group by device  
+Correct example: select * from root group by device  
+Correct example: select s0,s1,* from root.*.* group by device  
+Wrong example: select d0.s1, d0.s2, d1.s0 from root.sg1 group by device  
+Wrong example: select *.s0, *.s1 from root.* group by device  
+Wrong example: select *.*.* from root group by device
+
+4. The data types of the same measurement column should be the same across 
devices. 
+Note that when it comes to aggregated paths, the data type of the measurement 
column will reflect 
+the aggregation function rather than the original timeseries.
 
 Review comment:
   Good point.
   I have taken this future change into consideration. I check the same named 
measurements of different devices and reject service if they are not of the 
same data type. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] LeiRui commented on a change in pull request #433: [IOTDB-203]add "group by device" function for narrow table display

2019-10-09 Thread GitBox
LeiRui commented on a change in pull request #433: [IOTDB-203]add "group by 
device" function for narrow table display
URL: https://github.com/apache/incubator-iotdb/pull/433#discussion_r01434
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/PhysicalGenerator.java
 ##
 @@ -260,75 +269,122 @@ private PhysicalPlan transformQuery(QueryOperator 
queryOperator)
   queryPlan = new QueryPlan();
 }
 
-List paths = queryOperator.getSelectedPaths();
-queryPlan.setPaths(paths);
-queryPlan.checkPaths(executor);
+if (!queryOperator.isGroupByDevice()) {
+  List paths = queryOperator.getSelectedPaths();
+  queryPlan.setPaths(paths);
 
-if (queryOperator.isGroupByDevice()) {
-  queryPlan.setGroupByDevice(true);
-  // group paths and aggregations by devices
-  List aggregations = 
queryOperator.getSelectOperator().getAggregations();
-  TreeMap> pathsGroupByDevice = new TreeMap<>();
-  TreeMap> aggregationsGroupByDevice = new 
TreeMap<>();
-  TreeMap> sensorColumnsGroupByDevice = new 
TreeMap<>();
-  for (int i = 0; i < paths.size(); i++) {
-Path path = paths.get(i);
-String device = path.getDevice();
-if (!pathsGroupByDevice.containsKey(device)) {
-  pathsGroupByDevice.put(device, new ArrayList<>());
-  aggregationsGroupByDevice.put(device, new ArrayList<>());
-  sensorColumnsGroupByDevice.put(device, new ArrayList<>());
-}
-pathsGroupByDevice.get(device).add(path);
-if (aggregations != null && !aggregations.isEmpty()) {
-  aggregationsGroupByDevice.get(device).add(aggregations.get(i));
-  sensorColumnsGroupByDevice.get(device)
-  .add(aggregations.get(i) + "(" + paths.get(i).getMeasurement() + 
")");
-} else {
-  
sensorColumnsGroupByDevice.get(device).add(paths.get(i).getMeasurement());
+} else {
+  // below is the core realization of GROUP_BY_DEVICE sql
+  List prefixPaths = 
queryOperator.getFromOperator().getPrefixPaths();
+  List suffixPaths = 
queryOperator.getSelectOperator().getSuffixPaths();
+  List originAggregations = 
queryOperator.getSelectOperator().getAggregations();
+
+  List measurementColumnList = new ArrayList<>();
+  Map> measurementColumnsGroupByDevice = new 
LinkedHashMap<>();
+  Map dataTypeConsistencyChecker = new HashMap<>();
+  Set allSelectPaths = new HashSet<>();
+
+  for (int i = 0; i < suffixPaths.size(); i++) { // per suffix
+Path suffixPath = suffixPaths.get(i);
+Set deviceSetOfGivenSuffix = new HashSet<>();
+Set measurementSetOfGivenSuffix = new TreeSet<>();
+for (Path prefixPath : prefixPaths) { // per prefix
+  if (!prefixPath.startWith(SQLConstant.ROOT)) {
+throw new QueryProcessorException("illegal from clause : " + 
prefixPath.getFullPath());
+  }
 
 Review comment:
   Yeah you are right. I add this because the original code has this check. 
There are some redundant checks in the code. If it is important, I can remove 
this.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #433: [IOTDB-203]add "group by device" function for narrow table display

2019-10-09 Thread GitBox
jt2594838 commented on a change in pull request #433: [IOTDB-203]add "group by 
device" function for narrow table display
URL: https://github.com/apache/incubator-iotdb/pull/433#discussion_r332812726
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/PhysicalGenerator.java
 ##
 @@ -260,75 +269,122 @@ private PhysicalPlan transformQuery(QueryOperator 
queryOperator)
   queryPlan = new QueryPlan();
 }
 
-List paths = queryOperator.getSelectedPaths();
-queryPlan.setPaths(paths);
-queryPlan.checkPaths(executor);
+if (!queryOperator.isGroupByDevice()) {
+  List paths = queryOperator.getSelectedPaths();
+  queryPlan.setPaths(paths);
 
-if (queryOperator.isGroupByDevice()) {
-  queryPlan.setGroupByDevice(true);
-  // group paths and aggregations by devices
-  List aggregations = 
queryOperator.getSelectOperator().getAggregations();
-  TreeMap> pathsGroupByDevice = new TreeMap<>();
-  TreeMap> aggregationsGroupByDevice = new 
TreeMap<>();
-  TreeMap> sensorColumnsGroupByDevice = new 
TreeMap<>();
-  for (int i = 0; i < paths.size(); i++) {
-Path path = paths.get(i);
-String device = path.getDevice();
-if (!pathsGroupByDevice.containsKey(device)) {
-  pathsGroupByDevice.put(device, new ArrayList<>());
-  aggregationsGroupByDevice.put(device, new ArrayList<>());
-  sensorColumnsGroupByDevice.put(device, new ArrayList<>());
-}
-pathsGroupByDevice.get(device).add(path);
-if (aggregations != null && !aggregations.isEmpty()) {
-  aggregationsGroupByDevice.get(device).add(aggregations.get(i));
-  sensorColumnsGroupByDevice.get(device)
-  .add(aggregations.get(i) + "(" + paths.get(i).getMeasurement() + 
")");
-} else {
-  
sensorColumnsGroupByDevice.get(device).add(paths.get(i).getMeasurement());
+} else {
+  // below is the core realization of GROUP_BY_DEVICE sql
+  List prefixPaths = 
queryOperator.getFromOperator().getPrefixPaths();
+  List suffixPaths = 
queryOperator.getSelectOperator().getSuffixPaths();
+  List originAggregations = 
queryOperator.getSelectOperator().getAggregations();
+
+  List measurementColumnList = new ArrayList<>();
+  Map> measurementColumnsGroupByDevice = new 
LinkedHashMap<>();
+  Map dataTypeConsistencyChecker = new HashMap<>();
+  Set allSelectPaths = new HashSet<>();
+
+  for (int i = 0; i < suffixPaths.size(); i++) { // per suffix
+Path suffixPath = suffixPaths.get(i);
+Set deviceSetOfGivenSuffix = new HashSet<>();
+Set measurementSetOfGivenSuffix = new TreeSet<>();
+for (Path prefixPath : prefixPaths) { // per prefix
+  if (!prefixPath.startWith(SQLConstant.ROOT)) {
+throw new QueryProcessorException("illegal from clause : " + 
prefixPath.getFullPath());
+  }
+  Path fullPath = Path.addPrefixPath(suffixPath, prefixPath);
+  Set tmpDeviceSet = new HashSet<>();
+  try {
+List actualPaths = 
executor.getAllPaths(fullPath.getFullPath());
+for (String pathStr : actualPaths) {
+  Path path = new Path(pathStr);
+  String device = path.getDevice();
+  // update tmpDeviceSet for a full path
+  tmpDeviceSet.add(device);
+
+  // ignore the duplicate prefix device for a given suffix path
+  // e.g. select s0 from root.vehicle.d0, root.vehicle.d0 group by 
device,
+  // for the given suffix path "s0", the second prefix device 
"root.vehicle.d0" is
+  // duplicated and should be neglected.
+  if (deviceSetOfGivenSuffix.contains(device)) {
+continue;
+  }
+
+  // get pathForDataType and measurementColumn
+  String measurement = path.getMeasurement();
+  String pathForDataType;
+  String measurementColumn;
+  if (originAggregations != null && !originAggregations.isEmpty()) 
{
+pathForDataType = originAggregations.get(i) + "(" + 
path.getFullPath() + ")";
+measurementColumn = originAggregations.get(i) + "(" + 
measurement + ")";
+  } else {
+pathForDataType = path.getFullPath();
+measurementColumn = measurement;
+  }
+  // check the consistency of data types
+  // a example of inconsistency: select s0 from root.sg1.d1, 
root.sg2.d3 group by device,
+  // while root.sg1.d1.s0 is INT32 and root.sg2.d3.s0 is FLOAT.
+  TSDataType dataType = 
TSServiceImpl.getSeriesType(pathForDataType);
+  if (dataTypeConsistencyChecker.containsKey(measurementColumn)) {
+if 
(!dataType.equals(dataTypeConsistencyChecker.get(measurementColumn))) {
+  throw new QueryProcessorException(
+   

[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #433: [IOTDB-203]add "group by device" function for narrow table display

2019-10-09 Thread GitBox
jt2594838 commented on a change in pull request #433: [IOTDB-203]add "group by 
device" function for narrow table display
URL: https://github.com/apache/incubator-iotdb/pull/433#discussion_r332810229
 
 

 ##
 File path: 
server/src/main/java/org/apache/iotdb/db/qp/strategy/PhysicalGenerator.java
 ##
 @@ -260,75 +269,122 @@ private PhysicalPlan transformQuery(QueryOperator 
queryOperator)
   queryPlan = new QueryPlan();
 }
 
-List paths = queryOperator.getSelectedPaths();
-queryPlan.setPaths(paths);
-queryPlan.checkPaths(executor);
+if (!queryOperator.isGroupByDevice()) {
+  List paths = queryOperator.getSelectedPaths();
+  queryPlan.setPaths(paths);
 
-if (queryOperator.isGroupByDevice()) {
-  queryPlan.setGroupByDevice(true);
-  // group paths and aggregations by devices
-  List aggregations = 
queryOperator.getSelectOperator().getAggregations();
-  TreeMap> pathsGroupByDevice = new TreeMap<>();
-  TreeMap> aggregationsGroupByDevice = new 
TreeMap<>();
-  TreeMap> sensorColumnsGroupByDevice = new 
TreeMap<>();
-  for (int i = 0; i < paths.size(); i++) {
-Path path = paths.get(i);
-String device = path.getDevice();
-if (!pathsGroupByDevice.containsKey(device)) {
-  pathsGroupByDevice.put(device, new ArrayList<>());
-  aggregationsGroupByDevice.put(device, new ArrayList<>());
-  sensorColumnsGroupByDevice.put(device, new ArrayList<>());
-}
-pathsGroupByDevice.get(device).add(path);
-if (aggregations != null && !aggregations.isEmpty()) {
-  aggregationsGroupByDevice.get(device).add(aggregations.get(i));
-  sensorColumnsGroupByDevice.get(device)
-  .add(aggregations.get(i) + "(" + paths.get(i).getMeasurement() + 
")");
-} else {
-  
sensorColumnsGroupByDevice.get(device).add(paths.get(i).getMeasurement());
+} else {
+  // below is the core realization of GROUP_BY_DEVICE sql
+  List prefixPaths = 
queryOperator.getFromOperator().getPrefixPaths();
+  List suffixPaths = 
queryOperator.getSelectOperator().getSuffixPaths();
+  List originAggregations = 
queryOperator.getSelectOperator().getAggregations();
+
+  List measurementColumnList = new ArrayList<>();
+  Map> measurementColumnsGroupByDevice = new 
LinkedHashMap<>();
+  Map dataTypeConsistencyChecker = new HashMap<>();
+  Set allSelectPaths = new HashSet<>();
+
+  for (int i = 0; i < suffixPaths.size(); i++) { // per suffix
+Path suffixPath = suffixPaths.get(i);
+Set deviceSetOfGivenSuffix = new HashSet<>();
+Set measurementSetOfGivenSuffix = new TreeSet<>();
+for (Path prefixPath : prefixPaths) { // per prefix
+  if (!prefixPath.startWith(SQLConstant.ROOT)) {
+throw new QueryProcessorException("illegal from clause : " + 
prefixPath.getFullPath());
+  }
 
 Review comment:
   I think the definition of prefixPath has guaranteed this 
   
![image](https://user-images.githubusercontent.com/23610645/66447973-92242080-ea82-11e9-8248-645bae038882.png)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


  1   2   3   4   5   6   7   8   9   10   >