[GitHub] [incubator-iotdb] Genius-pig commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance
Genius-pig commented on a change in pull request #440: reconstruct antlrv3
grammar to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r335009423
##
File path: server/src/main/antlr3/org/apache/iotdb/db/sql/parse/TqlLexer.g
##
@@ -0,0 +1,539 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+lexer grammar TqlLexer;
+
+@header {
+package org.apache.iotdb.db.sql.parse;
+}
+
+//*** key words *
+K_SELECT
+: S E L E C T
+;
+
+K_INTO
+: I N T O
+;
+
+K_ROOT
+: R O O T
+;
+
+K_FROM
+: F R O M
+;
+
+K_WHERE
+: W H E R E
+;
+
+K_LIMIT
+: L I M I T
+;
+
+K_OFFSET
+: O F F S E T
+;
+
+K_SLIMIT
+: S L I M I T
+;
+
+K_SOFFSET
+: S O F F S E T
+;
+
+K_NOW
+: N O W
+;
+
+K_GROUP
+: G R O U P
+;
+
+K_BY
+: B Y
+;
+
+K_FILL
+: F I L L
+;
+
+K_PREVIOUS
+: P R E V I O U S
+;
+
+K_LINEAR
+: L I N E A R
+;
+
+K_INT32
+: I N T '3' '2'
+;
+
+K_INT64
+: I N T '6' '4'
+;
+
+K_FLOAT
+: F L O A T
+;
+
+K_DOUBLE
+: D O U B L E
+;
+
+K_BOOLEAN
+: B O O L E A N
+;
+
+K_TEXT
+: T E X T
+;
+
+K_INSERT
+: I N S E R T
+;
+
+K_VALUES
+: V A L U E S
+;
+
+K_TIMESTAMP
+: T I M E S T A M P
+;
+
+K_UPDATE
+: U P D A T E
+;
+
+K_SET
+: S E T
+;
+
+K_DELETE
+: D E L E T E
+;
+
+K_CREATE
+: C R E A T E
+;
+
+K_TIMESERIES
+: T I M E S E R I E S
+;
+
+K_WITH
+: W I T H
+;
+
+K_DATATYPE
+: D A T A T Y P E
+;
+
+K_ENCODING
+: E N C O D I N G
+;
+
+K_COMPRESSOR
+: C O M P R E S S O R
+;
+
+K_STORAGE
+: S T O R A G E
+;
+
+K_TO
+: T O
+;
+
+K_PROPERTY
+: P R O P E R T Y
+;
+
+K_LABEL
+: L A B E L
+;
+
+
+K_LINK
+: L I N K
+;
+
+K_UNLINK
+: U N L I N K
+;
+
+K_SHOW
+: S H O W
+;
+
+K_METADATA
+: M E T A D A T A
+;
+
+K_DESCRIBE
+: D E S C R I B E
+;
+
+K_INDEX
+: I N D E X
+;
+
+K_ON
+: O N
+;
+
+K_USING
+: U S I N G
+;
+
+K_DROP
+: D R O P
+;
+
+K_MERGE
+: M E R G E
+;
+
+K_LIST
+: L I S T
+;
+
+K_USER
+: U S E R
+;
+
+K_PRIVILEGES
+ : P R I V I L E G E S
+ ;
+
+K_ROLE
+: R O L E
+;
+
+K_ALL
+: A L L
+;
+
+K_OF
+: O F
+;
+
+K_QUIT
+: Q U I T
+;
+
+K_ALTER
+: A L T E R
+;
+
+K_PASSWORD
+: P A S S W O R D
+;
+
+K_GRANT
+: G R A N T
+;
+
+K_REVOKE
+: R E V O K E
+;
+
+K_PATH
+: P A T H
+;
+
+K_LOAD
+: L O A D
+;
+
+K_WATERMARK_EMBEDDING
+: W A T E R M A R K '_' E M B E D D I N G
+;
+
+//** logical operator***
+OPERATOR_AND
+: A N D
+| '&'
+| '&&'
+;
+
+OPERATOR_OR
+: O R
+| '|'
+| '||'
+;
+
+OPERATOR_NOT
+: N O T | '!'
+;
+
+
+// data type ***
+K_PLAIN
+ : P L A I N
+ ;
+K_PLAIN_DICTIONARY
+ : P L A I N '_' D I C T I O N A R Y
+ ;
+K_RLE
+ : R L E
+ ;
+K_DIFF
+ : D I F F
+ ;
+K_TS_2DIFF
+ : T S '_' '2' D I F F
+ ;
+K_BITMAP
+ : B I T M A P
+ ;
+K_GORILLA
+ : G O R I L L A
+ ;
+K_REGULAR
+ : R E G U L A R
+ ;
+
+K_ADD
+: A D D
+;
+// *** comparison ***
+OPERATOR_GT
+: '>'
+;
+
+OPERATOR_GTE
+: '>='
+;
+
+OPERATOR_LT
+: '<'
+;
+
+OPERATOR_LTE
+: '<='
+;
+
+OPERATOR_EQ
+: '=' | '=='
+;
+
+OPERATOR_NEQ
+: '!=' | '<>'
+;
+
+// operator ***
+STAR
+: '*'
+;
+
+MINUS
+: '-'
+;
+
+PLUS
+: '+'
+;
+
+DIVIDE
+: '/'
+;
+
+// symbol ***
+SEMI
+: ';'
+;
+
+DOT
+: '.'
+;
+
+COMMA
+: ','
+;
+
+LR_BRACKET
+: '('
+;
+
+RR_BRACKET
+: ')'
+;
+
+LS_BRACKET
+: '['
+;
+
+RS_BRACKET
+: ']'
+;
+
+STRING_LITERAL
+ : DQUOTA_STRING
+ | SQUOTA_STRING
+ ;
+
+fragment DQUOTA_STRING
+ : '"' ( '\\'. | '""' | ~('"'| '\\') )* '"'
+ ;
+
+fragment SQUOTA_STRING
+ :
[GitHub] [incubator-iotdb] jixuan1989 commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance
jixuan1989 commented on a change in pull request #440: reconstruct antlrv3
grammar to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334965163
##
File path: server/src/main/antlr3/org/apache/iotdb/db/sql/parse/TqlLexer.g
##
@@ -0,0 +1,539 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+lexer grammar TqlLexer;
+
+@header {
+package org.apache.iotdb.db.sql.parse;
+}
+
+//*** key words *
+K_SELECT
+: S E L E C T
+;
+
+K_INTO
+: I N T O
+;
+
+K_ROOT
+: R O O T
+;
+
+K_FROM
+: F R O M
+;
+
+K_WHERE
+: W H E R E
+;
+
+K_LIMIT
+: L I M I T
+;
+
+K_OFFSET
+: O F F S E T
+;
+
+K_SLIMIT
+: S L I M I T
+;
+
+K_SOFFSET
+: S O F F S E T
+;
+
+K_NOW
+: N O W
+;
+
+K_GROUP
+: G R O U P
+;
+
+K_BY
+: B Y
+;
+
+K_FILL
+: F I L L
+;
+
+K_PREVIOUS
+: P R E V I O U S
+;
+
+K_LINEAR
+: L I N E A R
+;
+
+K_INT32
+: I N T '3' '2'
+;
+
+K_INT64
+: I N T '6' '4'
+;
+
+K_FLOAT
+: F L O A T
+;
+
+K_DOUBLE
+: D O U B L E
+;
+
+K_BOOLEAN
+: B O O L E A N
+;
+
+K_TEXT
+: T E X T
+;
+
+K_INSERT
+: I N S E R T
+;
+
+K_VALUES
+: V A L U E S
+;
+
+K_TIMESTAMP
+: T I M E S T A M P
+;
+
+K_UPDATE
+: U P D A T E
+;
+
+K_SET
+: S E T
+;
+
+K_DELETE
+: D E L E T E
+;
+
+K_CREATE
+: C R E A T E
+;
+
+K_TIMESERIES
+: T I M E S E R I E S
+;
+
+K_WITH
+: W I T H
+;
+
+K_DATATYPE
+: D A T A T Y P E
+;
+
+K_ENCODING
+: E N C O D I N G
+;
+
+K_COMPRESSOR
+: C O M P R E S S O R
+;
+
+K_STORAGE
+: S T O R A G E
+;
+
+K_TO
+: T O
+;
+
+K_PROPERTY
+: P R O P E R T Y
+;
+
+K_LABEL
+: L A B E L
+;
+
+
+K_LINK
+: L I N K
+;
+
+K_UNLINK
+: U N L I N K
+;
+
+K_SHOW
+: S H O W
+;
+
+K_METADATA
+: M E T A D A T A
+;
+
+K_DESCRIBE
+: D E S C R I B E
+;
+
+K_INDEX
+: I N D E X
+;
+
+K_ON
+: O N
+;
+
+K_USING
+: U S I N G
+;
+
+K_DROP
+: D R O P
+;
+
+K_MERGE
+: M E R G E
+;
+
+K_LIST
+: L I S T
+;
+
+K_USER
+: U S E R
+;
+
+K_PRIVILEGES
+ : P R I V I L E G E S
+ ;
+
+K_ROLE
+: R O L E
+;
+
+K_ALL
+: A L L
+;
+
+K_OF
+: O F
+;
+
+K_QUIT
+: Q U I T
+;
+
+K_ALTER
+: A L T E R
+;
+
+K_PASSWORD
+: P A S S W O R D
+;
+
+K_GRANT
+: G R A N T
+;
+
+K_REVOKE
+: R E V O K E
+;
+
+K_PATH
+: P A T H
+;
+
+K_LOAD
+: L O A D
+;
+
+K_WATERMARK_EMBEDDING
+: W A T E R M A R K '_' E M B E D D I N G
+;
+
+//** logical operator***
+OPERATOR_AND
+: A N D
+| '&'
+| '&&'
+;
+
+OPERATOR_OR
+: O R
+| '|'
+| '||'
+;
+
+OPERATOR_NOT
+: N O T | '!'
+;
+
+
+// data type ***
+K_PLAIN
+ : P L A I N
+ ;
+K_PLAIN_DICTIONARY
+ : P L A I N '_' D I C T I O N A R Y
+ ;
+K_RLE
+ : R L E
+ ;
+K_DIFF
+ : D I F F
+ ;
+K_TS_2DIFF
+ : T S '_' '2' D I F F
+ ;
+K_BITMAP
+ : B I T M A P
+ ;
+K_GORILLA
+ : G O R I L L A
+ ;
+K_REGULAR
+ : R E G U L A R
+ ;
+
+K_ADD
+: A D D
+;
+// *** comparison ***
+OPERATOR_GT
+: '>'
+;
+
+OPERATOR_GTE
+: '>='
+;
+
+OPERATOR_LT
+: '<'
+;
+
+OPERATOR_LTE
+: '<='
+;
+
+OPERATOR_EQ
+: '=' | '=='
+;
+
+OPERATOR_NEQ
+: '!=' | '<>'
+;
+
+// operator ***
+STAR
+: '*'
+;
+
+MINUS
+: '-'
+;
+
+PLUS
+: '+'
+;
+
+DIVIDE
+: '/'
+;
+
+// symbol ***
+SEMI
+: ';'
+;
+
+DOT
+: '.'
+;
+
+COMMA
+: ','
+;
+
+LR_BRACKET
+: '('
+;
+
+RR_BRACKET
+: ')'
+;
+
+LS_BRACKET
+: '['
+;
+
+RS_BRACKET
+: ']'
+;
+
+STRING_LITERAL
+ : DQUOTA_STRING
+ | SQUOTA_STRING
+ ;
+
+fragment DQUOTA_STRING
+ : '"' ( '\\'. | '""' | ~('"'| '\\') )* '"'
+ ;
+
+fragment SQUOTA_STRING
+ :
[GitHub] [incubator-iotdb] SolomonAnn commented on a change in pull request #430: [IOTDB-193]Create schema automatically
SolomonAnn commented on a change in pull request #430: [IOTDB-193]Create schema
automatically
URL: https://github.com/apache/incubator-iotdb/pull/430#discussion_r334953104
##
File path: server/src/main/java/org/apache/iotdb/db/utils/DeviceIdCache.java
##
@@ -0,0 +1,53 @@
+package org.apache.iotdb.db.utils;
+
+import org.apache.iotdb.db.exception.PathErrorException;
+import org.apache.iotdb.db.exception.StorageGroupException;
+import org.apache.iotdb.tsfile.exception.cache.CacheException;
+
+import java.util.List;
+import java.util.concurrent.CopyOnWriteArrayList;
+
+public abstract class DeviceIdCache {
Review comment:
I'm confused that whether I should use `mNodeCache `directly because its key
is just `deviceId`. However, if I choose to use `mNodeCache`, I will have
trouble with handling exceptions since I need `PathErrorException `and
`StorageGroupException` to distinguish two different cases while `mNodeCache`
throws `CacheException` in any case. Do you have any suggestions?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] LeiRui commented on issue #429: [IOTDB-205]Support storage-group-level data ttl
LeiRui commented on issue #429: [IOTDB-205]Support storage-group-level data ttl URL: https://github.com/apache/incubator-iotdb/pull/429#issuecomment-542204822 Hi, This is a review on functionality. I find the operation `unset TTL to ` REALLY requires a seasoned user who knows exactly what he/she is doing. And I totally agree with what you have noted in the guidance: > Notice: When you set TTL to some storage groups, data out of the TTL will be made invisible immediately, but because the data files may contain both out-dated and living data or the data files may be being used by queries, the physical removal of data is stale. If you increase or unset TTL just after setting it previously, some previously invisible data may be seen again, but the physically removed one is lost forever. **In other word, different from delete statement, the atomicity of data deletion is not guaranteed for efficiency concerns.** **So we recommend that you do not change the TTL once it is set or at least do not reset it frequently, unless you are determined to suffer the unpredictability.** I want to draw attention to another point that can be easily misunderstood when using the operation `unset TTL to `. That is, the latest insertion time of a storage group, which is the split point used to determine whether the coming data point is sequential or unsequential, will not be “recycled" even if the data before `now()-TTL` are really deleted. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] qiaojialin closed pull request #447: [IOTDB-250] exclude .mvn/wrapper/maven-wrapper.jar
qiaojialin closed pull request #447: [IOTDB-250] exclude .mvn/wrapper/maven-wrapper.jar URL: https://github.com/apache/incubator-iotdb/pull/447 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] Ring-k commented on issue #440: reconstruct antlrv3 grammar to improve performance
Ring-k commented on issue #440: reconstruct antlrv3 grammar to improve performance URL: https://github.com/apache/incubator-iotdb/pull/440#issuecomment-542160163 > > No "=>" operators anymore. > > "=>" operator means always execute predicate. Removing all of them is a real improvement? Syntactic predicates were used to work around a prediction weakness in ANTLR 3. For each syntactic predicate, ANTLR defines a special method that returns true or false depending on whether the predicate’s grammar fragment matches the next input symbols. So if syntactic predicate is frequently used, the method will be frequently called. So, in my opinion, a simpler design leads to better performance. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] HTHou commented on issue #450: modify documents
HTHou commented on issue #450: modify documents URL: https://github.com/apache/incubator-iotdb/pull/450#issuecomment-542115088 > * about organization logic > > * I find that the TsFile docs are splitted into `6-API` and `7-System Design`. My questions are: > > * Why TsFile's installation and usage are classified as API? > * Is the chapter title `System Design` too big when there is only a doc about TsFile's hierarchy design in this chapter? > * Why splitting TsFile docs? > * about titles and outlines > > * Some articles under the directory `4-Operation Manual` have the stale chapter title `Chapter 3: Operation Manual`. > * It seems all articles under the directory `6-API` lack their article titles. > * Some files have outlines while some others don't. Best every file has its chapter title (if there is), article title and outline. > * about links > > * I find in QuickStart.md, some links link to the website 0.8.0 version docs. Shouldn't they link to master docs since the master docs are latest? > * about Chinese docs > > * Since they are written in Chinese, I wonder whether the file names and directory names can be in Chinese too. * for organization logic * The chapter 6 should be related to operation, the name `API` may not good enough. Any advice? * In chapter `System Design`, there is only one section now. There will be more documents in this chapter in the future. * The reason why I split TsFile docs is the TsFile hierarchy is more related to the introduction of design, and other docs are focus on user's operation. * about titles and outlines * The issues you listed are some mistakes. They should be fixed. * about links * It is a mistake too. I didn't check `QuickStart.md` when I modify the docs. Thank you. * about Chinese docs * If we change the file names and directory names, we have to change our website too, since our Chinese docs website is using same directory and file names. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] samperson1997 commented on a change in pull request #417: [IOTDB-234] Refactor TsFile storage on HDFS
samperson1997 commented on a change in pull request #417: [IOTDB-234] Refactor
TsFile storage on HDFS
URL: https://github.com/apache/incubator-iotdb/pull/417#discussion_r334832198
##
File path:
tsfile/src/main/java/org/apache/iotdb/tsfile/fileSystem/FileInputFactory.java
##
@@ -19,33 +19,39 @@
package org.apache.iotdb.tsfile.fileSystem;
+import java.io.IOException;
+import java.lang.reflect.InvocationTargetException;
+import java.nio.file.Paths;
import org.apache.iotdb.tsfile.common.conf.TSFileDescriptor;
import org.apache.iotdb.tsfile.read.reader.DefaultTsFileInput;
import org.apache.iotdb.tsfile.read.reader.TsFileInput;
-import org.apache.iotdb.tsfile.write.writer.TsFileIOWriter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import java.io.IOException;
-import java.nio.file.Paths;
-
public enum FileInputFactory {
Review comment:
Should I separate these different factories into three packages? like
`fsFactory`, `fileInputFactory` and `fileOutputFactory`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #417: [IOTDB-234] Refactor TsFile storage on HDFS
jt2594838 commented on a change in pull request #417: [IOTDB-234] Refactor
TsFile storage on HDFS
URL: https://github.com/apache/incubator-iotdb/pull/417#discussion_r334831041
##
File path:
tsfile/src/main/java/org/apache/iotdb/tsfile/fileSystem/FileInputFactory.java
##
@@ -19,33 +19,39 @@
package org.apache.iotdb.tsfile.fileSystem;
+import java.io.IOException;
+import java.lang.reflect.InvocationTargetException;
+import java.nio.file.Paths;
import org.apache.iotdb.tsfile.common.conf.TSFileDescriptor;
import org.apache.iotdb.tsfile.read.reader.DefaultTsFileInput;
import org.apache.iotdb.tsfile.read.reader.TsFileInput;
-import org.apache.iotdb.tsfile.write.writer.TsFileIOWriter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import java.io.IOException;
-import java.nio.file.Paths;
-
public enum FileInputFactory {
Review comment:
Better to perform your refactoring on these two factories, too.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance
Ring-k commented on a change in pull request #440: reconstruct antlrv3 grammar
to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334828519
##
File path: server/src/main/antlr3/org/apache/iotdb/db/sql/parse/TqlLexer.g
##
@@ -0,0 +1,539 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+lexer grammar TqlLexer;
+
+@header {
+package org.apache.iotdb.db.sql.parse;
+}
+
+//*** key words *
+K_SELECT
+: S E L E C T
+;
+
+K_INTO
+: I N T O
+;
+
+K_ROOT
+: R O O T
+;
+
+K_FROM
+: F R O M
+;
+
+K_WHERE
+: W H E R E
+;
+
+K_LIMIT
+: L I M I T
+;
+
+K_OFFSET
+: O F F S E T
+;
+
+K_SLIMIT
+: S L I M I T
+;
+
+K_SOFFSET
+: S O F F S E T
+;
+
+K_NOW
+: N O W
+;
+
+K_GROUP
+: G R O U P
+;
+
+K_BY
+: B Y
+;
+
+K_FILL
+: F I L L
+;
+
+K_PREVIOUS
+: P R E V I O U S
+;
+
+K_LINEAR
+: L I N E A R
+;
+
+K_INT32
+: I N T '3' '2'
+;
+
+K_INT64
+: I N T '6' '4'
+;
+
+K_FLOAT
+: F L O A T
+;
+
+K_DOUBLE
+: D O U B L E
+;
+
+K_BOOLEAN
+: B O O L E A N
+;
+
+K_TEXT
+: T E X T
+;
+
+K_INSERT
+: I N S E R T
+;
+
+K_VALUES
+: V A L U E S
+;
+
+K_TIMESTAMP
+: T I M E S T A M P
+;
+
+K_UPDATE
+: U P D A T E
+;
+
+K_SET
+: S E T
+;
+
+K_DELETE
+: D E L E T E
+;
+
+K_CREATE
+: C R E A T E
+;
+
+K_TIMESERIES
+: T I M E S E R I E S
+;
+
+K_WITH
+: W I T H
+;
+
+K_DATATYPE
+: D A T A T Y P E
+;
+
+K_ENCODING
+: E N C O D I N G
+;
+
+K_COMPRESSOR
+: C O M P R E S S O R
+;
+
+K_STORAGE
+: S T O R A G E
+;
+
+K_TO
+: T O
+;
+
+K_PROPERTY
+: P R O P E R T Y
+;
+
+K_LABEL
+: L A B E L
+;
+
+
+K_LINK
+: L I N K
+;
+
+K_UNLINK
+: U N L I N K
+;
+
+K_SHOW
+: S H O W
+;
+
+K_METADATA
+: M E T A D A T A
+;
+
+K_DESCRIBE
+: D E S C R I B E
+;
+
+K_INDEX
+: I N D E X
+;
+
+K_ON
+: O N
+;
+
+K_USING
+: U S I N G
+;
+
+K_DROP
+: D R O P
+;
+
+K_MERGE
+: M E R G E
+;
+
+K_LIST
+: L I S T
+;
+
+K_USER
+: U S E R
+;
+
+K_PRIVILEGES
+ : P R I V I L E G E S
+ ;
+
+K_ROLE
+: R O L E
+;
+
+K_ALL
+: A L L
+;
+
+K_OF
+: O F
+;
+
+K_QUIT
+: Q U I T
+;
+
+K_ALTER
+: A L T E R
+;
+
+K_PASSWORD
+: P A S S W O R D
+;
+
+K_GRANT
+: G R A N T
+;
+
+K_REVOKE
+: R E V O K E
+;
+
+K_PATH
+: P A T H
+;
+
+K_LOAD
+: L O A D
+;
+
+K_WATERMARK_EMBEDDING
+: W A T E R M A R K '_' E M B E D D I N G
+;
+
+//** logical operator***
+OPERATOR_AND
+: A N D
+| '&'
+| '&&'
+;
+
+OPERATOR_OR
+: O R
+| '|'
+| '||'
+;
+
+OPERATOR_NOT
+: N O T | '!'
+;
+
+
+// data type ***
+K_PLAIN
+ : P L A I N
+ ;
+K_PLAIN_DICTIONARY
+ : P L A I N '_' D I C T I O N A R Y
+ ;
+K_RLE
+ : R L E
+ ;
+K_DIFF
+ : D I F F
+ ;
+K_TS_2DIFF
+ : T S '_' '2' D I F F
+ ;
+K_BITMAP
+ : B I T M A P
+ ;
+K_GORILLA
+ : G O R I L L A
+ ;
+K_REGULAR
+ : R E G U L A R
+ ;
+
+K_ADD
+: A D D
+;
+// *** comparison ***
+OPERATOR_GT
+: '>'
+;
+
+OPERATOR_GTE
+: '>='
+;
+
+OPERATOR_LT
+: '<'
+;
+
+OPERATOR_LTE
+: '<='
+;
+
+OPERATOR_EQ
+: '=' | '=='
+;
+
+OPERATOR_NEQ
+: '!=' | '<>'
+;
+
+// operator ***
+STAR
+: '*'
+;
+
+MINUS
+: '-'
+;
+
+PLUS
+: '+'
+;
+
+DIVIDE
+: '/'
+;
+
+// symbol ***
+SEMI
+: ';'
+;
+
+DOT
+: '.'
+;
+
+COMMA
+: ','
+;
+
+LR_BRACKET
+: '('
+;
+
+RR_BRACKET
+: ')'
+;
+
+LS_BRACKET
+: '['
+;
+
+RS_BRACKET
+: ']'
+;
+
+STRING_LITERAL
+ : DQUOTA_STRING
+ | SQUOTA_STRING
+ ;
+
+fragment DQUOTA_STRING
+ : '"' ( '\\'. | '""' | ~('"'| '\\') )* '"'
+ ;
+
+fragment SQUOTA_STRING
+ : '\''
[GitHub] [incubator-iotdb] Genius-pig commented on a change in pull request #440: reconstruct antlrv3 grammar to improve performance
Genius-pig commented on a change in pull request #440: reconstruct antlrv3
grammar to improve performance
URL: https://github.com/apache/incubator-iotdb/pull/440#discussion_r334822659
##
File path: server/src/main/antlr3/org/apache/iotdb/db/sql/parse/TqlLexer.g
##
@@ -0,0 +1,539 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+lexer grammar TqlLexer;
+
+@header {
+package org.apache.iotdb.db.sql.parse;
+}
+
+//*** key words *
+K_SELECT
+: S E L E C T
+;
+
+K_INTO
+: I N T O
+;
+
+K_ROOT
+: R O O T
+;
+
+K_FROM
+: F R O M
+;
+
+K_WHERE
+: W H E R E
+;
+
+K_LIMIT
+: L I M I T
+;
+
+K_OFFSET
+: O F F S E T
+;
+
+K_SLIMIT
+: S L I M I T
+;
+
+K_SOFFSET
+: S O F F S E T
+;
+
+K_NOW
+: N O W
+;
+
+K_GROUP
+: G R O U P
+;
+
+K_BY
+: B Y
+;
+
+K_FILL
+: F I L L
+;
+
+K_PREVIOUS
+: P R E V I O U S
+;
+
+K_LINEAR
+: L I N E A R
+;
+
+K_INT32
+: I N T '3' '2'
+;
+
+K_INT64
+: I N T '6' '4'
+;
+
+K_FLOAT
+: F L O A T
+;
+
+K_DOUBLE
+: D O U B L E
+;
+
+K_BOOLEAN
+: B O O L E A N
+;
+
+K_TEXT
+: T E X T
+;
+
+K_INSERT
+: I N S E R T
+;
+
+K_VALUES
+: V A L U E S
+;
+
+K_TIMESTAMP
+: T I M E S T A M P
+;
+
+K_UPDATE
+: U P D A T E
+;
+
+K_SET
+: S E T
+;
+
+K_DELETE
+: D E L E T E
+;
+
+K_CREATE
+: C R E A T E
+;
+
+K_TIMESERIES
+: T I M E S E R I E S
+;
+
+K_WITH
+: W I T H
+;
+
+K_DATATYPE
+: D A T A T Y P E
+;
+
+K_ENCODING
+: E N C O D I N G
+;
+
+K_COMPRESSOR
+: C O M P R E S S O R
+;
+
+K_STORAGE
+: S T O R A G E
+;
+
+K_TO
+: T O
+;
+
+K_PROPERTY
+: P R O P E R T Y
+;
+
+K_LABEL
+: L A B E L
+;
+
+
+K_LINK
+: L I N K
+;
+
+K_UNLINK
+: U N L I N K
+;
+
+K_SHOW
+: S H O W
+;
+
+K_METADATA
+: M E T A D A T A
+;
+
+K_DESCRIBE
+: D E S C R I B E
+;
+
+K_INDEX
+: I N D E X
+;
+
+K_ON
+: O N
+;
+
+K_USING
+: U S I N G
+;
+
+K_DROP
+: D R O P
+;
+
+K_MERGE
+: M E R G E
+;
+
+K_LIST
+: L I S T
+;
+
+K_USER
+: U S E R
+;
+
+K_PRIVILEGES
+ : P R I V I L E G E S
+ ;
+
+K_ROLE
+: R O L E
+;
+
+K_ALL
+: A L L
+;
+
+K_OF
+: O F
+;
+
+K_QUIT
+: Q U I T
+;
+
+K_ALTER
+: A L T E R
+;
+
+K_PASSWORD
+: P A S S W O R D
+;
+
+K_GRANT
+: G R A N T
+;
+
+K_REVOKE
+: R E V O K E
+;
+
+K_PATH
+: P A T H
+;
+
+K_LOAD
+: L O A D
+;
+
+K_WATERMARK_EMBEDDING
+: W A T E R M A R K '_' E M B E D D I N G
+;
+
+//** logical operator***
+OPERATOR_AND
+: A N D
+| '&'
+| '&&'
+;
+
+OPERATOR_OR
+: O R
+| '|'
+| '||'
+;
+
+OPERATOR_NOT
+: N O T | '!'
+;
+
+
+// data type ***
+K_PLAIN
+ : P L A I N
+ ;
+K_PLAIN_DICTIONARY
+ : P L A I N '_' D I C T I O N A R Y
+ ;
+K_RLE
+ : R L E
+ ;
+K_DIFF
+ : D I F F
+ ;
+K_TS_2DIFF
+ : T S '_' '2' D I F F
+ ;
+K_BITMAP
+ : B I T M A P
+ ;
+K_GORILLA
+ : G O R I L L A
+ ;
+K_REGULAR
+ : R E G U L A R
+ ;
+
+K_ADD
+: A D D
+;
+// *** comparison ***
+OPERATOR_GT
+: '>'
+;
+
+OPERATOR_GTE
+: '>='
+;
+
+OPERATOR_LT
+: '<'
+;
+
+OPERATOR_LTE
+: '<='
+;
+
+OPERATOR_EQ
+: '=' | '=='
+;
+
+OPERATOR_NEQ
+: '!=' | '<>'
+;
+
+// operator ***
+STAR
+: '*'
+;
+
+MINUS
+: '-'
+;
+
+PLUS
+: '+'
+;
+
+DIVIDE
+: '/'
+;
+
+// symbol ***
+SEMI
+: ';'
+;
+
+DOT
+: '.'
+;
+
+COMMA
+: ','
+;
+
+LR_BRACKET
+: '('
+;
+
+RR_BRACKET
+: ')'
+;
+
+LS_BRACKET
+: '['
+;
+
+RS_BRACKET
+: ']'
+;
+
+STRING_LITERAL
+ : DQUOTA_STRING
+ | SQUOTA_STRING
+ ;
+
+fragment DQUOTA_STRING
+ : '"' ( '\\'. | '""' | ~('"'| '\\') )* '"'
+ ;
+
+fragment SQUOTA_STRING
+ :
[GitHub] [incubator-iotdb] Genius-pig commented on issue #440: reconstruct antlrv3 grammar to improve performance
Genius-pig commented on issue #440: reconstruct antlrv3 grammar to improve performance URL: https://github.com/apache/incubator-iotdb/pull/440#issuecomment-542104524 > No "=>" operators anymore. "=>" operator means always execute predicate. Removing all of them is a real improvement? Syntactic predicates were used to work around a prediction weakness in ANTLR 3. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] Genius-pig edited a comment on issue #440: reconstruct antlrv3 grammar to improve performance
Genius-pig edited a comment on issue #440: reconstruct antlrv3 grammar to improve performance URL: https://github.com/apache/incubator-iotdb/pull/440#issuecomment-542104524 > No "=>" operators anymore. "=>" operator means always execute predicate. Removing all of them is a real improvement? Syntactic predicates were used to work around a prediction weakness in ANTLR 3. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #417: [IOTDB-234] Refactor TsFile storage on HDFS
jt2594838 commented on a change in pull request #417: [IOTDB-234] Refactor
TsFile storage on HDFS
URL: https://github.com/apache/incubator-iotdb/pull/417#discussion_r334809838
##
File path:
tsfile/src/main/java/org/apache/iotdb/tsfile/fileSystem/HDFSFactory.java
##
@@ -0,0 +1,178 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.tsfile.fileSystem;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.lang.reflect.Constructor;
+import java.lang.reflect.InvocationTargetException;
+import java.net.URI;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class HDFSFactory implements FSFactory {
+
+ private static final Logger logger =
LoggerFactory.getLogger(HDFSFactory.class);
+ private static Class clazz;
+ private static Constructor constructorWithPathname;
+ private static Constructor constructorWithParentStringAndChild;
+ private static Constructor constructorWithParentFileAndChild;
+ private static Constructor constructorWithUri;
+
+ static {
+try {
+ clazz = Class.forName("org.apache.iotdb.tsfile.fileSystem.HDFSFile");
+ constructorWithPathname = clazz.getConstructor(String.class);
+ constructorWithParentStringAndChild = clazz.getConstructor(String.class,
String.class);
+ constructorWithParentFileAndChild = clazz.getConstructor(File.class,
String.class);
+ constructorWithUri = clazz.getConstructor(URI.class);
+} catch (ClassNotFoundException | NoSuchMethodException e) {
+ logger.error(
+ "Failed to get Hadoop file system. Please check your dependency of
Hadoop module.", e);
+}
+ }
+
+ public File getFile(String pathname) {
+try {
+ return (File) constructorWithPathname.newInstance(pathname);
+} catch (InstantiationException | InvocationTargetException |
IllegalAccessException e) {
+ logger.error(
+ "Failed to get file: {}. Please check your dependency of Hadoop
module.", pathname, e);
+ return null;
+}
+ }
+
+ public File getFile(String parent, String child) {
+try {
+ return (File) constructorWithParentStringAndChild.newInstance(parent,
child);
+} catch (InstantiationException | InvocationTargetException |
IllegalAccessException e) {
+ logger.error(
+ "Failed to get file: {}" + File.separator
+ + "{}. Please check your dependency of Hadoop module.", parent,
child, e);
+ return null;
+}
+ }
+
+ public File getFile(File parent, String child) {
+try {
+ return (File) constructorWithParentFileAndChild.newInstance(parent,
child);
+} catch (InstantiationException | InvocationTargetException |
IllegalAccessException e) {
+ logger.error(
+ "Failed to get file: {}" + File.separator
+ + "{}. Please check your dependency of Hadoop module.",
parent.getAbsolutePath(),
+ child, e);
+ return null;
+}
+ }
+
+ public File getFile(URI uri) {
+try {
+ return (File) constructorWithUri.newInstance(uri);
+} catch (InstantiationException | InvocationTargetException |
IllegalAccessException e) {
+ logger.error(
+ "Failed to get file: {}. Please check your dependency of Hadoop
module.",
+ uri.toString(), e);
+ return null;
+}
+ }
+
+ public BufferedReader getBufferedReader(String filePath) {
+try {
+ return (BufferedReader) clazz.getMethod("getBufferedReader",
String.class)
+ .invoke(clazz.newInstance(), filePath);
Review comment:
I think you can save the getter Methods as you have saved the Constructors.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] LeiRui opened a new pull request #451: [IOTDB-252]add/fix shell and bat for TsFileSketchTool/TsFileResourcePrinter
LeiRui opened a new pull request #451: [IOTDB-252]add/fix shell and bat for
TsFileSketchTool/TsFileResourcePrinter
URL: https://github.com/apache/incubator-iotdb/pull/451
Usage Examples:
- ./print-tsfile-resource-files.sh /home/rl/fileDir
```
-
Starting Printing the TsFileResources
-
15:44:46.640 [main] WARN org.apache.iotdb.db.conf.IoTDBDescriptor - Cannot
find IOTDB_HOME or IOTDB_CONF environment variable when loading config file
iotdb-engine.properties, use default configuration
15:44:46.655 [main] WARN
org.apache.iotdb.tsfile.common.conf.TSFileDescriptor - Failed to find config
file tsfile-format.properties at classpath, use default configuration
analyzing /home/rl/tmp/1571064257513-101.tsfile ...
device root.vehicle.d0, start time 3000
(1970-01-01T08:00:03+08:00[Asia/Shanghai]), end time 100999
(1970-01-01T08:01:40.999+08:00[Asia/Shanghai])
analyzing the resource file finished.
```
- print-tsfile-resource-files.bat /home/rl/fileDir is similar
- print-tsfile-sketch.sh /home/rl/fileDir/1571064257513-101.tsfile
```
-
Starting Printing the TsFile Sketch
-
filename:/home/rl/tmp/1571064257513-101.tsfile
outFile:TsFile_sketch_view.txt
15:47:24.881 [main] WARN
org.apache.iotdb.tsfile.common.conf.TSFileDescriptor - Failed to find config
file tsfile-format.properties at classpath, use default configuration
TsFile Sketch
file path: /home/rl/tmp/1571064257513-101.tsfile
file length: 187349
POSITION|CONTENT
---
0|[magic head] TsFilev0.8.0
>[Chunk Group] of root.vehicle.d0 begins at pos
12, ends at pos 186469, version:102, num of Chunks:6
12|[Chunk] of s3, numOfPoints:10600, time
range:[3000,13599], tsDataType:TEXT,
TsDigest:[min_value:A,max_value:E,first_value:A,last_value:E,sum_value:0.0]
|[marker] 1
|[ChunkHeader]
|num of pages: 11
55718|[Chunk] of s4, numOfPoints:10600, time
range:[3000,13599], tsDataType:BOOLEAN,
TsDigest:[min_value:false,max_value:true,first_value:true,last_value:false,sum_value:0.0]
|[marker] 1
|[ChunkHeader]
|num of pages: 11
68848|[Chunk] of s5, numOfPoints:10600, time
range:[3000,13599], tsDataType:DOUBLE,
TsDigest:[min_value:3000.0,max_value:13599.0,first_value:3000.0,last_value:13599.0,sum_value:8.79747E7]
|[marker] 1
|[ChunkHeader]
|num of pages: 11
98474|[Chunk] of s0, numOfPoints:21900, time
range:[3000,100999], tsDataType:INT32,
TsDigest:[min_value:0,max_value:99,first_value:0,last_value:19,sum_value:889750.0]
|[marker] 1
|[ChunkHeader]
|num of pages: 22
123369|[Chunk] of s1, numOfPoints:21900, time
range:[3000,100999], tsDataType:INT64,
TsDigest:[min_value:0,max_value:39,first_value:8,last_value:19,sum_value:300386.0]
|[marker] 1
|[ChunkHeader]
|num of pages: 22
144741|[Chunk] of s2, numOfPoints:21900, time
range:[3000,100999], tsDataType:FLOAT,
TsDigest:[min_value:0.0,max_value:122.0,first_value:8.0,last_value:52.0,sum_value:778581.0]
|[marker] 1
|[ChunkHeader]
|num of pages: 22
|[marker] 0
|[Chunk Group Footer]
<[Chunk Group] of root.vehicle.d0 ends
186469|[marker] 2
186470|[TsDeviceMetadata] of root.vehicle.d0,
startTime:3000, endTime:100999
|num of ChunkGroupMetaData: 1
187133|[TsFileMetaData]
|num of TsDeviceMetadataIndex: 1
|num of measurementSchema: 6
187333|[TsFileMetaDataSize] 200
187337|[magic tail] TsFilev0.8.0
187349|
[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #447: [IOTDB-250] exclude .mvn/wrapper/maven-wrapper.jar
qiaojialin commented on a change in pull request #447: [IOTDB-250] exclude .mvn/wrapper/maven-wrapper.jar URL: https://github.com/apache/incubator-iotdb/pull/447#discussion_r334776744 ## File path: .gitignore ## @@ -79,7 +82,6 @@ grafana/data/ grafana/data/test.csv **/lib/ -*.jar Review comment: it is duplicated... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
