yuzelin commented on code in PR #20931:
URL: https://github.com/apache/flink/pull/20931#discussion_r985271871
##########
flink-table/flink-sql-client/pom.xml:
##########
@@ -511,6 +511,12 @@ under the License.
<scope>provided</scope>
</dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-sql-parser</artifactId>
+ <version>${project.version}</version>
+ </dependency>
Review Comment:
Added configuration to the shade plugin. I pushed this commit and see if it
works.
##########
flink-table/flink-sql-client/pom.xml:
##########
@@ -511,6 +511,12 @@ under the License.
<scope>provided</scope>
</dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-sql-parser</artifactId>
+ <version>${project.version}</version>
+ </dependency>
Review Comment:
Added new configuration to shade plugin. I pushed a new commit to see if it
works.
##########
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/parser/ClientParser.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.client.cli.parser;
+
+import org.apache.flink.sql.parser.impl.FlinkSqlParserImplTokenManager;
+import org.apache.flink.sql.parser.impl.SimpleCharStream;
+import org.apache.flink.sql.parser.impl.Token;
+import org.apache.flink.table.client.gateway.SqlExecutionException;
+import org.apache.flink.table.operations.Operation;
+
+import javax.annotation.Nonnull;
+
+import java.io.StringReader;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Optional;
+
+/** ClientParser use {@link FlinkSqlParserImplTokenManager} to do lexical
analysis. */
Review Comment:
Added.
##########
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/parser/ClientParser.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.client.cli.parser;
+
+import org.apache.flink.sql.parser.impl.FlinkSqlParserImplTokenManager;
+import org.apache.flink.sql.parser.impl.SimpleCharStream;
+import org.apache.flink.sql.parser.impl.Token;
+import org.apache.flink.table.client.gateway.SqlExecutionException;
+import org.apache.flink.table.operations.Operation;
+
+import javax.annotation.Nonnull;
+
+import java.io.StringReader;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Optional;
+
+/** ClientParser use {@link FlinkSqlParserImplTokenManager} to do lexical
analysis. */
+public class ClientParser implements SqlCommandParser {
+
+ /** A dumb implementation. TODO: remove this after unifying the
SqlMultiLineParser. */
+ @Override
+ public Optional<Operation> parseCommand(String command) {
+ return Optional.empty();
+ }
+
+ public Optional<StatementType> parseStatement(@Nonnull String statement)
+ throws SqlExecutionException {
+ String trimmedStatement = statement.trim();
+ FlinkSqlParserImplTokenManager tokenManager =
+ new FlinkSqlParserImplTokenManager(
+ new SimpleCharStream(new
StringReader(trimmedStatement)));
Review Comment:
Did.
##########
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/parser/ClientParser.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.client.cli.parser;
+
+import org.apache.flink.sql.parser.impl.FlinkSqlParserImplTokenManager;
+import org.apache.flink.sql.parser.impl.SimpleCharStream;
+import org.apache.flink.sql.parser.impl.Token;
+import org.apache.flink.table.client.gateway.SqlExecutionException;
+import org.apache.flink.table.operations.Operation;
+
+import javax.annotation.Nonnull;
+
+import java.io.StringReader;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Optional;
+
+/** ClientParser use {@link FlinkSqlParserImplTokenManager} to do lexical
analysis. */
+public class ClientParser implements SqlCommandParser {
+
+ /** A dumb implementation. TODO: remove this after unifying the
SqlMultiLineParser. */
+ @Override
+ public Optional<Operation> parseCommand(String command) {
+ return Optional.empty();
+ }
+
+ public Optional<StatementType> parseStatement(@Nonnull String statement)
+ throws SqlExecutionException {
+ String trimmedStatement = statement.trim();
+ FlinkSqlParserImplTokenManager tokenManager =
+ new FlinkSqlParserImplTokenManager(
+ new SimpleCharStream(new
StringReader(trimmedStatement)));
Review Comment:
The `TokenManager` will close the `InputStream` when continuously getting
next token until the stream is consumed totally, so don't worry about it.
##########
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/parser/ClientParser.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.client.cli.parser;
+
+import org.apache.flink.sql.parser.impl.FlinkSqlParserImplTokenManager;
+import org.apache.flink.sql.parser.impl.SimpleCharStream;
+import org.apache.flink.sql.parser.impl.Token;
+import org.apache.flink.table.client.gateway.SqlExecutionException;
+import org.apache.flink.table.operations.Operation;
+
+import javax.annotation.Nonnull;
+
+import java.io.StringReader;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Optional;
+
+/** ClientParser use {@link FlinkSqlParserImplTokenManager} to do lexical
analysis. */
+public class ClientParser implements SqlCommandParser {
+
+ /** A dumb implementation. TODO: remove this after unifying the
SqlMultiLineParser. */
+ @Override
+ public Optional<Operation> parseCommand(String command) {
+ return Optional.empty();
+ }
+
+ public Optional<StatementType> parseStatement(@Nonnull String statement)
+ throws SqlExecutionException {
+ String trimmedStatement = statement.trim();
+ FlinkSqlParserImplTokenManager tokenManager =
+ new FlinkSqlParserImplTokenManager(
+ new SimpleCharStream(new
StringReader(trimmedStatement)));
+ List<Token> tokenList = new ArrayList<>();
+ Token token;
+ do {
+ token = tokenManager.getNextToken();
+ tokenList.add(token);
+ } while (token.endColumn != trimmedStatement.length());
Review Comment:
After checking the codes of `getNextToken`, I found that when the input
stream is over, it won't return `null` but a `EOF` token. So I think use
condition `token.kind != EOF` may be better?
##########
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/parser/ClientParser.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.client.cli.parser;
+
+import org.apache.flink.sql.parser.impl.FlinkSqlParserImplTokenManager;
+import org.apache.flink.sql.parser.impl.SimpleCharStream;
+import org.apache.flink.sql.parser.impl.Token;
+import org.apache.flink.table.client.gateway.SqlExecutionException;
+import org.apache.flink.table.operations.Operation;
+
+import javax.annotation.Nonnull;
+
+import java.io.StringReader;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Optional;
+
+/** ClientParser use {@link FlinkSqlParserImplTokenManager} to do lexical
analysis. */
+public class ClientParser implements SqlCommandParser {
+
+ /** A dumb implementation. TODO: remove this after unifying the
SqlMultiLineParser. */
+ @Override
+ public Optional<Operation> parseCommand(String command) {
+ return Optional.empty();
+ }
+
+ public Optional<StatementType> parseStatement(@Nonnull String statement)
+ throws SqlExecutionException {
+ String trimmedStatement = statement.trim();
+ FlinkSqlParserImplTokenManager tokenManager =
+ new FlinkSqlParserImplTokenManager(
+ new SimpleCharStream(new
StringReader(trimmedStatement)));
+ List<Token> tokenList = new ArrayList<>();
+ Token token;
+ do {
+ token = tokenManager.getNextToken();
+ tokenList.add(token);
+ } while (token.endColumn != trimmedStatement.length());
+ return getStatementType(tokenList);
+ }
+
+ //
---------------------------------------------------------------------------------------------
+ private Optional<StatementType> getStatementType(List<Token> tokenList) {
+ Token firstToken = tokenList.get(0);
+
+ if (firstToken.kind == EOF || firstToken.kind == EMPTY ||
firstToken.kind == SEMICOLON) {
+ return Optional.empty();
+ }
+
+ if (firstToken.kind == IDENTIFIER) {
+ // unrecognized token
+ return getPotentialCommandType(firstToken.image);
+ } else if (firstToken.kind == EXPLAIN) {
+ return Optional.of(StatementType.EXPLAIN);
+ } else if (firstToken.kind == SHOW) {
+ return getPotentialShowCreateType(tokenList);
+ } else {
+ return Optional.of(StatementType.OTHER);
+ }
+ }
+
+ private Optional<StatementType> getPotentialCommandType(String image) {
+ switch (image.toUpperCase()) {
+ case "QUIT":
+ case "EXIT":
+ return Optional.of(StatementType.QUIT);
+ case "CLEAR":
+ return Optional.of(StatementType.CLEAR);
+ case "HELP":
+ return Optional.of(StatementType.HELP);
+ default:
+ return Optional.of(StatementType.OTHER);
+ }
+ }
+
+ private Optional<StatementType> getPotentialShowCreateType(List<Token>
tokenList) {
Review Comment:
Modified.
##########
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/parser/ClientParser.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.client.cli.parser;
+
+import org.apache.flink.sql.parser.impl.FlinkSqlParserImplTokenManager;
+import org.apache.flink.sql.parser.impl.SimpleCharStream;
+import org.apache.flink.sql.parser.impl.Token;
+import org.apache.flink.table.client.gateway.SqlExecutionException;
+import org.apache.flink.table.operations.Operation;
+
+import javax.annotation.Nonnull;
+
+import java.io.StringReader;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Optional;
+
+/** ClientParser use {@link FlinkSqlParserImplTokenManager} to do lexical
analysis. */
+public class ClientParser implements SqlCommandParser {
+
+ /** A dumb implementation. TODO: remove this after unifying the
SqlMultiLineParser. */
+ @Override
+ public Optional<Operation> parseCommand(String command) {
+ return Optional.empty();
+ }
+
+ public Optional<StatementType> parseStatement(@Nonnull String statement)
+ throws SqlExecutionException {
+ String trimmedStatement = statement.trim();
+ FlinkSqlParserImplTokenManager tokenManager =
+ new FlinkSqlParserImplTokenManager(
+ new SimpleCharStream(new
StringReader(trimmedStatement)));
+ List<Token> tokenList = new ArrayList<>();
+ Token token;
+ do {
+ token = tokenManager.getNextToken();
+ tokenList.add(token);
+ } while (token.endColumn != trimmedStatement.length());
+ return getStatementType(tokenList);
+ }
+
+ //
---------------------------------------------------------------------------------------------
+ private Optional<StatementType> getStatementType(List<Token> tokenList) {
+ Token firstToken = tokenList.get(0);
+
+ if (firstToken.kind == EOF || firstToken.kind == EMPTY ||
firstToken.kind == SEMICOLON) {
+ return Optional.empty();
+ }
+
+ if (firstToken.kind == IDENTIFIER) {
+ // unrecognized token
+ return getPotentialCommandType(firstToken.image);
Review Comment:
Made it clear.
##########
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/parser/ClientParser.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.client.cli.parser;
+
+import org.apache.flink.sql.parser.impl.FlinkSqlParserImplTokenManager;
+import org.apache.flink.sql.parser.impl.SimpleCharStream;
+import org.apache.flink.sql.parser.impl.Token;
+import org.apache.flink.table.client.gateway.SqlExecutionException;
+import org.apache.flink.table.operations.Operation;
+
+import javax.annotation.Nonnull;
+
+import java.io.StringReader;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Optional;
+
+/** ClientParser use {@link FlinkSqlParserImplTokenManager} to do lexical
analysis. */
+public class ClientParser implements SqlCommandParser {
+
+ /** A dumb implementation. TODO: remove this after unifying the
SqlMultiLineParser. */
+ @Override
+ public Optional<Operation> parseCommand(String command) {
+ return Optional.empty();
+ }
+
+ public Optional<StatementType> parseStatement(@Nonnull String statement)
+ throws SqlExecutionException {
+ String trimmedStatement = statement.trim();
+ FlinkSqlParserImplTokenManager tokenManager =
+ new FlinkSqlParserImplTokenManager(
+ new SimpleCharStream(new
StringReader(trimmedStatement)));
+ List<Token> tokenList = new ArrayList<>();
+ Token token;
+ do {
+ token = tokenManager.getNextToken();
+ tokenList.add(token);
+ } while (token.endColumn != trimmedStatement.length());
+ return getStatementType(tokenList);
+ }
+
+ //
---------------------------------------------------------------------------------------------
+ private Optional<StatementType> getStatementType(List<Token> tokenList) {
+ Token firstToken = tokenList.get(0);
+
+ if (firstToken.kind == EOF || firstToken.kind == EMPTY ||
firstToken.kind == SEMICOLON) {
+ return Optional.empty();
+ }
+
+ if (firstToken.kind == IDENTIFIER) {
+ // unrecognized token
+ return getPotentialCommandType(firstToken.image);
+ } else if (firstToken.kind == EXPLAIN) {
+ return Optional.of(StatementType.EXPLAIN);
+ } else if (firstToken.kind == SHOW) {
+ return getPotentialShowCreateType(tokenList);
+ } else {
+ return Optional.of(StatementType.OTHER);
+ }
Review Comment:
To get a completed SQL statement is the function of the
`SqlMultiLineParser`. So I think we can make sure that statement here should
be ended with ';'. And now I think all the test data should be ended with ';'.
##########
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/parser/SqlCommandParser.java:
##########
@@ -16,17 +16,18 @@
* limitations under the License.
*/
-package org.apache.flink.table.client.cli;
+package org.apache.flink.table.client.cli.parser;
import org.apache.flink.annotation.Internal;
+import org.apache.flink.sql.parser.impl.FlinkSqlParserImplConstants;
import org.apache.flink.table.client.gateway.SqlExecutionException;
import org.apache.flink.table.operations.Operation;
import java.util.Optional;
-/** SqlClient command parser. */
+/** SqlClient command parser. Extends FlinkSqlParserImplConstants to use
defined token kinds. */
@Internal
-interface SqlCommandParser {
+public interface SqlCommandParser extends FlinkSqlParserImplConstants {
Review Comment:
Removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
