Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
korlov42 merged PR #4697: URL: https://github.com/apache/ignite-3/pull/4697 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@ignite.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
korlov42 commented on code in PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697#discussion_r1842057692
##
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/mapping/MappingServiceImpl.java:
##
@@ -220,7 +231,9 @@ private CompletableFuture mapFragments(
return res.thenApply(assignments -> {
Int2ObjectMap targetsById = new
Int2ObjectOpenHashMap<>();
-MappingContext context = new MappingContext(localNodeName, new
ArrayList<>(assignments.nodes()), template.cluster);
+MappingContext context = new MappingContext(
+localNodeName, new
ArrayList<>(assignments.nodes(nodeExclusionFilter)), template.cluster
Review Comment:
Change {{DistributionHolder}} to return list instead of set. Now the
collection is copied only once when filter is null, otherwise we create
filtered list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@ignite.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
AMashenkov commented on code in PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697#discussion_r1841979730
##
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/mapping/MappingServiceImpl.java:
##
@@ -220,7 +231,9 @@ private CompletableFuture mapFragments(
return res.thenApply(assignments -> {
Int2ObjectMap targetsById = new
Int2ObjectOpenHashMap<>();
-MappingContext context = new MappingContext(localNodeName, new
ArrayList<>(assignments.nodes()), template.cluster);
+MappingContext context = new MappingContext(
+localNodeName, new
ArrayList<>(assignments.nodes(nodeExclusionFilter)), template.cluster
Review Comment:
If we can't avoid copying, then the filter can be not null (@Nullable not
needed) and null value replaced with "always false" predicate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@ignite.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
AMashenkov commented on code in PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697#discussion_r1841963156
##
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/mapping/MappingServiceImpl.java:
##
@@ -220,7 +231,9 @@ private CompletableFuture mapFragments(
return res.thenApply(assignments -> {
Int2ObjectMap targetsById = new
Int2ObjectOpenHashMap<>();
-MappingContext context = new MappingContext(localNodeName, new
ArrayList<>(assignments.nodes()), template.cluster);
+MappingContext context = new MappingContext(
+localNodeName, new
ArrayList<>(assignments.nodes(nodeExclusionFilter)), template.cluster
Review Comment:
Do we need a copy to ArrayList here if filter is null?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@ignite.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
korlov42 commented on code in PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697#discussion_r1838148653
##
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/fsm/QueryExecutionProgram.java:
##
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.sql.engine.exec.fsm;
+
+import java.util.List;
+import org.apache.ignite.internal.sql.engine.AsyncSqlCursor;
+import org.apache.ignite.internal.sql.engine.InternalSqlRow;
+import org.apache.ignite.internal.sql.engine.SqlOperationContext;
+import org.apache.ignite.internal.sql.engine.message.UnknownNodeException;
+
+/**
+ * Generic execution program, which accepts query string and initialization
parameters as input, and returns cursor as result.
+ */
+class QueryExecutionProgram extends Program> {
+private static final String PROGRAM_NAME = "QUERY_EXECUTION";
+private static final List TRANSITIONS = List.of(
+new Transition(
+ExecutionPhase.REGISTERED,
+query -> ExecutionPhase.PARSING
+),
+new Transition(
+ExecutionPhase.PARSING,
+query -> query.parsedResult != null
+? ExecutionPhase.OPTIMIZING
+: ExecutionPhase.SCRIPT_INITIALIZATION),
+new Transition(
+ExecutionPhase.OPTIMIZING,
+query -> ExecutionPhase.CURSOR_INITIALIZATION
+),
+new Transition(
+ExecutionPhase.CURSOR_INITIALIZATION,
+query -> ExecutionPhase.EXECUTING
+),
+new Transition(
+ExecutionPhase.SCRIPT_INITIALIZATION,
+query -> ExecutionPhase.EXECUTING
+)
+);
+
+static final Program> INSTANCE = new
QueryExecutionProgram();
+
+private QueryExecutionProgram() {
+super(
+PROGRAM_NAME,
+TRANSITIONS,
+phase -> phase == ExecutionPhase.EXECUTING,
+query -> query.cursor,
+QueryExecutionProgram::errorHandler
+);
+}
+
+static boolean errorHandler(Query query, Throwable th) {
Review Comment:
Every program is supposed to have its own recovery rules.
For instance, to recover on CURSOR_INITIALISATION phase you need to make
sure that data cursor is closed which appeared only on this very phase. Another
phase may require different action to clean up state.
The same with exceptions which can be recovered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@ignite.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
zstan commented on code in PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697#discussion_r1838112199
##
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/fsm/QueryExecutionProgram.java:
##
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.sql.engine.exec.fsm;
+
+import java.util.List;
+import org.apache.ignite.internal.sql.engine.AsyncSqlCursor;
+import org.apache.ignite.internal.sql.engine.InternalSqlRow;
+import org.apache.ignite.internal.sql.engine.SqlOperationContext;
+import org.apache.ignite.internal.sql.engine.message.UnknownNodeException;
+
+/**
+ * Generic execution program, which accepts query string and initialization
parameters as input, and returns cursor as result.
+ */
+class QueryExecutionProgram extends Program> {
+private static final String PROGRAM_NAME = "QUERY_EXECUTION";
+private static final List TRANSITIONS = List.of(
+new Transition(
+ExecutionPhase.REGISTERED,
+query -> ExecutionPhase.PARSING
+),
+new Transition(
+ExecutionPhase.PARSING,
+query -> query.parsedResult != null
+? ExecutionPhase.OPTIMIZING
+: ExecutionPhase.SCRIPT_INITIALIZATION),
+new Transition(
+ExecutionPhase.OPTIMIZING,
+query -> ExecutionPhase.CURSOR_INITIALIZATION
+),
+new Transition(
+ExecutionPhase.CURSOR_INITIALIZATION,
+query -> ExecutionPhase.EXECUTING
+),
+new Transition(
+ExecutionPhase.SCRIPT_INITIALIZATION,
+query -> ExecutionPhase.EXECUTING
+)
+);
+
+static final Program> INSTANCE = new
QueryExecutionProgram();
+
+private QueryExecutionProgram() {
+super(
+PROGRAM_NAME,
+TRANSITIONS,
+phase -> phase == ExecutionPhase.EXECUTING,
+query -> query.cursor,
+QueryExecutionProgram::errorHandler
+);
+}
+
+static boolean errorHandler(Query query, Throwable th) {
Review Comment:
or you mean that in future this methods will be different ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@ignite.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] IGNITE-22768 Sql. Worker node left the cluster before fragment has been sent [ignite-3]
zstan commented on code in PR #4697:
URL: https://github.com/apache/ignite-3/pull/4697#discussion_r1838094639
##
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/fsm/QueryExecutionProgram.java:
##
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.sql.engine.exec.fsm;
+
+import java.util.List;
+import org.apache.ignite.internal.sql.engine.AsyncSqlCursor;
+import org.apache.ignite.internal.sql.engine.InternalSqlRow;
+import org.apache.ignite.internal.sql.engine.SqlOperationContext;
+import org.apache.ignite.internal.sql.engine.message.UnknownNodeException;
+
+/**
+ * Generic execution program, which accepts query string and initialization
parameters as input, and returns cursor as result.
+ */
+class QueryExecutionProgram extends Program> {
+private static final String PROGRAM_NAME = "QUERY_EXECUTION";
+private static final List TRANSITIONS = List.of(
+new Transition(
+ExecutionPhase.REGISTERED,
+query -> ExecutionPhase.PARSING
+),
+new Transition(
+ExecutionPhase.PARSING,
+query -> query.parsedResult != null
+? ExecutionPhase.OPTIMIZING
+: ExecutionPhase.SCRIPT_INITIALIZATION),
+new Transition(
+ExecutionPhase.OPTIMIZING,
+query -> ExecutionPhase.CURSOR_INITIALIZATION
+),
+new Transition(
+ExecutionPhase.CURSOR_INITIALIZATION,
+query -> ExecutionPhase.EXECUTING
+),
+new Transition(
+ExecutionPhase.SCRIPT_INITIALIZATION,
+query -> ExecutionPhase.EXECUTING
+)
+);
+
+static final Program> INSTANCE = new
QueryExecutionProgram();
+
+private QueryExecutionProgram() {
+super(
+PROGRAM_NAME,
+TRANSITIONS,
+phase -> phase == ExecutionPhase.EXECUTING,
+query -> query.cursor,
+QueryExecutionProgram::errorHandler
+);
+}
+
+static boolean errorHandler(Query query, Throwable th) {
Review Comment:
i think it become more clear if "errorHandler" and "canRecover" will be
moved into Program ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@ignite.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
