Vladsz83 commented on code in PR #13095:
URL: https://github.com/apache/ignite/pull/13095#discussion_r3585762458
##########
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java:
##########
@@ -1450,11 +1468,23 @@ private void processRegularMessage0(GridIoMessage msg,
UUID nodeId) {
if (lsnr == null)
return;
- Object obj = msg.message();
+ unmarshalPayload(msg);
+
+ invokeListener(msg.policy(), lsnr, nodeId, msg.message());
+ }
- assert obj != null;
+ /** */
+ private void unmarshalPayload(GridIoMessage msg) {
+ // Unmarshalled by GridCacheIoManager with the deployment loader; the
loader here can't see its peer classes.
Review Comment:
`deployment loader` -> some certain links?
##########
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java:
##########
@@ -1942,6 +1972,8 @@ private IgniteInternalFuture<Channel> openChannel(
false
);
+ MessageMarshaller.marshal(ctx.messageFactory(), ioMsg, ctx, null);
Review Comment:
Do we need to marshal with channel opening? Maybe somewhere later?
##########
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java:
##########
@@ -2011,25 +2042,102 @@ else if (async)
ackC.apply(null);
}
else {
+ MessageMarshaller.marshal(ctx.messageFactory(), ioMsg, ctx,
null);
+
+ sendMarshalled(node, ioMsg, ackC);
+ }
+ }
+ }
+
+ /**
+ * Wraps {@code msg} into a marshalled {@link GridIoMessage} without
sending it. Message marshalling is not
+ * idempotent (see {@code MessageMarshalOnceTest}), so a caller that
retries transmission must prepare the
+ * message once and re-send it via {@link #sendPrepared} on each attempt.
+ */
+ public GridIoMessage prepare(Object topic, Message msg, byte plc, boolean
ordered, long timeout,
+ boolean skipOnTimeout) throws IgniteCheckedException {
+ assert !ordered || timeout > 0 || skipOnTimeout;
+
+ GridIoMessage ioMsg = createGridIoMessage(topic, msg, plc, ordered,
timeout, skipOnTimeout);
+
+ MessageMarshaller.marshal(ctx.messageFactory(), ioMsg, ctx, null);
+
+ return ioMsg;
+ }
+
+ /**
+ * Transmits a message created by {@link #prepare} to a remote node.
Transmission does not mutate the message,
Review Comment:
Suggestion: `Transmits` -> `sends`
##########
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java:
##########
@@ -2011,25 +2042,102 @@ else if (async)
ackC.apply(null);
}
else {
+ MessageMarshaller.marshal(ctx.messageFactory(), ioMsg, ctx,
null);
+
+ sendMarshalled(node, ioMsg, ackC);
+ }
+ }
+ }
+
+ /**
+ * Wraps {@code msg} into a marshalled {@link GridIoMessage} without
sending it. Message marshalling is not
+ * idempotent (see {@code MessageMarshalOnceTest}), so a caller that
retries transmission must prepare the
+ * message once and re-send it via {@link #sendPrepared} on each attempt.
+ */
+ public GridIoMessage prepare(Object topic, Message msg, byte plc, boolean
ordered, long timeout,
Review Comment:
Is there any change to do `GridIoManager`'s job inside it? The
`prepare`/`sendPrepared` logic should be encapsulated into this class.
##########
modules/nio/src/main/java/org/apache/ignite/plugin/extensions/communication/MessageSerializer.java:
##########
@@ -38,4 +38,36 @@ public interface MessageSerializer<M extends Message> {
* @return Whether message was fully read.
*/
public boolean readFrom(M msg, MessageReader reader);
+
+ /**
+ * Writes the message using the serializer resolved from the factory.
+ *
+ * @param factory Message factory.
+ * @param msg Message instance.
+ * @param writer Writer.
+ * @param <M> Message type.
+ * @return Whether message was fully written.
+ */
+ static <M extends Message> boolean writeTo(MessageFactory factory, M msg,
MessageWriter writer) {
Review Comment:
We should keep either `public` for all exposed interface methods or for none.
##########
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoveryIoSession.java:
##########
@@ -196,6 +196,8 @@ <T> T readMessage() throws IgniteCheckedException,
IOException {
}
while (!finished);
+ MessageMarshaller.unmarshal(spi.messageFactory(), msg,
((IgniteEx)spi.ignite()).context());
Review Comment:
Why we do this not in the serializer? No we need to know how to process
messages outside io/sending logic.
##########
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/ErrorMessage.java:
##########
Review Comment:
Btw. Might be moved in some common package because is not only
`Communication` error. Might be a subticket.
##########
modules/core/src/test/java/org/apache/ignite/spi/discovery/tcp/DiscoveryUnmarshalVulnerabilityTest.java:
##########
Review Comment:
isn't used
##########
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java:
##########
@@ -1450,11 +1468,23 @@ private void processRegularMessage0(GridIoMessage msg,
UUID nodeId) {
if (lsnr == null)
return;
- Object obj = msg.message();
+ unmarshalPayload(msg);
+
+ invokeListener(msg.policy(), lsnr, nodeId, msg.message());
+ }
- assert obj != null;
+ /** */
+ private void unmarshalPayload(GridIoMessage msg) {
+ // Unmarshalled by GridCacheIoManager with the deployment loader; the
loader here can't see its peer classes.
+ if (msg.message() instanceof GridCacheMessage)
Review Comment:
Why we can't call those loader here?
##########
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java:
##########
@@ -2011,25 +2042,102 @@ else if (async)
ackC.apply(null);
}
else {
+ MessageMarshaller.marshal(ctx.messageFactory(), ioMsg, ctx,
null);
+
+ sendMarshalled(node, ioMsg, ackC);
+ }
+ }
+ }
+
+ /**
+ * Wraps {@code msg} into a marshalled {@link GridIoMessage} without
sending it. Message marshalling is not
+ * idempotent (see {@code MessageMarshalOnceTest}), so a caller that
retries transmission must prepare the
+ * message once and re-send it via {@link #sendPrepared} on each attempt.
+ */
+ public GridIoMessage prepare(Object topic, Message msg, byte plc, boolean
ordered, long timeout,
+ boolean skipOnTimeout) throws IgniteCheckedException {
+ assert !ordered || timeout > 0 || skipOnTimeout;
+
+ GridIoMessage ioMsg = createGridIoMessage(topic, msg, plc, ordered,
timeout, skipOnTimeout);
+
+ MessageMarshaller.marshal(ctx.messageFactory(), ioMsg, ctx, null);
+
+ return ioMsg;
+ }
+
+ /**
+ * Transmits a message created by {@link #prepare} to a remote node.
Transmission does not mutate the message,
+ * so it can be safely retried.
+ */
+ public void sendPrepared(ClusterNode node, GridIoMessage ioMsg) throws
IgniteCheckedException {
+ assert !locNodeId.equals(node.id()) : node;
Review Comment:
A check that message is prepared would be nice.
##########
modules/core/src/test/java/org/apache/ignite/spi/discovery/tcp/DiscoveryUnmarshalVulnerabilityTest.java:
##########
@@ -81,11 +93,14 @@ public class DiscoveryUnmarshalVulnerabilityTest extends
GridCommonAbstractTest
/** {@inheritDoc} */
@Override protected IgniteConfiguration getConfiguration(String
igniteInstanceName) throws Exception {
MessageFactoryProvider msgFactoryProvider = new
AbstractMarshallableMessageFactoryProvider() {
- @Override public void registerAll(MessageFactory factory) {
+ @Override public void registerAll(IgniteMessageFactory factory) {
+ serializer = new MessageSerializerWrapper(this);
Review Comment:
```suggestion
factory.register(
(short)(CoreMessagesProvider.MAX_MESSAGE_ID + 1),
ExploitMessage::new,
new MessageSerializerWrapper(this),
new MessageMarshallerWrapper(this)
);
```
and we can remove
```
/** */
private MessageSerializer<ExploitMessage> serializer;
/** */
private MessageMarshaller<ExploitMessage> marshaller;
```
##########
modules/zookeeper/src/main/java/org/apache/ignite/spi/discovery/zk/internal/DiscoveryMessageParser.java:
##########
@@ -44,8 +47,15 @@ public class DiscoveryMessageParser {
private final MessageFactory msgFactory;
/** */
- public DiscoveryMessageParser(MessageFactory msgFactory) {
+ private final GridKernalContext ctx;
Review Comment:
Suggestion: `ctx` -> `kctx`.
##########
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/IgniteIoTestMessage.java:
##########
@@ -258,10 +253,7 @@ public void onAfterRead() {
}
}
- /**
- * This method is called to initialize tracing variables.
- * TODO: introduce direct message lifecycle API?
- */
+ /** Captures the sent timestamp; repeated calls are no-ops. */
Review Comment:
Suggestion: Captures the sending timestamp once/at first call.
##########
modules/codegen/src/main/java/org/apache/ignite/internal/MessageGenerator.java:
##########
@@ -0,0 +1,292 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.PrintWriter;
+import java.io.Reader;
+import java.io.Writer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TreeSet;
+import java.util.function.Consumer;
+import java.util.stream.Collectors;
+import javax.annotation.processing.FilerException;
+import javax.annotation.processing.ProcessingEnvironment;
+import javax.lang.model.element.TypeElement;
+import javax.lang.model.element.VariableElement;
+import javax.lang.model.type.TypeMirror;
+import javax.lang.model.util.ElementFilter;
+import javax.lang.model.util.Elements;
+import javax.tools.Diagnostic;
+import javax.tools.FileObject;
+import javax.tools.JavaFileObject;
+import javax.tools.StandardLocation;
+import
org.apache.ignite.internal.systemview.SystemViewRowAttributeWalkerProcessor;
+
+/**
+ * Base class for message code generators ({@link MessageSerializerGenerator},
{@link MessageMarshallerGenerator},
+ * {@link MessageDeploymentGenerator}).
+ */
+public abstract class MessageGenerator {
+ /** Blank separator line in generated code. */
+ public static final String EMPTY = "";
+
+ /** Platform line separator used in generated code. */
+ public static final String NL = System.lineSeparator();
+
+ /** Single indentation unit. */
+ public static final String TAB = " ";
+
+ /** Javadoc stub emitted on every generated {@code @Override} method. */
+ public static final String METHOD_JAVADOC = "/** */";
+
+ /** Javadoc block emitted on every generated class. */
+ public static final String CLS_JAVADOC = "/**" + NL +
+ " * This class is generated automatically." + NL +
+ " *" + NL +
+ " * @see org.apache.ignite.internal.MessageProcessor" + NL +
+ " */";
+
+ /** */
+ final ProcessingEnvironment env;
Review Comment:
I vote for keeping the fields 'protected'. Seems to be normal,
already-in-use practice. `package-private` is usually set only when really
require.
##########
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/IgniteIoTestMessage.java:
##########
@@ -240,10 +238,7 @@ public long responseReceivedTsMillis() {
return resRcvTsMillis;
}
- /**
- * This method is called to initialize tracing variables.
- * TODO: introduce direct message lifecycle API?
- */
+ /** Captures the received timestamp; repeated calls are no-ops. */
Review Comment:
Suggestion: `Captures the receive timestamp once/at first call.`
##########
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/IgniteMessageFactoryImpl.java:
##########
@@ -42,6 +45,13 @@ public class IgniteMessageFactoryImpl implements
MessageFactory {
/** Message serializers. */
private final MessageSerializer[] msgSerializers =
(MessageSerializer[])Array.newInstance(MessageSerializer.class, ARR_SIZE);
+ /** Message marshallers (null entry = no marshaller registered for that
type). */
+ private final MessageMarshaller[] msgMarshallers =
(MessageMarshaller[])Array.newInstance(MessageMarshaller.class, ARR_SIZE);
Review Comment:
Could you please check this
[patch](https://github.com/user-attachments/files/30047160/MessageParametrizationPatch.patch)
? At least it compiles with ` mvn clean package -DskipTests -Pall-java`. While
the code becomes slightly wider, the parametrization, in contrast, is looks
clear. This is the final fix. Just an example. We might do smth. like this in a
separate ticket or a sub-ticket.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
