Vladsz83 commented on code in PR #13454:
URL: https://github.com/apache/ignite/pull/13454#discussion_r3748304941
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/datastreamer/DataStreamerRequest.java:
##########
@@ -26,16 +26,18 @@
import org.apache.ignite.internal.StripedMessage;
import org.apache.ignite.internal.processors.affinity.AffinityTopologyVersion;
import org.apache.ignite.internal.processors.cache.GridCacheUtils;
+import org.apache.ignite.internal.util.tostring.GridToStringExclude;
import org.apache.ignite.internal.util.tostring.GridToStringInclude;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.lang.IgniteUuid;
import org.apache.ignite.plugin.extensions.communication.CacheIdAware;
+import org.apache.ignite.stream.StreamReceiver;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import static org.apache.ignite.internal.GridTopic.TOPIC_DATASTREAM;
-/** */
+/** Batch of streamed entries. The receiver it carries is a user class, hence
the deferred unmarshalling. */
Review Comment:
It is not nesessary `is a user class`
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/datastreamer/DataStreamerImpl.java:
##########
@@ -1991,11 +1990,13 @@ private void submit(
if (topVer == null)
topVer =
ctx.cache().context().exchange().readyAffinityVersion();
+ StreamReceiverMessage rcvrMsg0 = rcvrMsg;
Review Comment:
No need of `rcvrMsg0` I think.
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/datastreamer/StreamReceiverMessage.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.datastreamer;
+
+import org.apache.ignite.internal.Marshalled;
+import org.apache.ignite.internal.Order;
+import org.apache.ignite.internal.UseBinaryMarshaller;
+import org.apache.ignite.plugin.extensions.communication.Message;
+import org.apache.ignite.stream.StreamReceiver;
+
+/**
+ * The receiver of a streamer on its way to the nodes that own the data: a
user object here, its serialized form on
+ * the wire. One instance serves every batch of a streamer, so the receiver is
marshalled once and the batches share
Review Comment:
`wire` is something new to Ignite in general.
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/datastreamer/StreamReceiverMessage.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.datastreamer;
+
+import org.apache.ignite.internal.Marshalled;
+import org.apache.ignite.internal.Order;
+import org.apache.ignite.internal.UseBinaryMarshaller;
+import org.apache.ignite.plugin.extensions.communication.Message;
+import org.apache.ignite.stream.StreamReceiver;
+
+/**
+ * The receiver of a streamer on its way to the nodes that own the data: a
user object here, its serialized form on
+ * the wire. One instance serves every batch of a streamer, so the receiver is
marshalled once and the batches share
+ * the result; a streamer given another receiver builds another instance.
Review Comment:
Let's shorten the comment. We should not explain her how DataStreamer works.
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/datastreamer/DataStreamerRequest.java:
##########
@@ -49,10 +51,10 @@ public class DataStreamerRequest implements
DeferredUnmarshalMessage, CacheIdAwa
@Order(2)
String cacheName;
- /** */
- // TODO: Refactor bytes serialization - IGNITE-27977
+ /** Cache receiver, in the message that carries it. Out of {@code
toString()}: it is a user object. */
Review Comment:
`Cache updater` sounds better to me. And it is not nessesary to be `it is a
user object`
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/datastreamer/DataStreamerRequest.java:
##########
@@ -180,8 +182,8 @@ String cacheName() {
}
/** @return Updater. */
- byte[] updaterBytes() {
- return updaterBytes;
+ StreamReceiver<?, ?> updater() {
Review Comment:
Suggestion: `@Nullable`.
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/datastreamer/StreamReceiverMessage.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.datastreamer;
+
+import org.apache.ignite.internal.Marshalled;
+import org.apache.ignite.internal.Order;
+import org.apache.ignite.internal.UseBinaryMarshaller;
+import org.apache.ignite.plugin.extensions.communication.Message;
+import org.apache.ignite.stream.StreamReceiver;
+
+/**
+ * The receiver of a streamer on its way to the nodes that own the data: a
user object here, its serialized form on
Review Comment:
It is not nessesary to be a `user object here`. Might be Ignite's updater.
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/datastreamer/StreamReceiverMessage.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.datastreamer;
+
+import org.apache.ignite.internal.Marshalled;
+import org.apache.ignite.internal.Order;
+import org.apache.ignite.internal.UseBinaryMarshaller;
+import org.apache.ignite.plugin.extensions.communication.Message;
+import org.apache.ignite.stream.StreamReceiver;
+
+/**
+ * The receiver of a streamer on its way to the nodes that own the data: a
user object here, its serialized form on
+ * the wire. One instance serves every batch of a streamer, so the receiver is
marshalled once and the batches share
+ * the result; a streamer given another receiver builds another instance.
+ */
+@UseBinaryMarshaller
+public class StreamReceiverMessage implements Message {
+ /** */
+ @Marshalled("rcvrBytes")
+ StreamReceiver<?, ?> rcvr;
+
+ /**
+ * Serialized {@link #rcvr}, written by whichever batch is marshalled
first and read by the rest. Those batches
Review Comment:
`Serialized {@link #rcvr}` is enough
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/datastreamer/StreamReceiverMessage.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.datastreamer;
+
+import org.apache.ignite.internal.Marshalled;
+import org.apache.ignite.internal.Order;
+import org.apache.ignite.internal.UseBinaryMarshaller;
+import org.apache.ignite.plugin.extensions.communication.Message;
+import org.apache.ignite.stream.StreamReceiver;
+
+/**
+ * The receiver of a streamer on its way to the nodes that own the data: a
user object here, its serialized form on
+ * the wire. One instance serves every batch of a streamer, so the receiver is
marshalled once and the batches share
+ * the result; a streamer given another receiver builds another instance.
+ */
+@UseBinaryMarshaller
+public class StreamReceiverMessage implements Message {
+ /** */
+ @Marshalled("rcvrBytes")
+ StreamReceiver<?, ?> rcvr;
+
+ /**
+ * Serialized {@link #rcvr}, written by whichever batch is marshalled
first and read by the rest. Those batches
+ * leave on different threads, hence the {@code volatile}: a reader seeing
the reference before the contents would
+ * skip the marshalling and send a half-written array.
+ */
+ @Order(0)
+ volatile byte[] rcvrBytes;
+
+ /** Empty constructor. */
Review Comment:
+ `for serialization purposes`
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/datastreamer/StreamReceiverMessage.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.datastreamer;
+
+import org.apache.ignite.internal.Marshalled;
+import org.apache.ignite.internal.Order;
+import org.apache.ignite.internal.UseBinaryMarshaller;
+import org.apache.ignite.plugin.extensions.communication.Message;
+import org.apache.ignite.stream.StreamReceiver;
+
+/**
+ * The receiver of a streamer on its way to the nodes that own the data: a
user object here, its serialized form on
+ * the wire. One instance serves every batch of a streamer, so the receiver is
marshalled once and the batches share
+ * the result; a streamer given another receiver builds another instance.
+ */
+@UseBinaryMarshaller
+public class StreamReceiverMessage implements Message {
Review Comment:
Suggestion: Lets mention `DataStreamer` like `DataStreamerReceiverMessage`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
