[
https://issues.apache.org/jira/browse/ARTEMIS-5573?focusedWorklogId=1015988&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-1015988
]
ASF GitHub Bot logged work on ARTEMIS-5573:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 16/Apr/26 16:39
Start Date: 16/Apr/26 16:39
Worklog Time Spent: 10m
Work Description: gemmellr commented on code in PR #6323:
URL: https://github.com/apache/artemis/pull/6323#discussion_r3094756606
##########
artemis-protocols/artemis-amqp-protocol/src/main/java/org/apache/activemq/artemis/protocol/amqp/broker/AMQPMessagePersisterV4.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.protocol.amqp.broker;
+
+import java.lang.invoke.MethodHandles;
+
+import org.apache.activemq.artemis.api.core.ActiveMQBuffer;
+import org.apache.activemq.artemis.api.core.Message;
+import org.apache.activemq.artemis.core.persistence.CoreMessageObjectPools;
+import org.apache.activemq.artemis.utils.DataConstants;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static
org.apache.activemq.artemis.core.persistence.PersisterIDs.AMQPMessagePersisterV4_ID;
+
+/**
+ * V4 adds a size field to determine persister boundaries, enabling
forward-compatible
+ * extensions without additional versioning.
+ **/
+public class AMQPMessagePersisterV4 extends AMQPMessagePersisterV3 {
+
+ private static final Logger logger =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ public static final byte ID = AMQPMessagePersisterV4_ID;
+
+ public static AMQPMessagePersisterV4 theInstance;
+
+ public static AMQPMessagePersisterV4 getInstance() {
+ if (theInstance == null) {
+ theInstance = new AMQPMessagePersisterV4();
+ }
+ return theInstance;
+ }
+
+ @Override
+ public byte getID() {
+ return ID;
+ }
+
+ public AMQPMessagePersisterV4() {
+ super();
+ }
+
+
+ protected static final int PERSISTER_SIZE = DataConstants.SIZE_INT + //
memory estimate
+ DataConstants.SIZE_BYTE +
+ DataConstants.SIZE_BOOLEAN; // message priority
+
+ @Override
+ public int getEncodeSize(Message record) {
+ int encodeSize = super.getEncodeSize(record) + PERSISTER_SIZE +
DataConstants.SIZE_INT; // the size delimiter and whatever is written in encode
+ return encodeSize;
+ }
+
+
+ @Override
+ public void encode(ActiveMQBuffer buffer, Message record) {
+ super.encode(buffer, record);
+
+ writeSizeDelimiter(buffer);
+ buffer.writeInt(record.getMemoryEstimate());
+ buffer.writeByte(record.getPriority());
+ buffer.writeBoolean(record.isDurable());
+ }
+
+ protected void writeSizeDelimiter(ActiveMQBuffer buffer) {
+ // this is to allow us to determine the boundary of this persister, for
future use.
+ buffer.writeInt(PERSISTER_SIZE);
Review Comment:
I can see is that this would in future allows older brokers unaware of
future changes to this new persister, to skip over anything added that they
dont understand...but, do we actually send these 'persister encodings' over the
wire at all? Wouldnt that make _these_ changes already incompatible with
existing versions if so? I kinda wonder if thats worth it if so, in order to
write information thats already available via the message.
Either way, if we ever changed this v4 persister (or the v2 large) in future
to do something else, and start writing a bigger value here so that old brokers
can skip it, then surely we end up having to have the new broker inspect the
size during decode and guess from any possible 'past variants of v4' what
specific things were added 'without changing he persister [version]' are
actually present or not in any given v4 (/v2 large) message encountered, purely
from the size it finds?
That seems like it could be a bit of a PITA. Would it not be simpler long
term to just have a basic list/map/array style structure so that we can always
easily determine/look-up the values the given version understands and not need
to be concerned with manually determining anything when additions are made? If
we did that at the start then perhaps it would still be the V1 persister in
use, unless we made some sort of drastic removal?
##########
tests/compatibility-tests/src/test/java/org/apache/activemq/artemis/tests/compatibility/MultiVersionReplicaTest.java:
##########
@@ -63,13 +64,11 @@ public static Collection getParameters() {
if (getJavaVersion() <= 22) {
// Old 2.x servers fail on JDK23+ without workarounds.
combinations.add(new Object[]{ARTEMIS_2_22_0, SNAPSHOT});
- combinations.add(new Object[]{SNAPSHOT, ARTEMIS_2_22_0});
combinations.add(new Object[]{ARTEMIS_2_17_0, SNAPSHOT});
- combinations.add(new Object[]{SNAPSHOT, ARTEMIS_2_17_0});
+ combinations.add(new Object[]{ARTEMIS_2_33_0, SNAPSHOT});
}
combinations.add(new Object[]{ARTEMIS_2_44_0, SNAPSHOT});
- combinations.add(new Object[]{SNAPSHOT, ARTEMIS_2_44_0});
Review Comment:
Is it expected there are now no combinations with a new main broker and old
backup? There were 3 before which seems a bit of a switch up. EDIT: ah, so is
that about persister incompatibility?
Issue Time Tracking
-------------------
Worklog Id: (was: 1015988)
Time Spent: 12h 50m (was: 12h 40m)
> Make AMQP Size immutable
> ------------------------
>
> Key: ARTEMIS-5573
> URL: https://issues.apache.org/jira/browse/ARTEMIS-5573
> Project: Artemis
> Issue Type: Improvement
> Components: AMQP
> Affects Versions: 2.41.0
> Reporter: Clebert Suconic
> Assignee: Clebert Suconic
> Priority: Major
> Labels: pull-request-available
> Time Spent: 12h 50m
> Remaining Estimate: 0h
>
> I have had a lot of issues, even recently on races between re-evaluating a
> message size in AMQP.
> Say a lazy decode happens at the wrong time and the memory estimates can be
> wrong.
> We have fixed issues along the years, but this is still a fragile process
> that is bound to fail. If an user for instance add a plugin breaking the
> chain of events.
> For that reason the memory estimate should already include enough estimation
> for any properties decoded and the process should be simplified.
> Less moving parts would mean less possibilities for bugs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
