[jira] [Commented] (KAFKA-6238) Issues with protocol version when applying a rolling upgrade to 1.0.0
[
https://issues.apache.org/jira/browse/KAFKA-6238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371154#comment-16371154
]
ASF GitHub Bot commented on KAFKA-6238:
---
dguy closed pull request #4583: KAFKA-6238; Fix inter-broker protocol message
format compatibility check
URL: https://github.com/apache/kafka/pull/4583
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/clients/src/main/java/org/apache/kafka/common/record/RecordFormat.java
b/clients/src/main/java/org/apache/kafka/common/record/RecordFormat.java
new file mode 100644
index 000..e71ec599f41
--- /dev/null
+++ b/clients/src/main/java/org/apache/kafka/common/record/RecordFormat.java
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.record;
+
+public enum RecordFormat {
+V0(0), V1(1), V2(2);
+
+public final byte value;
+
+RecordFormat(int value) {
+this.value = (byte) value;
+}
+
+public static RecordFormat lookup(byte version) {
+switch (version) {
+case 0: return V0;
+case 1: return V1;
+case 2: return V2;
+default: throw new IllegalArgumentException("Unknown format
version: " + version);
+}
+}
+
+public static RecordFormat current() {
+return V2;
+}
+
+}
diff --git a/core/src/main/scala/kafka/api/ApiVersion.scala
b/core/src/main/scala/kafka/api/ApiVersion.scala
index b8329c1ece2..9270a7a3b01 100644
--- a/core/src/main/scala/kafka/api/ApiVersion.scala
+++ b/core/src/main/scala/kafka/api/ApiVersion.scala
@@ -17,7 +17,7 @@
package kafka.api
-import org.apache.kafka.common.record.RecordBatch
+import org.apache.kafka.common.record.RecordFormat
/**
* This class contains the different Kafka versions.
@@ -90,11 +90,23 @@ object ApiVersion {
def latestVersion = versionNameMap.values.max
+ def allVersions: Set[ApiVersion] = {
+versionNameMap.values.toSet
+ }
+
+ def minVersionForMessageFormat(messageFormatVersion: RecordFormat): String =
{
+messageFormatVersion match {
+ case RecordFormat.V0 => "0.8.0"
+ case RecordFormat.V1 => "0.10.0"
+ case RecordFormat.V2 => "0.11.0"
+ case _ => throw new IllegalArgumentException(s"Invalid message format
version $messageFormatVersion")
+}
+ }
}
sealed trait ApiVersion extends Ordered[ApiVersion] {
val version: String
- val messageFormatVersion: Byte
+ val messageFormatVersion: RecordFormat
val id: Int
override def compare(that: ApiVersion): Int =
@@ -106,90 +118,90 @@ sealed trait ApiVersion extends Ordered[ApiVersion] {
// Keep the IDs in order of versions
case object KAFKA_0_8_0 extends ApiVersion {
val version: String = "0.8.0.X"
- val messageFormatVersion: Byte = RecordBatch.MAGIC_VALUE_V0
+ val messageFormatVersion = RecordFormat.V0
val id: Int = 0
}
case object KAFKA_0_8_1 extends ApiVersion {
val version: String = "0.8.1.X"
- val messageFormatVersion: Byte = RecordBatch.MAGIC_VALUE_V0
+ val messageFormatVersion = RecordFormat.V0
val id: Int = 1
}
case object KAFKA_0_8_2 extends ApiVersion {
val version: String = "0.8.2.X"
- val messageFormatVersion: Byte = RecordBatch.MAGIC_VALUE_V0
+ val messageFormatVersion = RecordFormat.V0
val id: Int = 2
}
case object KAFKA_0_9_0 extends ApiVersion {
val version: String = "0.9.0.X"
- val messageFormatVersion: Byte = RecordBatch.MAGIC_VALUE_V0
+ val messageFormatVersion = RecordFormat.V0
val id: Int = 3
}
case object KAFKA_0_10_0_IV0 extends ApiVersion {
val version: String = "0.10.0-IV0"
- val messageFormatVersion: Byte = RecordBatch.MAGIC_VALUE_V1
+ val messageFormatVersion = RecordFormat.V1
val id: Int = 4
}
case object KAFKA_0_10_0_IV1 extends ApiVersion {
val version: String = "0.10.0-IV1"
- val messageFormatVersion: Byte = RecordBatch.MAGIC_VALUE_V1
+ val messageFormatVersion = RecordFormat.V1
val id:
[jira] [Commented] (KAFKA-6238) Issues with protocol version when applying a rolling upgrade to 1.0.0
[
https://issues.apache.org/jira/browse/KAFKA-6238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367886#comment-16367886
]
Ivan Babrou commented on KAFKA-6238:
Thanks, that's precisely what we've done.
> Issues with protocol version when applying a rolling upgrade to 1.0.0
> -
>
> Key: KAFKA-6238
> URL: https://issues.apache.org/jira/browse/KAFKA-6238
> Project: Kafka
> Issue Type: Bug
> Components: documentation
>Affects Versions: 1.0.0
>Reporter: Diego Louzán
>Assignee: Jason Gustafson
>Priority: Major
> Fix For: 1.0.1
>
>
> Hello,
> I am trying to perform a rolling upgrade from 0.10.0.1 to 1.0.0, and
> according to the instructions in the documentation, I should only have to
> upgrade the "inter.broker.protocol.version" parameter in the first step. But
> after setting the value to "0.10.0" or "0.10.0.1" (tried both), the broker
> refuses to start with the following error:
> {code}
> [2017-11-20 08:28:46,620] FATAL (kafka.Kafka$)
> java.lang.IllegalArgumentException: requirement failed:
> log.message.format.version 1.0-IV0 cannot be used when
> inter.broker.protocol.version is set to 0.10.0.1
> at scala.Predef$.require(Predef.scala:224)
> at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:1205)
> at kafka.server.KafkaConfig.(KafkaConfig.scala:1170)
> at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:881)
> at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:878)
> at
> kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:28)
> at kafka.Kafka$.main(Kafka.scala:82)
> at kafka.Kafka.main(Kafka.scala)
> {code}
> I checked the instructions for rolling upgrades to previous versions (namely
> 0.11.0.0), and in here it's stated that is also needed to upgrade the
> "log.message.format.version" parameter in two stages. I have tried that and
> the upgrade worked. It seems it still applies to version 1.0.0, so I'm not
> sure if this is wrong documentation, or an actual issue with kafka since it
> should work as stated in the docs.
> Regards,
> Diego Louzán
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-6238) Issues with protocol version when applying a rolling upgrade to 1.0.0
[
https://issues.apache.org/jira/browse/KAFKA-6238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367880#comment-16367880
]
Jason Gustafson commented on KAFKA-6238:
[~bobrik] Thanks for reporting. It looks like there is a bug in the config
validation. I will submit a patch. I think a workaround would be to set the
message format to 0.11.0 at the same time as the inter-broker version. Then you
can remove them both after the second rolling restart.
> Issues with protocol version when applying a rolling upgrade to 1.0.0
> -
>
> Key: KAFKA-6238
> URL: https://issues.apache.org/jira/browse/KAFKA-6238
> Project: Kafka
> Issue Type: Bug
> Components: documentation
>Affects Versions: 1.0.0
>Reporter: Diego Louzán
>Assignee: Jason Gustafson
>Priority: Major
> Fix For: 1.0.1
>
>
> Hello,
> I am trying to perform a rolling upgrade from 0.10.0.1 to 1.0.0, and
> according to the instructions in the documentation, I should only have to
> upgrade the "inter.broker.protocol.version" parameter in the first step. But
> after setting the value to "0.10.0" or "0.10.0.1" (tried both), the broker
> refuses to start with the following error:
> {code}
> [2017-11-20 08:28:46,620] FATAL (kafka.Kafka$)
> java.lang.IllegalArgumentException: requirement failed:
> log.message.format.version 1.0-IV0 cannot be used when
> inter.broker.protocol.version is set to 0.10.0.1
> at scala.Predef$.require(Predef.scala:224)
> at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:1205)
> at kafka.server.KafkaConfig.(KafkaConfig.scala:1170)
> at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:881)
> at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:878)
> at
> kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:28)
> at kafka.Kafka$.main(Kafka.scala:82)
> at kafka.Kafka.main(Kafka.scala)
> {code}
> I checked the instructions for rolling upgrades to previous versions (namely
> 0.11.0.0), and in here it's stated that is also needed to upgrade the
> "log.message.format.version" parameter in two stages. I have tried that and
> the upgrade worked. It seems it still applies to version 1.0.0, so I'm not
> sure if this is wrong documentation, or an actual issue with kafka since it
> should work as stated in the docs.
> Regards,
> Diego Louzán
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-6238) Issues with protocol version when applying a rolling upgrade to 1.0.0
[
https://issues.apache.org/jira/browse/KAFKA-6238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264421#comment-16264421
]
ASF GitHub Bot commented on KAFKA-6238:
---
Github user asfgit closed the pull request at:
https://github.com/apache/kafka/pull/4256
> Issues with protocol version when applying a rolling upgrade to 1.0.0
> -
>
> Key: KAFKA-6238
> URL: https://issues.apache.org/jira/browse/KAFKA-6238
> Project: Kafka
> Issue Type: Bug
> Components: documentation
>Affects Versions: 1.0.0
>Reporter: Diego Louzán
>Assignee: Jason Gustafson
>
> Hello,
> I am trying to perform a rolling upgrade from 0.10.0.1 to 1.0.0, and
> according to the instructions in the documentation, I should only have to
> upgrade the "inter.broker.protocol.version" parameter in the first step. But
> after setting the value to "0.10.0" or "0.10.0.1" (tried both), the broker
> refuses to start with the following error:
> {code}
> [2017-11-20 08:28:46,620] FATAL (kafka.Kafka$)
> java.lang.IllegalArgumentException: requirement failed:
> log.message.format.version 1.0-IV0 cannot be used when
> inter.broker.protocol.version is set to 0.10.0.1
> at scala.Predef$.require(Predef.scala:224)
> at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:1205)
> at kafka.server.KafkaConfig.(KafkaConfig.scala:1170)
> at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:881)
> at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:878)
> at
> kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:28)
> at kafka.Kafka$.main(Kafka.scala:82)
> at kafka.Kafka.main(Kafka.scala)
> {code}
> I checked the instructions for rolling upgrades to previous versions (namely
> 0.11.0.0), and in here it's stated that is also needed to upgrade the
> "log.message.format.version" parameter in two stages. I have tried that and
> the upgrade worked. It seems it still applies to version 1.0.0, so I'm not
> sure if this is wrong documentation, or an actual issue with kafka since it
> should work as stated in the docs.
> Regards,
> Diego Louzán
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KAFKA-6238) Issues with protocol version when applying a rolling upgrade to 1.0.0
[
https://issues.apache.org/jira/browse/KAFKA-6238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259120#comment-16259120
]
Ismael Juma commented on KAFKA-6238:
[~hachikuji], can you please submit a fix?
> Issues with protocol version when applying a rolling upgrade to 1.0.0
> -
>
> Key: KAFKA-6238
> URL: https://issues.apache.org/jira/browse/KAFKA-6238
> Project: Kafka
> Issue Type: Bug
> Components: documentation
>Affects Versions: 1.0.0
>Reporter: Diego Louzán
>Assignee: Jason Gustafson
>
> Hello,
> I am trying to perform a rolling upgrade from 0.10.0.1 to 1.0.0, and
> according to the instructions in the documentation, I should only have to
> upgrade the "inter.broker.protocol.version" parameter in the first step. But
> after setting the value to "0.10.0" or "0.10.0.1" (tried both), the broker
> refuses to start with the following error:
> {code}
> [2017-11-20 08:28:46,620] FATAL (kafka.Kafka$)
> java.lang.IllegalArgumentException: requirement failed:
> log.message.format.version 1.0-IV0 cannot be used when
> inter.broker.protocol.version is set to 0.10.0.1
> at scala.Predef$.require(Predef.scala:224)
> at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:1205)
> at kafka.server.KafkaConfig.(KafkaConfig.scala:1170)
> at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:881)
> at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:878)
> at
> kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:28)
> at kafka.Kafka$.main(Kafka.scala:82)
> at kafka.Kafka.main(Kafka.scala)
> {code}
> I checked the instructions for rolling upgrades to previous versions (namely
> 0.11.0.0), and in here it's stated that is also needed to upgrade the
> "log.message.format.version" parameter in two stages. I have tried that and
> the upgrade worked. It seems it still applies to version 1.0.0, so I'm not
> sure if this is wrong documentation, or an actual issue with kafka since it
> should work as stated in the docs.
> Regards,
> Diego Louzán
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
