[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573:
URL: https://github.com/apache/kafka/pull/10573#discussion_r630601449
##
File path:
clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java
##
@@ -525,6 +525,11 @@ private TransactionManager
configureTransactionState(ProducerConfig config,
final int transactionTimeoutMs =
config.getInt(ProducerConfig.TRANSACTION_TIMEOUT_CONFIG);
final long retryBackoffMs =
config.getLong(ProducerConfig.RETRY_BACKOFF_MS_CONFIG);
final boolean autoDowngradeTxnCommit =
config.getBoolean(ProducerConfig.AUTO_DOWNGRADE_TXN_COMMIT);
+// Only log a warning if being used outside of Streams, which we
know includes "StreamThread-" in the client id
+if (autoDowngradeTxnCommit && !clientId.contains("StreamThread-"))
{
Review comment:
Done https://github.com/apache/kafka/pull/10675
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573:
URL: https://github.com/apache/kafka/pull/10573#discussion_r630593372
##
File path:
clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java
##
@@ -179,10 +179,18 @@ public void beginTransaction() throws
ProducerFencedException {
this.sentOffsets = false;
}
+@SuppressWarnings("deprecation")
Review comment:
Actually no, we do get a warning if we don't have either annotation.
I'll change it to `@Deprecated` then
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573:
URL: https://github.com/apache/kafka/pull/10573#discussion_r630553249
##
File path:
clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java
##
@@ -179,10 +179,18 @@ public void beginTransaction() throws
ProducerFencedException {
this.sentOffsets = false;
}
+@SuppressWarnings("deprecation")
Review comment:
I guess it doesn't need to have either, I can remove in a followup PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573:
URL: https://github.com/apache/kafka/pull/10573#discussion_r630552512
##
File path:
clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java
##
@@ -525,6 +525,11 @@ private TransactionManager
configureTransactionState(ProducerConfig config,
final int transactionTimeoutMs =
config.getInt(ProducerConfig.TRANSACTION_TIMEOUT_CONFIG);
final long retryBackoffMs =
config.getLong(ProducerConfig.RETRY_BACKOFF_MS_CONFIG);
final boolean autoDowngradeTxnCommit =
config.getBoolean(ProducerConfig.AUTO_DOWNGRADE_TXN_COMMIT);
+// Only log a warning if being used outside of Streams, which we
know includes "StreamThread-" in the client id
+if (autoDowngradeTxnCommit && !clientId.contains("StreamThread-"))
{
Review comment:
@ijuma I came across this after the question was raised around the
autodowngrade logic, apparently (according to the config's javadocs) it's an
"internal" config that's only used for Streams. The config itself is
package-private.
Given that, I thought we may want to log a warning to any plain client users
that saw this config and didn't notice that it was internal, and thus tried to
use it. But I'm happy to do a followup PR to remove this. Alternatively, we can
just take this config out -- I actually don't see any reason why it should be
necessary, AFAICT it's just a slight convenience config that saves Streams from
the ~5 lines of code it would take to do this downgrade itself (basically it
just erases the extra consumer group metadata that isn't understood by older
brokers). Not sure if this was vestigial from an older iteration of KIP-447, as
it seems rather unnecessary..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573:
URL: https://github.com/apache/kafka/pull/10573#discussion_r620726308
##
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java
##
@@ -603,7 +606,7 @@ boolean runLoop() {
log.error("Shutting down because the Kafka cluster seems
to be on a too old version. " +
"Setting {}=\"{}\" requires broker version 2.5
or higher.",
StreamsConfig.PROCESSING_GUARANTEE_CONFIG,
- EXACTLY_ONCE_BETA);
+ StreamsConfig.EXACTLY_ONCE_V2);
Review comment:
I'll just put in both of them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r620725616 ## File path: docs/streams/upgrade-guide.html ## @@ -93,6 +95,12 @@ Upgrade Guide and API Changes Streams API changes in 3.0.0 + + The StreamsConfig.EXACTLY_ONCE and StreamsConfig.EXACTLY_ONCE_BETA configs have been deprecated, and a new StreamsConfig.EXACTLY_ONCE_V2 config has been Review comment: I guess I'll just try to mention StreamsConfig in there somewhere... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r620722055 ## File path: docs/streams/upgrade-guide.html ## @@ -93,6 +95,12 @@ Upgrade Guide and API Changes Streams API changes in 3.0.0 + + The StreamsConfig.EXACTLY_ONCE and StreamsConfig.EXACTLY_ONCE_BETA configs have been deprecated, and a new StreamsConfig.EXACTLY_ONCE_V2 config has been + introduced. This is the same feature as eos-beta, but renamed to highlight its production-readiness. Users of exactly-once semantics should plan to migrate to the eos-v2 config and prepare for the removal of the deprecated configs in 4.0 or after at least a year Review comment: Well, I kind of thought that we did intentionally choose to call it `beta` because we weren't completely confident in it when it was first released. But we are now, and looking back we can say with hindsight that it turned out to be production-ready back then. Still, I see your point, I'll make it more explicit with something like that suggestion -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573:
URL: https://github.com/apache/kafka/pull/10573#discussion_r619559055
##
File path:
streams/src/test/java/org/apache/kafka/streams/processor/internals/RecordCollectorTest.java
##
@@ -126,10 +129,10 @@ public void setup() {
clientSupplier.setCluster(cluster);
streamsProducer = new StreamsProducer(
config,
-"threadId",
+processId + "-StreamThread-1",
Review comment:
StreamsProducer tries to parse the thread name to get the `processId`
for the null check that's only done for eos-v2 (see above)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r619558964 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/RecordCollectorTest.java ## @@ -111,6 +113,7 @@ private final StringSerializer stringSerializer = new StringSerializer(); private final ByteArraySerializer byteArraySerializer = new ByteArraySerializer(); +private final UUID processId = UUID.randomUUID(); Review comment: StreamsProducer requires the processID to be non-null, but only for eos-v2 for some reason -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573:
URL: https://github.com/apache/kafka/pull/10573#discussion_r619558819
##
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/ActiveTaskCreator.java
##
@@ -115,7 +115,7 @@ public void reInitializeThreadProducer() {
StreamsProducer streamsProducerForTask(final TaskId taskId) {
if (processingMode != EXACTLY_ONCE_ALPHA) {
-throw new IllegalStateException("Producer per thread is used.");
+throw new IllegalStateException("Expected EXACTLY_ONCE to be
enabled, but the processing mode was " + processingMode);
Review comment:
I'll do it the other way around to be consistent with how we print the
`processingMode` (ie change `eos-v2` to `EXACTLY_ONCE_V2` below instead)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573:
URL: https://github.com/apache/kafka/pull/10573#discussion_r619558551
##
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java
##
@@ -603,7 +606,7 @@ boolean runLoop() {
log.error("Shutting down because the Kafka cluster seems
to be on a too old version. " +
"Setting {}=\"{}\" requires broker version 2.5
or higher.",
StreamsConfig.PROCESSING_GUARANTEE_CONFIG,
- EXACTLY_ONCE_BETA);
+ StreamsConfig.EXACTLY_ONCE_V2);
Review comment:
My rationale was that if a user is hitting this then it's presumably a
new application that they just tried to enable eos-v2 with, without upgrading
their brokers. And if it's a new 3.0 application then why would they have
chosen the deprecated eos-beta config over the eos-v2 config.
But I can try to print the actual result, it just requires a little extra
code to look this up and save it since we don't track the configs after the
thread's creation. Not the end of the world, but didn't feel necessary to me.
Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573:
URL: https://github.com/apache/kafka/pull/10573#discussion_r619557786
##
File path: streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java
##
@@ -291,23 +293,35 @@
*
* Enabling exactly-once processing semantics requires broker version
0.11.0 or higher.
* If you enable this feature Kafka Streams will use more resources (like
broker connections)
- * compared to the {@link #AT_LEAST_ONCE} case.
+ * compared to {@link #AT_LEAST_ONCE "at_least_once"} and {@link
#EXACTLY_ONCE_V2 "exactly_once_v2"}.
*
- * @see #EXACTLY_ONCE_BETA
+ * @deprecated Since 3.0.0, will be removed in 4.0. Use {@link
#EXACTLY_ONCE_V2 "exactly_once_v2"} instead.
Review comment:
See comment above -- I'm ok with removing this for the
`sendOffsetsToTransaction` since all users have to do there is update one line
of code, but for eos that may require a broker upgrade I'd rather leave this in
to encourage them to do so
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r619557404 ## File path: docs/streams/upgrade-guide.html ## @@ -93,6 +95,12 @@ Upgrade Guide and API Changes Streams API changes in 3.0.0 + + The StreamsConfig.EXACTLY_ONCE and StreamsConfig.EXACTLY_ONCE_BETA configs have been deprecated, and a new StreamsConfig.EXACTLY_ONCE_V2 config has been + introduced. This is the same feature as eos-beta, but renamed to highlight its production-readiness. Users of exactly-once semantics should plan to migrate to the eos-v2 config and prepare for the removal of the deprecated configs in 4.0 or after at least a year + from the release of 3.0, whichever comes last. Note that eos-v2 requires broker version 2.5 or higher, like eos-beta, so users should begin to upgrade their kafka cluster if necessary. See Review comment: From my perspective, it's ok to be flexible or to backtrack and push the removal date out further than we claimed, but not the other way around. In other words we may as well be aggressive and say we plan to remove it in 4.0 now, if it hasn't been a year by the time 4.0 rolls around or we aren't ready for whatever reason then we don't have to do it. Users will actually probably be happy to have more time since let's be real, many of them will not yet have upgraded their brokers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r619557012 ## File path: docs/streams/upgrade-guide.html ## @@ -93,6 +95,12 @@ Upgrade Guide and API Changes Streams API changes in 3.0.0 + + The StreamsConfig.EXACTLY_ONCE and StreamsConfig.EXACTLY_ONCE_BETA configs have been deprecated, and a new StreamsConfig.EXACTLY_ONCE_V2 config has been + introduced. This is the same feature as eos-beta, but renamed to highlight its production-readiness. Users of exactly-once semantics should plan to migrate to the eos-v2 config and prepare for the removal of the deprecated configs in 4.0 or after at least a year + from the release of 3.0, whichever comes last. Note that eos-v2 requires broker version 2.5 or higher, like eos-beta, so users should begin to upgrade their kafka cluster if necessary. See Review comment: We specifically voted on this in the KIP. Personally I was a bit hesitant when someone suggested it, but I think the motivation there was to give users some slight sense of urgency so they don't put off upgrading their code or brokers, and then complain if/when we remove it because they haven't done so yet. Not sure if you saw the WW3 thread that John started when he synced over the commit that removed all the deprecated methods in 3.0... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r619556206 ## File path: docs/streams/upgrade-guide.html ## @@ -93,6 +95,12 @@ Upgrade Guide and API Changes Streams API changes in 3.0.0 + + The StreamsConfig.EXACTLY_ONCE and StreamsConfig.EXACTLY_ONCE_BETA configs have been deprecated, and a new StreamsConfig.EXACTLY_ONCE_V2 config has been Review comment: I feel like it does make more sense to include the actual variable here, since that's what's being deprecated. Also that way no one has to pause and think "hm, is that in StreamsConfig or ConsumerConfig or..." -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r619555279 ## File path: docs/streams/upgrade-guide.html ## @@ -93,6 +95,12 @@ Upgrade Guide and API Changes Streams API changes in 3.0.0 + + The StreamsConfig.EXACTLY_ONCE and StreamsConfig.EXACTLY_ONCE_BETA configs have been deprecated, and a new StreamsConfig.EXACTLY_ONCE_V2 config has been + introduced. This is the same feature as eos-beta, but renamed to highlight its production-readiness. Users of exactly-once semantics should plan to migrate to the eos-v2 config and prepare for the removal of the deprecated configs in 4.0 or after at least a year Review comment: Hm, I thought this phrasing did imply it was already production-ready and that we just changed the name to highlight that fact. Do you have suggestions for how to word this better? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r619554270 ## File path: docs/streams/upgrade-guide.html ## @@ -53,17 +53,19 @@ Upgrade Guide and API Changes -Starting in Kafka Streams 2.6.x, a new processing mode is available, named EOS version 2, which is configurable by setting -processing.guarantee to "exactly_once_beta". -NOTE: The "exactly_once_beta" processing mode is ready for production (i.e., it's not "beta" software). +Starting in Kafka Streams 2.6.x, a new processing mode is available, named EOS version 2. This can be configured +by setting StreamsConfig.PROCESSING_GUARANTEE to StreamsConfig.EXACTLY_ONCE_V2 for +application versions 3.0+, or setting it to StreamsConfig.EXACTLY_ONCE_BETA for versions between 2.6 and 2.8. To use this new feature, your brokers must be on version 2.5.x or newer. -A switch from "exactly_once" to "exactly_once_beta" (or the other way around) is -only possible if the application is on version 2.6.x. -If you want to upgrade your application from an older version and enable this feature, -you first need to upgrade your application to version 2.6.x, staying on "exactly_once", -and then do second round of rolling bounces to switch to "exactly_once_beta". -For a downgrade, do the reverse: first switch the config from "exactly_once_beta" to -"exactly_once" to disable the feature in your 2.6.x application. +If you want to upgrade your EOS application from an older version and enable this feature in version 3.0+, +you first need to upgrade your application to version 3.0.x, staying on StreamsConfig.EXACTLY_ONCE, Review comment: > If users have a config properties / text file, you would use the string. I just thought it was clearer, but I didn't think about that. I'll revert this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573:
URL: https://github.com/apache/kafka/pull/10573#discussion_r618914490
##
File path: streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java
##
@@ -1010,18 +1026,51 @@ public StreamsConfig(final Map props) {
protected StreamsConfig(final Map props,
final boolean doLog) {
super(CONFIG, props, doLog);
-eosEnabled = StreamThread.eosEnabled(this);
+eosEnabled = eosEnabled();
+
+final String processingModeConfig =
getString(StreamsConfig.PROCESSING_GUARANTEE_CONFIG);
+if (processingModeConfig.equals(EXACTLY_ONCE)) {
+log.warn("Configuration parameter `{}` is deprecated and will be
removed in the 4.0.0 release. " +
+ "Please use `{}` instead. Note that this requires
broker version 2.5+ so you should prepare "
+ + "to upgrade your brokers if necessary.",
EXACTLY_ONCE, EXACTLY_ONCE_V2);
+}
+if (processingModeConfig.equals(EXACTLY_ONCE_BETA)) {
+log.warn("Configuration parameter `{}` is deprecated and will be
removed in the 4.0.0 release. " +
+ "Please use `{}` instead.", EXACTLY_ONCE_BETA,
EXACTLY_ONCE_V2);
+}
+
if (props.containsKey(RETRIES_CONFIG)) {
-log.warn("Configuration parameter `{}` is deprecated and will be
removed in 3.0.0 release.", RETRIES_CONFIG);
+log.warn("Configuration parameter `{}` is deprecated and will be
removed in the 4.0.0 release.", RETRIES_CONFIG);
+}
+}
+
+public ProcessingMode processingMode() {
+if
(EXACTLY_ONCE.equals(getString(StreamsConfig.PROCESSING_GUARANTEE_CONFIG))) {
+return StreamThread.ProcessingMode.EXACTLY_ONCE_ALPHA;
+} else if
(EXACTLY_ONCE_BETA.equals(getString(StreamsConfig.PROCESSING_GUARANTEE_CONFIG)))
{
+return StreamThread.ProcessingMode.EXACTLY_ONCE_V2;
+} else if
(EXACTLY_ONCE_V2.equals(getString(StreamsConfig.PROCESSING_GUARANTEE_CONFIG))) {
+return StreamThread.ProcessingMode.EXACTLY_ONCE_V2;
+} else {
+return StreamThread.ProcessingMode.AT_LEAST_ONCE;
}
}
+public boolean eosEnabled() {
Review comment:
Ohh, I forgot this was public. Now the current code makes much more
sense, I moved it because I thought it was so awkward. I'll just put it back
and leave a comment so the next person doesn't fall into the same trap. Thanks
for the explanation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r618912905 ## File path: docs/streams/upgrade-guide.html ## @@ -53,17 +53,19 @@ Upgrade Guide and API Changes -Starting in Kafka Streams 2.6.x, a new processing mode is available, named EOS version 2, which is configurable by setting -processing.guarantee to "exactly_once_beta". -NOTE: The "exactly_once_beta" processing mode is ready for production (i.e., it's not "beta" software). +Starting in Kafka Streams 2.6.x, a new processing mode is available, named EOS version 2. This can be configured +by setting StreamsConfig.PROCESSING_GUARANTEE to StreamsConfig.EXACTLY_ONCE_V2 for +application versions 3.0+, or setting it to StreamsConfig.EXACTLY_ONCE_BETA for versions between 2.6 and 3.0. To use this new feature, your brokers must be on version 2.5.x or newer. -A switch from "exactly_once" to "exactly_once_beta" (or the other way around) is -only possible if the application is on version 2.6.x. -If you want to upgrade your application from an older version and enable this feature, -you first need to upgrade your application to version 2.6.x, staying on "exactly_once", -and then do second round of rolling bounces to switch to "exactly_once_beta". -For a downgrade, do the reverse: first switch the config from "exactly_once_beta" to -"exactly_once" to disable the feature in your 2.6.x application. +If you want to upgrade your EOS application from an older version and enable this feature, +you first need to upgrade your application to version 2.6.x, staying on StreamsConfig.EXACTLY_ONCE, Review comment: I just copied over the old instructions for upgrading to beta, and then mentioned how to upgrade to V2 down below. But since V2 is now the "real" version I suppose it should be the main one mentioned here ## File path: docs/streams/upgrade-guide.html ## @@ -53,17 +53,19 @@ Upgrade Guide and API Changes -Starting in Kafka Streams 2.6.x, a new processing mode is available, named EOS version 2, which is configurable by setting -processing.guarantee to "exactly_once_beta". -NOTE: The "exactly_once_beta" processing mode is ready for production (i.e., it's not "beta" software). +Starting in Kafka Streams 2.6.x, a new processing mode is available, named EOS version 2. This can be configured +by setting StreamsConfig.PROCESSING_GUARANTEE to StreamsConfig.EXACTLY_ONCE_V2 for +application versions 3.0+, or setting it to StreamsConfig.EXACTLY_ONCE_BETA for versions between 2.6 and 3.0. To use this new feature, your brokers must be on version 2.5.x or newer. -A switch from "exactly_once" to "exactly_once_beta" (or the other way around) is -only possible if the application is on version 2.6.x. -If you want to upgrade your application from an older version and enable this feature, -you first need to upgrade your application to version 2.6.x, staying on "exactly_once", -and then do second round of rolling bounces to switch to "exactly_once_beta". -For a downgrade, do the reverse: first switch the config from "exactly_once_beta" to -"exactly_once" to disable the feature in your 2.6.x application. +If you want to upgrade your EOS application from an older version and enable this feature, +you first need to upgrade your application to version 2.6.x, staying on StreamsConfig.EXACTLY_ONCE, +and then do second round of rolling bounces to switch to StreamsConfig.EXACTLY_ONCE_BETA. If you Review comment: see comment above ## File path: docs/streams/upgrade-guide.html ## @@ -53,17 +53,19 @@ Upgrade Guide and API Changes -Starting in Kafka Streams 2.6.x, a new processing mode is available, named EOS version 2, which is configurable by setting -processing.guarantee to "exactly_once_beta". -NOTE: The "exactly_once_beta" processing mode is ready for production (i.e., it's not "beta" software). +Starting in Kafka Streams 2.6.x, a new processing mode is available, named EOS version 2. This can be configured +by setting StreamsConfig.PROCESSING_GUARANTEE to StreamsConfig.EXACTLY_ONCE_V2 for +application versions 3.0+, or setting it to StreamsConfig.EXACTLY_ONCE_BETA for versions between 2.6 and 3.0. To use this new feature, your brokers must be on version 2.5.x or newer. -A switch from "exactly_once" to "exactly_once_beta" (or the other way around) is -only possible if the application is on version 2.6.x. -If you want to upgrade your application from an older version and enable this feature, -you first need to upgrade your
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r618912357 ## File path: docs/streams/developer-guide/config-streams.html ## @@ -667,12 +668,14 @@ probing.rebalance.interval.msprocessing.guarantee The processing guarantee that should be used. - Possible values are "at_least_once" (default), - "exactly_once" (for EOS version 1), - and "exactly_once_beta" (for EOS version 2). - Using "exactly_once" requires broker - version 0.11.0 or newer, while using "exactly_once_beta" - requires broker version 2.5 or newer. + Possible values are "at_least_once" (default) + and "exactly_once_v2" (for EOS version 2). + Deprecated config options are "exactly_once" (for EOS alpha), Review comment: I think given we have just rolled this out, and many users are likely still in the process of upgrading their brokers to a sufficient version, we should continue to mention EOS alpha. And then we may as well mention beta as well for consistency. Don't feel too strongly though -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2
ableegoldman commented on a change in pull request #10573:
URL: https://github.com/apache/kafka/pull/10573#discussion_r618911355
##
File path:
clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java
##
@@ -642,9 +647,13 @@ public void beginTransaction() throws
ProducerFencedException {
* to the partition leader. See the exception for more details
* @throws KafkaException if the producer has encountered a previous fatal
or abortable error, or for any
* other unexpected error
+ *
+ * @deprecated Since 3.0.0, will be removed in 4.0. Use {@link
#sendOffsetsToTransaction(Map, ConsumerGroupMetadata)} instead.
*/
+@Deprecated
public void sendOffsetsToTransaction(Map offsets,
String consumerGroupId) throws
ProducerFencedException {
+log.warn("This method has been deprecated and will be removed in 4.0,
please use #sendOffsetsToTransaction(Map, ConsumerGroupMetadata) instead");
Review comment:
Oh I thought Ismael had said we were meant to do so during the KIP
discussion, but I just re-read his message and I think he meant just for
configs. I'll take this out
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
