[
https://issues.apache.org/jira/browse/KAFKA-21144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Lau reassigned KAFKA-21144:
--------------------------------
Assignee: Alan Lau (was: Lianet Magrans)
> Producer auth errors may become stale but still be thrown later on cold/new
> topics
> ----------------------------------------------------------------------------------
>
> Key: KAFKA-21144
> URL: https://issues.apache.org/jira/browse/KAFKA-21144
> Project: Kafka
> Issue Type: Bug
> Components: clients, producer
> Reporter: Lianet Magrans
> Assignee: Alan Lau
> Priority: Major
>
> Whenever the producer encounters an auth failure, it's recorded as a fatal
> error in the producer's metadata, and it's thrown only when the producer has
> to wait for metadata for a topic (this would be when using a new topic or
> "idle" topics, with no produce for longer than the metadata.max.idle).
> These errors are only cleared when thrown, which means that it could remain
> in the internal cache for a long time if the producer lets some topics go
> idle. So a producer that keeps producing to its hot topics keeps the error
> indefinitely and fails the first time it produces to an idle topic or to a
> new topic (throwing the original fatal auth expcetion it kept in cache, it
> clears it at that point, and a retry would succeed). But this could be
> possibly hours or days after the actual auth issue was resolved.
> This initially made sense I expect given that auth errors were expected to be
> strictly non-transient, "bad credentials" that would require app action
> (change creds, restart). But the auth landscape has evolved, now we have
> oauth layers in between, and we've seen how a producer could see a
> token-related transient auth failure, that apps would still throw long after
> it got solved (e.g., if letting some topics go idle, then produce again)
> This task is to review if we could improve how the producer recovers from
> these kind of scenarios where auth failures could recover. We need to
> carefully consider that at the moment, fatal errors are an umbrella for
> several (Bootstrap, UnsupportedVersion, Auth).
> Even though the fatal error recording is at the network layer, I expect this
> "stale + throw late" issue can happen only in the producer (consumers do the
> maybeThrow on every poll)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)