Maybe I misunderstood this explanation.
Agreed. Spark relies on Kafka, Google Pub/Sub or any other messaging
systems to process the related streaming data via topic or topics and
present them to Spark. At this stage, Spark does not care to know how this
streaming data is produced. Spark relies on
If it requires a Kafka broker update, we should not simply bump the version
of Kafka client. Probably we should at least provide separate artifacts.
We should not enforce the upgrade of other component just because we want
to upgrade the dependency. At least it should not happen in minor versions
Well, yes, but if it requires a Kafka server-side update, it does, and that
is out of scope for us to document.
It is important that we document if and how (if we know) the client update
will impact existing Kafka installations (does it require a server-side
update or not?), and document the
Bump to try gathering more voices before taking action. For now, I see two
voices as option 2 & 5 (similar to option 2 but not in the migration node
but in the release note).
On Fri, Mar 18, 2022 at 7:15 PM Jungtaek Lim
wrote:
> CORRECTION: in option 2, we enumerate KIPs which may bring
