zsxwing commented on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-463008868 I'm thinking about what can be broken by this change (not block this PR, just to figure out what we should document in the release note). Here are two things on my list: - Add a new column to the Kafka source schema. - "select *" returns different schemas and results. - Streaming queries using "dropDuplicates(<empty parameters>)" cannot upgrade directly as the state store assumes no schema changes. - The minimum required version of Kafka client changes from 0.10.x to 0.11.x.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
