dongjoon-hyun commented on a change in pull request #31089:
URL: https://github.com/apache/spark/pull/31089#discussion_r553775138
##########
File path: docs/structured-streaming-kafka-integration.md
##########
@@ -878,7 +878,14 @@ group id, however, please read warnings for this option
and use it with caution.
where to start instead. Structured Streaming manages which offsets are
consumed internally, rather
than rely on the kafka Consumer to do it. This will ensure that no data is
missed when new
topics/partitions are dynamically subscribed. Note that `startingOffsets`
only applies when a new
- streaming query is started, and that resuming will always pick up from where
the query left off.
+ streaming query is started, and that resuming will always pick up from where
the query left off. Note
+ that when the offsets consumed by a streaming application is not in Kafka
(e.g., topics are deleted,
+ offsets are out of range, or offsets are removed after offset retention
period), because the offsets
+ are not reset and the streaming application will see data lost. In extream
cases, for example the
Review comment:
`extream` -> `extreme`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
