Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/18029#discussion_r143850553
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
---
@@ -182,14 +181,14 @@ object KinesisInputDStream {
/**
* Sets the initial position data is read from in the Kinesis stream.
Defaults to
- * [[InitialPositionInStream.LATEST]] if no custom value is specified.
+ * [[InitialPosition.latest]] if no custom value is specified.
*
- * @param initialPosition InitialPositionInStream value specifying
where Spark Streaming
+ * @param initialPosition [[InitialPosition]] value specifying where
Spark Streaming
* will start reading records in the Kinesis
stream from
* @return Reference to this [[KinesisInputDStream.Builder]]
*/
- def initialPositionInStream(initialPosition: InitialPositionInStream):
Builder = {
--- End diff --
don't remove this API, since it will break compatibility. Instead add an
API to take the `withTimestamp`. In the end if we see that `withTimestamp` has
been set, but initial position isn't `AtTimestamp`, then we throw an error.
Likewise if `AtTimestamp` is set, but no timestamp has been provided, also
throw an error.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
