Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20286#discussion_r161976455
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/streaming/reader/Offset.java
---
@@ -17,12 +17,20 @@
package org.apache.spark.sql.sources.v2.streaming.reader;
+import org.apache.spark.annotation.InterfaceStability;
+
/**
- * An abstract representation of progress through a [[MicroBatchReader]]
or [[ContinuousReader]].
- * During execution, Offsets provided by the data source implementation
will be logged and used as
- * restart checkpoints. Sources should provide an Offset implementation
which they can use to
- * reconstruct the stream position where the offset was taken.
+ * An abstract representation of progress through a {@link
MicroBatchReader} or
+ * {@link ContinuousReader}.
+ * During execution, offsets provided by the data source implementation
will be logged and used as
+ * restart checkpoints. Each source should provide an offset
implementation which the source can use
+ * to reconstruct a position in the stream up to which data has been
seen/processed.
+ *
+ * Note: This class currently extends {@link
org.apache.spark.sql.execution.streaming.Offset} to
+ * maintain compatibility with DataSource V1 APIs. This will be extension
will be removed once we
--- End diff --
ditto
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
