Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21145#discussion_r184235329
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala
---
@@ -299,13 +299,13 @@ private[kafka010] class KafkaMicroBatchReader(
}
}
-/** A [[DataReaderFactory]] for reading Kafka data in a micro-batch
streaming query. */
+/** A [[ReadTask]] for reading Kafka data in a micro-batch streaming
query. */
private[kafka010] case class KafkaMicroBatchDataReaderFactory(
--- End diff --
This fixes the API, not implementations, and it already touches 30+ files.
I'd rather not fix the downstream classes for two reasons. First, to avoid
this becoming really large. Second, we need to be able to evolve these APIs
without requiring changes to all implementations. This is still evolving and if
we need to update 20+ implementations to make simple changes, then I think
that's a problem.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
