Github user joseph-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/18973#discussion_r134367548
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -728,7 +729,16 @@ class FakeDefaultSource extends FakeSource {
override def getBatch(start: Option[Offset], end: Offset): DataFrame
= {
val startOffset =
start.map(_.asInstanceOf[LongOffset].offset).getOrElse(-1L) + 1
- spark.range(startOffset, end.asInstanceOf[LongOffset].offset +
1).toDF("a")
+ val ds = new Dataset[java.lang.Long](
--- End diff --
I've tried addressing this a few different ways, and I can't come up with
anything cleaner than the current solution. Directly creating a DF doesn't set
the isStreaming bit, and a bunch of copying and casting is required to get it
set; using LocalRelation requires explicitly handling the encoding of the rows,
since LocalRelation requires InternalRow input.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
