Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20828#discussion_r181180114
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala
---
@@ -53,32 +54,24 @@ class ContinuousSuiteBase extends StreamTest {
// A continuous trigger that will only fire the initial time for the
duration of a test.
// This allows clean testing with manual epoch advancement.
protected val longContinuousTrigger = Trigger.Continuous("1 hour")
+
+ override protected val defaultTrigger = Trigger.Continuous(100)
+ override protected val defaultUseV2Sink = true
}
class ContinuousSuite extends ContinuousSuiteBase {
import testImplicits._
- test("basic rate source") {
- val df = spark.readStream
- .format("rate")
- .option("numPartitions", "5")
- .option("rowsPerSecond", "5")
- .load()
- .select('value)
+ test("basic") {
+ val input = new ContinuousMemoryStream[Int](0, spark.sqlContext)
--- End diff --
ContinuousMemoryStream can have no-arg apply, similar to MemoryStream. Must
easier to use.
Having the user specify the id is a recipe of disaster when the user gives
the same number to multiple.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
