Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/20097#discussion_r159983634
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -392,6 +443,21 @@ class MicroBatchExecution(
cd.dataType, cd.timeZoneId)
}
+ val triggerLogicalPlan = sink match {
+ case _: Sink => newAttributePlan
+ case s: MicroBatchWriteSupport =>
+ val writer = s.createMicroBatchWriter(
+ s"$runId",
+ currentBatchId,
+ newAttributePlan.schema,
+ outputMode,
+ new DataSourceV2Options(extraOptions.asJava))
+ Option(writer.orElse(null)).map(WriteToDataSourceV2(_,
newAttributePlan)).getOrElse {
--- End diff --
I don't think the writer can ever be empty. Would you prefer an assert here?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
