JingsongLi commented on a change in pull request #11045:
[FLINK-15579][table-planner-blink] Support UpsertStreamTableSink on Blink batch
mode.
URL: https://github.com/apache/flink/pull/11045#discussion_r383798800
##########
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/nodes/physical/batch/BatchExecSink.scala
##########
@@ -81,13 +81,39 @@ class BatchExecSink[T](
override protected def translateToPlanInternal(
planner: BatchPlanner): Transformation[Any] = {
val resultTransformation = sink match {
- case _: RetractStreamTableSink[T] | _: UpsertStreamTableSink[T] =>
- throw new TableException("RetractStreamTableSink and
UpsertStreamTableSink is not" +
- " supported in Batch environment.")
-
case streamTableSink: StreamTableSink[T] =>
- // we can insert the bounded DataStream into a StreamTableSink
- val transformation = translateToTransformation(withChangeFlag = false,
planner)
+ val transformation = streamTableSink match {
+ case _: RetractStreamTableSink[T] =>
+ translateToTransformation(withChangeFlag = true, planner)
+
+ case upsertSink: UpsertStreamTableSink[T] =>
+ // check for append only table
+ val isAppendOnlyTable = UpdatingPlanChecker.isAppendOnly(this)
+ upsertSink.setIsAppendOnly(isAppendOnlyTable)
+ val tableKeys = {
+ val sinkFieldNames = upsertSink.getTableSchema.getFieldNames
+ UpdatingPlanChecker.getUniqueKeyFields(getInput, planner,
sinkFieldNames) match {
+ case Some(keys) => keys.sortBy(_.length).headOption
+ case None => None
+ }
+ }
+
+ // check that we have keys if the table has changes (is not
append-only)
+ tableKeys match {
+ case Some(keys) => upsertSink.setKeyFields(keys)
+ case None if isAppendOnlyTable => upsertSink.setKeyFields(null)
+ case None if !isAppendOnlyTable => throw new TableException(
Review comment:
In batch mode, maybe we don't need this check. Like HBaseUpsertSink, it not
depends on `setKeyFields`. Maybe sometimes JDBC also can work like append mode.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
