[
https://issues.apache.org/jira/browse/FLINK-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355986#comment-15355986
]
ASF GitHub Bot commented on FLINK-3943:
---------------------------------------
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2169#discussion_r69045985
--- Diff:
flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/plan/logical/operators.scala
---
@@ -236,6 +236,32 @@ case class Aggregate(
}
}
+case class SetMinus(left: LogicalNode, right: LogicalNode, all: Boolean)
extends BinaryNode {
+ override def output: Seq[Attribute] = left.output
+
+ override protected[logical] def construct(relBuilder: RelBuilder):
RelBuilder = {
+ left.construct(relBuilder)
+ right.construct(relBuilder)
+ relBuilder.minus(all)
+ }
+
+ override def validate(tableEnv: TableEnvironment): LogicalNode = {
+ val resolvedMinus = super.validate(tableEnv).asInstanceOf[SetMinus]
+ if (left.output.length != right.output.length) {
+ failValidation(s"Set minus two table of different column sizes:" +
+ s" ${left.output.size} and ${right.output.size}")
+ }
+ val sameSchema = left.output.zip(right.output).forall { case (l, r) =>
+ l.resultType == r.resultType && l.name == r.name }
--- End diff --
I think @wuchong refers exactly to the last case. If number of fields and
field types are identical, it should be possible to do a `EXCEPT` even if the
field names are not the same.
The situation for `UNION` is a bit special due to Flink's internals and the
way the `RowTypeInfo` is implemented but I think we can remove that restriction
in the future.
So, we must keep the checks for number of fields and field types but can
remove the check of the field names, IMO.
> Add support for EXCEPT (set minus)
> ----------------------------------
>
> Key: FLINK-3943
> URL: https://issues.apache.org/jira/browse/FLINK-3943
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
> Affects Versions: 1.1.0
> Reporter: Fabian Hueske
> Assignee: Ivan Mushketyk
> Priority: Minor
>
> Currently, the Table API and SQL do not support EXCEPT.
> EXCEPT can be executed as a coGroup on all fields that forwards records of
> the first input if the second input is empty.
> In order to add support for EXCEPT to the Table API and SQL we need to:
> - Implement a {{DataSetMinus}} class that translates an EXCEPT into a DataSet
> API program using a coGroup on all fields.
> - Implement a {{DataSetMinusRule}} that translates a Calcite {{LogicalMinus}}
> into a {{DataSetMinus}}.
> - Extend the Table API (and validation phase) to provide an except() method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
