kennknowles opened a new issue, #19176: URL: https://github.com/apache/beam/issues/19176
conjunction_clause: function_call(function_parameter, ...) | field_access | column function_parameter: function_call | field_access In Beam, equi-join is implemented by CoGBK, which requires both join inputs (assume binary join) to build PCollection of KV<Row, Row\>, where the key is join key. For equi-join, conjunction clause is essentially an equation. In order to build KV<Row, Row\>, it requires that columns from different sides of equation should come from different join input. For example, a **** b = 2 cannot be used to build join key but a = 2 - b can. So rewriting is required for clauses when it does not satisfy this property. It also implies that not every clause is rewritable. Say the clause is f(a, b) = 3, in which a is from left input and b is from right input. If this function f is not splittable, such that we cannot move a or b to right side of equation, then we cannot support this clause in BeamSQL's join. Imported from Jira [BEAM-7151](https://issues.apache.org/jira/browse/BEAM-7151). Original Jira may contain additional context. Reported by: amaliujia. Subtask of issue #19175 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
