RussellSpitzer opened a new pull request #2584:
URL: https://github.com/apache/iceberg/pull/2584
…ibuted
Previously there was an exception when both the source and target of a MERGE
INTO
command shared the same distribution, causing an join without an exchange but
partitioning was changed by DynamicFileFilterExec. This could only happen
if both the target and source happend to have a single partition
and were repesented with the distribution SinglePartition. If the filter
removes
the partition from the target then there is an unbalance between the number
of
partitions but the exchange is also missing. This situation breaks the join.
We do not currently expose any other distribution information so in most
cases
the source and target will return unknown distribution which avoids this
problem.
In those situations an exchange always follows the DynamicFileFilterExec
which makes
sure the number of partitions always matches for the join.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
