[GitHub] [spark] ulysses-you commented on pull request #32391: [SPARK-35264][SQL] Support AQE side broadcastJoin threshold

2021-05-20 Thread GitBox
ulysses-you commented on pull request #32391: URL: https://github.com/apache/spark/pull/32391#issuecomment-845603097 @Gabriel39 I guess you misunderstand the logic of AQE. > AQE should not optimize it to other join type since static stats (e.g sizeInBytes) is always larger or equal

[GitHub] [spark] ulysses-you commented on pull request #32391: [SPARK-35264][SQL] Support AQE side broadcastJoin threshold

2021-05-20 Thread GitBox
ulysses-you commented on pull request #32391: URL: https://github.com/apache/spark/pull/32391#issuecomment-845581748 @Gabriel39 Not sure I see your point. Do you mean a plan which can be broadcast in AQE is always smaller than the config we specified ? If so, What's the issue with that.

[GitHub] [spark] ulysses-you commented on pull request #32391: [SPARK-35264][SQL] Support AQE side broadcastJoin threshold

2021-04-30 Thread GitBox
ulysses-you commented on pull request #32391: URL: https://github.com/apache/spark/pull/32391#issuecomment-830004523 thanks for merging! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] ulysses-you commented on pull request #32391: [SPARK-35264][SQL] Support AQE side broadcastJoin threshold

2021-04-29 Thread GitBox
ulysses-you commented on pull request #32391: URL: https://github.com/apache/spark/pull/32391#issuecomment-829849295 @cloud-fan thank you for the review, address all comments: * inline the config check * simplify the test -- This is an automated message from the Apache Git

[GitHub] [spark] ulysses-you commented on pull request #32391: [SPARK-35264][SQL] Support AQE side broadcastJoin threshold

2021-04-29 Thread GitBox
ulysses-you commented on pull request #32391: URL: https://github.com/apache/spark/pull/32391#issuecomment-828972048 cc @maropu @cloud-fan @maryannxue do you have any thoughts about this feature ? -- This is an automated message from the Apache Git Service. To respond to the message,