EmmyMiao87 opened a new pull request #6226:
URL: https://github.com/apache/incubator-doris/pull/6226
## Proposed changes
The current JoinReorder algorithm mainly sorts according to the star model,
and only considers the query association relationship between the table and
the table.
The problems are following:
1. Only applicable to user data whose data model is a star model, data of
other models cannot be sorted.
2. Regardless of the cost of the table, it is impossible to determine the
size of the join table relationship,
and the real query optimization ability is weak.
3. It is impossible to avoid possible time-consuming joins such as cross
joins by sorting.
The new JoinReorder algorithm mainly introduces a new sorting algorithm for
Join
The new ranking algorithm introduces the cost evaluation model to Doris.
The sorting algorithm is mainly based on the following three principles:
1. The order is: Largest node, Smallest node. . . Second largest node
2. Cross join is better than Inner join
3. The right children of Outer join, semi join, and anti join do not move
PlanNode's cost model evaluation mainly relies on two values: cardinality
and selectivity.
cardinality: cardinality, can also be simply understood as the number of
rows.
selectivity: selectivity, a value between 0 and 1. Predicate generally has
selectivity.
The cost model generally calculates the final cardinality of a PlanNode
based on the pre-calculated
cardinality of PlanNode and the selectivity of the predicate to which it
belongs.
Currently, you can configure "enable_cost_based_join_reorder" to control the
opening and closing of JoinReorder.
When the configuration is turned on, the new sorting algorithm will take
effect, when it is turned off,
the old sorting algorithm will take effect, and it is turned off by default.
The new sorting algorithm currently has no cost base evaluation for external
tables (odbc, es)
and set calculations (intersect, except). When using these queries, it is
not recommended to enable cost base join reoder.
When using these queries, it is not recommended to enable cost base join
reoder.
At the code architecture level:
1. The new sorting algorithm occurs in the single-node execution planning
stage.
2. Refactored the init and finalize phases of PlanNode to ensure that
PlanNode planning
and cost evaluation have been completed before the sorting algorithm
occurs.
Change-Id: I6a89b20d70ff1d49909900f39337fa31cc8efd1e
## Types of changes
What types of changes does your code introduce to Doris?
_Put an `x` in the boxes that apply_
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [x] Documentation Update (if none of the other choices apply)
- [x] Code refactor (Modify the code structure, format the code, etc...)
- [x] Optimization. Including functional usability improvements and
performance improvements.
- [ ] Dependency. Such as changes related to third-party components.
- [ ] Other.
## Checklist
_Put an `x` in the boxes that apply. You can also fill these out after
creating the PR. If you're unsure about any of them, don't hesitate to ask.
We're here to help! This is simply a reminder of what we are going to look for
before merging your code._
- [x] I have created an issue on (Fix #6225) and described the bug/feature
there in detail
- [x] Compiling and unit tests pass locally with my changes
- [x] I have added tests that prove my fix is effective or that my feature
works
- [x] If these changes need document changes, I have updated the document
- [x] Any dependent changes have been merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
