morningman opened a new pull request #4904:
URL: https://github.com/apache/incubator-doris/pull/4904
## Proposed changes
This CL mainly changes:
1. Avoid repeated sending of common components in Fragments
In the previous implementation, a query may generate multiple Fragments,
these Fragments contain some common information, such as DescriptorTable.
Fragment will be sent to BE in a certain order, so these public information
will be sent repeatedly
and generated repeatedly on the BE side.
In some complex SQL, these public information may be very large,
thereby increasing the execution time of Fragment.
So I improved this. For multiple Fragments sent to the same BE, only the
first Fragment will carry
these public information, and it will be cached on the BE side, and
subsequent Fragments
no longer need to carry this information.
In the local test, the execution time of some complex SQL can be reduced
from 3 seconds to 1 second.
2. Add the time-consuming part of FE logic in Profile
Including SQL analysis, planning, Fragment scheduling and sending on the
FE side, and the time to fetch data.
## Types of changes
- [x] Code refactor (Modify the code structure, format the code, etc...)
## Checklist
- [x] I have create an issue on (Fix #4656), and have described the
bug/feature there in detail
## Further comments
The query planner is still time-consuming when processing complex SQL. It
will be optimized later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
