morningman opened a new pull request #6829:
URL: https://github.com/apache/incubator-doris/pull/6829
## Proposed changes
This CL mainly changes:
1. Use a lru cache to cache all opened segments
In the previous implementation, the segment instance is an object
belonging to the rowset,
so if the rowset exists, the segment will always exist, resulting in
memory usage.
In fact, the segment instance is only used in the data reading process.
In principle, the segment
can be closed after the data reading is completed.
So here I use a lru cache to cache all open segments, and use
SegmentLoader to uniformly manage
segment loading and closing.
After using cache, the memory will no longer increase continously.
2. Let BE report immediately the first time it receive heartbeat from FE
If not do so, even if the backend is alive with success heartbeat, user
can not create table because the
disk info is not reported yet.
And also add a random sleep for the report process to avoid too many
reports proceed at same time.
## Types of changes
What types of changes does your code introduce to Doris?
_Put an `x` in the boxes that apply_
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation Update (if none of the other choices apply)
- [ ] Code refactor (Modify the code structure, format the code, etc...)
- [ ] Optimization. Including functional usability improvements and
performance improvements.
- [ ] Dependency. Such as changes related to third-party components.
- [ ] Other.
## Checklist
_Put an `x` in the boxes that apply. You can also fill these out after
creating the PR. If you're unsure about any of them, don't hesitate to ask.
We're here to help! This is simply a reminder of what we are going to look for
before merging your code._
- [ ] I have created an issue on (Fix #6828 ) and described the bug/feature
there in detail
- [ ] Compiling and unit tests pass locally with my changes
- [ ] I have added tests that prove my fix is effective or that my feature
works
- [ ] If these changes need document changes, I have updated the document
- [ ] Any dependent changes have been merged
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[email protected] by explaining why you chose the solution you did and what
alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
