xinyiZzz opened a new pull request, #17820:
URL: https://github.com/apache/doris/pull/17820
# Proposed changes
Issue Number: close #xxx
## Problem summary
Arena can replace MemPool in most scenarios. Except for memory reuse,
MemPool supports reuse of previous memory chunks after `clear`, but Arena does
not.
Some comparisons between MemPool and Arena
1. Expansion
Arena is less than 128M index 2 alloc chunk; more than 128M memory,
allocate 128M * n > `size`, n is equal to the minimum value that satisfies the
expression;
MemPool less than 512K index 2 alloc chunk, greater than 512K
memory, separately apply for a `size` length chunk
After Arena applied for a chunk larger than 128M last time, the
minimum chunk applied for after that is 128M. Does this seem to be a waste of
memory? MemPool is also similar. After the chunk of 512K was applied for last
time, the minimum chunk of subsequent applications is 512K.
2. Alignment
MemPool defaults to 16 alignment, because memtable and other places
that use int128 require 16 alignment;
Arena has no default alignment;
3. Memory reuse
Arena only supports `rollback`, which reuses the memory of the
current chunk, usually the memory requested last time.
MemPool supports clear(), all chunks can be reused; or call
ReturnPartialAllocation() to roll back the last requested memory; if the last
chunk has no memory, search for the most free chunk for allocation
4. Realloc
Arena supports realloc contiguous memory; it also supports realloc
contiguous memory from any position at the time of the last allocation. The
difference between `alloc_continue` and `realloc` is:
1. Alloc_continue does not need to specify the old size, but
the default old size = head->pos - range_start
2. alloc_continue supports expansion from range_start when
additional_bytes is between head and pos, which is equivalent to reusing a part
of memory, while realloc completely allocates a new memory
MemPool does not support realloc, but supports transferring or
absorbing chunks between two MemPools
5. check mem limit
MemPool checks the mem limit, and Arena checks at the Allocator
layer.
6. Support for ASAN
Arena does something extra
7. Error handling
MemPool supports returning the error message of application failure
directly through `Status`, and Arena throws Exception.
## Checklist(Required)
* [ ] Does it affect the original behavior
* [ ] Has unit tests been added
* [ ] Has document been added or modified
* [ ] Does it need to update dependencies
* [ ] Is this PR support rollback (If NO, please explain WHY)
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[[email protected]](mailto:[email protected]) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
