ZhangYu0123 opened a new pull request #4234:
URL: https://github.com/apache/incubator-doris/pull/4234
## Proposed changes
The detail delete index design in storage level for batch delete function
feature #4051 .
We discuss the concrete delete index structure and update the design
document here.
When importing and deleting according to the key, we will generate a bitmap
index of the delete key for these keys. The content of the index is: the
deleted line number (ordinal).
(1) The specific format of delete index footer page(DeleteIndexFooterPB):
+--------------------+
| type |
|--------------------|
| uncompressed_size |
|--------------------|
| content_bytes |
|--------------------|
| num_items |
|--------------------|
| checksum |
+--------------------+
-type: The type is DELETE_INDEX_PAGE, which means that the page of the index
type is deleted
-uncompressed_size: Uncompressed page size
-content_bytes: Data size of index content
-num_items: Store the number of deleted entries recorded in the index
-checksum: Checksum of index page content
(2) SegmentFooterPB changes:
Add PagePointerPB delete_index_page to SegmentFooterPB, because delete index
is Segment level index.
(3) PageTypePB changes:
Add DELETE_INDEX_PAGE to PageTypePB to describe the type of delete index.
Add DeleteIndexFooterPB to PageFooterPB as optional.
(4) encoding
The encoding format of index using roaring bitmap format.
## Types of changes
What types of changes does your code introduce to Doris?
_Put an `x` in the boxes that apply_
- [] Bugfix (non-breaking change which fixes an issue)
- [] New feature (non-breaking change which adds functionality)
- [] Breaking change (fix or feature that would cause existing functionality
to not work as expected)
- [x] Documentation Update (if none of the other choices apply)
- [] Code refactor (Modify the code structure, format the code, etc...)
## Checklist
_Put an `x` in the boxes that apply. You can also fill these out after
creating the PR. If you're unsure about any of them, don't hesitate to ask.
We're here to help! This is simply a reminder of what we are going to look for
before merging your code._
- [x] I have create an issue on #ISSUE, and have described the bug/feature
there in detail
- [] Compiling and unit tests pass locally with my changes
- [] I have added tests that prove my fix is effective or that my feature
works
- [] If this change need a document change, I have updated the document
- [] Any dependent changes have been merged
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
