liaoxin01 opened a new pull request, #68063:
URL: https://github.com/apache/doris/pull/68063
### What problem does this PR solve?
Problem Summary:
When small files are packed (`enable_packed_file`, on by default in cloud
mode
on S3), a rowset's segment no longer exists as a standalone object: its bytes
are a slice inside a shared packed object, and the mapping from the segment
path to that slice lives in `RowsetMetaPB.packed_slice_locations`.
`GET /api/check_tablet_encryption?tablet_id=<id>&get_footer=true` fails on
such
a tablet with HTTP 500 and
NOT_FOUND: failed to head s3 file
<endpoint>/<bucket>/data/<tablet_id>/<rowset_id>_0.dat
Root cause: `CheckEncryptionAction` opens segments through
`RowsetMeta::physical_fs()`, which is the bare remote file system. The
`PackedFileSystem` wrapper that resolves a segment path to its slice is only
applied in `RowsetMeta::fs()`, so the action ends up asking S3 for an object
that was never written. It cannot simply switch to `fs()` either: `fs()` also
layers encryption on top, which would decrypt the segment and hide the very
magic code and `FileEncryptionInfoPB` footer this check inspects.
Fix: extract the packed wrapping out of `RowsetMeta::fs()` into a helper and
expose `RowsetMeta::packed_physical_fs()` - packed-aware, but without the
encryption layer - then use it at both read sites in `CheckEncryptionAction`.
`PackedFileReader` reports the slice size and takes slice-relative offsets,
so
the existing footer read logic keeps working unchanged. After the fix the
action returns the footer for packed and non-packed tablets alike.
Two defects found while reading the same code are fixed as well:
- `get_last_encrypt_footer()` dereferenced the file system without the null
check its sibling `is_tablet_encrypted()` has, so a rowset whose storage
resource cannot be resolved crashed the BE instead of returning an error.
- The footer length buffer used `reserve()` where `resize()` was meant,
writing
8 bytes past the end of an empty vector.
### Release note
None
### Check List (For Author)
- Test
- [x] Unit Test: added `RowsetMetaTest.TestPackedPhysicalFs`, which
builds a
packed file with the segment bytes at a non-zero offset and asserts
that
`physical_fs()` cannot open the segment path while
`packed_physical_fs()`
returns a reader with slice-relative size and offsets. Verified to
compile
locally; execution left to CI.
- Behavior changed:
- [x] No.
- Does this need documentation?
- [x] No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
