The GitHub Actions job "Metric Engine CI" on horaedb.git has failed. Run started by GitHub user jiacai2050 (triggered by jiacai2050).
Head commit for run: 25dbe738f384d36fe3a89a5f0f278b6dd106f144 / MianChen <[email protected]> feat: another format for manifest (#1604) ## Rationale Close #1600 ## Detailed Changes Creating a new data format to represent the manifest snapshot file. ```text | magic(u32) | version(u8) | flags(u8) | length(u64) | Record(N) ... | The Magic field (u32) is used to ensure the validity of the data source. The Flags field (u8) is reserved for future extensibility, such as enabling compression or supporting additional features. The length field (u64) represents the total length of the subsequent records and serves as a straightforward method for verifying their integrity. (length = record_length * record_count) # Record is a self-descriptive message | id(u64) | time_range(i64*2)| size(u32) | num_rows(u32)| ``` In do_merge, the snapshot data handle is like: ```text Old data flow in do_merge: delta_sstmetas | (extend vec) V object_store -> org_bytes -> org_pb -> Vec<sstmeta> -> dst_pb -> dst_bytes -> object_store New data flow in do_merge: delta_sstmetas -> bytes | (append) V object_store -> org_bytes -> dst_bytes -> object_store ```` Specifically, I create the SnapshotHeader and SnapshotRecordV1 to represent the corresponding data in snapshot bytes. Before merging delta sstfiles into new bytes, we allocate a larger Vec `<u8>` and copy each segment (header, old records, new records) into it. This RP DOES NOT address format upgrade logic which can be resolved in a separate PR. As for the upgrade, we could define a new SnapshotRecord format and perform data migration in Manifest::try_new. ## Test Plan UT --------- Co-authored-by: jiacai2050 <[email protected]> Report URL: https://github.com/apache/horaedb/actions/runs/12299267399 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
