zealchen commented on issue #1600:
URL: https://github.com/apache/horaedb/issues/1600#issuecomment-2515509429
Agreed, your proposal works for me. Specifically, I can implement a
MetaSerializer to convert Vec<SstFile> into bytes and merge them with the delta
metadata bytes. Additionally, we'll need a deserialization function to parse
the bytes back into Vec<SstFile>. To account for future evolution, we'll
include version verification and handle data migration when the versions differ
during startup. However the upgrade process can be tackled separately in
another issue.
Old data flow in do_merge:
delta_sstmetas
| (extend vec)
V
object_store -> org_bytes -> org_pb -> Vec<sstmeta> -> dst_pb -> dst_bytes
-> object_store
New data flow in do_merge:
delta_sstmetas -> bytes
| (append)
V
object_store -> org_bytes -> dst_bytes -> object_store
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
