GitHub user numinnex added a comment to the discussion: Arrow as zero-serialization target
We attach to each message our own metadata for the internal use, by having a zero copy over the `Message` struct, means that we don't need to deserialize it in order to access those metadata fields and read/update those. On top of that iggy uses batching for better tput, which means that realistically you end up having `Vec<Message>`, rather than individual message, by employing zero copy binary schema, we can avoid the Vec allocation at all and just work with raw bytes, that either provide an "view" into the array of messages or lazily deserializes those. GitHub link: https://github.com/apache/iggy/discussions/1602#discussioncomment-12478920 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
