liukun4515 commented on code in PR #1855:
URL: https://github.com/apache/arrow-rs/pull/1855#discussion_r903732476
##########
arrow/src/ipc/writer.rs:
##########
@@ -517,17 +610,20 @@ impl<W: Write> FileWriter<W> {
let data_gen = IpcDataGenerator::default();
let mut writer = BufWriter::new(writer);
// write magic to header
+ let mut header_size: usize = 0;
writer.write_all(&super::ARROW_MAGIC[..])?;
+ header_size += super::ARROW_MAGIC.len();
// create an 8-byte boundary after the header
writer.write_all(&[0, 0])?;
+ header_size += 2;
// write the schema, set the written bytes to the schema + header
let encoded_message = data_gen.schema_to_bytes(schema, &write_options);
let (meta, data) = write_message(&mut writer, encoded_message,
&write_options)?;
Ok(Self {
writer,
write_options,
schema: schema.clone(),
- block_offsets: meta + data + 8,
+ block_offsets: meta + data + header_size,
Review Comment:
`try_new_with_options` is used to write `schema` to the file or stream.
In the arrow format of IPC format, the layout is from the doc
https://arrow.apache.org/docs/format/Columnar.html#ipc-file-format
First, we will write
<magic number "ARROW1">
<empty padding bytes [to 8 byte boundary]>
the size of above part is 8bytes and is the length of header_size
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
