willtemperley commented on issue #82: URL: https://github.com/apache/arrow-swift/issues/82#issuecomment-3241450953
I was looking for a way to write .arrow files. I thought that `ArrowWriter().writeFile(info:) ` [1] would do so, but this does not appear to be writing valid .arrow files. It appends ARROW1 to the header and footer, however the file is unreadable in any readers I've tried, such as pyarrow. Looking at `testFileWriter_bool` in `IPCFileReaderTests`, it generates a 966 byte file using `ArrowWriter`, parses it and compare it to the expected results manually in `checkBoolRecordBatch `. It passes. The test `testFileReader_bool` uses the same routine `checkBoolRecordBatch` to read a 650 byte file generated by Go. So the flatbuffers message is decodes to the same thing but the first bytes in the Go written data are: 0x14000000000000000c0016001 Whereas the padding looks different in the Swift written message: 1c0000000000000000000000000000000c0016001 So I think there's almost certainly a writer bug here, and likely an issue with the reader being too lenient. [1] https://github.com/apache/arrow-swift/blob/a1fcb23a5157eb3bb79a55a667fe15c7c6ebaa6a/Arrow/Sources/Arrow/ArrowWriter.swift#L354 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
