pauldix commented on pull request #7064: URL: https://github.com/apache/arrow/pull/7064#issuecomment-628194857
@nevi-me they're not compared byte for byte, the Go tester is invoked, which has its own file reader read in the file and try to marshal it and compare with the JSON representation. The custom metadata definition in the first RecordBatch is where it trips up. Although I've reproduced this on a much smaller output than the already fairly small generated primitive test so I doubt it's related to the issue you're seeing. What seems to happen is that the Go reader tries to read the length of the metadata, which fails because that length isn't divisible by 8 (because of the padding restriction in the Arrow format spec). I'll have some more time to look into this specific failure tomorrow and I might be able to puzzle it out. At this point I'm guessing that the Flatbuffer serialization isn't taking Arrow's expected padding into account, but this test doesn't have actual metadata in it so I'm still not entirely sure what's going on. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
