Hey Neville, I had a chance to look at this. The debugging output is a separate, but misleading, issue. The real cause is the precision of the 32-bit floating point values. The JSON data has 3 decimal places and the array returned from the reader has more than 3, this might be due to the fact that we read in 64-bit floats and cast?
I implemented a quick fix to test and I can pass all tests locally, although I will leave it to you to change as I'm not sure where in your process it's best to adjust the precision. Regards, Paddy ________________________________ From: paddy horan <paddyho...@hotmail.com> Sent: Saturday, November 16, 2019 1:03 PM To: dev@arrow.apache.org <dev@arrow.apache.org> Subject: Re: [Help Needed] Arrow IPC Reader in Rust Hey Neville, I'll take a look if no-one beats me to it (I might not have time today or tomorrow). P ________________________________ From: Neville Dipale <nevilled...@gmail.com> Sent: Saturday, November 16, 2019 1:42 AM To: dev@arrow.apache.org <dev@arrow.apache.org> Subject: [Help Needed] Arrow IPC Reader in Rust Hi Arrow developers, I'm "done" with the Arrow IPC Reader in Rust (for supported data types), but am having issues with reading some of the test data. Specifically, I've noticed that when reading the integration test data (primitve_generated), where I expect an array with 17 values, the arrow array contains 20 values. To illustrate what's happening, I've added some debug statements to the unit test, and the behaviour can be seen at ( https://ci.ursalabs.org/#/builders/93/builds/1550/steps/3/logs/stdio). In the logs, there are a number of arrays which have a length of 17, but have 20 printed values. 3 of those values are duplicated. It's been hard trying to inspect the binary data to verify if there's an issue with them, and I'm able to correctly read 17 values with Python, so I suspect it has to be a Rust issue. Would anyone have some time to look into this with me? Thanks Neville
