SAMARTHJAISWAL opened a new pull request, #542: URL: https://github.com/apache/arrow-js/pull/542
## What's Changed A stream that ends inside the 8-byte prefix of a message (the continuation marker and the metadata length) was read as a complete stream. `tableFromIPC` and the stream readers returned the schema with 0 rows and no error, while a cut a few bytes later, inside the metadata, already threw. `MessageReader.readMetadataLength()` turned a missing or short read into the length 0, which is the end-of-stream value, and `readMetadata()` returned done for an empty read before its length check. Now, in both `MessageReader` and `AsyncMessageReader`: - a source that ends before a message is still the end of the stream, so a stream without the end-of-stream marker reads as before; - a short read of the continuation marker or of the metadata length throws; - a continuation marker with nothing after it throws; - a complete prefix with no metadata throws the existing "Expected to read N metadata bytes" error. The tests cut a stream 2, 4, 6 and 8 bytes into the prefix of its RecordBatch message and check `MessageReader`, `AsyncMessageReader` and `tableFromIPC`. One more test checks that a stream without the end-of-stream marker still reads all rows. Closes #535. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
