SAMARTHJAISWAL opened a new pull request, #542:
URL: https://github.com/apache/arrow-js/pull/542

   ## What's Changed
   
   A stream that ends inside the 8-byte prefix of a message (the continuation 
marker and the metadata length) was read as a complete stream. `tableFromIPC` 
and the stream readers returned the schema with 0 rows and no error, while a 
cut a few bytes later, inside the metadata, already threw.
   
   `MessageReader.readMetadataLength()` turned a missing or short read into the 
length 0, which is the end-of-stream value, and `readMetadata()` returned done 
for an empty read before its length check.
   
   Now, in both `MessageReader` and `AsyncMessageReader`:
   
   - a source that ends before a message is still the end of the stream, so a 
stream without the end-of-stream marker reads as before;
   - a short read of the continuation marker or of the metadata length throws;
   - a continuation marker with nothing after it throws;
   - a complete prefix with no metadata throws the existing "Expected to read N 
metadata bytes" error.
   
   The tests cut a stream 2, 4, 6 and 8 bytes into the prefix of its 
RecordBatch message and check `MessageReader`, `AsyncMessageReader` and 
`tableFromIPC`. One more test checks that a stream without the end-of-stream 
marker still reads all rows.
   
   Closes #535.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to