schaefer01 opened a new issue, #35477:
URL: https://github.com/apache/arrow/issues/35477

   ### Describe the bug, including details regarding any error messages, 
version, and platform.
   
   
   I found the index by searching for the initial data as a pattern in the data 
block
   
   But this isn't the right way to do this
   
   my code:
   
   pa_result = pa.array(result, type=pa.struct([('r', pa.int16()), ('i', 
pa.int16())]))
   print("len(pa_result)=",len(result))
     8192
   xfer_block = pa_to_bytes(pa_result)
   len(xfer_block)=",len(xfer_block)
     33224 
      8K bytes = (8*1024) * 4 bytes per complex value
      
     33224 - (1024*8*4) = 456  -> this is overhead, 448 in front and 8 
(checksum?) at the end
   
     I use NATS on a mac to xmit the bytes to a NATS server on ubuntu which 
sends the data back
      to a NATS receiver on the mac (xmit and receive are in the same python 
program using asynio), 
      and then use the arrow stream buffer reader to read the block. The 
purpose is to test py_arrow and
      NATS, then use py_arrow and MQTT for benchmarking which protocol, NATS or 
MQTT is better.
      The rational for py_arrow is a potential customer recommended arrow.
   
             buf = memoryview(data)
             with pa.input_stream(buf) as stream:
               size = stream.size()
               metadata = stream.metadata()
               print("size=",size)
   size= 33224
               print("metadata=",metadata)
   metadata= {}
               stream.seek(448)
               recovery = stream.read(16)
   
   The first 16 bytes match, I'm just looking for an explanation and perhaps a 
less "hacky" way to
   determine the size and placement of overhead.
   
   
   
   ### Component(s)
   
   Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to