When I try this I receive the following error: Expected to be able to read 824 bytes for message body, got 384
I'm assuming this is because the expected schema is missing? On Mon, Apr 15, 2024 at 1:49 PM Kevin Liu <kevin.jq....@gmail.com> wrote: > From the example in the Streaming, Serialization, and IPC > <https://arrow.apache.org/docs/python/ipc.html#using-streams> doc, it > looks like you don't need to create/open a stream with a schema, the schema > can be inferred from the RecordBatchStreamReader object. > > ``` > with pa.ipc.open_stream(buf) as reader: > schema = reader.schema > batches = [b for b in reader] > ``` > > On Mon, Apr 15, 2024 at 10:35 AM Amanda Weirich <acweiric...@gmail.com> > wrote: > >> Hello, >> >> I have an incoming arrow record batch without a schema attached coming in >> over a UDP port as buf.to_pybytes. We dont want to attach the schema >> because the schema is already known. So in my receive script I create my >> schema, and I am trying to create an arrow stream reader where I pass in >> the batch and the schema, but it says only one argument can be accepted >> (meaning it only expected to see the stream and not the schema): >> >> # Receive data >> received_data, addr = sock.recvfrom(1024) >> # make byte array >> byte_stream = bytearray(received_data) >> # Create a BytesIO object from the received data >> stream = io.BytesIO(byte_stream) >> schema = create_schema() >> reader = pa.ipc.open_stream(stream, schema=schema) >> >> How can I create an arrow stream reader for this use case? >> >> Thank you in advance! >> >> Amanda >> >
