sanatdeshpande1 opened a new issue, #14075:
URL: https://github.com/apache/arrow/issues/14075
I am currently working on developing a custom Athena Dynamodb connector
using Javascript. I am trying to get the ReadRecordsResponse implemented which
consists of a `records` block which contains `schema` and `records`.
Concatenating the buffers of `schema` and `records` returns a `RecordBatch`.
The `records` block looks like this:
```
"records" : {
"aId" : "test-allocator-id",
"schema" :
"BAEAABAAAAAAAAoADgAGAA0ACAAKAAAAAAADABAAAAAAAQoADAAAAAgABAAKAAAACAAAAAgAAAAAAAAAAwAAAIgAAAA8AAAABAAAAJb///8UAAAAFAAAABQAAAAAAAIBGAAAAAAAAAAAAAAAhP///wAAAAEgAAAAAwAAAGRheQDK////FAAAABQAAAAUAAAAAAACARgAAAAAAAAAAAAAALj///8AAAABIAAAAAUAAABtb250aAASABgAFAATABIADAAAAAgABAASAAAAFAAAABQAAAAcAAAAAAACASAAAAAAAAAAAAAAAAgADAAIAAcACAAAAAAAAAEgAAAABAAAAHllYXIAAAAA",
"records" :
"7AAAABQAAAAAAAAADAAWAA4AFQAQAAQADAAAAJAAAAAAAAAAAAADABAAAAAAAwoAGAAMAAgABAAKAAAAFAAAAHgAAAAKAAAAAAAAAAAAAAAGAAAAAAAAAAAAAAACAAAAAAAAAAgAAAAAAAAAKAAAAAAAAAAwAAAAAAAAAAIAAAAAAAAAOAAAAAAAAAAoAAAAAAAAAGAAAAAAAAAAAgAAAAAAAABoAAAAAAAAACgAAAAAAAAAAAAAAAMAAAAKAAAAAAAAAAAAAAAAAAAACgAAAAAAAAAAAAAAAAAAAAoAAAAAAAAAAAAAAAAAAAAAAAAA/wMAAAAAAADgBwAA4QcAAOIHAADjBwAA5AcAAOUHAADmBwAA5wcAAOgHAADpBwAA/wMAAAAAAAABAAAAAgAAAAMAAAAEAAAABQAAAAYAAAAHAAAACAAAAAkAAAAKAAAA/wMAAAAAAAABAAAAAgAAAAMAAAAEAAAABQAAAAYAAAAHAAAACAAAAAkAAAAKAAAA"
}
```
I am able to serialize the `schema` part but I was unable to find a direct
method to serialize just the `records` (data part) from the `RecordBatch`
without including the `schema`. I have tried using the RecordBatchStreamWriter
to write the record batches, however, because `RecordBatch` consists of
`schema` and `data`, the serialized output also contains a schema.
AWS Athena expects the `records` in ReadRecordsResponse to not have any
`schema` as the messsage header and contain only the data. Any help would be
appreciated!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
