Hi all,
Hoping for some advice here. If I have a very large struct (50+fields) that
I would like to represent using Arrow and then write out as a Parquet file.
Currently, I would use schema.NewSchemaFromStruct (
github.com/apache/arrow/go/v11/parquet/schema) to automatically generate a
schema from the Struct and then build a record manually using the record
builder.
like:
builder := array.NewRecordBuilder(pool, schema)
for _, struct := range structs {
builder.Field(0).(*array.BinaryBuilder).AppendString("test")
builder.Field(1).(*array.BinaryBuilder).AppendString("test")
builder.Field(2).(*array.BinaryBuilder).AppendString("test")
builder.Field(..).(*array.BinaryBuilder).AppendString("test")
}
is there instead some way of writing a struct directly to a record or I am
missing something? It stands to reason if the schema can be inferred it
shouldn't be a problem to also create a record directly from the struct,
instead of building it manually.
like:
builder.AppendStructs(structs)
It seems very tedious to have to write out the entire struct essentially
with the type assertions etc. I am hoping someone has come up with an
easier way to do this :)
Thanks for any insights people might have,
Gus
