Hi All, I took a stab at implementing something like this here: https://gist.github.com/gmintoco/3e65aa7b47ae37b0685db88b2755933f
I would appreciate any feedback or better approaches people might have :) Thanks! Gus On Thu, Mar 16, 2023 at 6:13 PM Gus Minto-Cowcher <[email protected]> wrote: > Hi all, > > Hoping for some advice here. If I have a very large struct (50+fields) > that I would like to represent using Arrow and then write out as a Parquet > file. Currently, I would use schema.NewSchemaFromStruct ( > github.com/apache/arrow/go/v11/parquet/schema) to automatically generate > a schema from the Struct and then build a record manually using the record > builder. > like: > > builder := array.NewRecordBuilder(pool, schema) > > for _, struct := range structs { > builder.Field(0).(*array.BinaryBuilder).AppendString("test") > builder.Field(1).(*array.BinaryBuilder).AppendString("test") > builder.Field(2).(*array.BinaryBuilder).AppendString("test") > builder.Field(..).(*array.BinaryBuilder).AppendString("test") > } > > is there instead some way of writing a struct directly to a record or I am > missing something? It stands to reason if the schema can be inferred it > shouldn't be a problem to also create a record directly from the struct, > instead of building it manually. > > like: > > builder.AppendStructs(structs) > > It seems very tedious to have to write out the entire struct essentially > with the type assertions etc. I am hoping someone has come up with an > easier way to do this :) > > Thanks for any insights people might have, > Gus >
