Hi Bryce, Thanks for the quick reply, that worked perfectly.
Thanks On 14 Oct 2023 at 3:58 AM +1030, Bryce Mecum <[email protected]>, wrote: > Hi Simon, take a look at the highlighted portion in scenario.go [1] > and see if that helps. I think it's pretty similar to what you're > wanting to do. > > PS: Your email shows up as a mostly-white wall of text. To read it I > had to paste it into a plain text editor. > > [1] > https://github.com/apache/arrow/blob/e9aac8a4ce4b8833667efe52957fb0881a488c4b/go/arrow/internal/flight_integration/scenario.go#L193-L221 > > > On Fri, Oct 13, 2023 at 4:39 AM Simon Knight <[email protected]> wrote: > > > > Hi, > > I have a simple Flight server in Go. I am able to receive data to a Go > > server from a Python client: > > > > import pyarrow.flight as flight > > import polars as pl > > df = pl.read_csv("test.csv") > > data_table = df.to_arrow() > > client = pa.flight.connect("grpc://0.0.0.0:58474") > > upload_descriptor = pa.flight.FlightDescriptor.for_path("uploaded.parquet") > > writer, _ = client.do_put(upload_descriptor, data_table.schema) > > writer.write_table(data_table) > > writer.close() > > > > This works perfectly. > > > > But I am getting very stuck with the methods to format the DoPut to include > > the descriptor and schema for the Go client. > > > > My approximate code is: > > > > client, err := flight.NewClientWithMiddleware(address, nil, > > nil, grpc.WithTransportCredentials(insecure.NewCredentials())) > > stream, err := client.DoPut(context.Background()) > > if err != nil { > > panic(err) > > } > > > > wr := flight.NewRecordWriter(stream) > > > > descriptor := &flight.FlightDescriptor{ > > Type: flight.DescriptorPATH, > > Path: []string{*output}, > > } > > > > wr.SetFlightDescriptor(descriptor) > > schema := flight.SerializeSchema(rr.Schema(), memory.DefaultAllocator) > > > > defer rr.Release() > > > > for rr.Next() { > > rec := rr.Record() > > print("rec", rec.NumRows()) > > err = wr.Write(rec) > > } > > > > My question is how to correctly insert the flight.FlightDescriptor and > > flight.SerializeSchema pieces. > > > > The comments for these methods suggest it can be a bit intricate. > > > > I can't find an example from the docs, the Flight SQL example in the > > repository, or docs such as > > https://voltrondata.com/resources/data-transfer-with-apache-arrow-and-golang > > or https://blog.djnavarro.net/posts/2022-10-18_arrow-flight/ > > > > All the examples I could find (blogs and tests) only show DoGet, where it > > appears the NewRecordWriter does most of the heavy lifting > > > > Thanks > > > > Thanks
