Hello! This repo [1] and this paper [2] may be relevant.
[1]: https://github.com/Sebastiaan-Alvarez-Rodriguez/arrow-spark-publication [2]: https://arxiv.org/pdf/2106.13020.pdf Aldrin Montana Computer Science PhD Student UC Santa Cruz On Wed, Feb 8, 2023 at 7:11 PM Vilayannur Sitaraman < [email protected]> wrote: > Hi, > > I just successfully wrote my first flight server and client that > transfers data read from and arrow file from Java Server to Python Client. > I would like to be able to transfer a Spark DataFrame created in Java to > Python using Arrow and ArrowFlight. If I can convert a Spark Dataframe > created in java to an Arrow file format then I can use the above created > flight server and python client to do the transfer. But I am not sure how > to convert a Spark dataframe created in Java to Arrow format in a Java > module. Any help/pointers appreciated. > > Sitaraman >
