I hope to get around to finishing my PR for reading in the next month or so. The work is mostly complete, it's just that it didn't get merged before the monorepo consolidation. I expect the write side of things to be easier, though it will probably take a month or so of work with testing.
On Fri, Nov 9, 2018 at 7:26 AM Wes McKinney <wesmck...@gmail.com> wrote: > Hi Paul, > > Yes, we are planning to support it eventually but I estimate there is at > least 100 hours of developer time involved in doing it thoroughly and with > both write and read support, and with good performance. There is a patch > available implementing reads (but not writes yet), but it has performance > regressions and so hasn't been merged until those are sorted out. > > If you are able to contribute in some way it would be appreciated. > > Thanks > > On Fri, Nov 9, 2018, 4:18 AM Paul Wais <p...@getcruise.com.invalid wrote: > > > Dear List, > > > > I'm writing about the ability to write nested data via pyarrow, as > > discussed in this StackOverflow posting: > > > > > https://stackoverflow.com/questions/45341182/nested-data-in-parquet-with-python > > > > I looked around JIRA and some of the PRs and I'm a bit confused: is > > pyarrow supposed to support writing nested data? Or is there planned > > support? I have tried writing such data using pyarrow and I get an > > exception; if I write the same data to Parquet using pyspark, the job > > completes successfully. I was hoping to use pyarrow for my task > > though. > > > > Cheers, > > -Paul > > > > -- > > > > > > *Confidentiality Note:* We care about protecting our proprietary > > information, confidential material, and trade secrets. This message may > > contain some or all of those things. Cruise will suffer material harm if > > anyone other than the intended recipient disseminates or takes any action > > based on this message. If you have received this message (including any > > attachments) in error, please delete it immediately and notify the sender > > promptly. > > >
