jorisvandenbossche commented on issue #44164: URL: https://github.com/apache/arrow/issues/44164#issuecomment-2367677686
> When I called it on the unnested data, it works and returns a dataframe. This is the thing I can't wrap my head around. If the issue is that one of the string columns is too large, how could it work here? Because the nested data is different from the unnested data? That's the only possible reason I think, and from the `.nbytes` you showed in an earlier comment, it is clear that this is the case? BTW, I think you should be able to avoid the conversion of the dict to a list of tuples back to a dict to use `Table.from_pydict`. The `Table.from_pylist` accepts a list of row dicts (as I think you start with?), and I think that should be able to handle missing keys as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
