davlee1972 commented on issue #50470: URL: https://github.com/apache/arrow/issues/50470#issuecomment-4938400829
In PyArrow, a NULL column inside a StructArray is often initialized with a default value like "" (empty string) or 0. This happens because Arrow does not allocate null buffers for nested struct fields; instead, it puts a placeholder in the null slot.To fix this, use PyArrow compute functions to replace the empty strings with proper null values. I guess using pa.null() or [None] to even out struct arrays won't work? I should be adding empty strings instead even though an empty strin g technically isn't a struct data type? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
