AlenkaF commented on issue #38809: URL: https://github.com/apache/arrow/issues/38809#issuecomment-1827836376
Looking at the last PR that was implementing the type promotion for concating pyarrow/Arrow Tables (see https://github.com/apache/arrow/issues/36845) that there is a structure of which types can be promoted and how: https://github.com/apache/arrow/pull/36846/files#diff-814ac6f43345f7d2f33e9249a1abf092c8078c62ec44cd782c49b676b94ec302 In Python promote options are passed to `unify_schemas` here: https://github.com/apache/arrow/blob/eb5de184a7e5d02f98526332ace54250417bd232/python/pyarrow/table.pxi#L5251-L5253 but note that `unify_schemas=True` in `pa.concat_tables([t1, t2], unify_schemas=True, promote_options="permissive")` doesn't have any effect. With the `MergeOptions` set the C++ handles the promotion for different cases (no struct mentioned): https://github.com/apache/arrow/pull/36846/files#diff-34166c61c1bd83226ad1277c13dca57ba9afa62dc7fba5206146ef6d2abf623e I am not sure but at first look it seems to me that struct type is not handled. I suggest looking at the python tests in the linked PR to see what kind of type promotion is possible in `concat_tables` https://github.com/apache/arrow/pull/36846/files#diff-72bd29ba764c85f18fbf9e74898b72d47ed8a1b04be879c1a5b2a59382e2eaef. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
