Anakin100100 commented on issue #49310: URL: https://github.com/apache/arrow/issues/49310#issuecomment-3938342805
I think I undestand the issue now, When we reserve the out array it has the string type which uses an int32 offset and has a roughly 2gb capacity but the result won't fit there so it segfaults. @pitrou Do we want to add a check so that UnsafeAppend exits cleanly rather than segfaulting or is it not worth the possible performance penalty? Can you confirm that it'd be ok to upcast the result to a large_string if it's not possible to fit the result in a string? I read in the docs for 23.0 that large_string may not be supported everywhere in arrow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
