mapleFU commented on PR #35825: URL: https://github.com/apache/arrow/pull/35825#issuecomment-1679016549
I think you change is reasonable, but I guess it would take sometime to merge it. So instead I guess: * If the string is greater than `INT32_MAX`, maybe dict read is not allowed before you patch being merged. Maybe we can cast to dictionary as a workaround * If the sum of string is greater than `INT32_MAX`, we can try to (1) shrink the batchSize in ClickHouse (2) in arrow, allow early return if we touch the limit? @pitrou @wgtmac @arthurpassos What do you think of this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
