benibus commented on PR #35047: URL: https://github.com/apache/arrow/pull/35047#issuecomment-1505546898
> Which mapping generator? Is this one of the ones used in the scanner? No, the one in the JSON reader. The reader uses two `MappingGenerator`s - one to parse/decode incoming buffers, then another to "unwrap" the final result and return the record batch to the caller. The latter was the one causing problems (I think because readahead was being applied in between, so finished futures were being buffered). That being said, the overflow only seemed to occur with the new scanner, but I suspect it's not really "at fault" of anything - just a difference in implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
