emkornfield commented on PR #14509: URL: https://github.com/apache/arrow/pull/14509#issuecomment-1297655387
> We need to consider the memory implications of this change. This means that if at least one Skip is requested, the scratch space will be allocated on the heap and kept until the column reader is destroyed. The scratch space can be as big as 12 KB. If we have 1024 column readers open at one time, that means a 12 MB overhead. Is it common to have this many readers open at the same time? If yes, is this overhead acceptable? That seems OK to me, another option would be to read the feature flag. This breaks only after a second call to skip? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
