OmBiradar commented on PR #50158: URL: https://github.com/apache/arrow/pull/50158#issuecomment-4687256334
Yes, even I faced the same IO limitation while testing the changes. But as this is only triggered when the user wants multi threaded reading. When the file is close to the processor i.e in memory there is a significant improvement (considering a normal case) An edge case like a heavy column (I mean w.r.t the computing time) the delay is inevitable unless there is a complex system to manage the building and loading of a single column using multiple threads. I would say that this change would not be the best case but give workable results? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
