Yicong-Huang commented on issue #5144: URL: https://github.com/apache/texera/issues/5144#issuecomment-4524637622
thanks for the analysis @eugenegujing, that's super helpful! Both suggested fixes make sense to me. Two more questions: 1. are we implementing and managing the logic to cut the file into parts/chunks? there are not open source solution for it? 2. the number of chunks/parts would also cost read time performance to differ. can we do some analysis on the "Dynamic part size based on file size" idea, see how would it impact read time as well? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
