ala commented on PR #37228: URL: https://github.com/apache/spark/pull/37228#issuecomment-1199112283
@sadikovi To answer the questions about performance: > Would you be able to provide performance numbers to see how row index affects reads and writes before and after the change? I don't quite see how this change would impact the write performance. There are no changes to the write path, only to the read path. > I think we may need to have a way to opt out depending on the first question. Would you be able to take a look into that too? I am not really sure how the opt-out would look like. The user can always just not read the `_metadata.row_index` and this bypasses pretty much all the code related to row indexes. If I introduce a config to disable row indexes altogether, the result will be very similar. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
