beliefer commented on PR #36295: URL: https://github.com/apache/spark/pull/36295#issuecomment-1146967033
> Can we have some kind of performance numbers for "push down OFFSET could improves the performance."? For most JDBC data source, push down OFFSET could improves the performance. There are three reasons: 1. JDBC data source knows the data distribution. 2. JDBC data source constructs index for data. 3. JDBC data source may have the query cache. 4. Some JDBC data source running in native. 5. push down OFFSET could reduce the I/O, then may reduce the shuffle size. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
