beliefer edited a comment on issue #24535: [SPARK-27640][SQL][SS] Avoid duplicate lookups for datasource through provider URL: https://github.com/apache/spark/pull/24535#issuecomment-493852958 > Thank you for ping me, @beliefer . However, the reviewers' comments are consistent. And, I also agree with that. AFAIK, you didn't provide more evidences for your claim about the performance problem. Without evidence, we can not make a progress. Could you investigate more and give us some numbers for your claim? > > > you know that it includes some Java SPI, traversing and reflection. > > I think there exists a little performance problems. Thanks for your reply. @dongjoon-hyun. AFAIK, java reflection will result in significant performance issue. The oracle website`https://docs.oracle.com/javase/tutorial/reflect/index.html` contains some description about performance: ``` Performance Overhead Because reflection involves types that are dynamically resolved, certain Java virtual machine optimizations can not be performed. Consequently, reflective operations have slower performance than their non-reflective counterparts, and should be avoided in sections of code which are called frequently in performance-sensitive applications. ``` `https://blog.frankel.ch/performance-cost-of-reflection/` contains performance cost test. `https://stackoverflow.com/questions/435553/java-reflection-performance` has discussion of java reflection.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
