zhenlineo commented on PR #39789: URL: https://github.com/apache/spark/pull/39789#issuecomment-1409593261
Hi @LuciferYang Thanks for fixing this error. Regarding the E2E test, I want to clarify the purpose of E2E: It is used by scala client to verify client methods are returning the correct results. Due to the nature of the client development, we can hardly verify a lot behaviors without a real server, thus the E2E tests contains a lot of very basic tests and need to run daily. It was not to testing the server shading or client shading. The reason to use the server shaded jar is because that's the only way to start spark with spark connect. I am not super favor the idea to use E2E also for client/server shading test. Can I suggest we have a separate test to test the client/server shading, following the idea that each test only test one thing? Really great thanks to help to look into better E2E tests. I also tried to put the E2E in a separate module, I did not get too much benefits but created an empty module that only contains tests. It would be really great if you could work out a way to run the tests as if they are unit tests. Thanks again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
