juliuszsompolski commented on PR #43745: URL: https://github.com/apache/spark/pull/43745#issuecomment-1805957328
> So before making any changes, let me investigate how to directly test the API to be fixed on the client-side. Sure, you should be able to put a very similar test into client suites as well; I put it into ReattachableExecuteSuite because it has handy helpers like "getReattachableIterator" that you would need to reimplement on the client side, I think a very similar test could be added to SparkConnectClientSuite > In the future, if a server-side test case fails and we need to fix some client API, this logic seems a bit non-intuitive ReattachableExecuteSuite is on the server side because it also tests and asserts what happens on the server when reattachable execution is used. But there are many client things that it tests as well... I think it's closely enough coupled that it's unavoidable. You may remember that I had a whole series of PRs in August where I was trying to restructure the module structure of this, to be able to test both in-process client and in-process server but communicating over RPCs at the same time... In the end had to move a bunch of client code to "common". This is not the cleanest I admit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
