viirya commented on pull request #31219: URL: https://github.com/apache/spark/pull/31219#issuecomment-764883827
To be clear, before we mention the plan to have other StateStore, there is no so-called internal information, right? So, can we just talk about this test change? Even there is no plan to have other StateStore, I don't think the veto makes sense. > Doing refactoring like this PR increases the backporting cost. So I would like to see the value of this first. I already mentioned it. Moved tests are for general StateStore behavior. So, is there any reason to keep them in a HDFS-specific StateStore test suite? I don't think there is. > Test codes are not public APIs. They are changed frequently. I would recommend a third-party StateStore implementation to build their own tests. This point conflicts with your first point. If test codes are changed frequently, it is pretty frequent to see conflicts in test when backporting. Why we don't do it better? We can reduce the time for third-party StateStore implementation and make thing easier for further StateStore development if any. Sorry, but I still don't see any strong reason for veto. There are some values in this refactoring, and the only concern is backporting cost. Actually this PR did less changing and more moving test code, so I don't think it increases a lot backporting cost at all. We frequently refactor main and test code, this is not a special one. And, finally this is only for test code and test code changes frequently. I doubt the concern is enough for veto. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
