Hi Mang Zhang, For question 1: Yes, there will be large amount of Spark test file, but most of the code is simply some chore work to inherit the vanilla Spark test with native engine enabled, we are NOT copying the Spark tests into Auron code. We will depend on the Spark's test JAR as you mentioned, but we need to do some inheritance, enable the native engine, and sometimes disable some tests(due to Auron's bug).
For question 2: First the test code of a specific released version of Spark will not change. And if there is change in Spark, e.g. some bugs are fixed, Spark change, we also need to change, this is the purpose of correctness testing. On 2025/12/18 04:18:02 Mang Zhang wrote: > Hi James, > Thanks for driving this. +1! > I see the proposal mentions migrating a large amount of Spark test code. > There are two issues here: > 1. There will be a significant amount of migration work. > 2. Code maintenance work: If Spark code changes, Auron may also require > corresponding modifications. > > > Can we achieve our testing objectives by introducing Spark's test JAR? > This approach would only require updating Spark's dependency version, saving > us a significant amount of work. > Similarly, Flink could adopt the same pattern in the future. > > > > -- > > Best regards, > Mang Zhang > > > > At 2025-12-18 11:38:17, "Shreyesh Arangath" <[email protected]> > wrote: > >Great effort! Thanks for driving this. +1! > > > >PS: Could you also provide comment access for the document so that we can > >ask questions? Thanks > > > >Best, > >Shreyesh > > > >On Tue, Dec 16, 2025 at 8:05 AM James <[email protected]> wrote: > > > >> Hi, everyone I'd like to start a discussion about AIP-2: Enhance Auron’s > >> Correctness Testing [1]. Looking forward to your feedback. > >> > >> [1]. > >> > >> https://docs.google.com/document/d/1v8wMyLZXuA7tmDSJysAo8CRqWX36SWR5ZgjyOMSfm94/edit?tab=t.0 > >> >
