There are many factors: - Typically it is a race between multiple PRs, where they all pass CI without the other changes, and get merged at the same time. - Differences between (the nightly job and the PR job) environments (e.g. size of the machine) can also cause these issues. - In the case of my PR there were some classpath subtleties (apparently the testing classpath is not fully hermetic), which PR testing did not uncover. - ...
On Tue, Jan 28, 2025 at 1:53 PM Asif Shahid <asif.sha...@gmail.com> wrote: > I am genuinely curious to know, as to how do those commits which are > reliably failing the build, end up in master ? Is there some window of race > where two conflicting PRs in terms of logic ,tend to mess up the final > state in master ? > I have seen in past few months, while synching up my open PRs, fail due to > issues in master. > I suppose many times, env. issues etc can cause. But scala unit tests > failing on master should not be common. > > On Mon, Jan 27, 2025 at 12:37 PM Dongjoon Hyun <dongjoon.h...@gmail.com> > wrote: > >> Did you see the PR, Martin? SBT is also broken like the following and >> we've been waiting for actions over two days on the original PR. >> >> $ build/sbt clean "catalyst/testOnly >> org.apache.spark.sql.catalyst.encoders.EncoderResolutionSuite" >> ... >> [info] *** 1 SUITE ABORTED *** >> [error] Error during tests: >> [error] org.apache.spark.sql.catalyst.encoders.EncoderResolutionSuite >> [error] (catalyst / Test / testOnly) sbt.TestsFailedException: Tests >> unsuccessful >> [error] Total time: 66 s (01:06), completed Jan 27, 2025, 9:52:35 AM >> >> For the record, we keep it in the `master` branch still. >> >> Dongjoon. >> >> >> >> >> On Mon, Jan 27, 2025 at 12:33 PM Martin Grund >> <mar...@databricks.com.invalid> wrote: >> >>> Would it not have been mindful to wait for the original author to >>> investigate the PR and do a forward fix instead of reverting such a big >>> change? Since this was only blocking the Maven test we could have waited >>> probably a few more days without any issues. >>> >>> On Mon, Jan 27, 2025 at 8:32 PM Dongjoon Hyun <dongj...@apache.org> >>> wrote: >>> >>>> This is reverted from branch-4.0 via the following. >>>> >>>> - https://github.com/apache/spark/pull/49696 >>>> Revert "[SPARK-49700][CONNECT][SQL] Unified Scala Interface for Connect >>>> and Classic" >>>> >>>> Dongjoon. >>>> >>>> On 2025/01/26 16:58:45 Dongjoon Hyun wrote: >>>> > Thank you! >>>> > >>>> > Dongjoon >>>> > >>>> > On Sat, Jan 25, 2025 at 20:01 Yang Jie <yangji...@apache.org> wrote: >>>> > >>>> > > I reported a test issue that is suspected to be related to this pr： >>>> > > >>>> > > - https://github.com/apache/spark/pull/48818/files#r1929652392 >>>> > > >>>> > > and it seems to be causing the failure of the Maven daily test. >>>> > > >>>> > > Thanks, >>>> > > Jie Yang >>>> > > >>>> > > On 2025/01/24 20:24:57 Dongjoon Hyun wrote: >>>> > > > Hi, All. >>>> > > > >>>> > > > SPARK-49700 landed one hour ago. >>>> > > > >>>> > > > Since this is another huge package redesign across 399 files in >>>> Spark >>>> > > 4.0, >>>> > > > please check if you are not affected accidentally. >>>> > > > >>>> > > > Best Regards, >>>> > > > Dongjoon. >>>> > > > >>>> > > >>>> > > >>>> --------------------------------------------------------------------- >>>> > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>> > > >>>> > > >>>> > >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>> >>>>
