iemejia commented on pull request #14640: URL: https://github.com/apache/beam/pull/14640#issuecomment-828917047
Well maybe it is the way the job was designed I am not familiar with how the cache key is determined, so if you have suggestions on how to guarantee the execution of the VR tests only when needed (when the commit of the git master has changed between runs) I would be ok to fix it. Notice that I discovered this when looking at the execution time of the Postcommit jobs, for example https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Java11/184/ took 2m 55s to run which was noticeable short for the full VR set that is around 20 mins long https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Java11/190/ I suspected this might be due to the fact that the git SHA of the previous job did not change so gradle was smart and did not run it, but it does not seem to be the case: https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Java11/184/ - e13e5481adb60c4d14321003eb51a16d59362cf9 https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Java11/183/ - 38e879b8f63b5ca2bbd055024bdca230ff92aae2 Notice that this was also happening for the normal Java 8 Spark VR tests suite too: https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/8823/ - 2m 26s - e13e5481adb60c4d14321003eb51a16d59362cf9 https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/8822/ - 2m 24s - 38e879b8f63b5ca2bbd055024bdca230ff92aae2 And I just looked at Flink and it has the same issue: https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/8853/ - 2m 5s - 9ea38849772a91b328a812355a153de96042625f https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/8852/ - 15m - 325471d9ea7834f92284e1b4a95490b3f22e8b5a -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
