[
https://issues.apache.org/jira/browse/BEAM-9283?focusedWorklogId=590810&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-590810
]
ASF GitHub Bot logged work on BEAM-9283:
----------------------------------------
Author: ASF GitHub Bot
Created on: 29/Apr/21 03:31
Start Date: 29/Apr/21 03:31
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #14640:
URL: https://github.com/apache/beam/pull/14640#issuecomment-828917047
Well maybe it is the way the job was designed I am not familiar with how the
cache key is determined, so if you have suggestions on how to guarantee the
execution of the VR tests only when needed (when the commit of the git master
has changed between runs) I would be ok to fix it.
Notice that I discovered this when looking at the execution time of the
Postcommit jobs, for example
https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Java11/184/
took 2m 55s to run which was noticeable short for the full VR set that is
around 20 mins long
https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Java11/190/
I suspected this might be due to the fact that the git SHA of the previous
job did not change so gradle was smart and did not run it, but it does not seem
to be the case:
https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Java11/184/
- e13e5481adb60c4d14321003eb51a16d59362cf9
https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Java11/183/
- 38e879b8f63b5ca2bbd055024bdca230ff92aae2
Notice that this was also happening for the normal Java 8 Spark VR tests
suite too:
https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/8823/
- 2m 26s - e13e5481adb60c4d14321003eb51a16d59362cf9
https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/8822/
- 2m 24s - 38e879b8f63b5ca2bbd055024bdca230ff92aae2
And I just looked at Flink and it has the same issue:
https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/8853/
- 2m 5s - 9ea38849772a91b328a812355a153de96042625f
https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/8852/
- 15m -
325471d9ea7834f92284e1b4a95490b3f22e8b5a
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 590810)
Time Spent: 4h 10m (was: 4h)
> Add Spark 3 test jobs to the CI (Java 11)
> -----------------------------------------
>
> Key: BEAM-9283
> URL: https://issues.apache.org/jira/browse/BEAM-9283
> Project: Beam
> Issue Type: Sub-task
> Components: runner-spark
> Reporter: Ismaël Mejía
> Assignee: Ismaël Mejía
> Priority: P3
> Fix For: 2.29.0
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Spark 3 runner introduces support for Java 11. We need to add a job in the CI
> that covers the complete suite of Spark 3 runner tests on Java 11.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
