HyukjinKwon commented on code in PR #59431:
URL: https://github.com/apache/spark/pull/59431#discussion_r4282758738
##########
.github/workflows/build_and_test.yml:
##########
@@ -614,6 +620,7 @@ jobs:
- modules: ${{
fromJson(needs.precondition.outputs.required).pyspark-pandas != 'true' &&
'pyspark-pandas-connect' }}
- modules: ${{
fromJson(needs.precondition.outputs.required).pyspark-pandas != 'true' &&
'pyspark-pandas-slow-connect' }}
env:
+ DEVELOCITY_ACCESS_KEY: ${{ secrets.SPARK_DEVELOCITY_ACCESS_KEY }}
Review Comment:
**Non-blocking (P2):** This job never runs sbt or Maven. `Run tests` exports
`SKIP_SCALA_BUILD=true`, the job env sets `SKIP_MIMA=true`, and no `pyspark-*`
module defines sbt test goals or build tests, so `dev/run-tests` only runs the
Python suites against the precompiled artifact. The sparkr job (line 767) is
the same: it runs `./dev/run-tests --modules sparkr` on the precompiled
artifact.
In both jobs, this line enables no Build Scan, but it puts the publish- and
cache-write-capable key into the environment of every step. That includes
`codecov/codecov-action` (lines 716 and 727, when the scheduled coverage
workflow sets `PYSPARK_CODECOV`), `test-summary/action`, and every Python and R
test process. The description lists pyspark and sparkr among the job-level
exceptions because their test step runs sbt or Maven, which isn't true for
these two.
Could you drop the entry from both jobs? The `build` and
`docker-integration-tests` jobs do run sbt tests, so their job-level entries
are still needed.
**Verification:**
- **Inspection:** Every DEVELOCITY_ACCESS_KEY entry left in
build_and_test.yml is scoped to a job or step that runs build/sbt, build/mvn,
or dev/run-tests with a Scala build or sbt test goal. The pyspark and sparkr
jobs have none.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
