[
https://issues.apache.org/jira/browse/SPARK-57420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía updated SPARK-57420:
---------------------------------
Description:
Two improvements to the benchmark workflow:
1. Add explicit generate-tpcds boolean input (default: true) to control TPC-DS
data generation. Users running non-TPC-DS benchmarks can set it to false to
skip the expensive generation step (~5-10 min saved per run). This replaces the
previous heuristic pattern matching which had edge cases with wildcard patterns.
2. Add early CPU model check step that runs immediately after checkout, before
compilation. Prints the CPU as a notice annotation for live visibility in the
Actions UI, and optionally fails fast if the runner CPU does not match the
expected-cpu input parameter (~20-30 min saved on CPU mismatch).
Summary: Add generate-tpcds input and early CPU check to benchmark
workflow (was: Only generate TPC-DS data when required and check CPU
compatibility early in benchmark workflow)
> Add generate-tpcds input and early CPU check to benchmark workflow
> ------------------------------------------------------------------
>
> Key: SPARK-57420
> URL: https://issues.apache.org/jira/browse/SPARK-57420
> Project: Spark
> Issue Type: Sub-task
> Components: Project Infra
> Affects Versions: 5.0.0
> Reporter: Ismaël Mejía
> Priority: Major
> Labels: pull-request-available
>
> Two improvements to the benchmark workflow:
> 1. Add explicit generate-tpcds boolean input (default: true) to control
> TPC-DS data generation. Users running non-TPC-DS benchmarks can set it to
> false to skip the expensive generation step (~5-10 min saved per run). This
> replaces the previous heuristic pattern matching which had edge cases with
> wildcard patterns.
> 2. Add early CPU model check step that runs immediately after checkout,
> before compilation. Prints the CPU as a notice annotation for live visibility
> in the Actions UI, and optionally fails fast if the runner CPU does not match
> the expected-cpu input parameter (~20-30 min saved on CPU mismatch).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
