kazuyukitanimura opened a new pull request, #37020: URL: https://github.com/apache/spark/pull/37020
### What changes were proposed in this pull request? Currently `org.apache.spark.sql.execution.benchmark.TPCDSQueryBenchmark` cannot start via Github Actions. That is due to the extra required `--data-location` argument. This PR proposes to provide an option to pass the argument by `SPARK_TPCDS_DATA` environment variable. This is the same approach taken by `TPCDSQueryTestSuite`. This PR also adds the Github workflow step to generate the TPC-DS data and set the `SPARK_TPCDS_DATA` location. Developers still can use `--data-location` argument to `TPCDSQueryBenchmark` just like before for manual testing. This PR is a refinement of [the earlier attempt](https://github.com/apache/spark/pull/33544) ### Why are the changes needed? Currently `TPCDSQueryBenchmark` is excluded from benchmark runs via Github Actions. Therefore, benchmark results e.g. TPCDSQueryBenchmark-results.txt are not updated regularly and obsolete. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Tested on Github Actions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
