This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 4e2e9476c79c [SPARK-54693][CORE][TESTS][FOLLOWUP] Minor fix
TPCDSDataBenchmark
4e2e9476c79c is described below
commit 4e2e9476c79cc8b8e066130d92f698eb2292ee91
Author: Cheng Pan <[email protected]>
AuthorDate: Wed Mar 4 14:30:40 2026 +0800
[SPARK-54693][CORE][TESTS][FOLLOWUP] Minor fix TPCDSDataBenchmark
### What changes were proposed in this pull request?
- Use https://github.com/databricks/tpcds-kit.git to match the GHA
benchmark workflow.
- Replace `assert` with `require` for better semantics.
### Why are the changes needed?
Minor improvements to the tests.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Review.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #54586 from pan3793/SPARK-54693-followup.
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
---
core/src/test/scala/org/apache/spark/io/TPCDSDataBenchmark.scala | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/core/src/test/scala/org/apache/spark/io/TPCDSDataBenchmark.scala
b/core/src/test/scala/org/apache/spark/io/TPCDSDataBenchmark.scala
index b6b419e0201a..30b4be898b09 100644
--- a/core/src/test/scala/org/apache/spark/io/TPCDSDataBenchmark.scala
+++ b/core/src/test/scala/org/apache/spark/io/TPCDSDataBenchmark.scala
@@ -24,7 +24,7 @@ import org.apache.spark.benchmark.BenchmarkBase
/**
* TPC-DS data preparation:
* <p>
- * 1. Follow https://github.com/gregrahn/tpcds-kit.git to set up tpcds-kit
+ * 1. Follow https://github.com/databricks/tpcds-kit.git to set up tpcds-kit
* <p>
* 2. Create a folder and export environment variable SPARK_TPCDS_DATA_TEXT
* {{{
@@ -48,13 +48,13 @@ abstract class TPCDSDataBenchmark extends BenchmarkBase {
protected def prepareData(): Unit = {
val tpcDsDataDir = sys.env.get("SPARK_TPCDS_DATA_TEXT")
- assert(tpcDsDataDir.nonEmpty, "Can not find env var SPARK_TPCDS_DATA_TEXT")
+ require(tpcDsDataDir.nonEmpty, "Can not find env var
SPARK_TPCDS_DATA_TEXT")
val catalogSalesDatPath = Paths.get(tpcDsDataDir.get, "catalog_sales.dat")
- assert(Files.exists(catalogSalesDatPath), s"File $catalogSalesDatPath does
not exists, " +
+ require(Files.exists(catalogSalesDatPath), s"File $catalogSalesDatPath
does not exists, " +
s"please follow instruction to generate the TPC-DS (SF1) text data
first.")
- // the size of TPCDS catalog_sales.dat (SF1) is about 283M
+ // the size of TPC-DS catalog_sales.dat (SF1) is about 283M
data = Files.readAllBytes(catalogSalesDatPath)
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
