[
https://issues.apache.org/jira/browse/SPARK-35266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338878#comment-17338878
]
Apache Spark commented on SPARK-35266:
--------------------------------------
User 'byungsoo-oh' has created a pull request for this issue:
https://github.com/apache/spark/pull/32432
> Fix an error in BenchmarkBase.scala that occurs when creating a benchmark
> file in a non-existent directory
> ----------------------------------------------------------------------------------------------------------
>
> Key: SPARK-35266
> URL: https://issues.apache.org/jira/browse/SPARK-35266
> Project: Spark
> Issue Type: Bug
> Components: Tests
> Affects Versions: 3.2.0
> Reporter: Byungsoo Oh
> Assignee: Byungsoo Oh
> Priority: Minor
> Labels: easyfix
> Fix For: 3.2.0
>
>
> When submitting a benchmark job using _org.apache.spark.benchmark.Benchmarks_
> class with _SPARK_GENERATE_BENCHMARK_FILES=1_ option, an exception is raised
> if the directory where the benchmark file will be generated does not exist.
> For example, if you execute _BLASBenchmark_ like the command below, you get
> an error unless you manually create _benchmarks/_ directory under
> _spark/mllib-local/_.
> {code:java}
> SPARK_GENERATE_BENCHMARK_FILES=1 bin/spark-submit \
> --driver-memory 6g --class org.apache.spark.benchmark.Benchmarks \
> --jars "`find . -name '*-SNAPSHOT-tests.jar' -o -name '*avro*-SNAPSHOT.jar' |
> paste -sd ',' -`" \
> "`find . -name 'spark-core*-SNAPSHOT-tests.jar'`" \
> "org.apache.spark.ml.linalg.BLASBenchmark"
> {code}
> This is caused by the code in _BenchmarkBase.scala_ where an attempt is made
> to create the benchmark file without validating the path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
