pratham76 commented on code in PR #59213:
URL: https://github.com/apache/spark/pull/59213#discussion_r4188005782
##########
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/StringFunctionsBenchmark.scala:
##########
@@ -53,6 +53,18 @@ object StringFunctionsBenchmark extends SqlBasedBenchmark {
doRegexpReplaceBenchmark()
}
benchmark.run()
+
+ val splitBenchmark = new Benchmark("split", N, output = output)
+ splitBenchmark.addCase("split('*-*', '-')", M) { _ =>
+ df.selectExpr("split(subject, '-')").noop()
+ }
+ splitBenchmark.addCase("split('*-*', '[-]')", M) { _ =>
+ df.selectExpr("split(subject, '[-]')").noop()
+ }
+ splitBenchmark.addCase("split('*-*' COLLATE UTF8_LCASE, '-')", M) { _ =>
+ df.selectExpr("split(subject COLLATE UTF8_LCASE, '-')").noop()
+ }
+ splitBenchmark.run()
Review Comment:
This PR bundles three independent things: (1) the core Pattern cache fix,
(2) replicating JDK's single-char fast-path heuristic (splitsWithoutRegex,
UTF8String.java), and (3) this new benchmark. Could we split the fast-path
replication and the benchmark into follow-ups so the core cache fix is easier
to review/backport on its own?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
