mgaido91 commented on a change in pull request #24505: [SPARK-27607][SQL]
Improve Row.toString performance
URL: https://github.com/apache/spark/pull/24505#discussion_r280110036
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala
##########
@@ -465,16 +465,34 @@ trait Row extends Serializable {
}
/** Displays all elements of this sequence in a string (without a
separator). */
- def mkString: String = toSeq.mkString
+ def mkString: String = mkString("")
/** Displays all elements of this sequence in a string using a separator
string. */
- def mkString(sep: String): String = toSeq.mkString(sep)
+ def mkString(sep: String): String = mkString("", sep, "")
/**
* Displays all elements of this traversable or iterator in a string using
* start, end, and separator strings.
*/
- def mkString(start: String, sep: String, end: String): String =
toSeq.mkString(start, sep, end)
+ def mkString(start: String, sep: String, end: String): String = {
+ var first = true
+ var i = 0
+ val n = length
+ val builder = new StringBuilder
+ builder.append(start)
+ while (i < n) {
+ if (first) {
Review comment:
no, I don't think it is a good idea. This is implementation is actually
similar to what is done in scala's code for `mkString` and I tried running the
benchmark above with the change and it takes ~ 50 ms vs ~ 48 ms. The overhead
of the comparison added for each execution of the cycle seems to be dominant
wrt saving one variable and assigning it once...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
