David Mollitor created SPARK-59507:
--------------------------------------
Summary: Parse interval fractional seconds without building a
padded string in IntervalUtils
Key: SPARK-59507
URL: https://issues.apache.org/jira/browse/SPARK-59507
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.1.0
Reporter: David Mollitor
{{IntervalUtils.parseNanos}} converts the fractional-second digits of a
day-time interval into nanoseconds by right-padding them to 9 characters with a
string concatenation and a substring, then parsing the result:
{code:scala}
val alignedStr = if (nanos.length < maxNanosLen) {
(nanos + "000000000").substring(0, maxNanosLen)
} else nanos
val nanoSecond = toLongWithRange(nanosStr, alignedStr, 0L, 999999999L)
{code}
This allocates two throwaway strings per call – one for the concatenation and
one for the substring – purely to zero-pad the value before an integer parse.
h3. Proposed change
The fractional part is guaranteed to be 1-9 ASCII digits by the grammar, so the
value can be parsed directly and scaled by the corresponding power of ten
instead of building a padded string:
{code:scala}
val raw = toLongWithRange(nanosStr, nanos, 0L, 999999999L) // 1..9 digits
val nanoSecond = raw * pow10(maxNanosLen - nanos.length) // e.g. "5" -> 5 *
1e8
{code}
This removes both string allocations on the fractional-seconds path. The result
is
identical: {{raw}} is always in {{[0, 999999999]}} and the scaled value stays
within the
same range, so the range check and error behavior are unchanged. The change
also folds the {{length == 9}} case in for free ({{{}pow10(0) == 1{}}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
