[
https://issues.apache.org/jira/browse/SPARK-59507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59507:
-----------------------------------
Labels: pull-request-available (was: )
> Parse interval fractional seconds without building a padded string in
> IntervalUtils
> -----------------------------------------------------------------------------------
>
> Key: SPARK-59507
> URL: https://issues.apache.org/jira/browse/SPARK-59507
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.1.0
> Reporter: David Mollitor
> Priority: Trivial
> Labels: pull-request-available
>
> {{IntervalUtils.parseNanos}} converts the fractional-second digits of a
> day-time interval into nanoseconds by right-padding them to 9 characters with
> a string concatenation and a substring, then parsing the result:
> {code:scala}
> val alignedStr = if (nanos.length < maxNanosLen) {
> (nanos + "000000000").substring(0, maxNanosLen)
> } else nanos
> val nanoSecond = toLongWithRange(nanosStr, alignedStr, 0L, 999999999L)
> {code}
> This allocates two throwaway strings per call – one for the concatenation and
> one for the substring – purely to zero-pad the value before an integer parse.
> h3. Proposed change
> The fractional part is guaranteed to be 1-9 ASCII digits by the grammar, so
> the value can be parsed directly and scaled by the corresponding power of ten
> instead of building a padded string:
> {code:scala}
> val raw = toLongWithRange(nanosStr, nanos, 0L, 999999999L) // 1..9 digits
> val nanoSecond = raw * pow10(maxNanosLen - nanos.length) // e.g. "5" -> 5
> * 1e8
> {code}
> This removes both string allocations on the fractional-seconds path. The
> result is
> identical: {{raw}} is always in {{[0, 999999999]}} and the scaled value stays
> within the
> same range, so the range check and error behavior are unchanged. The change
> also folds the {{length == 9}} case in for free ({{{}pow10(0) == 1{}}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
