[
https://issues.apache.org/jira/browse/IMPALA-10680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17401098#comment-17401098
]
ASF subversion and git services commented on IMPALA-10680:
----------------------------------------------------------
Commit 0bde6b443cb1f1eea9fb9cb2fd3dc98acb92cd62 in impala's branch
refs/heads/master from Amogh Margoor
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0bde6b4 ]
IMPALA-10680: Replace StringToFloatInternal using fast_double_parser library
StringToFloatInternal is used to parse string into float. It had logic
to ensure it is faster than standard functions like strtod in many
cases, but it was not as accurate. We are replacing it by a third
party library named fast_double_parser which is both fast and doesn't
sacrifise the accuracy for speed. On benchmarking on more than
1 million rows where string is cast to double, it is found that new
patch is on par with the earlier algorithm.
Results:
W/O library: Fetched 1222386 row(s) in 32.10s
With library: Fetched 1222386 row(s) in 31.71s
Testing:
1. Added test to check for accuracy improvement.
2. Ran existing Backend tests for correctness.
Change-Id: Ic105ad38a2fcbf2fb4e8ae8af6d9a8e251a9c141
Reviewed-on: http://gerrit.cloudera.org:8080/17389
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Replace StringToFloatInternal that converts String to Float using
> fast_double_parser library
> --------------------------------------------------------------------------------------------
>
> Key: IMPALA-10680
> URL: https://issues.apache.org/jira/browse/IMPALA-10680
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Amogh Margoor
> Assignee: Amogh Margoor
> Priority: Major
>
> Based on the comment made by [~csringhofer]
> [here|https://issues.apache.org/jira/browse/IMPALA-10350?focusedCommentId=17324270&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17324270],
> we can use fast_double_parser (introduced by IMPALA-10654) to do String to
> Double conversion
> [here|https://github.com/apache/impala/blob/master/be/src/util/string-parser.h#L459].
> This would ensure precision loss can be avoided in cases like below:
> {code:java}
> select cast("0.43149576573887316" as double);
> result: 0.4314957657388731
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
