[
https://issues.apache.org/jira/browse/ARROW-13028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397395#comment-17397395
]
Nate Clark commented on ARROW-13028:
------------------------------------
>> Is this by design or accident?
> I would say by accident, but I'm not sure what you mean with "the precision
> of the string is too much for a float". Strictly speaking, some very short
> decimal numbers are not exactly representable in binary floating-point, for
> example "0.3". Should we reject them?
I was thinking something like `3.78946546156984798497501e10` can be better
represented as double than a float. But as you point out there are values which
cannot be fully represented in either, so there might not be a good way to
detect when double should be used instead of float.
> [C++] CSV add convert option to attempt 32bit number inferences
> ---------------------------------------------------------------
>
> Key: ARROW-13028
> URL: https://issues.apache.org/jira/browse/ARROW-13028
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Nate Clark
> Assignee: Nate Clark
> Priority: Major
>
> When types are being inferred by CSV the numbers are always 64 bit. For large
> data sets it could be better to use 32 bit types to save over all memory. To
> do this it would be useful to add an option to ConvertOptions to try 32 bit
> numbers before 64 bit. By default this option would be disabled.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
