[
https://issues.apache.org/jira/browse/CALCITE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426277#comment-17426277
]
Julian Hyde edited comment on CALCITE-4841 at 10/8/21, 4:51 PM:
----------------------------------------------------------------
I get your point that floating point types ({{DOUBLE}} and {{REAL}}) are not
suitable for financial data. I'm always frustrated with JSON for this reason.
And I see a decimal type – perhaps fixed point, perhaps floating point – as the
answer.
But the problem is that Calcite only has a {{DECIMAL}} type (fixed point
decimal). (There is a SQL type {{NUMERIC}} but Calcite doesn't support it very
well: it's in {{java.sql.Types}} but not in
{{org.apache.sql.type.SqlTypeName}}.) Therefore weird things are going to
happen when the data leaves the file adapter.
To be practical, I think you should either support the {{DECIMAL}} type in the
file adapter, or should kick off a project to add support for {{NUMERIC}} to
Calcite.
was (Author: julianhyde):
I get your point that floating point types ({{DOUBLE}} and {{REAL}}) are not
suitable for financial data. I'm always frustrated with JSON for this reason.
And I see a decimal type – perhaps fixed point, perhaps floating point – as the
answer.
But the problem is that Calcite only has a {{DECIMAL}} type (fixed point
decimal). (There is a SQL type {{NUMERIC}} but Calcite doesn't support it very
well: it's in {{java.sql.Types}} but not in
{{org.apache.sql.type.SqlTypeName}}.) Therefore weird things are going to
happen when the data leaves the file adapter.
To be practical, I think you should either support the {{DECIMAL}} type in the
file adapter, or should kick of a project to add support for {{NUMERIC}} to
Calcite.
> Support numeric column type in CSV and File adapter
> ---------------------------------------------------
>
> Key: CALCITE-4841
> URL: https://issues.apache.org/jira/browse/CALCITE-4841
> Project: Calcite
> Issue Type: Improvement
> Components: csv-adapter, file-adapter
> Reporter: Louis Kuang
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently, the csv adapter does not support the numeric column type (see
> supported types in
> [CsvFieldType|https://github.com/apache/calcite/blob/4bc916619fd286b2c0cc4d5c653c96a68801d74e/file/src/main/java/org/apache/calcite/adapter/file/CsvFieldType.java#L35].
> Any type that is not supported will be interpreted by the
> [CsvEnumerator|https://github.com/apache/calcite/blob/master/file/src/main/java/org/apache/calcite/adapter/file/CsvEnumerator.java]
> as string.
> When dealing with decimal numbers, the currently most appropriate type is
> `double`. However, this is not accurate enough for financial data. This
> feature request proposes adding a `numeric` column type that will be
> implemented by the Java `BigDecimal` type (and by conversion in
> [JavaToSqlTypeConversionRules|https://github.com/apache/calcite/blob/4bc916619fd286b2c0cc4d5c653c96a68801d74e/core/src/main/java/org/apache/calcite/sql/type/JavaToSqlTypeConversionRules.java#L74]
> be represented as a `Decimal` SQL type). This allow financial data to be
> represented and computed more accurately (`BigDecimal` has higher precision
> than `double`).
> Please see sample implementation in PR.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
