[
https://issues.apache.org/jira/browse/FLINK-30092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hong Liang Teoh updated FLINK-30092:
------------------------------------
Description:
Currently, the DynamoDB sink's Table API allows users to input records with
Flink type of `DOUBLE`. However, the limitations on Double on Flink/Java side
[1] is more permissive than DynamoDB's limitations on the Number type [2].
Java Double limitations:
MIN_VALUE: −1.7976931348623157×10{^}308{^}
MAX_VALUE: 1.7976931348623157×10{^}308{^}
DynamoDB number type limitations:
MIN_VALUE: -9.9999999999999999999999999999999999999E+125 to -1E-130
MAX_VALUE: 1E-130 to 9.9999999999999999999999999999999999999E+125
This means that a user can write values of type `DOUBLE` into the DDB sink, as
long as they are within the limitations of DynamoDB. Once they exceed the
limitation, the record will fail to write.
We should consider improving this experience by adding a configuration to allow
users to map DOUBLE to a specified DDB type (e.g. DDB String, DDB number)
instead. We will be able to do this easily once FLINK-29267 is implemented.
[1] [https://docs.oracle.com/javase/7/docs/api/java/lang/Double.html#MAX_VALUE]
[2]
[https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.NamingRulesDataTypes.html#:~:text=Boolean%2C%20and%20null.-,Number,-Numbers%20can%20be]
was:
Currently, the DynamoDB sink's Table API allows users to input records with
Flink type of `DOUBLE`. However, the limitations on Double on Flink/Java side
[1] is more permissive than DynamoDB's limitations on the Number type [2].
Java Double limitations:
MIN_VALUE: −1.7976931348623157×10{^}308{^}
MAX_VALUE: 1.7976931348623157×10{^}308{^}
DynamoDB number type limitations:
MIN_VALUE: -9.9999999999999999999999999999999999999E+125 to -1E-130
MAX_VALUE: 1E-130 to 9.9999999999999999999999999999999999999E+125
This means that a user can write values of type `DOUBLE` into the DDB sink, as
long as they are within the limitations of DynamoDB. Once they exceed the
limitation, the record will fail to write.
We should consider improving this experience by adding a configuration to allow
users to map DOUBLE to DDB String type instead.
[1] [https://docs.oracle.com/javase/7/docs/api/java/lang/Double.html#MAX_VALUE]
[2]
[https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.NamingRulesDataTypes.html#:~:text=Boolean%2C%20and%20null.-,Number,-Numbers%20can%20be]
> Improve Table API experience for Flink DOUBLE type
> --------------------------------------------------
>
> Key: FLINK-30092
> URL: https://issues.apache.org/jira/browse/FLINK-30092
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / DynamoDB
> Reporter: Hong Liang Teoh
> Priority: Major
>
> Currently, the DynamoDB sink's Table API allows users to input records with
> Flink type of `DOUBLE`. However, the limitations on Double on Flink/Java side
> [1] is more permissive than DynamoDB's limitations on the Number type [2].
>
> Java Double limitations:
> MIN_VALUE: −1.7976931348623157×10{^}308{^}
> MAX_VALUE: 1.7976931348623157×10{^}308{^}
> DynamoDB number type limitations:
> MIN_VALUE: -9.9999999999999999999999999999999999999E+125 to -1E-130
> MAX_VALUE: 1E-130 to 9.9999999999999999999999999999999999999E+125
>
> This means that a user can write values of type `DOUBLE` into the DDB sink,
> as long as they are within the limitations of DynamoDB. Once they exceed the
> limitation, the record will fail to write.
>
> We should consider improving this experience by adding a configuration to
> allow users to map DOUBLE to a specified DDB type (e.g. DDB String, DDB
> number) instead. We will be able to do this easily once FLINK-29267 is
> implemented.
>
>
> [1]
> [https://docs.oracle.com/javase/7/docs/api/java/lang/Double.html#MAX_VALUE]
> [2]
> [https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.NamingRulesDataTypes.html#:~:text=Boolean%2C%20and%20null.-,Number,-Numbers%20can%20be]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
