[
https://issues.apache.org/jira/browse/AVRO-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17785909#comment-17785909
]
Martin Tzvetanov Grigorov commented on AVRO-3902:
-------------------------------------------------
{quote}I understand that you may not want to change the default behavior of the
library for backward compatibility reasons but could you at least mention this
important limitation in the main documentation page?
{quote}
Which main page do you mean ? This one -
[https://github.com/apache/avro/blob/main/lang/js/doc/Home.md] ?
IMO this case is an "advanced usage" and it is correct to be explained at
[https://github.com/apache/avro/blob/main/lang/js/doc/Advanced-usage.md] !
I like your CUSTOM_LONG_AVRO_TYPE ! It seems to be backward compatible! Please
send a Pull Request with some unit tests!
> 64bit long support for JavaScript
> ---------------------------------
>
> Key: AVRO-3902
> URL: https://issues.apache.org/jira/browse/AVRO-3902
> Project: Apache Avro
> Issue Type: Bug
> Components: javascript, js
> Affects Versions: 1.11.3
> Reporter: Philippe Ozil
> Priority: Major
>
> The latest version of avro-js cannot deserialize large numeric values that
> fit in a 64 bit Long. It's limited to the values that fit in the [maximum
> safe integer in
> JavaScript|https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/MAX_SAFE_INTEGER]
> (2^53 – 1).
> Attempting to read larger values such as 1698100138468892700 triggers a
> deserialization error:
> ?? Error: invalid "long": 1698100138468892700??
>
> The avro-js documentation provides a solution for this: [specifying a custom
> Long
> type|https://github.com/apache/avro/blob/main/lang/js/doc/Advanced-usage.md#custom-long-types]
> however, this documentation page is not easy to find and can easily be
> missed.
> I understand that you may not want to change the default behavior of the
> library for backward compatibility reasons but could you at least mention
> this important limitation in the main documentation page?
>
> Also, there's now a native JS solution for supporting large values in long
> type:
> [BigInt|https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt].
> I believe that this should be the preferred approach since it doesn't
> introduce extra dependencies.
> This is the native long-safe implementation (inspired from
> [avsc|https://github.com/mtth/avsc/wiki/Advanced-usage#custom-long-types]):
> {code:js}
> const longType = avro.types.LongType.using({
> fromBuffer: (buf) => buf.readBigInt64LE(),
> toBuffer: (n) => {
> const buf = Buffer.alloc(8);
> buf.writeBigInt64LE(n);
> return buf;
> },
> fromJSON: BigInt,
> toJSON: Number,
> isValid: (n) => typeof n === 'bigint',
> compare: (n1, n2) => { return n1 === n2 ? 0 : (n1 < n2 ? -1 : 1); }
> });{code}
>
> This is an alternate implementation that I wrote. It uses regular Number
> whenever possible and falls back to BigInt when the value exceeds the max
> safe value for Number. This could be a good built-in alternative for avro-js
> in which we avoid throwing a deserialization error.
> {code:js}
> const CUSTOM_LONG_AVRO_TYPE = avro.types.LongType.using({
> fromBuffer: (buf) => {
> const big = buf.readBigInt64LE();
> if (big > Number.MAX_SAFE_INTEGER) {
> return big;
> }
> return Number(BigInt.asIntN(64, big));
> },
> toBuffer: (n) => {
> const buf = Buffer.alloc(8);
> if (n instanceof BigInt) {
> buf.writeBigInt64LE(n);
> } else {
> buf.writeBigInt64LE(BigInt(n));
> }
> return buf;
> },
> fromJSON: BigInt,
> toJSON: Number,
> isValid: (n) => {
> const type = typeof n;
> return type === 'bigint' || type === 'number';
> },
> compare: (n1, n2) => {
> return n1 === n2 ? 0 : n1 < n2 ? -1 : 1;
> }
> });{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
