With regards to scale, my colleague discovered some inconsistencies and filed a JIRA with a proposed fix (a PR should be attached shortly).
I think this is an edge case that should be fixed but if someone with more historical context has opinions, I'd like to here them. [1] https://issues.apache.org/jira/browse/ARROW-9913 On 2020/07/02 14:51:34, Wes McKinney <wesmck...@gmail.com> wrote: > I think the intention so far has been to support precision between 0 > and 38 and scale <= precision. 128-bit integers max out at 38 digits, > I think that's the rationale for the limit. See e.g. the Impala docs > (also uses 128-bit decimals) [1] > > [1]: https://impala.apache.org/docs/build/html/topics/impala_decimal.html > > On Wed, Jul 1, 2020 at 10:16 AM Kazuaki Ishizaki <ishiz...@jp.ibm.com> wrote: > > > > Hi, > > According to https://arrow.apache.org/docs/cpp/api/utilities.html, > > Decimal128 comes from the Apache ORC C++ implementation. > > > > When I see the Hive document at > > https://hive.apache.org/javadocs/r1.2.2/api/index.html?org/apache/hadoop/hive/common/type/Decimal128.html > > , there is the following statement. Does it help you? > > > A 128-bit fixed-length Decimal value in the ANSI SQL Numeric semantics, > > representing unscaledValue / 10**scale where scale is 0 or positive. > > > > Regards, > > Kazuaki Ishizaki > > > > > > > > From: Jacek Pliszka <jacek.plis...@gmail.com> > > To: dev@arrow.apache.org > > Date: 2020/07/02 00:08 > > Subject: [EXTERNAL] Re: Decimal128 scale limits > > > > > > > > Hi! > > > > I am aware about at least 2 different decimal128 things: > > > > a) the one we have - where we use 128 bits to store integer which is > > later shifted by scale - 38 is number of digits of significand i.e. > > digits fitting in 128 bits > > (2**128/10**38) - IMHO it is completely unrelated to scale which we > > store separately > > > > b) IEEE 754 one which has exponent from -6143 to +6144 > > > > BR, > > > > Jacek > > > > śr., 1 lip 2020 o 16:16 Antoine Pitrou <anto...@python.org> napisał(a): > > > > > > > > > Hello, > > > > > > Are there limits to the value of the scale for either decimal128 or > > > decimal? Can it be negative? Can it be greater than 38 (and/or lower > > > than -38)? > > > > > > It's not clear from looking either at the spec or at the C++ code... > > > > > > Regards > > > > > > Antoine. > > > > > > > > >
