Sounds fine to me. I guess one question is what needs to be formalized in the Schema.fbs files or elsewhere in the columnar format documentation (and we will need to hold an associated vote for that I think)
On Mon, Aug 3, 2020 at 11:30 PM Micah Kornfield <emkornfi...@gmail.com> wrote: > > Given no objections, we'll go ahead and start implementing support for > 256-bit decimals. > > I'm considering setting up another branch to develop all the components so > they can be merged to master atomically. > > Thanks, > Micah > > On Tue, Jul 28, 2020 at 6:39 AM Wes McKinney <wesmck...@gmail.com> wrote: >> >> Generally this sounds fine to me. At some point it would be good to >> add 32-bit and 64-bit decimal support but this can be done in the >> future. >> >> On Tue, Jul 28, 2020 at 7:28 AM Fan Liya <liya.fa...@gmail.com> wrote: >> > >> > Hi Micah, >> > >> > Thanks for opening the discussion. >> > I am aware of some scenarios where decimal requires more than 16 bytes, so >> > I think it would be beneficial to support this in Arrow. >> > >> > Best, >> > Liya Fan >> > >> > >> > On Tue, Jul 28, 2020 at 11:12 AM Micah Kornfield <emkornfi...@gmail.com> >> > wrote: >> > >> > > Hi Arrow Dev, >> > > ZetaSQL (Google's open source standard SQL library) recently introduced a >> > > BigNumeric [1] type which requires a 256 bit width to properly support >> > > it. >> > > I'd like to add support (possibly in collaboration with some of my >> > > colleagues) to add support for 256 bit width Decimals in Arrow to >> > > support a >> > > type corresponding to BigNumeric. >> > > >> > > In past discussions on this, I don't think we established a minimum bar >> > > for >> > > supporting additional bit-widths within Arrow. >> > > >> > > I'd like to propose the following requirements: >> > > 1. A vote agreeing on adding support for a new bitwidth (we can discuss >> > > any objections here). >> > > 2. Support in Java and C++ for integration tests verifying the ability >> > > to >> > > round-trip the value. >> > > 3. Support in Java for conversion to/from BigDecimal [2] >> > > 4. Support in Python converting to/from Decimal [3] >> > > >> > > Is there anything else that people feel like is a requirement for basic >> > > support of an additional bit width for Decimal's? >> > > >> > > Thanks, >> > > Micah >> > > >> > > >> > > [1] >> > > >> > > https://github.com/google/zetasql/blob/1aefaa7c62fc7a50def879bb7c4225ec6974b7ef/zetasql/public/numeric_value.h#L486 >> > > [2] https://docs.oracle.com/javase/7/docs/api/java/math/BigDecimal.html >> > > [3] https://docs.python.org/3/library/decimal.html >> > >
