It does not, which might've been a mistake. The user can pass in an arbitrary BigDecimal object, and we will encode whatever scale parameter is encoded. This means that for DECIMAL, each record encodes the scale.
On Thu, Mar 18, 2021 at 12:33 PM Mingyu Zhong <myzh...@google.com> wrote: > Just wanted to clarify: BigQuery BIGNUMERIC type costs more than NUMERIC > type, so if NUMERIC is sufficient, the users likely won't want to switch to > BIGNUMERIC. > > Does Beam DECIMAL datatype contain the precision/scale parameters in the > metadata? If so, can we use those parameters to determine the mapped type? > > On Thu, Mar 18, 2021 at 12:08 PM Brian Hulette <bhule...@google.com> > wrote: > >> Hi Vachan, >> I don't think Beam DECIMAL is really a great mapping for either >> BigQuery's NUMERIC or BIGNUMERIC type. Beam's DECIMAL represents arbitrary >> precision decimals [1] to map well to Java's BigDecimal class [2]. >> >> Maybe we should add a fixed-precision decimal logical type [3], then have >> specific instances of it with the appropriate precision that map to NUMERIC >> and BIGNUMERIC. We could also shunt Beam DECIMAL to BIGNUMERIC for >> compatibility. >> >> [1] >> https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/Schema.java#L424 >> [2] https://docs.oracle.com/javase/8/docs/api/java/math/BigDecimal.html >> [3] >> https://github.com/apache/beam/tree/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/logicaltypes >> >> On Thu, Mar 18, 2021 at 12:00 PM Vachan Shetty <vac...@google.com> wrote: >> >>> Hello, I am currently trying to add support for BigQuery's new >>> BIGNUMERIC datatype [1] in Beam's BigQueryIO. I am currently following the >>> steps that were used for adding the NUMERIC datatype [2]. AFAICT Beam's >>> DECIMAL is the most appropriate datatype to map to BIGNUMERIC in BQ. >>> However, the Beam DECIMAL datatype is already mapped to NUMERIC in BQ >>> [2, 3]. Given this, should I simply map all Beam DECIMAL to BQ BIGNUMERIC? >>> Or should this conversion be done based on other information? [1]: >>> https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#decimal_types >>> [2]: https://issues.apache.org/jira/browse/BEAM-4417 [3]: >>> https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java#L188 >>> >> > > -- > Thanks, > > Mingyu >
