RyuSA opened a new pull request, #27413: URL: https://github.com/apache/beam/pull/27413
**Current issue:** As per the specifications, BigQuery is capable of receiving timestamps up to `9999-12-31T23:59:59`. https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#timestamp_type There is a bug in the `java.time.Instant.until`. https://bugs.java.com/bugdatabase/view_bug?bug_id=8199095 This causes there is an overflow issue with timestamps beyond `2262-04-11T23:47:16Z`. This prevents correct processing of timestamps beyond this point. https://github.com/apache/beam/blob/v2.48.0/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProto.java#L764 <details> <summary>Details</summary> `Instant.until` calculates the period using `ChronoUnit.UNIT.between`. `ChronoUnit.MICROS.between` will multiply 1000_000_000(=Instant.NANOS_PER_SECOND) to the micro-second based Instant object and convert it to nano-second based one. https://github.com/frohoff/jdk8u-jdk/blob/master/src/share/classes/java/time/Instant.java#L1149 This means, the maximum timestamp which `ChronoUnit.MICROS.between` can handle is - long.max = 9223372036854775807 - Timestamp(long.max, UNIT.NANO) ~ 2262-04-11T23:47:16Z `TableRowToStorageApiProto.singularFieldToProtoValue` will use `ChronoUnit.MICROS.between(Instant.EPOCH, Instant.from(timestamp)`. This will be overflowed if timestamp > `2262-04-11T23:47:16Z` and it throw `java.lang.ArithmeticException: long overflow`. </details> **Proposed Fix:** This pull request applies a workaround to the section of code where the overflow occurs. With this fix, the StorageWriteAPI can correctly transmit timestamps up to `9999-12-31 23:59:59.999999 UTC` as originally intended by BigQuery's specifications. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier). To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md) GitHub Actions Tests Status (on master branch) ------------------------------------------------------------------------------------------------ [](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule) [](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule) [](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule) [](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule) See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
