[ https://issues.apache.org/jira/browse/BEAM-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Udi Meiri reassigned BEAM-2969: ------------------------------- Assignee: Udi Meiri > BigQueryIO fails when reading then writing timestamps before 1970 > ----------------------------------------------------------------- > > Key: BEAM-2969 > URL: https://issues.apache.org/jira/browse/BEAM-2969 > Project: Beam > Issue Type: Bug > Components: io-java-gcp > Reporter: Kevin Peterson > Assignee: Udi Meiri > Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > I have a batch pipeline which reads from BigQuery (via standard sql query), > does a small transform, and writes the data back to BigQuery. > This fails if any timestamps are present in the BQ data from before 1970: > {{"message" : "JSON parsing error in row starting at position 0: Couldn't > convert value to timestamp: Could not parse '1969-12-28 02:52:54.-484 UTC' as > a timestamp. Required format is YYYY-MM-DD HH:MM[:SS[.SSSSSS]] Field: > observed_timestamp; Value: 1969-12-28 02:52:54.-484 UTC",}} > It appears the TableRow coder doesn't handle negative timestamps properly, > using a negative number for the fractions of a second, which BQ considers > invalid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)