kennknowles opened a new issue, #18680:
URL: https://github.com/apache/beam/issues/18680
Following is the code that writes to BigQuery:
```
BigQueryIO.writeTableRows()
.to(destination)
.withCreateDisposition(CREATE_IF_NEEDED)
.withWriteDisposition(WRITE_APPEND)
.withSchema(tableSchema)
.expand(tableRows);
```
Here's the destination's implementation:
```
public TableDestination apply(ValueInSingleWindow<TableRow> input) {
String partition = timestampExtractor.apply(input.getValue())
.toString(DateTimeFormat.forPattern("yyyyMMdd").withZoneUTC());
TableReference tableReference = new
TableReference();
tableReference.setDatasetId(dataset);
tableReference.setProjectId(projectId);
tableReference.setTableId(String.format("%s_%s", table, partition));
log.debug("Will write to BigQuery
table: %s", tableReference);
return new TableDestination(tableReference, null);
}
```
When the dataflow tries to write to this table, I see the following message:
```
"errors" : [ {
"domain" : "global",
"message" : "Cannot read partition information from a table
that is not partitioned: <project_id>:<dataset>.<table>$19730522",
"reason" : "invalid"
} ]
```
So, it looks like it's not creating tables with partition in the first
place?
Apache beam version : 2.2.0
Imported from Jira
[BEAM-3501](https://issues.apache.org/jira/browse/BEAM-3501). Original Jira may
contain additional context.
Reported by: darshanmehta2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
