jonathan-lemos commented on code in PR #26873:
URL: https://github.com/apache/beam/pull/26873#discussion_r1205863520
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java:
##########
@@ -694,7 +694,16 @@ public static Row toBeamRow(Schema rowSchema, TableSchema
bqSchema, TableRow jso
if (jsonBQValue instanceof List) {
return ((List<Object>) jsonBQValue)
.stream()
- .map(v -> ((Map<String, Object>) v).get("v"))
+ .map(
+ v -> {
+ if (v instanceof Map) {
+ Map<String, Object> m = (Map<String, Object>) v;
+ if (m.size() == 1 && m.containsKey("v")) {
+ return m.get("v");
+ }
+ }
+ return v;
+ })
Review Comment:
That makes more sense. I've applied the change and formatted it.
Trying to make a test case for this scenario, it seems that Beam's BigQuery
IO will flatten any list of structs e.g. querying
```
[
{make: honda, metadata: [{color: red, model: civic}, {color: blue, model:
accord}]}
{make: toyota, metadata: [{color: green, model: corolla}]}
]
```
will result in the following:
```
[
{make: honda, metadata_color: red, metadata_model: civic}
{make: honda, metadata_color: blue, metadata_model: accord}
{make: toyota, metadata_color: green, metadata_model: corolla}
]
```
The underlying BigQuery API returns the two rows above, but Beam's
[`readTableRows()`](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.html#readTableRows--)
will flatten the structure and return the 3 below.
Given that behavior, it doesn't seem like it's possible for a value of
`List<Struct>` to make it to this function, so I haven't added a test case for
that scenario.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
