Hi everyone!! I'm working on BEAM-4257 issue and the approach I'm following is to create a new class 'BigQueryInsertError' that also extends 'GenericJson' and that contains three keys 'TableRow row', 'TableDataInsertAllResponse.InsertErrors error', and 'TableReference ref' and use this type as the contained items returned by WriteResults.getFailedInserts
I have now to create a Coder for this new type and I'm following the TableRowJsonCoder way https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowJsonCoder.java#L34 by relying on Jackson's ObjectMapper and StringUtf8Encoder. The problem is that I always get errors when deserialising as it deserialises the inner TableRow as a LinkedHashMap and fails when trying to assign it. Here you can see the full stacktrace: https://pastebin.com/MkUD9L3W Testing it a bit further I've spotted other GenericJson subclasses that cannot be encoded/decoded following that method. For example TableDataInsertAllResponse.InsertErrors itself. See the example below: TableDataInsertAllResponse.InsertErrors err = new TableDataInsertAllResponse.InsertErrors().setIndex(0L); ObjectMapper mapper = new ObjectMapper().disable(SerializationFeature.FAIL_ON_EMPTY_BEANS); mapper.readValue(mapper.writeValueAsString(err), TableDataInsertAllResponse.InsertErrors.class); Fails with a similar error, but in this case is because it deserialises the index into an int: https://pastebin.com/bzXMR3z5 So a couple of questions here: * Which is the appropriate way of encoding/decoding GenericJson subclasses? (Maybe this issues can be tackled using Jackson's type annotations, but I'm quite a newbie on Jackson and I couldn't figure out how) * This will (hopefully) be my very first contribution to Apache Beam and I'd like to get some feedback/comments/ideas/... on the issue and the suggested solution. Thanks everyone!
