kkulczak commented on issue #18445:
URL: https://github.com/apache/beam/issues/18445#issuecomment-1447905587
Hi,
I've encountered same issue while running a high throughput (2GB/s) job on
DataflowRunner.
22 000 records out of 54 000 000 were decoded as `LinkedHashMap` resulting
in multiple uncaught `CastException`.
```
class java.util.LinkedHashMap cannot be cast to class
com.google.api.services.bigquery.model.TableRow (java.util.LinkedHashMap is in
module java.base of loader 'bootstrap';
com.google.api.services.bigquery.model.TableRow is in unnamed module of loader
'app')
```
It was very rare (0,04%), but my client cannot allow for any loss of data.
As a workaround for now we switched to cast with `Map<String, Object>`
It is very hard to debug this error as it is not mention in the docs, that
to safely access nested fields you should not cast them to `TableRow` type.
Can we mention this fact in Beam Java Documentation of TableRow?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
