lastomato edited a comment on issue #11151: [BEAM-9468] Hl7v2 io URL: https://github.com/apache/beam/pull/11151#issuecomment-606940440 Sorry for the delay. Please see my comments inline. > Open Questions: > > 1. Should we remove adaptive throttling? I think it is fine to remove it since the API has quota enabled by default, and retry logic is in place. > > * Seems that we're using retries in the client request initializer and right now a "bad record" will slow down the Read / Write (even though the error has nothing to do with the HL7v2 store being overwhelmed). Originally we wanted to be safe with overwhelming QPS on the HL7v2 store in batch scenarios. > 2. Should we add more to the `HealthcareIOError`? The second point below would be very helpful, but I am fine with adding it to next PR (if that's easier). > > * Add (processing time) Timestamp? > * Add a convenience DoFn `HealthcareIOErrrorToTableRowFn` to ease writing deadletter queue to BigQuery. > 3. Would it be more useful to expose an error rate metric than an error count? This functionality is probably already provided by services like stackdriver, we might want to wait until there is a concrete use case.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
