brucearctor commented on issue #22986: URL: https://github.com/apache/beam/issues/22986#issuecomment-1234909480
> The deadletter pattern is also important in batch systems - to prevent a single malformed row from failing the entire job. Had you seen: https://cloud.google.com/java/docs/reference/google-cloud-bigquery/latest/com.google.cloud.bigquery.LoadJobConfiguration.Builder#com_google_cloud_bigquery_LoadJobConfiguration_Builder_setMaxBadRecords_java_lang_Integer_ or ( since I think you prefer python ) --> https://googleapis.dev/python/bigquery/latest/generated/google.cloud.bigquery.job.LoadJobConfig.html#google.cloud.bigquery.job.LoadJobConfig.max_bad_records @cozos -- with a higher number in that config number ( more than the number of bad records ) the job will succeed. It sounds like that is what you need based on the quote. As it relates to using FILE_LOADS: I *think* the BQ API that FILE_LOADS calls would not support that functionality. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
