shunping commented on PR #34910: URL: https://github.com/apache/beam/pull/34910#issuecomment-2869131488
> > For my education, when validation is present, what happened if some of the files do not exist due to some reasons? Will BigQuey retry those files or simply ignore them? > > Also want to make sure that skipping validation will not lead to any data loss. > > It will fail the pipeline eventually. validate=True just does more early checks to fail the jobs. I see. So GCS complains about we check existence of those files, but it has no problem when BigQuery processing them. That's weird. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
