Re: java.io.IOException: Failed to close some writers

2018-05-11 Thread Eugene Kirpichov
Do you see any other exceptions in Stackdriver, e.g. (guess) OOM errors? "worker lost contact with the service" is commonly caused by OOM crashes, and GC thrashing could also cause network issues of the sort that you're seeing. On Fri, May 11, 2018 at 2:54 AM Jose Ignacio Honrado Benítez <

java.io.IOException: Failed to close some writers

2018-05-11 Thread Jose Ignacio Honrado Benítez
Hi there, I'm running a pipeline in Google Cloud Dataflow that reads from GCS and writes into several BQ tables. This pipeline has been successfully used to load several TB of data from GCS to BQ. In one particular execution, I am loading documents from GCS that are "exploded" into several