[
https://issues.apache.org/jira/browse/BEAM-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chamikara Madhusanka Jayalath updated BEAM-9486:
------------------------------------------------
Summary: Python 2 PostCommit Failures due to BQ IT failures (was: Python 2
PostCommit Failures)
> Python 2 PostCommit Failures due to BQ IT failures
> --------------------------------------------------
>
> Key: BEAM-9486
> URL: https://issues.apache.org/jira/browse/BEAM-9486
> Project: Beam
> Issue Type: Bug
> Components: io-py-gcp
> Reporter: Rui Wang
> Assignee: Pablo Estrada
> Priority: Blocker
> Fix For: 2.20.0
>
>
> https://builds.apache.org/job/beam_PostCommit_Python2_PR/195/
> https://builds.apache.org/job/beam_PostCommit_Python2_PR/196/
> https://builds.apache.org/job/beam_PostCommit_Python2_PR/198/
> They might be related to this error message:
> {code:java}
> 03:42:13 RuntimeError: BigQuery job
> beam_load_2020_03_10_034115_50_bc104bcc3054360def24bf4ca8246e92_428184e062fc4771848d3c187f46f941
> failed. Error Result: <ErrorProto
> 03:42:13 message: u'No schema specified on job or table.'
> 03:42:13 reason: u'invalid'> [while running
> 'WriteWithMultipleDests/BigQueryBatchFileLoads/WaitForTempTableLoadJobs/WaitForTempTableLoadJobs']
> 03:42:13
> 03:42:18
> INFO:apache_beam.runners.dataflow.dataflow_runner:2020-03-10T03:42:15.301Z:
> JOB_MESSAGE_ERROR: Traceback (most recent call last):
> 03:42:18 File
> "/usr/local/lib/python2.7/site-packages/dataflow_worker/batchworker.py", line
> 647, in do_work
> 03:42:18 work_executor.execute()
> 03:42:18 File
> "/usr/local/lib/python2.7/site-packages/dataflow_worker/executor.py", line
> 176, in execute
> 03:42:18 op.start()
> 03:42:18 File "dataflow_worker/native_operations.py", line 38, in
> dataflow_worker.native_operations.NativeReadOperation.start
> 03:42:18 def start(self):
> 03:42:18 File "dataflow_worker/native_operations.py", line 39, in
> dataflow_worker.native_operations.NativeReadOperation.start
> 03:42:18 with self.scoped_start_state:
> 03:42:18 File "dataflow_worker/native_operations.py", line 44, in
> dataflow_worker.native_operations.NativeReadOperation.start
> 03:42:18 with self.spec.source.reader() as reader:
> 03:42:18 File "dataflow_worker/native_operations.py", line 54, in
> dataflow_worker.native_operations.NativeReadOperation.start
> 03:42:18 self.output(windowed_value)
> 03:42:18 File "apache_beam/runners/worker/operations.py", line 329, in
> apache_beam.runners.worker.operations.Operation.output
> 03:42:18 cython.cast(Receiver,
> self.receivers[output_index]).receive(windowed_value)
> 03:42:18 File "apache_beam/runners/worker/operations.py", line 135, in
> apache_beam.runners.worker.operations.ConsumerSet.receive
> 03:42:18 cython.cast(Operation, consumer).process(windowed_value)
> 03:42:18 File "apache_beam/runners/worker/operations.py", line 682, in
> apache_beam.runners.worker.operations.DoOperation.process
> 03:42:18 with self.scoped_process_state:
> 03:42:18 File "apache_beam/runners/worker/operations.py", line 683, in
> apache_beam.runners.worker.operations.DoOperation.process
> 03:42:18 delayed_application = self.dofn_runner.process(o)
> 03:42:18 File "apache_beam/runners/common.py", line 952, in
> apache_beam.runners.common.DoFnRunner.process
> 03:42:18 self._reraise_augmented(exn)
> 03:42:18 File "apache_beam/runners/common.py", line 1028, in
> apache_beam.runners.common.DoFnRunner._reraise_augmented
> 03:42:18 raise_with_traceback(new_exn)
> 03:42:18 File "apache_beam/runners/common.py", line 950, in
> apache_beam.runners.common.DoFnRunner.process
> 03:42:18 return self.do_fn_invoker.invoke_process(windowed_value)
> 03:42:18 File "apache_beam/runners/common.py", line 720, in
> apache_beam.runners.common.PerWindowInvoker.invoke_process
> 03:42:18 self._invoke_process_per_window(
> 03:42:18 File "apache_beam/runners/common.py", line 803, in
> apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
> 03:42:18 self.process_method(*args_for_process),
> 03:42:18 File
> "/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery_file_loads.py",
> line 543, in process
> 03:42:18 self.bq_wrapper.wait_for_bq_job(ref, sleep_duration_sec=10,
> max_retries=0)
> 03:42:18 File
> "/usr/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery_tools.py",
> line 424, in wait_for_bq_job
> 03:42:18 job_reference.jobId, job.status.errorResult))
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
