[ 
https://issues.apache.org/jira/browse/BEAM-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Valentyn Tymofieiev updated BEAM-3824:
--------------------------------------
    Affects Version/s: 2.4.0

> BQ sink fails on Direct Runner for 2.4.0 RC2
> --------------------------------------------
>
>                 Key: BEAM-3824
>                 URL: https://issues.apache.org/jira/browse/BEAM-3824
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-direct
>    Affects Versions: 2.4.0
>            Reporter: Valentyn Tymofieiev
>            Assignee: David Cavazos
>            Priority: Blocker
>
> python -m apache_beam.examples.complete.game.hourly_team_score 
> --project=$PROJECT --dataset=beam_release_2_4_0 
> --input=gs://$BUCKET/mobile/first_5000_gaming_data.csv
> The pipeline fails with:
> INFO:root:finish <DoOperation WriteTeamScoreSums/WriteToBigQuery 
> output_tags=['out'], 
> receivers=[ConsumerSet[WriteTeamScoreSums/WriteToBigQuery.out0, 
> coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=0]]> 
> Traceback (most recent call last):
>  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
>  "__main__", fname, loader, pkg_name) 
>  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
>  exec code in run_globals 
>  File 
> "/tmp/release_testing/r2.4.0_env/lib/python2.7/site-packages/apache_beam/examples/complete/game/hourly_team_score.py",
>  line 276, in <
> module> 
>  run() 
>  File 
> "/tmp/release_testing/r2.4.0_env/lib/python2.7/site-packages/apache_beam/examples/complete/game/hourly_team_score.py",
>  line 270, in r
> un 
>  write_disposition=beam.io.BigQueryDisposition.WRITE_APPEND)) 
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/pipeline.py",
>  line 389, in __exit__
>  self.run().wait_until_finish() 
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/pipeline.py",
>  line 369, in run
>  self.to_runner_api(), self.runner, self._options).run(False) 
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/pipeline.py",
>  line 382, in run
>  return self.runner.run_pipeline(self) 
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/direct/direct_runner.py",
>  line 129, in run_pip
> eline 
>  return runner.run_pipeline(pipeline)
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/portability/fn_api_runner.py",
>  line 215, in ru
> n_pipeline 
>  return self.run_via_runner_api(pipeline.to_runner_api())
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/portability/fn_api_runner.py",
>  line 218, in ru
> n_via_runner_api 
>  return self.run_stages(*self.create_stages(pipeline_proto))
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/portability/fn_api_runner.py",
>  line 837, in ru
> n_stages 
>  pcoll_buffers, safe_coders).process_bundle.metrics
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/portability/fn_api_runner.py",
>  line 938, in ru
> n_stage 
>  self._progress_frequency).process_bundle(data_input, data_output)
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/portability/fn_api_runner.py",
>  line 1110, in p
> rocess_bundle 
>  result_future = self._controller.control_handler.push(process_bundle)
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/portability/fn_api_runner.py",
>  line 1003, in p
> ush 
>  response = self.worker.do_instruction(request)
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
>  line 185, in do_instruc
> tion 
>  request.instruction_id) 
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
>  line 202, in process_bu
> ndle 
>  processor.process_bundle(instruction_id)
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
>  line 298, in proc
> ess_bundle 
>  op.finish() 
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/worker/operations.py",
>  line 389, in finish
>  self.dofn_runner.finish()
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
>  line 517, in finish
>  self._invoke_bundle_method(self.do_fn_invoker.invoke_finish_bundle)
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
>  line 508, in _invoke_bundle_method
>  self._reraise_augmented(exn) 
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
>  line 537, in _reraise_augmented
>  six.raise_from(new_exn, original_traceback)
>  File 
> "/tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/six.py", 
> line 737, in raise_from
>  raise value 
> RuntimeError: Could not successfully insert rows to BigQuery table 
> [google.com:clouddfe:beam_release_2_4_0.leader_board]. Errors: [<InsertEr
> rorsValueListEntry
>  errors: [<ErrorProto 
>  debugInfo: u'' 
>  location: u'processing_time' 
>  message: u'no such field.'
>  reason: u'invalid'>]
>  index: 0>, <InsertErrorsValueListEntry
>  errors: [<ErrorProto
>  debugInfo: u''
>  location: u'processing_time'
>  message: u'no such field.'
>  reason: u'invalid'>]
>  index: 1>, <InsertErrorsValueListEntry
>  errors: [<ErrorProto
>  debugInfo: u''
>  location: u'processing_time'
>  message: u'no such field.'
>  reason: u'invalid'>]
>  index: 2>, <InsertErrorsValueListEntry
>  errors: [<ErrorProto
>  debugInfo: u''
>  location: u'processing_time'
>  message: u'no such field.'
>  reason: u'invalid'>]
>  index: 3>, <InsertErrorsValueListEntry
>  errors: [<ErrorProto
>  debugInfo: u''
>  location: u'processing_time'
>  message: u'no such field.'
>  reason: u'invalid'>]
> ...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to