[
https://issues.apache.org/jira/browse/BEAM-8965?focusedWorklogId=393841&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-393841
]
ASF GitHub Bot logged work on BEAM-8965:
----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Feb/20 23:35
Start Date: 26/Feb/20 23:35
Worklog Time Spent: 10m
Work Description: pabloem commented on issue #10901: [BEAM-8965] Remove
duplicate sideinputs in ConsumerTrackingPipelineVisitor
URL: https://github.com/apache/beam/pull/10901#issuecomment-591699495
I see. Thanks for pointing that out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 393841)
Time Spent: 2h 10m (was: 2h)
> WriteToBigQuery failed in BundleBasedDirectRunner
> -------------------------------------------------
>
> Key: BEAM-8965
> URL: https://issues.apache.org/jira/browse/BEAM-8965
> Project: Beam
> Issue Type: Bug
> Components: io-py-gcp
> Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
> Reporter: Wenbing Bai
> Assignee: Wenbing Bai
> Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> *{{WriteToBigQuery}}* fails in *{{BundleBasedDirectRunner}}* with error
> {{PCollection of size 2 with more than one element accessed as a singleton
> view.}}
> Here is the code
>
> {code:python}
> with Pipeline() as p:
> query_results = (
> p
> | beam.io.Read(beam.io.BigQuerySource(
> query='SELECT ... FROM ...')
> )
> query_results | beam.io.gcp.WriteToBigQuery(
> table=<your_table_name>,
> method=WriteToBigQuery.Method.FILE_LOADS,
> schema={"fields": []}
> )
> {code}
>
> Here is the error
>
> {code:none}
> File "apache_beam/runners/common.py", line 778, in
> apache_beam.runners.common.DoFnRunner.process
> def process(self, windowed_value):
> File "apache_beam/runners/common.py", line 782, in
> apache_beam.runners.common.DoFnRunner.process
> self._reraise_augmented(exn)
> File "apache_beam/runners/common.py", line 849, in
> apache_beam.runners.common.DoFnRunner._reraise_augmented
> raise_with_traceback(new_exn)
> File "apache_beam/runners/common.py", line 780, in
> apache_beam.runners.common.DoFnRunner.process
> return self.do_fn_invoker.invoke_process(windowed_value)
> File "apache_beam/runners/common.py", line 587, in
> apache_beam.runners.common.PerWindowInvoker.invoke_process
> self._invoke_process_per_window(
> File "apache_beam/runners/common.py", line 610, in
> apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
> [si[global_window] for si in self.side_inputs]))
> File
> "/home/wbai/terra/terra_py2/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
> line 65, in __getitem__
> _FilteringIterable(self._iterable, target_window), self._view_options)
> File
> "/home/wbai/terra/terra_py2/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 443, in _from_runtime_iterable
> len(head), str(head[0]), str(head[1])))
> ValueError: PCollection of size 2 with more than one element accessed as a
> singleton view. First two elements encountered are
> "gs://temp-dev/temp/bq_load/3edbf2172dd540edb5c8e9597206b10f",
> "gs://temp-dev/temp/bq_load/3edbf2172dd540edb5c8e9597206b10f". [while running
> 'WriteToBigQuery/BigQueryBatchFileLoads/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)']
> {code}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
