kennknowles opened a new issue, #19540: URL: https://github.com/apache/beam/issues/19540
GameStats example is failing with following logs ``` ***************************************************** * Running GameStats with DirectRunner ***************************************************** INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. /usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported method_to_use = self._compute_method(p, p.options) INFO:root:Running pipeline with DirectRunner. INFO:oauth2client.transport:Attempting refresh to obtain initial access_token INFO:oauth2client.client:Refreshing access_token INFO:root:Created table google.com:clouddfe.goenka_python_validations.game_stats_teams with schema <TableSchema fields: [<TableFieldSchema fields: [] mode: u'NULLABLE' name: u'processing_time' type: u'STRING'>, <TableFieldSchema fields: [] mode: u'NULLABLE' name: u'total_score' type: u'INTEGER'>, <TableFieldSchema fields: [] mode: u'NULLABLE' name: u'window_start' type: u'STRING'>, <TableFieldSchema fields: [] mode: u'NULLABLE' name: u'team' type: u'STRING'>]>. Result: <Table creationTime: 1559341252301 etag: u'tEQeN0Sr4FvttBRZuXn7zw==' id: u'google.com:clouddfe:goenka_python_validations.game_stats_teams' kind: u'bigquery#table' lastModifiedTime: 1559341252336 location: u'US' numBytes: 0 numLongTermBytes: 0 numRows: 0 schema: <TableSchema fields: [<TableFieldSchema fields: [] mode: u'NULLABLE' name: u'processing_time' type: u'STRING'>, <TableFieldSchema fields: [] mode: u'NULLABLE' name: u'total_score' type: u'INTEGER'>, <TableFieldSchema fields: [] mode: u'NULLABLE' name: u'window_start' type: u'STRING'>, <TableFieldSchema fields: [] mode: u'NULLABLE' name: u'team' type: u'STRING'>]> selfLink: u'https://www.googleapis.com/bigquery/v2/projects/google.com:clouddfe/datasets/goenka_python_validations/tables/game_stats_teams' tableReference: <TableReference datasetId: u'goenka_python_validations' projectId: u'google.com:clouddfe' tableId: u'game_stats_teams'> type: u'TABLE'>. INFO:oauth2client.transport:Attempting refresh to obtain initial access_token INFO:oauth2client.client:Refreshing access_token ERROR:root:Exception at bundle <apache_beam.runners.direct.bundle_factory._Bundle object at 0x7fb0f1cfb6d0>, due to an exception. Traceback (most recent call last): File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 343, in call finish_state) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 380, in attempt_call evaluator.process_element(value) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 633, in process_element self.runner.process(element) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 753, in process self._reraise_augmented(exn) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 807, in _reraise_augmented raise_with_traceback(new_exn) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 751, in process return self.do_fn_invoker.invoke_process(windowed_value) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 564, in invoke_process windowed_value, additional_args, additional_kwargs, output_processor) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 571, in _invoke_process_per_window side_inputs = [si[window] for si in self.side_inputs] File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py", line 65, in __getitem__ _FilteringIterable(self._iterable, target_window), self._view_options) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 346, in _from_runtime_iterable return options['data'].view_fn(it) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable, view_options)) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1]))) ValueError: PCollection of size 2 with more than one element accessed as a singleton view. First two elements encountered are "181.611538462", "9.1". [while running 'CalculateSpammyUsers/ProcessAndFilter'] ERROR:root:Exception at bundle <apache_beam.runners.direct.bundle_factory._Bundle object at 0x7fb0f1cfb6d0>, due to an exception. Traceback (most recent call last): File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 343, in call finish_state) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 380, in attempt_call evaluator.process_element(value) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 633, in process_element self.runner.process(element) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 753, in process self._reraise_augmented(exn) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 807, in _reraise_augmented raise_with_traceback(new_exn) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 751, in process return self.do_fn_invoker.invoke_process(windowed_value) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 564, in invoke_process windowed_value, additional_args, additional_kwargs, output_processor) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 571, in _invoke_process_per_window side_inputs = [si[window] for si in self.side_inputs] File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py", line 65, in __getitem__ _FilteringIterable(self._iterable, target_window), self._view_options) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 346, in _from_runtime_iterable return options['data'].view_fn(it) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable, view_options)) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1]))) ValueError: PCollection of size 2 with more than one element accessed as a singleton view. First two elements encountered are "181.611538462", "9.1". [while running 'CalculateSpammyUsers/ProcessAndFilter'] ERROR:root:Exception at bundle <apache_beam.runners.direct.bundle_factory._Bundle object at 0x7fb0f1cfb6d0>, due to an exception. Traceback (most recent call last): File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 343, in call finish_state) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 380, in attempt_call evaluator.process_element(value) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 633, in process_element self.runner.process(element) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 753, in process self._reraise_augmented(exn) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 807, in _reraise_augmented raise_with_traceback(new_exn) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 751, in process return self.do_fn_invoker.invoke_process(windowed_value) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 564, in invoke_process windowed_value, additional_args, additional_kwargs, output_processor) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 571, in _invoke_process_per_window side_inputs = [si[window] for si in self.side_inputs] File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py", line 65, in __getitem__ _FilteringIterable(self._iterable, target_window), self._view_options) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 346, in _from_runtime_iterable return options['data'].view_fn(it) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable, view_options)) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1]))) ValueError: PCollection of size 2 with more than one element accessed as a singleton view. First two elements encountered are "181.611538462", "9.1". [while running 'CalculateSpammyUsers/ProcessAndFilter'] ERROR:root:Exception at bundle <apache_beam.runners.direct.bundle_factory._Bundle object at 0x7fb0f1cfb6d0>, due to an exception. Traceback (most recent call last): File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 343, in call finish_state) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 380, in attempt_call evaluator.process_element(value) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 633, in process_element self.runner.process(element) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 753, in process self._reraise_augmented(exn) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 807, in _reraise_augmented raise_with_traceback(new_exn) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 751, in process return self.do_fn_invoker.invoke_process(windowed_value) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 564, in invoke_process windowed_value, additional_args, additional_kwargs, output_processor) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 571, in _invoke_process_per_window side_inputs = [si[window] for si in self.side_inputs] File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py", line 65, in __getitem__ _FilteringIterable(self._iterable, target_window), self._view_options) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 346, in _from_runtime_iterable return options['data'].view_fn(it) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable, view_options)) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1]))) ValueError: PCollection of size 2 with more than one element accessed as a singleton view. First two elements encountered are "181.611538462", "9.1". [while running 'CalculateSpammyUsers/ProcessAndFilter'] ERROR:root:Giving up after 4 attempts. WARNING:root:A task failed with exception: PCollection of size 2 with more than one element accessed as a singleton view. First two elements encountered are "181.611538462", "9.1". [while running 'CalculateSpammyUsers/ProcessAndFilter'] INFO:oauth2client.transport:Attempting refresh to obtain initial access_token INFO:oauth2client.client:Refreshing access_token Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/lib/python2.7/site-packages/apache_beam/examples/complete/game/game_stats.py", line 399, in <module> run() File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/lib/python2.7/site-packages/apache_beam/examples/complete/game/game_stats.py", line 393, in run }, options.view_as(GoogleCloudOptions).project)) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 426, in __exit__ self.run().wait_until_finish() File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/direct_runner.py", line 430, in wait_until_finish self._executor.await_completion() File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 400, in await_completion self._executor.await_completion() File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 446, in await_completion raise_(t, v, tb) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 343, in call finish_state) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 380, in attempt_call evaluator.process_element(value) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 633, in process_element self.runner.process(element) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 753, in process self._reraise_augmented(exn) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 807, in _reraise_augmented raise_with_traceback(new_exn) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 751, in process return self.do_fn_invoker.invoke_process(windowed_value) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 564, in invoke_process windowed_value, additional_args, additional_kwargs, output_processor) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py", line 571, in _invoke_process_per_window side_inputs = [si[window] for si in self.side_inputs] File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py", line 65, in __getitem__ _FilteringIterable(self._iterable, target_window), self._view_options) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 346, in _from_runtime_iterable return options['data'].view_fn(it) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable, view_options)) File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py", line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1]))) ValueError: PCollection of size 2 with more than one element accessed as a singleton view. First two elements encountered are "181.611538462", "9.1". [while running 'CalculateSpammyUsers/ProcessAndFilter'] ``` Imported from Jira [BEAM-7471](https://issues.apache.org/jira/browse/BEAM-7471). Original Jira may contain additional context. Reported by: angoenka. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
