[
https://issues.apache.org/jira/browse/BEAM-7471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankur Goenka updated BEAM-7471:
-------------------------------
Description:
GameStats example is failing with following logs
{code:java}
***************************************************** * Running GameStats with
DirectRunner *****************************************************
INFO:root:Missing pipeline option (runner). Executing pipeline using the
default runner: DirectRunner.
/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py:1137:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported method_to_use =
self._compute_method(p, p.options) INFO:root:Running pipeline with
DirectRunner. INFO:oauth2client.transport:Attempting refresh to obtain initial
access_token INFO:oauth2client.client:Refreshing access_token INFO:root:Created
table google.com:clouddfe.goenka_python_validations.game_stats_teams with
schema <TableSchema fields: [<TableFieldSchema fields: [] mode: u'NULLABLE'
name: u'processing_time' type: u'STRING'>, <TableFieldSchema fields: [] mode:
u'NULLABLE' name: u'total_score' type: u'INTEGER'>, <TableFieldSchema fields:
[] mode: u'NULLABLE' name: u'window_start' type: u'STRING'>, <TableFieldSchema
fields: [] mode: u'NULLABLE' name: u'team' type: u'STRING'>]>. Result: <Table
creationTime: 1559341252301 etag: u'tEQeN0Sr4FvttBRZuXn7zw==' id:
u'google.com:clouddfe:goenka_python_validations.game_stats_teams' kind:
u'bigquery#table' lastModifiedTime: 1559341252336 location: u'US' numBytes: 0
numLongTermBytes: 0 numRows: 0 schema: <TableSchema fields: [<TableFieldSchema
fields: [] mode: u'NULLABLE' name: u'processing_time' type: u'STRING'>,
<TableFieldSchema fields: [] mode: u'NULLABLE' name: u'total_score' type:
u'INTEGER'>, <TableFieldSchema fields: [] mode: u'NULLABLE' name:
u'window_start' type: u'STRING'>, <TableFieldSchema fields: [] mode:
u'NULLABLE' name: u'team' type: u'STRING'>]> selfLink:
u'https://www.googleapis.com/bigquery/v2/projects/google.com:clouddfe/datasets/goenka_python_validations/tables/game_stats_teams'
tableReference: <TableReference datasetId: u'goenka_python_validations'
projectId: u'google.com:clouddfe' tableId: u'game_stats_teams'> type:
u'TABLE'>. INFO:oauth2client.transport:Attempting refresh to obtain initial
access_token INFO:oauth2client.client:Refreshing access_token
ERROR:root:Exception at bundle
<apache_beam.runners.direct.bundle_factory._Bundle object at 0x7fb0f1cfb6d0>,
due to an exception. Traceback (most recent call last): File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 343, in call finish_state) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 380, in attempt_call evaluator.process_element(value) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
line 633, in process_element self.runner.process(element) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 753, in process self._reraise_augmented(exn) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 807, in _reraise_augmented raise_with_traceback(new_exn) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 751, in process return self.do_fn_invoker.invoke_process(windowed_value)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 564, in invoke_process windowed_value, additional_args,
additional_kwargs, output_processor) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 571, in _invoke_process_per_window side_inputs = [si[window] for si in
self.side_inputs] File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
line 65, in __getitem__ _FilteringIterable(self._iterable, target_window),
self._view_options) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 346, in _from_runtime_iterable return options['data'].view_fn(it) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable,
view_options)) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1])))
ValueError: PCollection of size 2 with more than one element accessed as a
singleton view. First two elements encountered are "181.611538462", "9.1".
[while running 'CalculateSpammyUsers/ProcessAndFilter'] ERROR:root:Exception at
bundle <apache_beam.runners.direct.bundle_factory._Bundle object at
0x7fb0f1cfb6d0>, due to an exception. Traceback (most recent call last): File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 343, in call finish_state) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 380, in attempt_call evaluator.process_element(value) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
line 633, in process_element self.runner.process(element) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 753, in process self._reraise_augmented(exn) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 807, in _reraise_augmented raise_with_traceback(new_exn) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 751, in process return self.do_fn_invoker.invoke_process(windowed_value)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 564, in invoke_process windowed_value, additional_args,
additional_kwargs, output_processor) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 571, in _invoke_process_per_window side_inputs = [si[window] for si in
self.side_inputs] File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
line 65, in __getitem__ _FilteringIterable(self._iterable, target_window),
self._view_options) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 346, in _from_runtime_iterable return options['data'].view_fn(it) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable,
view_options)) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1])))
ValueError: PCollection of size 2 with more than one element accessed as a
singleton view. First two elements encountered are "181.611538462", "9.1".
[while running 'CalculateSpammyUsers/ProcessAndFilter'] ERROR:root:Exception at
bundle <apache_beam.runners.direct.bundle_factory._Bundle object at
0x7fb0f1cfb6d0>, due to an exception. Traceback (most recent call last): File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 343, in call finish_state) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 380, in attempt_call evaluator.process_element(value) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
line 633, in process_element self.runner.process(element) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 753, in process self._reraise_augmented(exn) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 807, in _reraise_augmented raise_with_traceback(new_exn) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 751, in process return self.do_fn_invoker.invoke_process(windowed_value)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 564, in invoke_process windowed_value, additional_args,
additional_kwargs, output_processor) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 571, in _invoke_process_per_window side_inputs = [si[window] for si in
self.side_inputs] File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
line 65, in __getitem__ _FilteringIterable(self._iterable, target_window),
self._view_options) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 346, in _from_runtime_iterable return options['data'].view_fn(it) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable,
view_options)) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1])))
ValueError: PCollection of size 2 with more than one element accessed as a
singleton view. First two elements encountered are "181.611538462", "9.1".
[while running 'CalculateSpammyUsers/ProcessAndFilter'] ERROR:root:Exception at
bundle <apache_beam.runners.direct.bundle_factory._Bundle object at
0x7fb0f1cfb6d0>, due to an exception. Traceback (most recent call last): File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 343, in call finish_state) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 380, in attempt_call evaluator.process_element(value) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
line 633, in process_element self.runner.process(element) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 753, in process self._reraise_augmented(exn) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 807, in _reraise_augmented raise_with_traceback(new_exn) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 751, in process return self.do_fn_invoker.invoke_process(windowed_value)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 564, in invoke_process windowed_value, additional_args,
additional_kwargs, output_processor) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 571, in _invoke_process_per_window side_inputs = [si[window] for si in
self.side_inputs] File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
line 65, in __getitem__ _FilteringIterable(self._iterable, target_window),
self._view_options) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 346, in _from_runtime_iterable return options['data'].view_fn(it) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable,
view_options)) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1])))
ValueError: PCollection of size 2 with more than one element accessed as a
singleton view. First two elements encountered are "181.611538462", "9.1".
[while running 'CalculateSpammyUsers/ProcessAndFilter'] ERROR:root:Giving up
after 4 attempts. WARNING:root:A task failed with exception: PCollection of
size 2 with more than one element accessed as a singleton view. First two
elements encountered are "181.611538462", "9.1". [while running
'CalculateSpammyUsers/ProcessAndFilter'] INFO:oauth2client.transport:Attempting
refresh to obtain initial access_token INFO:oauth2client.client:Refreshing
access_token Traceback (most recent call last): File
"/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main "__main__",
fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in
_run_code exec code in run_globals File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/lib/python2.7/site-packages/apache_beam/examples/complete/game/game_stats.py",
line 399, in <module> run() File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/lib/python2.7/site-packages/apache_beam/examples/complete/game/game_stats.py",
line 393, in run }, options.view_as(GoogleCloudOptions).project)) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pipeline.py",
line 426, in __exit__ self.run().wait_until_finish() File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/direct_runner.py",
line 430, in wait_until_finish self._executor.await_completion() File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 400, in await_completion self._executor.await_completion() File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 446, in await_completion raise_(t, v, tb) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 343, in call finish_state) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 380, in attempt_call evaluator.process_element(value) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
line 633, in process_element self.runner.process(element) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 753, in process self._reraise_augmented(exn) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 807, in _reraise_augmented raise_with_traceback(new_exn) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 751, in process return self.do_fn_invoker.invoke_process(windowed_value)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 564, in invoke_process windowed_value, additional_args,
additional_kwargs, output_processor) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 571, in _invoke_process_per_window side_inputs = [si[window] for si in
self.side_inputs] File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
line 65, in __getitem__ _FilteringIterable(self._iterable, target_window),
self._view_options) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 346, in _from_runtime_iterable return options['data'].view_fn(it) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable,
view_options)) File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1])))
ValueError: PCollection of size 2 with more than one element accessed as a
singleton view. First two elements encountered are "181.611538462", "9.1".
[while running 'CalculateSpammyUsers/ProcessAndFilter']
{code}
was:
After generating the archetype for the 2.13.0 RC2, the following quick start
command fails:
{code:java}
mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount
-Dexec.args="--inputFile=pom.xml --output=counts --runner=GearpumpRunner"
-Pgearpump-runner{code}
I also tried:
{code:java}
mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount
-Dexec.args="--inputFile=pom.xml --output=counts --runner=GearpumpRunner"
-Pgearpump-runner{code}
Log:
{code:java}
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/google/home/lcwik/.m2/repository/org/slf4j/slf4j-jdk14/1.7.25/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/google/home/lcwik/.m2/repository/org/slf4j/slf4j-log4j12/1.7.16/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
May 31, 2019 9:38:26 AM akka.event.slf4j.Slf4jLogger$$anonfun$receive$1
applyOrElse
INFO: Slf4jLogger started
May 31, 2019 9:38:26 AM akka.event.slf4j.Slf4jLogger$$anonfun$receive$1
$anonfun$applyOrElse$3
INFO: Starting remoting
May 31, 2019 9:38:26 AM akka.event.slf4j.Slf4jLogger$$anonfun$receive$1
$anonfun$applyOrElse$3
INFO: Remoting started; listening on addresses
:[akka.tcp://[email protected]:38849]
May 31, 2019 9:38:26 AM io.gearpump.metrics.Metrics$ createExtension
INFO: Metrics is enabled..., false
May 31, 2019 9:38:26 AM io.gearpump.cluster.master.MasterProxy <init>
INFO: Master Proxy is started...
[WARNING]
java.lang.NoClassDefFoundError: com/gs/collections/api/block/procedure/Procedure
at io.gearpump.streaming.dsl.plan.WindowOp.chain (OP.scala:273)
at io.gearpump.streaming.dsl.plan.Planner.merge (Planner.scala:86)
at io.gearpump.streaming.dsl.plan.Planner.$anonfun$optimize$2
(Planner.scala:71)
at io.gearpump.streaming.dsl.plan.Planner.$anonfun$optimize$2$adapted
(Planner.scala:70)
at scala.collection.mutable.HashSet.foreach (HashSet.scala:77)
at io.gearpump.streaming.dsl.plan.Planner.$anonfun$optimize$1
(Planner.scala:70)
at io.gearpump.streaming.dsl.plan.Planner.$anonfun$optimize$1$adapted
(Planner.scala:68)
at scala.collection.immutable.List.foreach (List.scala:388)
at io.gearpump.streaming.dsl.plan.Planner.optimize (Planner.scala:68)
at io.gearpump.streaming.dsl.plan.Planner.plan (Planner.scala:48)
at io.gearpump.streaming.dsl.scalaapi.StreamApp.plan (StreamApp.scala:59)
at io.gearpump.streaming.dsl.scalaapi.StreamApp$.streamAppToApplication
(StreamApp.scala:82)
at io.gearpump.streaming.dsl.javaapi.JavaStreamApp.submit
(JavaStreamApp.scala:44)
at org.apache.beam.runners.gearpump.GearpumpRunner.run (GearpumpRunner.java:83)
at org.apache.beam.runners.gearpump.GearpumpRunner.run (GearpumpRunner.java:44)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:313)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:299)
at org.apache.beam.examples.WordCount.runWordCount (WordCount.java:185)
at org.apache.beam.examples.WordCount.main (WordCount.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
at java.lang.Thread.run (Thread.java:748)
Caused by: java.lang.ClassNotFoundException:
com.gs.collections.api.block.procedure.Procedure
at java.net.URLClassLoader.findClass (URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass (ClassLoader.java:424)
at java.lang.ClassLoader.loadClass (ClassLoader.java:357)
at io.gearpump.streaming.dsl.plan.WindowOp.chain (OP.scala:273)
at io.gearpump.streaming.dsl.plan.Planner.merge (Planner.scala:86)
at io.gearpump.streaming.dsl.plan.Planner.$anonfun$optimize$2
(Planner.scala:71)
at io.gearpump.streaming.dsl.plan.Planner.$anonfun$optimize$2$adapted
(Planner.scala:70)
at scala.collection.mutable.HashSet.foreach (HashSet.scala:77)
at io.gearpump.streaming.dsl.plan.Planner.$anonfun$optimize$1
(Planner.scala:70)
at io.gearpump.streaming.dsl.plan.Planner.$anonfun$optimize$1$adapted
(Planner.scala:68)
at scala.collection.immutable.List.foreach (List.scala:388)
at io.gearpump.streaming.dsl.plan.Planner.optimize (Planner.scala:68)
at io.gearpump.streaming.dsl.plan.Planner.plan (Planner.scala:48)
at io.gearpump.streaming.dsl.scalaapi.StreamApp.plan (StreamApp.scala:59)
at io.gearpump.streaming.dsl.scalaapi.StreamApp$.streamAppToApplication
(StreamApp.scala:82)
at io.gearpump.streaming.dsl.javaapi.JavaStreamApp.submit
(JavaStreamApp.scala:44)
at org.apache.beam.runners.gearpump.GearpumpRunner.run (GearpumpRunner.java:83)
at org.apache.beam.runners.gearpump.GearpumpRunner.run (GearpumpRunner.java:44)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:313)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:299)
at org.apache.beam.examples.WordCount.runWordCount (WordCount.java:185)
at org.apache.beam.examples.WordCount.main (WordCount.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
at java.lang.Thread.run (Thread.java:748){code}
The mvn dependency:tree doesn't show gs-collections or eclipse-collections as
part of the dependency tree:
{code:java}
mvn dependency:tree -Pgearpump-runner
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building word-count-beam 0.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ word-count-beam ---
[INFO] org.example:word-count-beam:jar:0.1
[INFO] +- org.apache.beam:beam-sdks-java-core:jar:2.13.0:compile
[INFO] | +- org.apache.beam:beam-model-pipeline:jar:2.13.0:compile
[INFO] | | \- com.google.errorprone:error_prone_annotations:jar:2.1.2:compile
[INFO] | +- org.apache.beam:beam-model-job-management:jar:2.13.0:compile
[INFO] | +- org.apache.beam:beam-vendor-guava-20_0:jar:0.1:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.9.8:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.9.8:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.9.8:compile
[INFO] | +- org.apache.avro:avro:jar:1.8.2:compile
[INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.7:compile
[INFO] | | \- org.apache.commons:commons-compress:jar:1.8.1:compile
[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.4:compile
[INFO] | \- org.tukaani:xz:jar:1.8:compile
[INFO] +-
org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.13.0:compile
[INFO] | +-
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:jar:2.13.0:compile
[INFO] | | +- com.google.cloud.bigdataoss:gcsio:jar:1.9.16:compile
[INFO] | | +-
com.google.apis:google-api-services-cloudresourcemanager:jar:v1-rev20181015-1.27.0:compile
[INFO] | | \-
com.google.apis:google-api-services-storage:jar:v1-rev20181013-1.27.0:compile
[INFO] | +-
org.apache.beam:beam-sdks-java-extensions-protobuf:jar:2.13.0:compile
[INFO] | +- io.grpc:grpc-core:jar:1.17.1:compile
[INFO] | | +- io.grpc:grpc-context:jar:1.17.1:compile
[INFO] | | +- com.google.code.gson:gson:jar:2.7:compile
[INFO] | | +- org.codehaus.mojo:animal-sniffer-annotations:jar:1.17:compile
[INFO] | | +- io.opencensus:opencensus-api:jar:0.17.0:compile
[INFO] | | \- io.opencensus:opencensus-contrib-grpc-metrics:jar:0.17.0:compile
[INFO] | +- com.google.api:gax-grpc:jar:1.38.0:compile
[INFO] | | +- com.google.api:gax:jar:1.38.0:compile
[INFO] | | +- io.grpc:grpc-protobuf:jar:1.17.1:compile
[INFO] | | | \- io.grpc:grpc-protobuf-lite:jar:1.17.1:compile
[INFO] | | +- org.threeten:threetenbp:jar:1.3.3:compile
[INFO] | | +- com.google.api:api-common:jar:1.7.0:compile
[INFO] | | +- io.grpc:grpc-netty-shaded:jar:1.17.1:compile
[INFO] | | \- io.grpc:grpc-alts:jar:1.17.1:compile
[INFO] | | +- org.apache.commons:commons-lang3:jar:3.5:compile
[INFO] | | \- io.grpc:grpc-grpclb:jar:1.17.1:runtime
[INFO] | +- com.google.cloud:google-cloud-core-grpc:jar:1.61.0:compile
[INFO] | | +- com.google.protobuf:protobuf-java-util:jar:3.6.1:compile
[INFO] | | \- javax.annotation:javax.annotation-api:jar:1.2:compile
[INFO] | +- com.google.api.grpc:grpc-google-cloud-pubsub-v1:jar:1.43.0:compile
[INFO] | +- com.google.api.grpc:proto-google-cloud-pubsub-v1:jar:1.43.0:compile
[INFO] | | \- com.google.api.grpc:proto-google-iam-v1:jar:0.12.0:compile
[INFO] | +- com.google.cloud.bigdataoss:util:jar:1.9.16:compile
[INFO] | | +- com.google.api-client:google-api-client-java6:jar:1.27.0:compile
[INFO] | | +-
com.google.api-client:google-api-client-jackson2:jar:1.27.0:compile
[INFO] | | +- com.google.auto.value:auto-value-annotations:jar:1.6.3:compile
[INFO] | | +-
com.google.oauth-client:google-oauth-client-java6:jar:1.27.0:compile
[INFO] | | +- com.google.flogger:google-extensions:jar:0.3.1:compile
[INFO] | | | \- com.google.flogger:flogger:jar:0.3.1:compile
[INFO] | | \- com.google.flogger:flogger-system-backend:jar:0.3.1:runtime
[INFO] | +-
com.google.cloud.datastore:datastore-v1-proto-client:jar:1.6.0:compile
[INFO] | | +-
com.google.http-client:google-http-client-protobuf:jar:1.20.0:compile
[INFO] | | \-
com.google.http-client:google-http-client-jackson:jar:1.20.0:compile
[INFO] | +-
com.google.api.grpc:proto-google-cloud-datastore-v1:jar:0.44.0:compile
[INFO] | +- io.grpc:grpc-auth:jar:1.17.1:compile
[INFO] | +- io.grpc:grpc-netty:jar:1.17.1:compile
[INFO] | | +- io.netty:netty-codec-http2:jar:4.1.30.Final:compile (version
selected from constraint [4.1.30.Final,4.1.30.Final])
[INFO] | | | \- io.netty:netty-codec-http:jar:4.1.30.Final:compile
[INFO] | | \- io.netty:netty-handler-proxy:jar:4.1.30.Final:compile
[INFO] | | \- io.netty:netty-codec-socks:jar:4.1.30.Final:compile
[INFO] | +- io.netty:netty-handler:jar:4.1.30.Final:compile
[INFO] | | +- io.netty:netty-buffer:jar:4.1.30.Final:compile
[INFO] | | | \- io.netty:netty-common:jar:4.1.30.Final:compile
[INFO] | | +- io.netty:netty-transport:jar:4.1.30.Final:compile
[INFO] | | | \- io.netty:netty-resolver:jar:4.1.30.Final:compile
[INFO] | | \- io.netty:netty-codec:jar:4.1.30.Final:compile
[INFO] | +- io.grpc:grpc-stub:jar:1.17.1:compile
[INFO] | +-
com.google.cloud:google-cloud-bigquerystorage:jar:0.79.0-alpha:compile
[INFO] | | \-
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:jar:0.44.0:compile
[INFO] | +-
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:jar:0.44.0:compile
[INFO] | +- com.google.cloud:google-cloud-core:jar:1.61.0:compile
[INFO] | +- com.google.cloud:google-cloud-spanner:jar:1.6.0:compile
[INFO] | | +-
com.google.api.grpc:proto-google-cloud-spanner-v1:jar:1.6.0:compile
[INFO] | | +-
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:jar:1.6.0:compile
[INFO] | | +- com.google.api.grpc:grpc-google-cloud-spanner-v1:jar:1.6.0:compile
[INFO] | | +-
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:jar:1.6.0:compile
[INFO] | | +-
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:jar:1.6.0:compile
[INFO] | | +- com.google.api.grpc:grpc-google-common-protos:jar:1.12.0:compile
[INFO] | | \- io.opencensus:opencensus-contrib-grpc-util:jar:0.17.0:compile
[INFO] | +- com.google.api.grpc:grpc-google-cloud-bigtable-v2:jar:0.44.0:compile
[INFO] | | \-
com.google.api.grpc:proto-google-cloud-bigtable-v2:jar:0.44.0:compile
[INFO] | +- com.google.cloud.bigtable:bigtable-client-core:jar:1.8.0:compile
[INFO] | | +- com.google.cloud:google-cloud-bigtable:jar:0.73.0-alpha:compile
[INFO] | | +-
com.google.cloud:google-cloud-bigtable-admin:jar:0.73.0-alpha:compile
[INFO] | | +-
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:jar:0.38.0:compile
[INFO] | | +-
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:jar:0.38.0:compile
[INFO] | | +- commons-logging:commons-logging:jar:1.2:compile
[INFO] | | +- com.google.cloud:google-cloud-core-http:jar:1.55.0:compile
[INFO] | | | +-
com.google.http-client:google-http-client-appengine:jar:1.27.0:compile
[INFO] | | | +- com.google.api:gax-httpjson:jar:0.52.0:compile
[INFO] | | | \- io.opencensus:opencensus-contrib-http-util:jar:0.15.0:compile
[INFO] | | \- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile
[INFO] | +-
com.google.http-client:google-http-client-jackson2:jar:1.27.0:compile
[INFO] | +- com.google.auth:google-auth-library-credentials:jar:0.12.0:compile
[INFO] | +- com.google.auth:google-auth-library-oauth2-http:jar:0.12.0:compile
[INFO] | +- com.google.protobuf:protobuf-java:jar:3.6.0:compile
[INFO] | +-
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:jar:1.6.0:compile
[INFO] | +- com.google.api.grpc:proto-google-common-protos:jar:1.12.0:compile
[INFO] | +- io.grpc:grpc-all:jar:1.17.1:compile
[INFO] | | +- io.grpc:grpc-okhttp:jar:1.17.1:compile
[INFO] | | | +- com.squareup.okhttp:okhttp:jar:2.5.0:compile
[INFO] | | | \- com.squareup.okio:okio:jar:1.13.0:compile
[INFO] | | +- io.grpc:grpc-protobuf-nano:jar:1.17.1:compile
[INFO] | | | \-
com.google.protobuf.nano:protobuf-javanano:jar:3.0.0-alpha-5:compile
[INFO] | | \- io.grpc:grpc-testing:jar:1.17.1:compile
[INFO] | \- io.netty:netty-tcnative-boringssl-static:jar:2.0.17.Final:compile
[INFO] +- com.google.api-client:google-api-client:jar:1.27.0:compile
[INFO] | \- com.google.oauth-client:google-oauth-client:jar:1.27.0:compile
[INFO] +-
com.google.apis:google-api-services-bigquery:jar:v2-rev20181104-1.27.0:compile
[INFO] +- com.google.http-client:google-http-client:jar:1.27.0:compile
[INFO] | +- com.google.code.findbugs:jsr305:jar:3.0.2:compile
[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.5:compile
[INFO] | | +- org.apache.httpcomponents:httpcore:jar:4.4.9:compile
[INFO] | | \- commons-codec:commons-codec:jar:1.10:compile
[INFO] | \- com.google.j2objc:j2objc-annotations:jar:1.1:compile
[INFO] +-
com.google.apis:google-api-services-pubsub:jar:v1-rev20181105-1.27.0:compile
[INFO] +- joda-time:joda-time:jar:2.10.1:compile
[INFO] +- com.google.guava:guava:jar:20.0:compile
[INFO] +- org.slf4j:slf4j-api:jar:1.7.25:compile
[INFO] +- org.slf4j:slf4j-jdk14:jar:1.7.25:runtime
[INFO] +- org.hamcrest:hamcrest-core:jar:2.1:compile
[INFO] | \- org.hamcrest:hamcrest:jar:2.1:compile
[INFO] +- org.hamcrest:hamcrest-library:jar:2.1:compile
[INFO] +- junit:junit:jar:4.13-beta-1:compile
[INFO] +- org.apache.beam:beam-runners-direct-java:jar:2.13.0:test
[INFO] | +- org.apache.beam:beam-vendor-grpc-1_13_1:jar:0.2:compile
[INFO] | \- args4j:args4j:jar:2.33:test
[INFO] +- org.mockito:mockito-core:jar:1.10.19:test
[INFO] | \- org.objenesis:objenesis:jar:2.1:test
[INFO] \- org.apache.beam:beam-runners-gearpump:jar:2.13.0:runtime
[INFO] +- org.apache.beam:beam-runners-core-java:jar:2.13.0:runtime
[INFO] | +- org.apache.beam:beam-model-fn-execution:jar:2.13.0:runtime
[INFO] | \- org.apache.beam:beam-sdks-java-fn-execution:jar:2.13.0:runtime
[INFO] +- org.apache.beam:beam-runners-core-construction-java:jar:2.13.0:runtime
[INFO] +- io.github.gearpump:gearpump-core_2.12:jar:0.9.0:runtime
[INFO] | +- org.scala-lang:scala-library:jar:2.12.7:runtime
[INFO] | +- org.slf4j:slf4j-log4j12:jar:1.7.16:runtime
[INFO] | | \- log4j:log4j:jar:1.2.17:runtime
[INFO] | +- commons-lang:commons-lang:jar:2.6:runtime
[INFO] | +- io.netty:netty:jar:3.8.0.Final:runtime
[INFO] | +- com.typesafe.akka:akka-remote_2.12:jar:2.5.18:runtime
[INFO] | | +- com.typesafe.akka:akka-stream_2.12:jar:2.5.18:runtime
[INFO] | | | +- org.reactivestreams:reactive-streams:jar:1.0.2:runtime
[INFO] | | | \- com.typesafe:ssl-config-core_2.12:jar:0.3.6:runtime
[INFO] | | | \-
org.scala-lang.modules:scala-parser-combinators_2.12:jar:1.1.1:runtime
[INFO] | | +- com.typesafe.akka:akka-protobuf_2.12:jar:2.5.18:runtime
[INFO] | | +- io.aeron:aeron-driver:jar:1.11.2:runtime
[INFO] | | \- io.aeron:aeron-client:jar:1.11.2:runtime
[INFO] | | \- org.agrona:agrona:jar:0.9.25:runtime
[INFO] | +- com.typesafe.akka:akka-cluster_2.12:jar:2.5.18:runtime
[INFO] | +- com.typesafe.akka:akka-cluster-tools_2.12:jar:2.5.18:runtime
[INFO] | +- com.typesafe.akka:akka-distributed-data_2.12:jar:2.5.18:runtime
[INFO] | | \- org.lmdbjava:lmdbjava:jar:0.6.1:runtime
[INFO] | | +- com.github.jnr:jnr-ffi:jar:2.1.7:runtime
[INFO] | | | +- com.github.jnr:jffi:jar:1.2.16:runtime
[INFO] | | | +- com.github.jnr:jffi:jar:native:1.2.16:runtime
[INFO] | | | +- org.ow2.asm:asm:jar:5.0.3:runtime
[INFO] | | | +- org.ow2.asm:asm-commons:jar:5.0.3:runtime
[INFO] | | | +- org.ow2.asm:asm-analysis:jar:5.0.3:runtime
[INFO] | | | +- org.ow2.asm:asm-tree:jar:5.0.3:runtime
[INFO] | | | +- org.ow2.asm:asm-util:jar:5.0.3:runtime
[INFO] | | | \- com.github.jnr:jnr-x86asm:jar:1.0.2:runtime
[INFO] | | \- com.github.jnr:jnr-constants:jar:0.9.9:runtime
[INFO] | +- com.typesafe.akka:akka-actor_2.12:jar:2.5.18:runtime
[INFO] | | +- com.typesafe:config:jar:1.3.3:runtime
[INFO] | | \- org.scala-lang.modules:scala-java8-compat_2.12:jar:0.8.0:runtime
[INFO] | +- com.typesafe.akka:akka-agent_2.12:jar:2.5.18:runtime
[INFO] | | \- org.scala-stm:scala-stm_2.12:jar:0.8:runtime
[INFO] | +- com.typesafe.akka:akka-slf4j_2.12:jar:2.5.18:runtime
[INFO] | +- com.typesafe.akka:akka-http_2.12:jar:10.1.3:runtime
[INFO] | | \- com.typesafe.akka:akka-http-core_2.12:jar:10.1.3:runtime
[INFO] | | \- com.typesafe.akka:akka-parsing_2.12:jar:10.1.3:runtime
[INFO] | +- com.typesafe.akka:akka-http-spray-json_2.12:jar:10.1.3:runtime
[INFO] | | \- io.spray:spray-json_2.12:jar:1.3.4:runtime
[INFO] | \- org.scala-lang:scala-reflect:jar:2.12.7:runtime
[INFO] \- io.github.gearpump:gearpump-streaming_2.12:jar:0.9.0:runtime
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.321 s
[INFO] Finished at: 2019-05-31T09:43:53-07:00
[INFO] Final Memory: 20M/238M
[INFO] ------------------------------------------------------------------------
{code}
> DirectRunner GameStats fails, ValueError: PCollection of size 2 with more
> than one element accessed as a singleton view. First two elements encountered
> are "181.611538462", "9.1". [while running
> 'CalculateSpammyUsers/ProcessAndFilter']
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-7471
> URL: https://issues.apache.org/jira/browse/BEAM-7471
> Project: Beam
> Issue Type: Improvement
> Components: runner-gearpump
> Affects Versions: 2.13.0
> Reporter: Ankur Goenka
> Assignee: Ankur Goenka
> Priority: Blocker
>
> GameStats example is failing with following logs
>
> {code:java}
> ***************************************************** * Running GameStats
> with DirectRunner *****************************************************
> INFO:root:Missing pipeline option (runner). Executing pipeline using the
> default runner: DirectRunner.
> /usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py:1137:
> BeamDeprecationWarning: options is deprecated since First stable release.
> References to <pipeline>.options will not be supported method_to_use =
> self._compute_method(p, p.options) INFO:root:Running pipeline with
> DirectRunner. INFO:oauth2client.transport:Attempting refresh to obtain
> initial access_token INFO:oauth2client.client:Refreshing access_token
> INFO:root:Created table
> google.com:clouddfe.goenka_python_validations.game_stats_teams with schema
> <TableSchema fields: [<TableFieldSchema fields: [] mode: u'NULLABLE' name:
> u'processing_time' type: u'STRING'>, <TableFieldSchema fields: [] mode:
> u'NULLABLE' name: u'total_score' type: u'INTEGER'>, <TableFieldSchema fields:
> [] mode: u'NULLABLE' name: u'window_start' type: u'STRING'>,
> <TableFieldSchema fields: [] mode: u'NULLABLE' name: u'team' type:
> u'STRING'>]>. Result: <Table creationTime: 1559341252301 etag:
> u'tEQeN0Sr4FvttBRZuXn7zw==' id:
> u'google.com:clouddfe:goenka_python_validations.game_stats_teams' kind:
> u'bigquery#table' lastModifiedTime: 1559341252336 location: u'US' numBytes: 0
> numLongTermBytes: 0 numRows: 0 schema: <TableSchema fields:
> [<TableFieldSchema fields: [] mode: u'NULLABLE' name: u'processing_time'
> type: u'STRING'>, <TableFieldSchema fields: [] mode: u'NULLABLE' name:
> u'total_score' type: u'INTEGER'>, <TableFieldSchema fields: [] mode:
> u'NULLABLE' name: u'window_start' type: u'STRING'>, <TableFieldSchema fields:
> [] mode: u'NULLABLE' name: u'team' type: u'STRING'>]> selfLink:
> u'https://www.googleapis.com/bigquery/v2/projects/google.com:clouddfe/datasets/goenka_python_validations/tables/game_stats_teams'
> tableReference: <TableReference datasetId: u'goenka_python_validations'
> projectId: u'google.com:clouddfe' tableId: u'game_stats_teams'> type:
> u'TABLE'>. INFO:oauth2client.transport:Attempting refresh to obtain initial
> access_token INFO:oauth2client.client:Refreshing access_token
> ERROR:root:Exception at bundle
> <apache_beam.runners.direct.bundle_factory._Bundle object at 0x7fb0f1cfb6d0>,
> due to an exception. Traceback (most recent call last): File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
> line 343, in call finish_state) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
> line 380, in attempt_call evaluator.process_element(value) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
> line 633, in process_element self.runner.process(element) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 753, in process self._reraise_augmented(exn) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 807, in _reraise_augmented raise_with_traceback(new_exn) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 751, in process return
> self.do_fn_invoker.invoke_process(windowed_value) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 564, in invoke_process windowed_value, additional_args,
> additional_kwargs, output_processor) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 571, in _invoke_process_per_window side_inputs = [si[window] for si in
> self.side_inputs] File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
> line 65, in __getitem__ _FilteringIterable(self._iterable, target_window),
> self._view_options) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 346, in _from_runtime_iterable return options['data'].view_fn(it) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable,
> view_options)) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1])))
> ValueError: PCollection of size 2 with more than one element accessed as a
> singleton view. First two elements encountered are "181.611538462", "9.1".
> [while running 'CalculateSpammyUsers/ProcessAndFilter'] ERROR:root:Exception
> at bundle <apache_beam.runners.direct.bundle_factory._Bundle object at
> 0x7fb0f1cfb6d0>, due to an exception. Traceback (most recent call last): File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
> line 343, in call finish_state) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
> line 380, in attempt_call evaluator.process_element(value) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
> line 633, in process_element self.runner.process(element) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 753, in process self._reraise_augmented(exn) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 807, in _reraise_augmented raise_with_traceback(new_exn) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 751, in process return
> self.do_fn_invoker.invoke_process(windowed_value) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 564, in invoke_process windowed_value, additional_args,
> additional_kwargs, output_processor) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 571, in _invoke_process_per_window side_inputs = [si[window] for si in
> self.side_inputs] File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
> line 65, in __getitem__ _FilteringIterable(self._iterable, target_window),
> self._view_options) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 346, in _from_runtime_iterable return options['data'].view_fn(it) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable,
> view_options)) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1])))
> ValueError: PCollection of size 2 with more than one element accessed as a
> singleton view. First two elements encountered are "181.611538462", "9.1".
> [while running 'CalculateSpammyUsers/ProcessAndFilter'] ERROR:root:Exception
> at bundle <apache_beam.runners.direct.bundle_factory._Bundle object at
> 0x7fb0f1cfb6d0>, due to an exception. Traceback (most recent call last): File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
> line 343, in call finish_state) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
> line 380, in attempt_call evaluator.process_element(value) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
> line 633, in process_element self.runner.process(element) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 753, in process self._reraise_augmented(exn) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 807, in _reraise_augmented raise_with_traceback(new_exn) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 751, in process return
> self.do_fn_invoker.invoke_process(windowed_value) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 564, in invoke_process windowed_value, additional_args,
> additional_kwargs, output_processor) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 571, in _invoke_process_per_window side_inputs = [si[window] for si in
> self.side_inputs] File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
> line 65, in __getitem__ _FilteringIterable(self._iterable, target_window),
> self._view_options) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 346, in _from_runtime_iterable return options['data'].view_fn(it) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable,
> view_options)) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1])))
> ValueError: PCollection of size 2 with more than one element accessed as a
> singleton view. First two elements encountered are "181.611538462", "9.1".
> [while running 'CalculateSpammyUsers/ProcessAndFilter'] ERROR:root:Exception
> at bundle <apache_beam.runners.direct.bundle_factory._Bundle object at
> 0x7fb0f1cfb6d0>, due to an exception. Traceback (most recent call last): File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
> line 343, in call finish_state) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
> line 380, in attempt_call evaluator.process_element(value) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
> line 633, in process_element self.runner.process(element) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 753, in process self._reraise_augmented(exn) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 807, in _reraise_augmented raise_with_traceback(new_exn) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 751, in process return
> self.do_fn_invoker.invoke_process(windowed_value) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 564, in invoke_process windowed_value, additional_args,
> additional_kwargs, output_processor) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 571, in _invoke_process_per_window side_inputs = [si[window] for si in
> self.side_inputs] File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
> line 65, in __getitem__ _FilteringIterable(self._iterable, target_window),
> self._view_options) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 346, in _from_runtime_iterable return options['data'].view_fn(it) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable,
> view_options)) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1])))
> ValueError: PCollection of size 2 with more than one element accessed as a
> singleton view. First two elements encountered are "181.611538462", "9.1".
> [while running 'CalculateSpammyUsers/ProcessAndFilter'] ERROR:root:Giving up
> after 4 attempts. WARNING:root:A task failed with exception: PCollection of
> size 2 with more than one element accessed as a singleton view. First two
> elements encountered are "181.611538462", "9.1". [while running
> 'CalculateSpammyUsers/ProcessAndFilter']
> INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
> INFO:oauth2client.client:Refreshing access_token Traceback (most recent call
> last): File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
> "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line
> 72, in _run_code exec code in run_globals File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/lib/python2.7/site-packages/apache_beam/examples/complete/game/game_stats.py",
> line 399, in <module> run() File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/lib/python2.7/site-packages/apache_beam/examples/complete/game/game_stats.py",
> line 393, in run }, options.view_as(GoogleCloudOptions).project)) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pipeline.py",
> line 426, in __exit__ self.run().wait_until_finish() File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/direct_runner.py",
> line 430, in wait_until_finish self._executor.await_completion() File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
> line 400, in await_completion self._executor.await_completion() File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
> line 446, in await_completion raise_(t, v, tb) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
> line 343, in call finish_state) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
> line 380, in attempt_call evaluator.process_element(value) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
> line 633, in process_element self.runner.process(element) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 753, in process self._reraise_augmented(exn) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 807, in _reraise_augmented raise_with_traceback(new_exn) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 751, in process return
> self.do_fn_invoker.invoke_process(windowed_value) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 564, in invoke_process windowed_value, additional_args,
> additional_kwargs, output_processor) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
> line 571, in _invoke_process_per_window side_inputs = [si[window] for si in
> self.side_inputs] File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
> line 65, in __getitem__ _FilteringIterable(self._iterable, target_window),
> self._view_options) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 346, in _from_runtime_iterable return options['data'].view_fn(it) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 323, in <lambda> lambda iterable: from_runtime_iterable(iterable,
> view_options)) File
> "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
> line 433, in _from_runtime_iterable len(head), str(head[0]), str(head[1])))
> ValueError: PCollection of size 2 with more than one element accessed as a
> singleton view. First two elements encountered are "181.611538462", "9.1".
> [while running 'CalculateSpammyUsers/ProcessAndFilter']
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
