[
https://issues.apache.org/jira/browse/BEAM-3889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405789#comment-16405789
]
Valentyn Tymofieiev commented on BEAM-3889:
-------------------------------------------
Correction: I was able to run the injector to populate a pubsub topic after
fixing credential file, but was not able to produce an output in BigQuery with
game_stats pipeline running on DirectRunner.
> Revise python streaming mobile gaming examples.
> -----------------------------------------------
>
> Key: BEAM-3889
> URL: https://issues.apache.org/jira/browse/BEAM-3889
> Project: Beam
> Issue Type: Bug
> Components: examples-python
> Reporter: Valentyn Tymofieiev
> Assignee: David Cavazos
> Priority: Major
>
> I spent some time trying to run streaming examples [1] on Direct runner and
> ran into the following issues:
> * Examples use deprecated code as per warnings
> ** DeprecationWarning: options is deprecated since First stable release..
> References to <pipeline>.options will not be supported
> project = pipeline.options.view_as(GoogleCloudOptions).project
> /tmp/release_testing/r2.4.0_env/local/lib/python2.7/site-packages/apache_beam/runners/direct/direct_runner.py:337:
> DeprecationWarning: options is deprecated since First stable release..
> References to <pipeline>.options will not be supported
> pipeline.replace_all(_get_transform_overrides(pipeline.options))
> * Path to Injector has a typo (java8 instead of java)
> * I was not able to use the injector to produce an output on a pubsub topic,
> but perhaps I missed some instructions. Injector would exit after:
> ** Warning! You are not using service account credentials to authenticate.
> You need to use service account credentials for this example,
> since user-level credentials do not have enough pubsub quota,
> and so you will run out of PubSub quota very quickly.
> ** Note that I set an environment variable GOOGLE_APPLICATION_CREDENTIALS
> and also tried passing
> -DGOOGLE_APPLICATION_CREDENTIALS=/path/to/credential.json.
> * Examples say they are not applicable for Dataflow runner, is it still the
> case?
> * Running injector requires mvn, which is being replaced with gradle and
> also requires familiarity with java toolchain, even though the example is for
> Python SDK.
>
> [1]
> https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/complete/game/game_stats.py.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
