[
https://issues.apache.org/jira/browse/BEAM-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16902580#comment-16902580
]
Ahmet Altay commented on BEAM-7885:
-----------------------------------
[~nikenano] could you add a log statement to your setup and check to see
whether Dataflow logs have it or not?
The related PR (https://github.com/apache/beam/pull/7994) added a validates
runner test called (DoFnLifecycleTest). I checked for a recent run
(https://builds.apache.org/view/A-D/view/Beam/view/PostCommit/job/beam_PostCommit_Py_VR_Dataflow/4226/consoleFull)
and verified that it worked for batch and streaming. This is the streaming one
(https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-06_17_48_28-1792094113770579496?project=apache-beam-testing)
(For everyone's benefit, there is additional context on the dev@ list
https://lists.apache.org/thread.html/ce2a149e978bde252da8a1cc1c5257465a0e456e56453824594908a3@%3Cdev.beam.apache.org%3E)
> DoFn.setup() don't run for streaming jobs.
> -------------------------------------------
>
> Key: BEAM-7885
> URL: https://issues.apache.org/jira/browse/BEAM-7885
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Affects Versions: 2.14.0
> Environment: Python
> Reporter: niklas Hansson
> Priority: Minor
>
> From version 2.14.0 Python have introduced setup and teardown for DoFn in
> order to "Called to prepare an instance for processing bundles of
> elements.This is a good place to initialize transient in-memory resources,
> such as network connections."
> However when trying to use it for a unbounded job (pubsub source) it seams
> like the DoFn.setup() is never called and the resources are never
> initialize. Instead I get:
> """"
> AttributeError: 'NoneType' object has no attribute 'predict' [while running
> 'transform the data']
> """
> My source code: [https://github.com/NikeNano/DataflowSklearnStreaming]
>
> I am happy to contribute with example code for how to use setup as soon as I
> get it running :)
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
