[
https://issues.apache.org/jira/browse/BEAM-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16904190#comment-16904190
]
Ahmet Altay commented on BEAM-7885:
-----------------------------------
> I got it working on Dataflow, but actually still have problems to run it with
> the direct runner which i used initially Is it expected to work on the direct
> runner?
It should work on fnapi based direct runner. There is another (older) direct
runner, that was not updated for this. For streaming we are using the older
direct runner. At some point we need to unify those. (Feel free to rename the
title here to specific that this is direct runenr related.)
> However it seams like the setup function where run multiple times during
> start up of the pipeline but then never again. Is this expected behaviour?
Possible. setup will run once dofn is loaded. As long as there is a reference
to it there is no need to run setup again. (Setup is actually good for this,
for doing heavy tasks only a few times.)
> DoFn.setup() don't run for streaming jobs.
> -------------------------------------------
>
> Key: BEAM-7885
> URL: https://issues.apache.org/jira/browse/BEAM-7885
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Affects Versions: 2.14.0
> Environment: Python
> Reporter: niklas Hansson
> Priority: Minor
>
> From version 2.14.0 Python have introduced setup and teardown for DoFn in
> order to "Called to prepare an instance for processing bundles of
> elements.This is a good place to initialize transient in-memory resources,
> such as network connections."
> However when trying to use it for a unbounded job (pubsub source) it seams
> like the DoFn.setup() is never called and the resources are never
> initialize. Instead I get:
> """"
> AttributeError: 'NoneType' object has no attribute 'predict' [while running
> 'transform the data']
> """
> My source code: [https://github.com/NikeNano/DataflowSklearnStreaming]
>
> I am happy to contribute with example code for how to use setup as soon as I
> get it running :)
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
