[
https://issues.apache.org/jira/browse/BEAM-3884?focusedWorklogId=114377&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-114377
]
ASF GitHub Bot logged work on BEAM-3884:
----------------------------------------
Author: ASF GitHub Bot
Created on: 21/Jun/18 16:20
Start Date: 21/Jun/18 16:20
Worklog Time Spent: 10m
Work Description: lukecwik commented on issue #5685: [BEAM-3884] Use
Impulse primitive for Read.
URL: https://github.com/apache/beam/pull/5685#issuecomment-399161937
I believe this is causing the python postcommit to fail with:
======================================================================
ERROR: test_wordcount_fnapi_it
(apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/examples/wordcount_it_test.py",
line 79, in test_wordcount_fnapi_it
on_success_matcher=PipelineStateMatcher()))
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/examples/wordcount_fnapi.py",
line 128, in run
result = p.run()
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
line 395, in run
self.to_runner_api(), self.runner, self._options).run(False)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
line 408, in run
return self.runner.run_pipeline(self)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
line 48, in run_pipeline
self.result = super(TestDataflowRunner, self).run_pipeline(pipeline)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
line 350, in run_pipeline
super(DataflowRunner, self).run_pipeline(pipeline)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/runners/runner.py",
line 170, in run_pipeline
pipeline.visit(RunVisitor(self))
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
line 436, in visit
self._root_transform().visit(visitor, self, visited)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
line 791, in visit
part.visit(visitor, pipeline, visited)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
line 791, in visit
part.visit(visitor, pipeline, visited)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
line 791, in visit
part.visit(visitor, pipeline, visited)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
line 794, in visit
visitor.visit_transform(self)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/runners/runner.py",
line 165, in visit_transform
self.runner.run_transform(transform_node)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/runners/runner.py",
line 208, in run_transform
return m(transform_node)
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
line 586, in run_ParDo
input_step = self._cache.get_pvalue(transform_node.inputs[0])
File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/runners/runner.py",
line 286, in get_pvalue
return self._cache[self.key(pvalue)]
KeyError: (u'read/Read/Impulse', None)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 114377)
Time Spent: 2h (was: 1h 50m)
> Python SDK supports Impulse as a primitive transform
> ----------------------------------------------------
>
> Key: BEAM-3884
> URL: https://issues.apache.org/jira/browse/BEAM-3884
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Ben Sidhom
> Assignee: Robert Bradshaw
> Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Portable runners require Impulse to be the only root nodes of pipelines. The
> Python SDK should provide this for pipeline construction.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
