shunping commented on code in PR #39161:
URL: https://github.com/apache/beam/pull/39161#discussion_r3651397129
##########
sdks/python/apache_beam/runners/interactive/recording_manager.py:
##########
@@ -710,19 +725,16 @@ def task():
return async_result
def _get_pipeline_graph(self):
- """Lazily initializes and returns the PipelineGraph."""
- if self._pipeline_graph is None:
- try:
- # Try to create the graph.
- self._pipeline_graph = PipelineGraph(self.user_pipeline)
- except (ImportError, NameError, AttributeError):
- # If pydot is missing, PipelineGraph() might crash.
- _LOGGER.warning(
- "Could not create PipelineGraph (pydot missing?). " \
- "Async features disabled."
- )
- self._pipeline_graph = None
- return self._pipeline_graph
+ """Initializes and returns the PipelineGraph."""
+ try:
+ # Try to create the graph.
+ return PipelineGraph(self.user_pipeline)
Review Comment:
I understand we want to make sure the graph is up-to-date, but I think
building pipeline graphs could be slow with pydot, and that's why we had a
cache here previously.
Have you tested the new implementation on different graph sizes? I am
worried about a performance regression here without a cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
