KevinGG commented on a change in pull request #15490:
URL: https://github.com/apache/beam/pull/15490#discussion_r707624430
##########
File path:
sdks/python/apache_beam/runners/interactive/interactive_environment.py
##########
@@ -235,9 +239,16 @@ def is_in_notebook(self):
@property
def inspector(self):
"""Gets the singleton InteractiveEnvironmentInspector to retrieve
- information consumable by other applications."""
+ information consumable by other applications such as a notebook
+ extension."""
return self._inspector
+ @property
+ def inspector_with_synthetic(self):
+ """Gets the singleton InteractiveEnvironmentInspector with additional
+ synthetic variables generated by Interactive Beam. Internally used."""
+ return self._inspector_with_synthetic
+
Review comment:
The synthetic PCollections are the ones created when caching unbounded
source outputs.
The inspector was originally created to list all the PCollections and
pipelines defined in the notebook and then display them in Jupyter. Since the
synthetic ones are not defined by the user, the should be hidden from them in
that use case.
However, internally, when looking for the cacheables, the synthetic ones are
needed to correctly create WriteCache transforms. Thus, we need a separate
inspector. Or at least the inspector should produce 2 views of PCollections:
one with the synthetics and one without.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
