IMO returning the input PCollection in a PTransform should be a valid
albeit trivial use case. I have put a suggested fix for supporting
these kinds of transforms in the interactive runner here:
https://github.com/apache/beam/pull/9865 . I'm also new to beam dev so
if there's something I'm missing
+Sam Rohde is working on streaming support for
Interactive Beam.
The high level idea is to capture a bounded segment of the unbounded data
source for replayablity and determinism, and to use TestStream, which has
the ability to control the clock of the pipeline, to replay the data, so
streaming
Thanks, +1 to adding support for streaming on Interactive Beam (+David Yan
)
On Thu, Oct 24, 2019 at 1:45 PM Hai Lu wrote:
> Hi Robert,
>
> We're trying out iBeam at LinkedIn for Python. As Igor mentioned, there
> seems to be some inconsistency in the behavior of interactive beam. We can
>
Hi Robert,
We're trying out iBeam at LinkedIn for Python. As Igor mentioned, there
seems to be some inconsistency in the behavior of interactive beam. We can
suggest some fixes from our end but we would need some support from the
community.
Also, is there a plan to support iBeam for streaming
Thanks for trying this out. Yes, this is definitely something that
should be supported (and tested).
On Mon, Oct 21, 2019 at 3:40 PM Igor Durovic wrote:
>
> Hi everyone,
>
> The interactive beam example using the DirectRunner fails after execution of
> the last cell. The recursion limit is
Hi everyone,
The interactive beam example using the DirectRunner fails after execution
of the last cell. The recursion limit is exceeded during the calculation of
the cache label because of a circular reference in the PipelineInfo object.
The constructor
