[
https://issues.apache.org/jira/browse/BEAM-8335?focusedWorklogId=398028&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-398028
]
ASF GitHub Bot logged work on BEAM-8335:
----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Mar/20 00:42
Start Date: 05/Mar/20 00:42
Worklog Time Spent: 10m
Work Description: KevinGG commented on issue #11050: [BEAM-8335]
Implemented Capture Size limitation
URL: https://github.com/apache/beam/pull/11050#issuecomment-594967107
R: @rohdesamuel
R: @aaltay
Data captured from sources are stored in cache just like intermediate
PCollections that are assigned to variables.
The capture_size limit is only applied to disk usage of data captured from
sources.
The implementation of getting a capture cache file's size
`os.stat(self._path).st_size`
The implementation of summing up all capture cache file's sizes
`sum([sink.size_in_bytes for _, sink in self._capture_sinks.items()])`
They both locate in
[this](https://github.com/apache/beam/pull/11050/commits/a6d9e2382eeea148b3f667726f8e8e8933a7196c#diff-e15d1558a3154511b759ef711deeaddb)
change.
Everything else is wiring, logging and testing.
The first commit is a patch from Sam's ongoing PR, there is no need to
review diff of it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 398028)
Time Spent: 93h 10m (was: 93h)
> Add streaming support to Interactive Beam
> -----------------------------------------
>
> Key: BEAM-8335
> URL: https://issues.apache.org/jira/browse/BEAM-8335
> Project: Beam
> Issue Type: Improvement
> Components: runner-py-interactive
> Reporter: Sam Rohde
> Assignee: Sam Rohde
> Priority: Major
> Time Spent: 93h 10m
> Remaining Estimate: 0h
>
> This issue tracks the work items to introduce streaming support to the
> Interactive Beam experience. This will allow users to:
> * Write and run a streaming job in IPython
> * Automatically cache records from unbounded sources
> * Add a replay experience that replays all cached records to simulate the
> original pipeline execution
> * Add controls to play/pause/stop/step individual elements from the cached
> records
> * Add ability to inspect/visualize unbounded PCollections
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
