KevinGG commented on issue #11050: [BEAM-8335] Implemented Capture Size limitation URL: https://github.com/apache/beam/pull/11050#issuecomment-594967107 R: @rohdesamuel R: @aaltay Data captured from sources are stored in cache just like intermediate PCollections that are assigned to variables. The capture_size limit is only applied to disk usage of data captured from sources. The implementation of getting a capture cache file's size `os.stat(self._path).st_size` The implementation of summing up all capture cache file's sizes `sum([sink.size_in_bytes for _, sink in self._capture_sinks.items()])` Everything else is wiring, logging and testing.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
