Amit Sela created BEAM-542:
------------------------------
Summary: Spark batch interval should be a configuration instead of
an interpretation of the Pipeline's windows
Key: BEAM-542
URL: https://issues.apache.org/jira/browse/BEAM-542
Project: Beam
Issue Type: Bug
Components: runner-spark
Reporter: Amit Sela
Assignee: Amit Sela
Currently, the SparkRunner extracts the batch interval from the duration of the
first window.
This is wrong in several ways:
# GlobalWindow pipelines
# It's an engine specific property and should not be expressed as a part of the
logic but rather as a configuration for execution of the pipeline.
# Effectively forces the definition of Fixed/SlidingWindows even when they are
not needed (stateless processing), which also makes the pipeline code not
portable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
