Christian Tzolov created ZEPPELIN-274:
-----------------------------------------
Summary: Add Support for Streaming (long-running) Tasks.
Key: ZEPPELIN-274
URL: https://issues.apache.org/jira/browse/ZEPPELIN-274
Project: Zeppelin
Issue Type: New Feature
Components: Core, Interpreters
Affects Versions: 0.6.0
Reporter: Christian Tzolov
The Zeppelin's Paragraph State Machines (SM) defines 3 active states (PENDING,
FINISHED and FAILED). At high level the SM looks like this:
FINISHED - (On Run) -> PENDING - (On Success) -> FINISHED /or (On Failure) ->
FAILED
The Stream (long-running) processes are continuous as opposed to typical
Paragraph's batch-like jobs. Currently if you start a twitter search stream
task in a paragraph (by clicking the Run button) the underlying streaming
platform will open a stream and will return some information for the newly
created stream. The Paragraph SM will consider the job done and will transfer
from PENDING into FINISHED state. In reality the actual state should be
STARTED (or alike) to reflect that the stream is running and should transfer
into FINISHED only after the stream is closed.
To support long-running (aka streaming) jobs Zeppelin need to extend its SM
with at least one additional state: 'STARTED'.
Such improvement would benefit streaming platforms like Spark Streaming, Flink,
SpringXD and alike.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
