Shivaram Venkataraman created SPARK-19487:
---------------------------------------------
Summary: Low latency execution for Spark
Key: SPARK-19487
URL: https://issues.apache.org/jira/browse/SPARK-19487
Project: Spark
Issue Type: Umbrella
Components: ML, Scheduler, Structured Streaming
Affects Versions: 2.1.0
Reporter: Shivaram Venkataraman
This JIRA tracks the design discussion for supporting low latency execution in
Apache Spark. The motivation for this comes from need to support lower latency
stream processing and lower latency iterations for sparse ML workloads.
Overview of proposed design (in the format of Spark Improvement Proposal) is at
https://docs.google.com/document/d/1m_q83DjQcWQonEz4IsRUHu4QSjcDyqRpl29qE4LJc4s/edit?usp=sharing
Source code prototype is at: https://github.com/amplab/drizzle-spark
Lets use this JIRA to discuss high level design and we can create subtasks as
we break this down into smaller PRs.
This is joint work with [~kayousterhout]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
