[
https://issues.apache.org/jira/browse/SPARK-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225223#comment-14225223
]
Chip Senkbeil commented on SPARK-4605:
--------------------------------------
Actually, the kernel itself is concrete today as we already have applications
running on top of it (including the IPython notebook) and demonstrated those
applications at the StrataConf in Barcelona, Spain. It does not involve any
change in the Spark core codebase. We believe that the kernel represents a
fundamental/core piece of Spark that can be used as the foundation for
interactive applications and should not be a separate package. It provides the
hooks into Spark in the same way that the JDBC server allows connecting to a
database.
> Proposed Contribution: Spark Kernel to enable interactive Spark applications
> ----------------------------------------------------------------------------
>
> Key: SPARK-4605
> URL: https://issues.apache.org/jira/browse/SPARK-4605
> Project: Spark
> Issue Type: New Feature
> Reporter: Chip Senkbeil
> Attachments: Kernel Architecture Widescreen.pdf, Kernel
> Architecture.pdf
>
>
> This architecture is describing running code that was demonstrated at the
> StrataConf in Barcelona, Spain.
> ----
> Enables applications to interact with a Spark cluster using Scala in several
> ways:
> * Defining and running core Spark Tasks
> * Collecting results from a cluster without needing to write to external data
> store
> ** Ability to stream results using well-defined protocol
> * Arbitrary Scala code definition and execution (without submitting
> heavy-weight jars)
> Applications can be hosted and managed separate from the Spark cluster using
> the kernel as a proxy to communicate requests.
> The Spark Kernel implements the server side of the IPython Kernel protocol,
> the rising “de-facto” protocol for language (Python, Haskell, etc.) execution.
> Inherits a suite of industry adopted clients such as the IPython Notebook.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
