[
https://issues.apache.org/jira/browse/BEAM-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276724#comment-15276724
]
Frances Perry commented on BEAM-262:
------------------------------------
I'm not sure why you think using Flink or Spark for execution is overkill for
what Beam does? Creating a backend that can handle all Beam pipelines at scale
is a huge undertaking! I agree with Davor that building backends is generally
beyond the scope of Beam currently.
Right now we're looking at creating the best programming model for writing data
processing pipelines that generalizes functionality of a number of current
distributed processing backends. Each backend has its own strengths in terms of
what use cases it handles well, and users can choose which one fits their
needs. Having a single backend that automatically does the best thing would be
great, but I don't think it's feasible yet.
> Native Runners | Direct Compiler
> ---------------------------------
>
> Key: BEAM-262
> URL: https://issues.apache.org/jira/browse/BEAM-262
> Project: Beam
> Issue Type: Improvement
> Components: runner-ideas
> Reporter: Suminda Dharmasena
> Assignee: Davor Bonaci
>
> Having to depend on other frameworks to do the heavy lifting means that the
> quakes, limitation and overhead of the other platform limits what can be
> achieved. Hence is it possible to have Beam directly generate code for LLVM,
> JVM and .Net platforms without dependence on any other platform.
> Also perhaps there can be code generation than directly native code in high
> level languages like C/C++, Java, C#, F#, Rust, Julia, D, Nim, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
