[
https://issues.apache.org/jira/browse/BEAM-265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275697#comment-15275697
]
Davor Bonaci commented on BEAM-265:
-----------------------------------
Fair point -- in a (very) limited scope, user-defined functions can be
language-independent.
The real question is what are the tradeoffs: limiting scope of user-defined
functions is a big downside. On the other hand, language-independence at that
level is not providing us appropriate value. Once we clarify the Runner / Fn
APIs, there'll be a set of core SDKs that will have to provide
language-specific "Fn harness". This achieves total independence between
runners and SDKs, without sacrifices in scope.
In this sense, you are challenging top-level project vision / architecture. It
is a very complex and long discussion -- it would be better to have this part
of the discussion on the dev@ mailing list. Even better, during one of the Beam
(virtual) meetings.
> Dynamic Pipeline, Transformation, etc. / DAG Changes DSL | Full Specification
> Language Independence, Transo
> -----------------------------------------------------------------------------------------------------------
>
> Key: BEAM-265
> URL: https://issues.apache.org/jira/browse/BEAM-265
> Project: Beam
> Issue Type: Improvement
> Components: beam-model
> Reporter: Suminda Dharmasena
> Assignee: Davor Bonaci
>
> There should conditional / dynamic changes to pipelines and DAGs. The Beam
> specification should ideally be:
> - fully language neutral to support many languages
> - should be specified statically so code generation can work. E.g. pipeline
> specified in Java but run as native. In this case the conversion would be
> like that GWT does with Java.
> In addition the transformation code though expressed in a given language
> should be transformable into an language independent form.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
