Hi guys,
To sum up what we said, I just opened this
ticket:https://issues.apache.org/jira/browse/BEAM-5849
Etienne
Le jeudi 18 octobre 2018 à 12:44 +0200, Maximilian Michels a écrit :
> Plugins for IDEs would be amazing because they could provide feedback already
> during pipeline construction, but I'm
> not sure about the effort required to develop/maintain such plugins.
> Ultimately, Runners have to decide whether they can translate the given
> pipeline or not. So I'm leaning more towards
> an approach to make intermediate checking of the pipeline translation easier,
> e.g. by
> - providing the target Runner already during development- running check of
> the Runner alongside with the DirectRunner
> (which is typically used when developing pipelines)
> On 17.10.18 15:57, Etienne Chauchot wrote:
> Hey Max, Kenn,
> Thanks for your feedback !
> Yes the idea was to inform the user as soon as possible, ideally while coding
> the pipeline. It could be done with a
> IDE plugin (like checkstyle) that is configured with the targeted runner;
> that way the targeted runner conf is not
> part of the pipeline code in an annotation which would be against Beam
> portability philosophy. Such a plugin could
> color unsupported features while coding.
> Another way could be to implement it as a javadoc but it seems weak because
> not automatic enough.Another way could be
> to implement it as a validation plugin in the build system but IMHO it is
> already too late for the user.
> So, long story short, I'm more in favor of an IDE plugin or similar
> coding-time solution.
> BestEtienne
> Le mercredi 17 octobre 2018 à 12:11 +0200, Maximilian Michels a écrit :
> This is a good idea. It needs to be fleshed out how the capability of aRunner
> would be visible to the user (apart from
> the compatibility matrix).
> A dry-run feature would be useful, i.e. the user can run an inspectionon the
> pipeline to see if it contains any
> features which are notsupported by the Runner.
> On 17.10.18 00:03, Rui Wang wrote:Sounds like a good idea.
> Sounds like while coding, user gets a list to show if a feature issupported
> on different runners. User can check the
> list for the answer.Is my understanding correct? Will this approach become
> slow as number ofrunner grows? (it's just a
> question as I am not familiar the performanceof combination of a long list,
> annotation and IDE)
>
> -Rui
> On Sat, Oct 13, 2018 at 11:56 PM Reuven Lax <re...@google.com
> <mailto:re...@google.com> <mailto:re...@google.com
> <mailto:re...@google.com>>> wrote:
> Sounds like a good idea. I don't think it will work for all
> capabilities (e.g. some of them such as "exactly
> once" apply to all of the API surface), but useful for the ones that we
> can capture.
> On Thu, Oct 4, 2018 at 2:43 AM Etienne Chauchot
> <echauc...@apache.org <mailto:echauc...@apache.org> <mailto:
> echauc...@apache.org <mailto:echauc...@apache.org>>> wrote:
> Hi guys, As part of our user experience improvement to
> attract new Beam users, I would like
> to suggest something:
> Today we only have the capability matrix to inform users about
> features support among runners. But,
> they might discover only when the pipeline runs, when they receive an
> exception, that a given feature
> is not supported by the targeted runner. I would like to suggest to
> translate the capability matrix
> into the API with annotations for example, so that, while coding, the
> user could know that, for now, a
> given feature is not supported on the runner he targets.
> I know that the runner is only specified at pipeline runtime,
> and that adding code would be a leak of
> runner implementation and against portability. So it could be just
> informative annotations
> like @Experimental for example with no annotation processor.
> WDYT?
> Etienne
