HI guys,
As part of our user growth, I'd like to revive this subject.I have sketched up
a 2 pages proposal on this:
https://docs.google.com/document/d/1eXt54ht0h7-pPbP-MJR0N5nzmxRRlAwbFod-LXI1x0A/edit?usp=sharing
Unfortunately I have no knowledge on IDE plugin developement. Does someone have
this knowledge and the envy to code it ?
Best
Etienne
Le mercredi 24 octobre 2018 à 09:45 +0200, Etienne Chauchot a écrit :
> Hi guys,
> To sum up what we said, I just opened this
> ticket:https://issues.apache.org/jira/browse/BEAM-5849
> Etienne
> Le jeudi 18 octobre 2018 à 12:44 +0200, Maximilian Michels a écrit :
> > Plugins for IDEs would be amazing because they could provide feedback
> > already during pipeline construction, but I'm
> > not sure about the effort required to develop/maintain such plugins.
> > Ultimately, Runners have to decide whether they can translate the given
> > pipeline or not. So I'm leaning more towards
> > an approach to make intermediate checking of the pipeline translation
> > easier, e.g. by
> > - providing the target Runner already during development- running check of
> > the Runner alongside with the
> > DirectRunner (which is typically used when developing pipelines)
> > On 17.10.18 15:57, Etienne Chauchot wrote:Hey Max, Kenn,
> > Thanks for your feedback !
> > Yes the idea was to inform the user as soon as possible, ideally while
> > coding the pipeline. It could be done with a
> > IDE plugin (like checkstyle) that is configured with the targeted runner;
> > that way the targeted runner conf is not
> > part of the pipeline code in an annotation which would be against Beam
> > portability philosophy. Such a plugin could
> > color unsupported features while coding.
> > Another way could be to implement it as a javadoc but it seems weak because
> > not automatic enough.Another way could
> > be to implement it as a validation plugin in the build system but IMHO it
> > is already too late for the user.
> > So, long story short, I'm more in favor of an IDE plugin or similar
> > coding-time solution.
> > BestEtienne
> > Le mercredi 17 octobre 2018 à 12:11 +0200, Maximilian Michels a écrit :This
> > is a good idea. It needs to be fleshed
> > out how the capability of aRunner would be visible to the user (apart from
> > the compatibility matrix).
> > A dry-run feature would be useful, i.e. the user can run an inspectionon
> > the pipeline to see if it contains any
> > features which are notsupported by the Runner.
> > On 17.10.18 00:03, Rui Wang wrote:Sounds like a good idea.
> > Sounds like while coding, user gets a list to show if a feature issupported
> > on different runners. User can check the
> > list for the answer.Is my understanding correct? Will this approach become
> > slow as number ofrunner grows? (it's just
> > a question as I am not familiar the performanceof combination of a long
> > list, annotation and IDE)
> >
> > -Rui
> > On Sat, Oct 13, 2018 at 11:56 PM Reuven Lax <[email protected]
> > <mailto:[email protected]> <mailto:[email protected]
> > <mailto:[email protected]>>> wrote:
> > Sounds like a good idea. I don't think it will work for all
> > capabilities (e.g. some of them such as
> > "exactly once" apply to all of the API surface), but useful for the
> > ones that we can capture.
> > On Thu, Oct 4, 2018 at 2:43 AM Etienne Chauchot
> > <[email protected] <mailto:[email protected]> <mailt
> > o:[email protected] <mailto:[email protected]>>> wrote:
> > Hi guys, As part of our user experience improvement to
> > attract new Beam users, I would like
> > to suggest something:
> > Today we only have the capability matrix to inform users about
> > features support among runners. But,
> > they might discover only when the pipeline runs, when they receive
> > an exception, that a given
> > feature is not supported by the targeted runner. I would like to
> > suggest to translate the capability matrix
> > into the API with annotations for example, so that, while coding,
> > the user could know that, for now,
> > a given feature is not supported on the runner he targets.
> > I know that the runner is only specified at pipeline runtime,
> > and that adding code would be a leak
> > of runner implementation and against portability. So it could be
> > just informative annotations
> > like @Experimental for example with no annotation processor.
> > WDYT?
> > Etienne
