Plugins for IDEs would be amazing because they could provide feedback
already during pipeline construction, but I'm not sure about the effort
required to develop/maintain such plugins.
Ultimately, Runners have to decide whether they can translate the given
pipeline or not. So I'm leaning more towards an approach to make
intermediate checking of the pipeline translation easier, e.g. by
- providing the target Runner already during development
- running check of the Runner alongside with the DirectRunner (which is
typically used when developing pipelines)
On 17.10.18 15:57, Etienne Chauchot wrote:
Hey Max, Kenn,
Thanks for your feedback !
Yes the idea was to inform the user as soon as possible, ideally while
coding the pipeline. It could be done with a IDE plugin (like
checkstyle) that is configured with the targeted runner; that way the
targeted runner conf is not part of the pipeline code in an annotation
which would be against Beam portability philosophy. Such a plugin could
color unsupported features while coding.
Another way could be to implement it as a javadoc but it seems weak
because not automatic enough.
Another way could be to implement it as a validation plugin in the build
system but IMHO it is already too late for the user.
So, long story short, I'm more in favor of an IDE plugin or similar
coding-time solution.
Best
Etienne
Le mercredi 17 octobre 2018 à 12:11 +0200, Maximilian Michels a écrit :
This is a good idea. It needs to be fleshed out how the capability of a
Runner would be visible to the user (apart from the compatibility matrix).
A dry-run feature would be useful, i.e. the user can run an inspection
on the pipeline to see if it contains any features which are not
supported by the Runner.
On 17.10.18 00:03, Rui Wang wrote:
Sounds like a good idea.
Sounds like while coding, user gets a list to show if a feature is
supported on different runners. User can check the list for the answer.
Is my understanding correct? Will this approach become slow as number of
runner grows? (it's just a question as I am not familiar the performance
of combination of a long list, annotation and IDE)
-Rui
On Sat, Oct 13, 2018 at 11:56 PM Reuven Lax <re...@google.com <mailto:re...@google.com>
<mailto:re...@google.com <mailto:re...@google.com>>> wrote:
Sounds like a good idea. I don't think it will work for all
capabilities (e.g. some of them such as "exactly once" apply to all
of the API surface), but useful for the ones that we can capture.
On Thu, Oct 4, 2018 at 2:43 AM Etienne Chauchot
<echauc...@apache.org <mailto:echauc...@apache.org> <mailto:echauc...@apache.org
<mailto:echauc...@apache.org>>> wrote:
Hi guys,
As part of our user experience improvement to attract new Beam
users, I would like to suggest something:
Today we only have the capability matrix to inform users about
features support among runners. But, they might discover only
when the pipeline runs, when they receive an exception, that a
given feature is not supported by the targeted runner.
I would like to suggest to translate the capability matrix into
the API with annotations for example, so that, while coding, the
user could know that, for now, a given feature is not supported
on the runner he targets.
I know that the runner is only specified at pipeline runtime,
and that adding code would be a leak of runner implementation
and against portability. So it could be just informative
annotations like @Experimental for example with no annotation
processor.
WDYT?
Etienne
