Closing the loop on this thread, I've summarized the suggestions into a
mega-ticket at https://issues.apache.org/jira/browse/BEAM-2888
Eventually, we'll need a redesign, but there is a lot that we can do
incrementally.
If you want to help, make a subtask for the piece you are handling, or I
can
Agree, it sounds like a good idea to me.
Regards
JB
On 08/31/2017 10:35 AM, Etienne Chauchot wrote:
Hi,
I think Nexmark (https://github.com/apache/beam/tree/master/sdks/java/nexmark)
could help in getting quantitative benchmark metrics for all the runners like
Tyler suggested.
Another
+1 for having plain English feature descriptions.
Nitpick: the capability matrix uses the "~" symbol, the meaning of which is
not entirely clear from the context. I think a legend would be helpful
given things have gone beyond ✘ and ✓.
-Stas
On Mon, Aug 28, 2017 at 7:23 PM Lukasz Cwik
I like where this is going!
Regarding benchmarking, I think we could do this if we had common benchmarking
infrastructure and pipelines that regularly run on different Runners so that we
have up-to-date data.
I think we can also have a more technical section where we show stats on the
level
I would like to have an API compatibility testing. AFAIK there's still gap
to achieve our goal (one job for any runner), that means developers should
notice the limitation when writing the job. For example PCollectionView is
not well supported in FlinkRunner(not quite sure the current status as my
Oh, I missed
11. Quantitative properties. This seems like an interesting and important
project all on its own. Since Beam is so generic, we need pretty diverse
measurements for a user to have a hope of extrapolating to their use case.
Kenn
On Tue, Aug 22, 2017 at 7:22 PM, Kenneth Knowles
OK, so adding these good ideas to the list:
8. Plain-English summary that comes before the nitty-gritty.
9. Comment on production readiness from maintainers. Maybe testimonials are
helpful if they can be obtained?
10. Versioning of all of the above
Any more thoughts? I'll summarize in a JIRA in
Hi, I'd also like to ask if versioning as proposed in BEAM-166 <
https://issues.apache.org/jira/browse/BEAM-166> is still relevant? If it
is, would this be something we want to add to this proposal?
G
On 21 August 2017 at 08:31, Tyler Akidau wrote:
> Is there any
Is there any way we could add quantitative runner metrics to this as well?
Like by having some benchmarks that process X amount of data, and then
detailing in the matrix latency, throughput, and (where possible) cost,
etc, numbers for each of the given runners? Semantic support is one thing,
but
It'd be awesome to see these updated. I'd add two more:
1. A plain English summary of the runner's support in Beam. People who
are new to Beam won't understand the in-depth coverage and need a general
idea of how it is supported.
2. The production readiness of the runner. Does the
Hi all,
I want to revamp
https://beam.apache.org/documentation/runners/capability-matrix/
When Beam first started, we didn't work on feature branches for the core
runners, and they had a lot more gaps compared to what goes on `master`
today, so this tracked our progress in a way that was easy
11 matches
Mail list logo