[
https://issues.apache.org/jira/browse/BEAM-2888?focusedWorklogId=241863&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-241863
]
ASF GitHub Bot logged work on BEAM-2888:
----------------------------------------
Author: ASF GitHub Bot
Created on: 14/May/19 16:55
Start Date: 14/May/19 16:55
Worklog Time Spent: 10m
Work Description: kennknowles commented on pull request #8576:
[BEAM-2888] Add the not-yet-fully-designed drain and checkpoint to runner
comparison
URL: https://github.com/apache/beam/pull/8576
These features have been discussed a bit on the mailing list and I get asked
about them a bit. I think it is OK to list them here like we list retractions /
metadata triggers, both of which are further from reality than checkpoint and
drain.
I would have preferred to do the whole comparison rewrite of
https://issues.apache.org/jira/browse/BEAM-2888 because the tabular form
misleadingly implies these are yes/no features, when in reality they have many
facets. But I haven't had time, and haven't convinced anyone else to take on
that work.
So this is a low-effort compromise.
What do you think?
------------------------
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [x] [**Choose
reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and
mention them in a comment (`R: @username`).
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [x] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
| --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python3_Verify/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
<br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)
| --- | --- | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/)
| --- | ---
See
[.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md)
for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 241863)
Time Spent: 10m
Remaining Estimate: 0h
> Runner Comparison / Capability Matrix revamp
> --------------------------------------------
>
> Key: BEAM-2888
> URL: https://issues.apache.org/jira/browse/BEAM-2888
> Project: Beam
> Issue Type: Improvement
> Components: website
> Reporter: Kenneth Knowles
> Assignee: Griselda Cuevas Zambrano
> Priority: Major
> Labels: gsod, gsod2019
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Discussion:
> https://lists.apache.org/thread.html/8aff7d70c254356f2dae3109fb605e0b60763602225a877d3dadf8b7@%3Cdev.beam.apache.org%3E
> Summarizing that discussion, we have a lot of issues/wishes. Some can be
> addressed as one-off and some need a unified reorganization of the runner
> comparison.
> Basic corrections:
> - Remove rows that impossible to not support (ParDo)
> - Remove rows where "support" doesn't really make sense (Composite
> transforms)
> - Deduplicate rows are actually the same model feature (all non-merging
> windowing / all merging windowing)
> - Clearly separate rows that represent optimizations (Combine)
> - Correct rows in the wrong place (Timers are actually a "what...?" row)
> - Separate or remove rows have not been designed ([Meta]Data driven
> triggers, retractions)
> - Rename rows with names that appear no where else (Timestamp control, which
> is called a TimestampCombiner in Java)
> - Switch to a more distinct color scheme for full/partial support (currently
> just solid/faded colors)
> - Switch to something clearer than "~" for partial support, versus ✘ and ✓
> for none and full.
> - Correct Gearpump support for merging windows (see BEAM-2759)
> - Correct Spark support for non-merging and merging windows (see BEAM-2499)
> Minor rewrites:
> - Lump all the basic stuff (ParDo, GroupByKey, Read, Window) into one row
> - Make sections as users see them, like "ParDo" / "side Inputs" not "What?"
> / "side inputs"
> - Add rows for non-model things, like portability framework support, metrics
> backends, etc
> Bigger rewrites:
> - Add versioning to the comparison, as in BEAM-166
> - Find a way to fit in a plain English summary of runner's support in Beam.
> It should come first, as it is what new users need before getting to details.
> - Find a way to describe production readiness of runners and/or testimonials
> of who is using it in production.
> - Have a place to compare non-model differences between runners
> Changes requiring engineering efforts:
> - Gather and add quantitative runner metrics, perhaps Nexmark results for
> mid-level, smaller benchmarks for measuring aspects of specific features, and
> larger end-to-end benchmarks to get an idea how it might actually perform on
> a use case
> - Tighter coupling of the matrix portion of the comparison with tags on
> ValidatesRunner tests
> If you care to address some aspect of this, please reach out and/or just file
> a subtask and address it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
