Hi Stuart,
thanks for sharing the document. I left a comment. I think we might want
to contact Kafka community with some of the design decisions /
questions, because there might be related effort - e.g. quite recently I
came across idea that watermarks could be natively supported by Kafka,
which would make the runner design somewhat easier [1].
Jan
[1] https://lists.apache.org/thread/9wktgkz9y3lv3t08q65hrvm0h1mwshgn
On 12/27/21 13:24, Stuart Perks wrote:
Thanks Dan,
All we have created a starting point of a design doc be great to start
getting comments to fill out the key points so please get involved any
thoughts, questions, ideas, content suggestions please feel free to
comment on the doc.
The doc is linked to the JIRA
https://issues.apache.org/jira/browse/BEAM-2466
And directly you can access it here where you can add comments:
https://docs.google.com/document/d/1mNqERvvV8oGI_O4tGewH2Kgkq6PQGv3ylmxnaTRBqH8/edit?usp=sharing
I will take relevant comments and add to the document as appropriate.
Looking forward to collaborating on this.
Thanks
Stuart
On 25 Dec 2021, at 15:55, Daniel Collins <[email protected]> wrote:
Hi Stuart,
I think most design documents I've seen recently are in fact google
docs- indeed, most of the links on that page just link out to google
docs. The most productive way for you to start this might be to start
writing the outlines of a design in a google doc that is open for
comments by anyone- if you end up having collaborators, you can grant
them edit access. Posting the document here and in the jira would
then allow interested developers to comment on it. It can be added to
that confluence page later once approved/implemented for reference.
Interested to see your thoughts on how to best implement this!
-Daniel
On Fri, Dec 24, 2021 at 5:31 PM Stuart Perks
<[email protected]> wrote:
Hi Beam Community,
Happy Holidays.
I was hoping I could be granted permissions to create a design
document under the “Runner” section of this page, so we can
start designing JIRA
https://issues.apache.org/jira/browse/BEAM-2466 “Add Kafka
Streams Runner”
Alternatively could someone create this and then I can edit the
google doc its linked to?
https://cwiki.apache.org/confluence/display/BEAM/Design+Documents
Thanks,
Stuart Perks
https://github.com/perkss
