FYI: Finally, it was merged and you can find this page here: https://beam.apache.org/contribute/design-documents/ <https://beam.apache.org/contribute/design-documents/>
Thank you everybody who helped me to compile this list! I’ll try to do my best to update this with new coming docs. In the same time, please, feel free to add your new docs (or notify me if I missed this) once they are finished and ready to be published. WBR, Alexey > On 31 May 2018, at 18:52, Eugene Kirpichov <[email protected]> wrote: > > Thank you! > > On Thu, May 31, 2018 at 8:30 AM Alexey Romanenko <[email protected] > <mailto:[email protected]>> wrote: > Thank you everybody for provided links. I collected all of them (please, > correct me if I missed something), categorized and created a dedicated page > for Beam website. > > Here is a PR for that (please, review): > https://github.com/apache/beam-site/pull/456 > <https://github.com/apache/beam-site/pull/456> > > WBR, > Alexey > >> On 30 May 2018, at 13:17, Łukasz Gajowy <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi, >> >> I just wanted to add those two (sorry for being kinda late with this): >> >> https://docs.google.com/document/d/1dA-5s6OHiP_cz-NRAbwapoKF5MEC1wKps4A5tFbIPKE/edit?usp=sharing >> >> <https://docs.google.com/document/d/1dA-5s6OHiP_cz-NRAbwapoKF5MEC1wKps4A5tFbIPKE/edit?usp=sharing> >> https://docs.google.com/document/d/1Cb7XVmqe__nA_WCrriAifL-3WCzbZzV4Am5W_SkQLeA/edit?usp=sharing >> >> <https://docs.google.com/document/d/1Cb7XVmqe__nA_WCrriAifL-3WCzbZzV4Am5W_SkQLeA/edit?usp=sharing> >> >> Thanks, >> Łukasz >> >> 2018-05-29 22:42 GMT+02:00 Lukasz Cwik <[email protected] >> <mailto:[email protected]>>: >> Providing ownership to the PMC account allows others to take over ownership >> of the document once a contributor stops being active. This allows docs to >> be updated (even if just to point to a newer doc). >> >> On Tue, May 29, 2018 at 1:20 PM Kenneth Knowles <[email protected] >> <mailto:[email protected]>> wrote: >> My position on ownership is design docs are really documents "of the moment" >> and authored by a particular individual or group. Experience shows that even >> if you try, keeping it fresh is not likely to happen. Anything that needs >> freshness (like end-user docs) should be in a different medium. I would just >> date the gdoc so readers know how to interpret it (the automated "last edit" >> date is not sufficient for understanding how stale something is). >> >> So it seems like it makes little difference if the project or PMC has >> ownership or even write access. Of course I have no objections if someone >> wants to transfer ownership, but is there a reason to encourage it? >> >> Kenn >> >> On Tue, May 29, 2018 at 1:11 PM Lukasz Cwik <[email protected] >> <mailto:[email protected]>> wrote: >> I transferred ownership of the docs that I owned to the [email protected] >> <mailto:[email protected]> PMC account and put the ones that I owned into >> the drive folder. >> >> Would it be a good idea for others to follow suit? >> >> Instructions on how to transfer ownership are here: >> http://support.it.mtu.edu/Accounts/E-Mail/75946047/How-do-I-transfer-ownership-of-a-Google-Doc.htm >> >> <http://support.it.mtu.edu/Accounts/E-Mail/75946047/How-do-I-transfer-ownership-of-a-Google-Doc.htm> >> >> >> >> On Tue, May 29, 2018 at 11:23 AM Lukasz Cwik <[email protected] >> <mailto:[email protected]>> wrote: >> I created a PR for the beam-site to link to the design docs and template >> from the contribution guide: >> https://github.com/apache/beam-site/pull/454 >> <https://github.com/apache/beam-site/pull/454> >> >> On Fri, May 25, 2018 at 10:23 AM Lukasz Cwik <[email protected] >> <mailto:[email protected]>> wrote: >> Here are some more links related to portability efforts: >> >> https://s.apache.org/beam-fn-api <https://s.apache.org/beam-fn-api> >> https://s.apache.org/beam-fn-api-processing-a-bundle >> <https://s.apache.org/beam-fn-api-processing-a-bundle> >> https://s.apache.org/beam-fn-api-send-and-receive-data >> <https://s.apache.org/beam-fn-api-send-and-receive-data> >> https://s.apache.org/beam-fn-state-api-and-bundle-processing >> <https://s.apache.org/beam-fn-state-api-and-bundle-processing> >> https://s.apache.org/beam-fn-api-progress-reporting >> <https://s.apache.org/beam-fn-api-progress-reporting> >> https://s.apache.org/beam-fn-api-container-contract >> <https://s.apache.org/beam-fn-api-container-contract> >> https://s.apache.org/beam-breaking-fusion >> <https://s.apache.org/beam-breaking-fusion> >> https://s.apache.org/beam-runner-api-combine-model >> <https://s.apache.org/beam-runner-api-combine-model> >> https://s.apache.org/beam-fn-api-metrics >> <https://s.apache.org/beam-fn-api-metrics> >> >> >> On Thu, May 24, 2018 at 2:11 PM Scott Wegner <[email protected] >> <mailto:[email protected]>> wrote: >> Thanks for sharing these. I also put together a design doc template based on >> common styling / sections I saw in the docs listed above. Others are free to >> use it as they'd like. >> >> https://docs.google.com/document/d/1kVePqjt2daZd0bQHGUwghlcLbhvrny7VpflAzk9sjUg/edit?usp=sharing >> >> <https://docs.google.com/document/d/1kVePqjt2daZd0bQHGUwghlcLbhvrny7VpflAzk9sjUg/edit?usp=sharing> >> >> >> On Thu, May 24, 2018 at 6:23 AM Kenneth Knowles <[email protected] >> <mailto:[email protected]>> wrote: >> OK, I will also put a list here of those I know off the top of my head. Some >> are redundant with Etienne's but short links that I can think of: >> >> https://s.apache.org/a-new-dofn <https://s.apache.org/a-new-dofn> >> https://s.apache.org/beam-triggers <https://s.apache.org/beam-triggers> >> https://s.apache.org/beam-sink-triggers >> <https://s.apache.org/beam-sink-triggers> >> https://s.apache.org/beam-runner-composites >> <https://s.apache.org/beam-runner-composites> >> https://s.apache.org/beam-lateness <https://s.apache.org/beam-lateness> >> https://s.apache.org/beam-runner-api <https://s.apache.org/beam-runner-api> >> https://s.apache.org/beam-state >> <https://s.apache.org/beam-state>https://s.apache.org/beam-side-inputs-1-pager >> <https://s.apache.org/beam-side-inputs-1-pager> >> >> Kenn >> >> On Thu, May 24, 2018 at 6:08 AM Etienne Chauchot <[email protected] >> <mailto:[email protected]>> wrote: >> Great that you take this action Alexey ! >> Here are the links I have, there is duplicates with the ones you already >> received and maybe old docs as well: >> >> https://docs.google.com/document/d/1MtBZYV7NAcfbwyy9Op8STeFNBxtljxgy69FkHMvhTMA/edit >> >> <https://docs.google.com/document/d/1MtBZYV7NAcfbwyy9Op8STeFNBxtljxgy69FkHMvhTMA/edit> >> https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit >> >> <https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit> >> https://docs.google.com/document/d/1Fl_LM918j7ZxAmCSkm43GBjV8knsZAIA1tRhvJ4DneM/edit#heading=h.1lcfuwfvxg2 >> >> <https://docs.google.com/document/d/1Fl_LM918j7ZxAmCSkm43GBjV8knsZAIA1tRhvJ4DneM/edit#heading=h.1lcfuwfvxg2> >> https://docs.google.com/document/d/1zcF4ZGtq8pxzLZxgD_JMWAouSszIf9LnFANWHKBsZlg/edit >> >> <https://docs.google.com/document/d/1zcF4ZGtq8pxzLZxgD_JMWAouSszIf9LnFANWHKBsZlg/edit> >> https://docs.google.com/document/d/1AQmx-T9XjSi1PNoEp5_L-lT0j7BkgTbmQnc6uFEMI4c/edit#heading=h.dtl8cwoybr2y >> >> <https://docs.google.com/document/d/1AQmx-T9XjSi1PNoEp5_L-lT0j7BkgTbmQnc6uFEMI4c/edit#heading=h.dtl8cwoybr2y> >> https://docs.google.com/document/d/1u-4o_0uj8uKa2SVNPBNxIKfvcJ4t66ecCoU1M2yVoDA/edit#heading=h.c1deqkr0bp31 >> >> <https://docs.google.com/document/d/1u-4o_0uj8uKa2SVNPBNxIKfvcJ4t66ecCoU1M2yVoDA/edit#heading=h.c1deqkr0bp31> >> https://docs.google.com/document/d/17H2sBEtnoTSxjzlrz7rmKtX5E3F0mW1NpFQzWzSYOpY/edit#heading=h.1lcfuwfvxg2 >> >> <https://docs.google.com/document/d/17H2sBEtnoTSxjzlrz7rmKtX5E3F0mW1NpFQzWzSYOpY/edit#heading=h.1lcfuwfvxg2> >> https://docs.google.com/document/d/1tnG2DPHZYbsomvihIpXruUmQ12pHGK0QIvXS1FOTgRc/edit#heading=h.puuotbien1gf >> >> <https://docs.google.com/document/d/1tnG2DPHZYbsomvihIpXruUmQ12pHGK0QIvXS1FOTgRc/edit#heading=h.puuotbien1gf> >> https://docs.google.com/document/d/1KEtUPE5d6oCZFvFlR7PtSDvHpx5kDj2dTlIxF7UXa5w/edit#heading=h.p6lvszfbmyj6 >> >> <https://docs.google.com/document/d/1KEtUPE5d6oCZFvFlR7PtSDvHpx5kDj2dTlIxF7UXa5w/edit#heading=h.p6lvszfbmyj6> >> https://docs.google.com/document/d/1IGduUqmhWDi_69l9nG8kw73HZ5WI5wOps9Tshl5wpQA/edit#heading=h.gh88g5y0rekp >> >> <https://docs.google.com/document/d/1IGduUqmhWDi_69l9nG8kw73HZ5WI5wOps9Tshl5wpQA/edit#heading=h.gh88g5y0rekp> >> https://docs.google.com/document/d/1NehB__CxJTmjxkt7hDYIJWNDLJyEoE0b-AUJvInHPCI/edit#heading=h.gh88g5y0rekp >> >> <https://docs.google.com/document/d/1NehB__CxJTmjxkt7hDYIJWNDLJyEoE0b-AUJvInHPCI/edit#heading=h.gh88g5y0rekp> >> https://docs.google.com/document/d/1BOozW0bzBuz4oHJEuZNDOHdzaV5Y56ix58Ozrqm2jFg/edit#heading=h.gh88g5y0rekp >> >> <https://docs.google.com/document/d/1BOozW0bzBuz4oHJEuZNDOHdzaV5Y56ix58Ozrqm2jFg/edit#heading=h.gh88g5y0rekp> >> https://docs.google.com/document/d/1XYzb1Fnt2sam7u2MsGFaZp-2qSIGxUn66VLer-bcXAk/edit#heading=h.p6lvszfbmyj6 >> >> <https://docs.google.com/document/d/1XYzb1Fnt2sam7u2MsGFaZp-2qSIGxUn66VLer-bcXAk/edit#heading=h.p6lvszfbmyj6> >> https://docs.google.com/document/d/1voyUIQ2DrWkoY-BsJwM8YvF4gGKB76CDG8BYL8XBc7A/edit#heading=h.vv2fbulkp7t >> >> <https://docs.google.com/document/d/1voyUIQ2DrWkoY-BsJwM8YvF4gGKB76CDG8BYL8XBc7A/edit#heading=h.vv2fbulkp7t> >> >> Etienne >> >> >> >> Le jeudi 24 mai 2018 à 14:53 +0200, Alexey Romanenko a écrit : >>> Thank you everybody for positive feedback and sending me links to design >>> docs. I’m going to complete the full list and create PR for review. >>> >>> Griselda >>> Perhaps, it was a bit of misunderstanding here, let me explain what I >>> assume under creating of such page. >>> >>> Actually, for now I’m going to collect all known design docs under one >>> umbrella (the page on Beam web site) to make it easy to discover. For >>> example, as a new contributor, I’d like to know more details what was >>> initial design of new FileIO, SDF, Metrics, etc. >>> >>> I'm not going to copy the content of this docs to one page or even web >>> site, let’s keep this as it is, no changes here for the moment. I think, >>> moving to something else than Google docs is a tough question and requires >>> another discussion. >>> >>> So, in this case, this task seems not so hard since we don’t add such docs >>> too often - I'll just have to update this index page on web site. In >>> addition, the authors will be always welcomed to update this page by >>> themselves. In my turn, I’ll try to keep an eye on this to keep it synced. >>> And of course, any help will be welcomed too =) >>> >>> WBR, >>> Alexey >>> >>>> On 24 May 2018, at 00:01, Griselda Cuevas <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> Hi Everyone, >>>> >>>> @Alexey, I think this is a great idea, I'd like to understand more of the >>>> motivation behind having all the designs doc under a single page. In my >>>> opinion it could become a challenge to maintain a page, so knowing what >>>> you want to accomplish could help us think of alternative solutions? >>>> >>>> On Wed, 23 May 2018 at 14:08, Daniel Oliveira <[email protected] >>>> <mailto:[email protected]>> wrote: >>>>> +1 to web site page (not Google Doc). >>>>> >>>>> Definitely agree that a common entry point would be excellent. I don't >>>>> like the idea of the Google Doc so much because it's not very good for >>>>> having changes reviewed and keeping track of who added what, unlike >>>>> Github. Adding an entry to the list in the website would require reviews >>>>> and leave behind a commit history, which I think is important for an >>>>> authoritative source like this. >>>>> >>>>> PS: I also have a doc I proposed that I didn't see in the lists: >>>>> https://s.apache.org/beam-runner-api-combine-model >>>>> <https://s.apache.org/beam-runner-api-combine-model> >>>>> >>>>> On Wed, May 23, 2018 at 12:52 PM Lukasz Cwik <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>>> +1, Thanks for picking this up Alexey >>>>>> >>>>>> On Wed, May 23, 2018 at 10:41 AM Huygaa Batsaikhan <[email protected] >>>>>> <mailto:[email protected]>> wrote: >>>>>>> +1. That is great, Alexey. Robin and I are working on documenting some >>>>>>> missing pieces of Java SDK. We will let you know when we create >>>>>>> polished documents. >>>>>>> >>>>>>> On Wed, May 23, 2018 at 9:28 AM Ismaël Mejía <[email protected] >>>>>>> <mailto:[email protected]>> wrote: >>>>>>>> +1 and thanks for volunteering for this Alexey. >>>>>>>> We really need to make this more accesible. >>>>>>>> On Wed, May 23, 2018 at 6:00 PM Alexey Romanenko >>>>>>>> <[email protected] <mailto:[email protected]>> >>>>>>>> wrote: >>>>>>>> >>>>>>>> > Joseph, Eugene - thank you very much for the links! >>>>>>>> >>>>>>>> > All, regarding one common entry point for all design documents. >>>>>>>> > Could we >>>>>>>> just have a dedicated page on Beam web site with a list of links to >>>>>>>> every >>>>>>>> proposed document? Every entry (optionally) might contain, in addition, >>>>>>>> short abstract and list of author(s). In this case, it would be easily >>>>>>>> searchable and available for those who are interested in this. >>>>>>>> >>>>>>>> > In the same time, using a Google doc for writing/discussing the >>>>>>>> > documents >>>>>>>> seems more than reasonable since it’s quite native and easy to use. I >>>>>>>> only >>>>>>>> propose to have a common entry point to fall of them. >>>>>>>> >>>>>>>> > If this idea looks feasible, I’d propose myself to collect the links >>>>>>>> > to >>>>>>>> already created documents, create such page and update this list in the >>>>>>>> future. >>>>>>>> >>>>>>>> > WBR, >>>>>>>> > Alexey >>>>>>>> >>>>>>>> > On 22 May 2018, at 21:34, Eugene Kirpichov <[email protected] >>>>>>>> > <mailto:[email protected]>> wrote: >>>>>>>> >>>>>>>> > Making it easier to manage indeed would be good. Could someone from >>>>>>>> > PMC >>>>>>>> please add the following documents of mine to it? >>>>>>>> >>>>>>>> > SDF related documents: >>>>>>>> > http://s.apache.org/splittable-do-fn >>>>>>>> > <http://s.apache.org/splittable-do-fn> >>>>>>>> > http://s.apache.org/sdf-via-source >>>>>>>> > <http://s.apache.org/sdf-via-source> >>>>>>>> > http://s.apache.org/textio-sdf <http://s.apache.org/textio-sdf> >>>>>>>> > http://s.apache.org/beam-watch-transform >>>>>>>> > <http://s.apache.org/beam-watch-transform> >>>>>>>> > http://s.apache.org/beam-breaking-fusion >>>>>>>> > <http://s.apache.org/beam-breaking-fusion> >>>>>>>> >>>>>>>> > Non SDF related: >>>>>>>> > http://s.apache.org/context-fn <http://s.apache.org/context-fn> >>>>>>>> > http://s.apache.org/fileio-write <http://s.apache.org/fileio-write> >>>>>>>> >>>>>>>> > A suggestion: maybe we can establish a convention to send design >>>>>>>> > document >>>>>>>> proposals to [email protected] >>>>>>>> <mailto:dev%[email protected]>? Does the Apache mailing list >>>>>>>> management software support this kind of stuff? Then they'd be quite >>>>>>>> easy >>>>>>>> to find and filter. >>>>>>>> >>>>>>>> > On Tue, May 22, 2018 at 10:57 AM Kenneth Knowles <[email protected] >>>>>>>> > <mailto:[email protected]>> wrote: >>>>>>>> >>>>>>>> >> It is owned by the Beam PMC collectively. Any PMC member can add >>>>>>>> >> things >>>>>>>> to it. Ideas for making it easy to manage are welcome. >>>>>>>> >>>>>>>> >> Probably easier to have a markdown file somewhere with a list of >>>>>>>> >> docs so >>>>>>>> we can issue and review PRs. Not sure the web site is the right place >>>>>>>> for >>>>>>>> it - we have a history of porting docs to markdown but really that is >>>>>>>> high >>>>>>>> overhead and users/community probably don't gain from it so much. Some >>>>>>>> have >>>>>>>> suggested a wiki. >>>>>>>> >>>>>>>> >> Kenn >>>>>>>> >>>>>>>> >> On Tue, May 22, 2018 at 10:22 AM Scott Wegner <[email protected] >>>>>>>> >> <mailto:[email protected]>> wrote: >>>>>>>> >>>>>>>> >>> Thanks for the links. Any details on that Google drive folder? Who >>>>>>>> maintains it? Is it possible for any contributor to add their design >>>>>>>> doc? >>>>>>>> >>>>>>>> >>> On Mon, May 21, 2018 at 8:15 AM Joseph PENG >>>>>>>> >>> <[email protected] <mailto:[email protected]>> >>>>>>>> wrote: >>>>>>>> >>>>>>>> >>>> Alexey, >>>>>>>> >>>>>>>> >>>> I do not know where you can find all design docs, but I know a >>>>>>>> >>>> blog >>>>>>>> that has collected some of the major design docs. Hope it helps. >>>>>>>> >>>>>>>> >>>> https://wtanaka.com/beam/design-doc >>>>>>>> >>>> <https://wtanaka.com/beam/design-doc> >>>>>>>> >>>>>>>> >>>> https://drive.google.com/drive/folders/0B-IhJZh9Ab52OFBVZHpsNjc4eXc >>>>>>>> >>>> >>>>>>>> >>>> <https://drive.google.com/drive/folders/0B-IhJZh9Ab52OFBVZHpsNjc4eXc> >>>>>>>> >>>>>>>> >>>> On Mon, May 21, 2018 at 9:28 AM Alexey Romanenko < >>>>>>>> [email protected] <mailto:[email protected]>> wrote: >>>>>>>> >>>>>>>> >>>>> Hi all, >>>>>>>> >>>>>>>> >>>>> Is it possible to obtain somewhere a list of all proposals / >>>>>>>> prototype documents that have been published as a technical / design >>>>>>>> documents for new features? I have links to only some of them (found in >>>>>>>> mail list discussions by chance) but I’m not aware of others. >>>>>>>> >>>>>>>> >>>>> If yes, could someone share it or point me out where it is >>>>>>>> >>>>> located in >>>>>>>> case if I missed this? >>>>>>>> >>>>>>>> >>>>> If not, don’t you think it would make sense to have such index of >>>>>>>> these documents? I believe it can be useful for Beam contributors since >>>>>>>> these proposals contain information which is absent or not so detailed >>>>>>>> on >>>>>>>> Beam web site documentation. >>>>>>>> >>>>>>>> >>>>> WBR, >>>>>>>> >>>>> Alexey >>> >> >
