I think it is important to clarify that the developer documentation discussed in this thread is of two kinds:
6.1. Documents with proposals and new designs, those covered by the Beam Improvement Proposal (BEAM-566), and that we need to put with a single file index (I remember there was a google dir for this but not sure it is still valid, and in any case probably the website is a better place for this). Is there any progress on this? 6.2. Documentation about how things work, so new developers can get into developing features/fixes for the project, those are the kind that Kenneth/Etienne mention and include Stephen’s IO guide but could be definitely expanded to include things like how does the different runner translation works, or some details on triggers/materialization of panes/windows from the SDK point of view. However the hard part of this documents is that they should be maintained e.g. updated when the code evolves so they don’t get outdated as JB mentions. On Tue, Apr 25, 2017 at 10:47 AM, Wesley Tanaka <wtan...@yahoo.com.invalid> wrote: > These are the ones I've come across so far, are there others? > > * Dynamic DoFn https://s.apache.org/a-new-dofn > > ** Splittable DoFn (Obsoletes Source API) http://s.apache.org/splittable-do-fn > > ** State and Timers for DoFn: https://s.apache.org/beam-state > > > * Lateness https://s.apache.org/beam-lateness > > > * Metrics API http://s.apache.org/beam-metrics-api > > ** I/O Metrics https://s.apache.org/standard-io-metrics > > > * Runner API http://s.apache.org/beam-runner-api > > ** https://s.apache.org/beam-runner-composites > > ** https://s.apache.org/beam-side-inputs-1-pager > > > * Fn API http://s.apache.org/beam-fn-api > > --- > Wesley Tanaka > https://wtanaka.com/ > > > On Monday, April 24, 2017, 2:45:45 PM HST, Sourabh Bajaj > <sourabhba...@google.com.INVALID> wrote: > For 6. I think having them in one page on the website where we can find the > design docs more easily would be great. > > 7. For low-hanging-fruit, one thing I really liked from some Mozilla > projects was assigning a mentor on the ticket. Someone you can reach out to > if you have questions. I think this makes the entry barrier really low for > first time contributors who might feel intimidated asking questions > completely in public. > > On Mon, Apr 24, 2017 at 10:06 AM Kenneth Knowles <k...@google.com.invalid> > wrote: > >> I like the subject Etienne has brought up, and will give it a number in >> this list :-) >> >> 6. Have more technical reference docs (not just workspace set up) for >> contributors. >> >> I think this overlaps a lot with a prior discussion about where to collect >> design proposals [1]. Design docs used to be just dropped into a public >> folder, but that got disorganized. And that thread was about work in >> progress, so JIRA was a good place for details after a dev@ thread agrees >> on a proposal. At this point, the designs are pretty solid conceptually or >> even implemented and we could start to build out deeper technical bits on >> the web site, or at least some place that people can find it. We do have >> the Testing Guide and the PTransform Style Guide and somewhere near there >> we could have deeper references. I think we need a broader vision for the >> "table of contents" here. >> >> For my docs (triggers, lateness, runner API, side inputs, state, coders) I >> haven't had time, but I do intend to both translate from GDoc to some other >> format and also rewrite versions for users where appropriate. Probably this >> will mean coming up with that table of contents. >> >> Kenn >> >> [1] >> >> https://lists.apache.org/thread.html/%3c6bc60c88-cf91-4fff-eae6-fea6ee06f...@nanthrax.net%3E >> >> >> On Mon, Apr 24, 2017 at 9:33 AM, Neelesh Salian <neeleshssal...@gmail.com> >> wrote: >> >> > Agreed. I have some old JIRAs that I am cleaning up. >> > >> > Thank you for bringing this up. >> > >> > On Mon, Apr 24, 2017 at 9:29 AM, Jean-Baptiste Onofré <j...@nanthrax.net> >> > wrote: >> > >> > > Same also for Slack, github comments, etc. >> > > >> > > From a Apache perspective, it should happen on the mailing list, >> > > eventually referencing a central wiki/faq/whatever. >> > > >> > > Regards >> > > JB >> > > >> > > >> > > On 04/24/2017 06:23 PM, Mingmin Xu wrote: >> > > >> > >> many design documents are mixed in maillist, jira comments, it would >> be >> > a >> > >> big help to put them in a centralized list. Also I would expect more >> > >> wiki/blogs to provide in-depth analysis, like the translation from >> > >> pipeline >> > >> to runner specified topology, window/trigger implementation. Without >> > these >> > >> knowledge, it's hard to touch the core concepts. >> > >> >> > >> On Mon, Apr 24, 2017 at 6:03 AM, Jean-Baptiste Onofré < >> j...@nanthrax.net> >> > >> wrote: >> > >> >> > >> Got it. By experience on other Apache projects, it's really hard to >> > >>> maintain ;) >> > >>> >> > >>> Regards >> > >>> JB >> > >>> >> > >>> >> > >>> On 04/24/2017 02:56 PM, Etienne Chauchot wrote: >> > >>> >> > >>> Hi JB, >> > >>>> >> > >>>> I was proposing a FAQ (or another form), not something about IDE >> > setup. >> > >>>> The FAQ >> > >>>> could group in the same place Q/A like for example "what is a >> source, >> > >>>> how >> > >>>> do I >> > >>>> use it to implement an IO" >> > >>>> >> > >>>> Etienne >> > >>>> >> > >>>> >> > >>>> Le 24/04/2017 à 14:19, Jean-Baptiste Onofré a écrit : >> > >>>> >> > >>>> Hi Etienne, >> > >>>>> >> > >>>>> What about the contribution guide ? I think it's covered in the >> > >>>>> IntelliJ >> > >>>>> and >> > >>>>> Eclipse setup sections. >> > >>>>> >> > >>>>> Regards >> > >>>>> JB >> > >>>>> >> > >>>>> On 04/24/2017 02:12 PM, Etienne Chauchot wrote: >> > >>>>> >> > >>>>> Hi all, >> > >>>>>> >> > >>>>>> I definitely agree with everything that is said in this thread. >> > >>>>>> >> > >>>>>> I might suggest another good to have: >> > >>>>>> >> > >>>>>> to ease the work of a new contributor, it would be nice to have >> some >> > >>>>>> sort of >> > >>>>>> programming guide but not oriented to pipeline writers but to >> > >>>>>> sdk/runner/io/... >> > >>>>>> writers. >> > >>>>>> >> > >>>>>> I know that new contributors have the docs available in the google >> > >>>>>> drive, the >> > >>>>>> ML, the code base, and the availability of beamers, but maybe >> having >> > >>>>>> key points >> > >>>>>> in a common place (like FAQ for sdk/runner/io/... writers, for >> > >>>>>> example) >> > >>>>>> would be >> > >>>>>> interesting. >> > >>>>>> >> > >>>>>> Best, >> > >>>>>> >> > >>>>>> Etienne >> > >>>>>> >> > >>>>>> >> > >>>>>> Le 24/04/2017 à 09:14, Jean-Baptiste Onofré a écrit : >> > >>>>>> >> > >>>>>> Hi, >> > >>>>>>> >> > >>>>>>> I think we already tag the newbie jira ("low hanging fruit" ;)). >> > >>>>>>> >> > >>>>>>> Good idea for domain of interest/concept. >> > >>>>>>> >> > >>>>>>> Regards >> > >>>>>>> JB >> > >>>>>>> >> > >>>>>>> On 04/24/2017 09:01 AM, Ankur Chauhan wrote: >> > >>>>>>> >> > >>>>>>> Might I suggest adding tags to projects based on area of >> intetest, >> > >>>>>>>> concept >> > >>>>>>>> and if it's a good "first bug". >> > >>>>>>>> >> > >>>>>>>> Sent from my iPhone >> > >>>>>>>> >> > >>>>>>>> On Apr 23, 2017, at 23:03, Davor Bonaci <da...@apache.org> >> wrote: >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> 1. Have people unassign themselves from issues they're not >> > actively >> > >>>>>>>>>> working on. >> > >>>>>>>>>> 2. Have the community engage more in triage, improving tickets >> > >>>>>>>>>> descriptions and raising concerns. >> > >>>>>>>>>> 3. Clean house - apply (2) to currently open issues (over >> 800). >> > >>>>>>>>>> Perhaps >> > >>>>>>>>>> some can be closed. >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> +1 on all three of these, and will do my part shortly! >> > >>>>>>>>> >> > >>>>>>>>> Also, it is worth noting that we have improved as a project in >> > >>>>>>>>> tracking >> > >>>>>>>>> issues in the last 1-2 months. There are more resolved issues >> > than >> > >>>>>>>>> opened >> > >>>>>>>>> in this period, whereas in the past we'd have a hundred more >> > opened >> > >>>>>>>>> than >> > >>>>>>>>> resolved. >> > >>>>>>>>> >> > >>>>>>>>> I would also propose to not assign new Jira automatically: now, >> > the >> > >>>>>>>>> Jira is >> > >>>>>>>>> >> > >>>>>>>>> automatically assigned to the Jira component leader. >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> Imagine a user discovering an issue and filing a new JIRA >> issue. >> > >>>>>>>>> It >> > >>>>>>>>> wouldn't be assigned to anyone, significantly reducing the >> chance >> > >>>>>>>>> somebody >> > >>>>>>>>> will actually help. >> > >>>>>>>>> >> > >>>>>>>>> Of course, somebody could search for new issues periodically, >> > etc. >> > >>>>>>>>> -- but >> > >>>>>>>>> that just won't happen. The final outcome would be -- instead >> of >> > a >> > >>>>>>>>> lot of >> > >>>>>>>>> issues assigned to component leads, we'd have (much) more >> > >>>>>>>>> unassigned >> > >>>>>>>>> issues, which were *never* looked at. Assigning an issue just >> > sets >> > >>>>>>>>> a >> > >>>>>>>>> community expectation that a committer should look -- and it >> does >> > >>>>>>>>> help move >> > >>>>>>>>> things along! >> > >>>>>>>>> >> > >>>>>>>>> I think a better approach of addressing the current state would >> > be >> > >>>>>>>>> increase >> > >>>>>>>>> the number of components / component leads. With more people >> > >>>>>>>>> involved and >> > >>>>>>>>> lower per-person load, I think we'd be more effective. >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>> >> > >>>>>>> >> > >>>>>> >> > >>>>> >> > >>>> -- >> > >>> Jean-Baptiste Onofré >> > >>> jbono...@apache.org >> > >>> http://blog.nanthrax.net >> > >>> Talend - http://www.talend.com >> > >>> >> > >>> >> > >> >> > >> >> > >> >> > > -- >> > > Jean-Baptiste Onofré >> > > jbono...@apache.org >> > > http://blog.nanthrax.net >> > > Talend - http://www.talend.com >> > > >> > >> > >> > >> > -- >> > Regards, >> > Neelesh S. Salian >> > >>
