I found a slightly hacky way to enable :inherited-members: just for the DataFrame API. I can add the option to the .rst output generated by sphinx-apidoc, before we run sphinx-build [1].
I'm fine just doing that instead of turning it on globally. [1] https://github.com/TheNeuralBit/beam/blob/e26760937f7a34fd72578b65f716098c74e4380b/sdks/python/scripts/generate_pydoc.sh#L86 On Tue, Apr 6, 2021 at 1:50 PM Brian Hulette <[email protected]> wrote: > Sure, I can try cutting out PTransform. > > We could also look into reducing noise by: > - removing undoc-members from the config [1] (this would make it so only > objects with a docstring are added to the generated docs) > - adding :meta private:` to docstrings for objects we don't want publicly > visible > > [1] > https://github.com/apache/beam/blob/243128a8fc52798e1b58b0cf1a271d95ee7aa241/sdks/python/scripts/generate_pydoc.sh#L48 > > On Tue, Apr 6, 2021 at 1:17 PM Robert Bradshaw <[email protected]> > wrote: > >> Way too many things are inherited from PTransform, can we at least cut >> that out? >> >> On Tue, Apr 6, 2021 at 1:09 PM Brian Hulette <[email protected]> wrote: >> >>> Just wanted to bump this - does anyone have concerns with the way the >>> API docs look when inherited members are included? >>> >>> On Wed, Mar 31, 2021 at 5:23 PM Brian Hulette <[email protected]> >>> wrote: >>> >>>> I staged my current working copy built from head here [1], see >>>> CombinePerKey here [2]. Note it also has a few other changes, most notably >>>> I excluded several internal-only modules that are currently in our API docs >>>> (I will PR this soon regardless). >>>> >>>> > are these inherited members grouped in such a way that it makes it >>>> easy to ignore them once they get to "low" in the stack? >>>> There doesn't seem to be any grouping, but it does look like inherited >>>> members are added at the end. >>>> >>>> > If it can't be per-module, is there a "nice" set of ancestors to >>>> avoid (as it seems this option takes such an argument). >>>> Ah good point, I missed this. I suppose we could avoid basic constructs >>>> like PTransform, DoFn, etc. I'm not sure how realistic that is though. It >>>> would be nice if this argument worked the other way >>>> >>>> [1] https://theneuralbit.github.io/beam-site/pydoc/inherited-members >>>> [2] >>>> https://theneuralbit.github.io/beam-site/pydoc/inherited-members/apache_beam.transforms.core.html#apache_beam.transforms.core.CombinePerKey >>>> >>>> On Wed, Mar 31, 2021 at 4:45 PM Robert Bradshaw <[email protected]> >>>> wrote: >>>> >>>>> +1 to an example. In particular, are these inherited members grouped >>>>> in such a way that it makes it easy to ignore them once they get to "low" >>>>> in the stack? If it can't be per-module, is there a "nice" set of >>>>> ancestors >>>>> to avoid (as it seems this option takes such an argument). >>>>> >>>>> On Wed, Mar 31, 2021 at 4:23 PM Pablo Estrada <[email protected]> >>>>> wrote: >>>>> >>>>>> Do you have an example of what it would look like when released? >>>>>> >>>>>> On Wed, Mar 31, 2021 at 4:16 PM Brian Hulette <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> I'm working on generating useful API docs for the DataFrame API >>>>>>> (BEAM-12074). In doing so, one thing I've found would be very helpful >>>>>>> is if >>>>>>> we could include docstrings for inherited members in the API docs. That >>>>>>> way >>>>>>> docstrings for operations defined in DeferredDataFrameOrSeries [1], >>>>>>> will be >>>>>>> propagated to DeferredDataFrame [2] and DeferredSeries, and the former >>>>>>> can >>>>>>> be hidden entirely. This would be more consistent with the pandas >>>>>>> documentation [3]. >>>>>>> >>>>>>> It looks like we can do this by specifying :inherited-members: [4], >>>>>>> but this will apply to _all_ of our API docs, there doesn't seem to be a >>>>>>> way to restrict it to a particular module. This seems generally useful >>>>>>> to >>>>>>> me, but it would be a significant change, so I wanted to see if there >>>>>>> are >>>>>>> any objections from dev@ before doing this. >>>>>>> >>>>>>> An example of the kind of change this would produce: any PTransform >>>>>>> sub-classes, e.g. CombinePerKey [5], would now include docstrings for >>>>>>> every >>>>>>> PTransform member, e.g. with_input_types [6], and display_data [7]. >>>>>>> >>>>>>> Would there be any objections to that? >>>>>>> >>>>>>> Thanks, >>>>>>> Brian >>>>>>> >>>>>>> [1] >>>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.dataframe.frames.html#apache_beam.dataframe.frames.DeferredDataFrameOrSeries >>>>>>> [2] >>>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.dataframe.frames.html#apache_beam.dataframe.frames.DeferredDataFrame >>>>>>> [3] https://pandas.pydata.org/docs/reference/frame.html >>>>>>> [4] >>>>>>> https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html >>>>>>> [5] >>>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.transforms.core.html?highlight=combineperkey#apache_beam.transforms.core.CombinePerKey >>>>>>> [6] >>>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.transforms.ptransform.html#apache_beam.transforms.ptransform.PTransform.with_input_types >>>>>>> [7] >>>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.transforms.display.html#apache_beam.transforms.display.HasDisplayData.display_data >>>>>>> >>>>>>
