I used to, before each release during the RC phase, go through every single doc page to make sure we don’t unintentionally leave things public. I no longer have time to do that unfortunately. I find that very useful because I always catch some mistakes through organic development.
> On Nov 13, 2018, at 8:00 PM, Wenchen Fan <cloud0...@gmail.com> wrote: > > > Could you clarify what you mean here? Mima has some known limitations such > > as not handling "private[blah]" very well > > Yes that's what I mean. > > What I want to know here is, which classes/methods we expect them to be > private. I think things marked as "private[blabla]" are expected to be > private for sure, it's just the MiMa and doc generator can't handle it well. > We can fix them later, by using the @Private annotation probably. > > > seems like it's tracked by a bunch of exclusions in the Unidoc object > > That's good. At least we have a clear definition about which packages are > meant to be private. We should make it consistent between MiMa and doc > generator though. > >> On Wed, Nov 14, 2018 at 10:41 AM Marcelo Vanzin <van...@cloudera.com> wrote: >> On Tue, Nov 13, 2018 at 6:26 PM Wenchen Fan <cloud0...@gmail.com> wrote: >> > Recently I updated the MiMa exclusion rules, and found MiMa tracks some >> > private classes/methods unexpectedly. >> >> Could you clarify what you mean here? Mima has some known limitations >> such as not handling "private[blah]" very well (because that means >> public in Java). Spark has (had?) this tool to generate an exclusions >> file for Mima, but not sure how up-to-date it is. >> >> > AFAIK, we have several rules: >> > 1. everything which is really private that end users can't access, e.g. >> > package private classes, private methods, etc. >> > 2. classes under certain packages. I don't know if we have a list, the >> > catalyst package is considered as a private package. >> > 3. everything which has a @Private annotation. >> >> That's my understanding of the scope of the rules. >> >> (2) to me means "things that show up in the public API docs". That's, >> AFAIK, tracked in SparkBuild.scala; seems like it's tracked by a bunch >> of exclusions in the Unidoc object (I remember that being different in >> the past). >> >> (3) might be a limitation of the doc generation tool? Not sure if it's >> easy to say "do not document classes that have @Private". At the very >> least, that annotation seems to be missing the "@Documented" >> annotation, which would make that info present in the javadoc. I do >> not know if the scala doc tool handles that. >> >> -- >> Marcelo
