zero323 commented on pull request #29879: URL: https://github.com/apache/spark/pull/29879#issuecomment-704288127
> Hmm, but technically `python/join.pyi` and `python/rddsampler.pyi` are not documented so far. I believe they are for internal purposes. I got that many undocumented stuffs can be useful (also in Scala or Java sides) but I would prefer to hide it unless we explicitly document and expose. Fair point. If you feel strong about these, I'll be happy to remove them. But I think we need a bit precise criteria of inclusion / exclusion in a long run. In general the most important factors are: - Stability, which influences cost of maintenance. - Precision of annotations, which influences potential usefulness (I avoided annotating many parts of the "internal" API, primarily because possible annotations would be far to generic to be useful). - Likelihood that given part of the API will be used by the end users. - Direct cost of not including (amount of ignores required for things to type check now, and possibly in the future, if we decide to switch to inline variants). (I am primarily thinking about SPARK-33003 here) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
