Yeah I think this could be a great project now that we're only Python 3.5+. One potential is making this an Outreachy project to get more folks from different backgrounds involved in Spark.
On Tue, Jul 21, 2020 at 12:33 PM Driesprong, Fokko <fo...@driesprong.frl> wrote: > Since we've recently dropped support for Python <=3.5 > <https://github.com/apache/spark/pull/28957>, I think it would be nice to > add support for type annotations. Having this in the main repository allows > us to do type checking using MyPy <http://mypy-lang.org/> in the CI > itself. <http://mypy-lang.org/> > > This is now handled by the Stub file: > https://www.python.org/dev/peps/pep-0484/#stub-files However I think it > is nicer to integrate the types with the code itself to keep everything in > sync, and make it easier for the people who work on the codebase itself. A > first step would be to move the stubs into the codebase. First step would > be to cover the public API which is the most important one. Having the > types with the code itself makes it much easier to understand. For example, > if you can supply a str or column here: > https://github.com/apache/spark/pull/29122/files#diff-f5295f69bfbdbf6e161aed54057ea36dR2486 > > One of the implications would be that future PR's on Python should cover > annotations on the public API's. Curious what the rest of the community > thinks. > > Cheers, Fokko > > > > > > > > > > Op di 21 jul. 2020 om 20:04 schreef zero323 <mszymkiew...@gmail.com>: > >> Given a discussion related to SPARK-32320 PR >> <https://github.com/apache/spark/pull/29122> I'd like to resurrect this >> thread. Is there any interest in migrating annotations to the main >> repository? >> >> >> >> -- >> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >> >> -- Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
