Re: [PySpark] Revisiting PySpark type annotations

I think the annotations are compatible with Python 2 since Maciej implemented them via stub files , which Python 2 simply ignores. Folks using mypy to check types will get the benefit whether they're on Python 2 or 3,

Re: Clean out https://dist.apache.org/repos/dist/dev/spark/ ?

https://issues.apache.org/jira/browse/SPARK-26731 On Fri, Jan 25, 2019 at 10:27 AM shane knapp wrote: > On Thu, Jan 24, 2019 at 6:13 PM Sean Owen wrote: > >> No and we could retire 2.2 now too but wouldn't hurt to keep it a bit >> longer in case we have to make a critical release even though

Re: Clean out https://dist.apache.org/repos/dist/dev/spark/ ?

On Thu, Jan 24, 2019 at 6:13 PM Sean Owen wrote: > No and we could retire 2.2 now too but wouldn't hurt to keep it a bit > longer in case we have to make a critical release even though it's EOL. > btw, the 2.2 jobs have been disabled (though not yet deleted from jenkins):

Re: [PySpark] Revisiting PySpark type annotations

If we can make the annotation compatible with Python 2, why don’t we add type annotation to make life easier for users of Python 3 (with type)? On Fri, Jan 25, 2019 at 7:53 AM Maciej Szymkiewicz wrote: > > Hello everyone, > > I'd like to revisit the topic of adding PySpark type annotations in

[PySpark] Revisiting PySpark type annotations

Hello everyone, I'd like to revisit the topic of adding PySpark type annotations in 3.0. It has been discussed before ( http://apache-spark-developers-list.1001551.n3.nabble.com/Python-friendly-API-for-Spark-3-0-td25016.html and

Re: Custom Window Function

Hi, https://stackoverflow.com/questions/32100973/how-to-define-and-use-a-user-defined-aggregate-function-in-spark-sql has a good overview and the best sample I have found so far. (besides spark source code). Best, Georg Am Mi., 23. Jan. 2019 um 17:16 Uhr schrieb Georg Heiler <