Hi Herman,
That’s great and thanks for quick reply. The JIRA has an example of transform
and refers to a presto doc with lots of functions. Do you know which functions
will be supported. I am interested in using filter for example.
Cheers
Mick
> On 19 Apr 2018, at 10:46, Herman van Hövell
Hi,
Regarding higher order functions
> Yes, we intend to contribute this to open source.
It doesn't look like this is in 2.3.0, at least I can't find it.
Do you know when it might reach open source.
Thanks
Mick
--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
Hello,
At Netflix's algorithm team, we work on ranking problems a lot where
we naturally deal with the dataset with nested list of the structs. We
built Scala APIs like map, filter, drop, withColumn that can work on
the nested list of structs efficiently using SQL expression with
codegen.
Here
Good news :) Thx Sameer.
On Friday, June 9, 2017, Sameer Agarwal wrote:
> * As a heavy user of complex data types I was wondering if there was
>> any plan to push those changes upstream?
>>
>
> Yes, we intend to contribute this to open source.
>
>
>> * In addition, I was
>
> * As a heavy user of complex data types I was wondering if there was
> any plan to push those changes upstream?
>
Yes, we intend to contribute this to open source.
> * In addition, I was wondering if as part of this change it also tries
> to solve the column pruning / filter pushdown issues
+1 for the question
2017-06-07 19:50 GMT+02:00 Antoine HOM :
> Hey guys,
>
> Databricks released higher order functions as part of their runtime
> 3.0 beta (https://databricks.com/blog/2017/05/24/working-with-
>