Yes, that's what I was looking for. Thanks!
On Mon, Nov 21, 2016 at 6:56 PM, Michael Armbrust
wrote:
> You are looking for org.apache.spark.sql.functions.expr()
>
> On Sat, Nov 19, 2016 at 6:12 PM, Stuart White
> wrote:
>>
>> I'd like to allow for runtime-configured Column expressions in my
>>
You are looking for org.apache.spark.sql.functions.expr()
On Sat, Nov 19, 2016 at 6:12 PM, Stuart White
wrote:
> I'd like to allow for runtime-configured Column expressions in my
> Spark SQL application. For example, if my application needs a 5-digit
> zip code, but the file I'm processing cont
Are you looking for UDFs?
https://jaceklaskowski.gitbooks.io/mastering-apache-spark/content/spark-sql-udfs.html
On Sun, Nov 20, 2016 at 3:12 AM Stuart White
wrote:
> I'd like to allow for runtime-configured Column expressions in my
> Spark SQL application. For example, if my application needs
I'd like to allow for runtime-configured Column expressions in my
Spark SQL application. For example, if my application needs a 5-digit
zip code, but the file I'm processing contains a 9-digit zip code, I'd
like to be able to configure my application with the expression
"substring('zipCode, 0, 5)"