https://github.com/DataSystemsLab/GeoSpark should be public.
Am Do., 22. Aug. 2019 um 09:50 Uhr schrieb Arwin Tio <arwin....@hotmail.com >: > Hey, > > It seems like the GeoSpark repo is not publicly accessible? > > But from the filepath it seems like the Spark codebase itself was forked > or modified. > > The examples that I've seen seem to suggest that you need to register > custom Spark-Native functions inside Spark's private namespace like you > said (FunctionRegistry.scala I believe). > > I was wondering if it was possible to add the more efficient Spark-Native > functions in my user application without having to fork or modify Spark > itself. > > Thanks, > > Arwin > > From: Georg Heiler > Sent: Wednesday, August 21, 11:18 PM > Subject: Re: Creating custom Spark-Native catalyst/codegen functions > To: Arwin Tio > Cc: user@spark.apache.org > > > Look at > https://github.com/DataSystemsLab/GeoSpark/tree/master/sql/src/main/scala/org/apache/spark/sql/geospark > sql for an example. > > > Using custom function registration and functions residing inside sparks > private namespace should work. > > But I am not aware of a public user facing API. > Is there any I am missing? > > > Arwin Tio < arwin....@hotmail.com> schrieb am Do. 22. Aug. 2019 04:28: > > Hi friends, > > I am looking into converting some UDFs/UDAFs to Spark-Native functions to > leverage Catalyst and codegen. > > Looking through some examples (for example: > https://github.com/apache/spark/pull/7214/files for Levenshtein) it seems > like we need to add these functions to the Spark framework itself. > > Is there a way to add custom Spark-Native functions in "userspace"? > > Thank you! > > Arwin > > > >
