ASF GitHub Bot commented on S2GRAPH-225:

GitHub user elric-k opened a pull request:


    [S2GRAPH-225] support custom udf class

    I simply added udf trait that can be implemented to use custom UDF class.
    trait Udf extends Serializable with Logger {
      def register(ss: SparkSession, name:String, options:Map[String, String])
    If you implement the  *register()* method by inheriting the above udf 
trait, you can create custom UDF and use it in SQL by specifying the udf in the 
job description
    udfs: [
            name: udf_name
            class: org.apache.s2graph.s2jobs.udfs.MyUdfClass
            params: {
              "param": "dummy"

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/elric-k/incubator-s2graph S2GRAPH-225

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #185
commit d67adc5ecb43a4b9c5c9961c44330dd72250b058
Author: Chul Kang <elric@...>
Date:   2018-06-21T03:03:03Z

    add udfs option

commit 26321aca24afbfbf0f44bd197d91d017e06e1ac0
Author: Chul Kang <elric@...>
Date:   2018-06-21T05:33:30Z

    add grok udf

commit 0ae2cb2516264f37d07425bb7b59981aabd8600a
Author: Chul Kang <elric@...>
Date:   2018-07-24T05:41:14Z

    add schema option for grok udf

commit 34b46f6e4da9da793c5055ce38ed3b1202174eca
Author: Chul Kang <elric@...>
Date:   2018-07-27T09:50:38Z

    fix bug


> support custom udf class
> ------------------------
>                 Key: S2GRAPH-225
>                 URL: https://issues.apache.org/jira/browse/S2GRAPH-225
>             Project: S2Graph
>          Issue Type: Sub-task
>          Components: s2jobs
>            Reporter: Chul Kang
>            Assignee: Chul Kang
>            Priority: Minor
> We need to support custom UDFs that is available on SQL queries.
> UDFs allow enabling new functions in SQL by abstracting their lower level 
> language implementations. 
> Spark SQL offers integrating custom udf easily.
> I think it would be nice to provide an interface to register the custom UDFs 
> in the Job Description.

This message was sent by Atlassian JIRA

Reply via email to