[
https://issues.apache.org/jira/browse/PIG-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457499#comment-13457499
]
Cheolsoo Park commented on PIG-2904:
------------------------------------
Given that the goal of this jira is to support function closures in scripting
UDFs, here is how I'd like to implement this feature. (This is basically my
understanding of Julien's proposal in PIG-928.)
Let's say we have a Python UDF 'logn' as follows:
{code:title=PyScript.py}
def logn(base):
def log(x):
return math.log(x, base)
return log
{code}
Now in Pig script, I can define two aliases 'log2' and 'log10' as follows:
{code:title=PigScript.pig}
REGISTER PyScript.py USING jython AS py;
DEFINE log2 py.logn('2');
DEFINE log10 py.logn('10');
...
log_2 = FOREACH in GENERATE log2($0); // log2($0) => math.log($0, 2)
log_10 = FOREACH in GENERATE log10($0); // log10($0) => math.log($0, 10)
{code}
To achieve that, I believe that the following work is required (Python only for
now):
- Add a non-default constructor with varargs to JythonFunction: e.g.
JythonFunction(String filename, String functionName, String...ctorargs)
- Update LogicalPlanGenerater/LogicalPlanBuilder so that FuncSpecs for
scripting UDFs are built from DEFINE statements: currently, they are built
directly from REGISTER statements using the default constructor, so it is not
possible to pass any constructor parameters to them via DEFINE statements.
- Update the exec() method of JythonFunction so that if a function closure is
given (i.e. the list of constructor args is not null), we first invoke the
function (e.g. logn) to get the 2nd function (e.g. log2), and then invoke the
2nd function with parameters.
I am sure that there will be many more details that I need to sort out, but I
believe that this is doable at least for Python. Once I complete implementing
function closure support in Python, I am going to add it to other supported
scripting languages too.
Please let me know if you have any concerns/questions.
Thanks!
> Scripting UDFs should allow DEFINE statements to pass parameters to the UDF's
> constructor
> -----------------------------------------------------------------------------------------
>
> Key: PIG-2904
> URL: https://issues.apache.org/jira/browse/PIG-2904
> Project: Pig
> Issue Type: New Feature
> Reporter: Julien Le Dem
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
