[jira] [Commented] (FLINK-5792) Improve “UDF/UDTF" to support constructor with parameter.
[ https://issues.apache.org/jira/browse/FLINK-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871584#comment-15871584 ] sunjincheng commented on FLINK-5792: HI,[~clarkyzl] Thanks for your attention to this JIRA. After https://github.com/apache/flink/pull/3330 merge into master, I'll open the PR of FLINK-5794. > Improve “UDF/UDTF" to support constructor with parameter. > - > > Key: FLINK-5792 > URL: https://issues.apache.org/jira/browse/FLINK-5792 > Project: Flink > Issue Type: Improvement > Components: Table API & SQL >Reporter: sunjincheng >Assignee: sunjincheng > > Currently UDF/UDTF in the codegen phase using a nonparametric constructor to > create the instance, causing the user can not include the state value in the > UDF/UDTF. The UDF/UDTF's codegen phase can use a serialized mechanism so that > the UDTF can contain state values. > 1. UserDefinedFunction inherits Serializable. > 2. Modify CodeGenerator about UDF/UDTF part. > 3. Modify TableAPI about UDF/UDTF > 4. Add Test. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5792) Improve “UDF/UDTF" to support constructor with parameter.
[ https://issues.apache.org/jira/browse/FLINK-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871230#comment-15871230 ] Zhuoluo Yang commented on FLINK-5792: - Hi [~sunjincheng121], I think this feature is very important to FLINK-5802. Because we need to pass something (eg, Hive Udf ) to the Flink's UDF/UDTF. A serialization will be a good idea, IMHO. > Improve “UDF/UDTF" to support constructor with parameter. > - > > Key: FLINK-5792 > URL: https://issues.apache.org/jira/browse/FLINK-5792 > Project: Flink > Issue Type: Improvement > Components: Table API & SQL >Reporter: sunjincheng >Assignee: sunjincheng > > Currently UDF/UDTF in the codegen phase using a nonparametric constructor to > create the instance, causing the user can not include the state value in the > UDF/UDTF. The UDF/UDTF's codegen phase can use a serialized mechanism so that > the UDTF can contain state values. > 1. UserDefinedFunction inherits Serializable. > 2. Modify CodeGenerator about UDF/UDTF part. > 3. Modify TableAPI about UDF/UDTF > 4. Add Test. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5792) Improve “UDF/UDTF" to support constructor with parameter.
[ https://issues.apache.org/jira/browse/FLINK-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15869488#comment-15869488 ] sunjincheng commented on FLINK-5792: BYW, when I implement the issue, I had try to use `kryo` serialize&deserialize the UDTF/UDF object, It will make byte [] very small. But unfortunately, It asked the serialized member must have a zero-parameter constructor which not friendly to user. > Improve “UDF/UDTF" to support constructor with parameter. > - > > Key: FLINK-5792 > URL: https://issues.apache.org/jira/browse/FLINK-5792 > Project: Flink > Issue Type: Improvement > Components: Table API & SQL >Reporter: sunjincheng >Assignee: sunjincheng > > Currently UDF/UDTF in the codegen phase using a nonparametric constructor to > create the instance, causing the user can not include the state value in the > UDF/UDTF. The UDF/UDTF's codegen phase can use a serialized mechanism so that > the UDTF can contain state values. > 1. UserDefinedFunction inherits Serializable. > 2. Modify CodeGenerator about UDF/UDTF part. > 3. Modify TableAPI about UDF/UDTF > 4. Add Test. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5792) Improve “UDF/UDTF" to support constructor with parameter.
[ https://issues.apache.org/jira/browse/FLINK-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15869473#comment-15869473 ] sunjincheng commented on FLINK-5792: HI, [~fhueske] Thanks for your attention to this JIRA. Thanks [~jark] review the PR. I had serialize UDF object at code-gen stage and deserialize in the `open()` method. JackWu is right. the current implement is similar to the approach-2. What do you think about the current PR? > Improve “UDF/UDTF" to support constructor with parameter. > - > > Key: FLINK-5792 > URL: https://issues.apache.org/jira/browse/FLINK-5792 > Project: Flink > Issue Type: Improvement > Components: Table API & SQL >Reporter: sunjincheng >Assignee: sunjincheng > > Currently UDF/UDTF in the codegen phase using a nonparametric constructor to > create the instance, causing the user can not include the state value in the > UDF/UDTF. The UDF/UDTF's codegen phase can use a serialized mechanism so that > the UDTF can contain state values. > 1. UserDefinedFunction inherits Serializable. > 2. Modify CodeGenerator about UDF/UDTF part. > 3. Modify TableAPI about UDF/UDTF > 4. Add Test. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5792) Improve “UDF/UDTF" to support constructor with parameter.
[ https://issues.apache.org/jira/browse/FLINK-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15869413#comment-15869413 ] Fabian Hueske commented on FLINK-5792: -- Ah, great. Thanks for pointing to the PR [~jark]. > Improve “UDF/UDTF" to support constructor with parameter. > - > > Key: FLINK-5792 > URL: https://issues.apache.org/jira/browse/FLINK-5792 > Project: Flink > Issue Type: Improvement > Components: Table API & SQL >Reporter: sunjincheng >Assignee: sunjincheng > > Currently UDF/UDTF in the codegen phase using a nonparametric constructor to > create the instance, causing the user can not include the state value in the > UDF/UDTF. The UDF/UDTF's codegen phase can use a serialized mechanism so that > the UDTF can contain state values. > 1. UserDefinedFunction inherits Serializable. > 2. Modify CodeGenerator about UDF/UDTF part. > 3. Modify TableAPI about UDF/UDTF > 4. Add Test. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5792) Improve “UDF/UDTF" to support constructor with parameter.
[ https://issues.apache.org/jira/browse/FLINK-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15869404#comment-15869404 ] Jark Wu commented on FLINK-5792: Hi [~fhueske], [~sunjincheng121] has create a PR for this issue: https://github.com/apache/flink/pull/3330 Sun's way is similar to approach-2, which serialized the UDF object into a string. Maybe we can move the discussion under the PR. > Improve “UDF/UDTF" to support constructor with parameter. > - > > Key: FLINK-5792 > URL: https://issues.apache.org/jira/browse/FLINK-5792 > Project: Flink > Issue Type: Improvement > Components: Table API & SQL >Reporter: sunjincheng >Assignee: sunjincheng > > Currently UDF/UDTF in the codegen phase using a nonparametric constructor to > create the instance, causing the user can not include the state value in the > UDF/UDTF. The UDF/UDTF's codegen phase can use a serialized mechanism so that > the UDTF can contain state values. > 1. UserDefinedFunction inherits Serializable. > 2. Modify CodeGenerator about UDF/UDTF part. > 3. Modify TableAPI about UDF/UDTF > 4. Add Test. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5792) Improve “UDF/UDTF" to support constructor with parameter.
[ https://issues.apache.org/jira/browse/FLINK-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15869395#comment-15869395 ] Fabian Hueske commented on FLINK-5792: -- I agree, it makes sense to ship a serialized UDF object. Regular Flink function are also serialized and distributed to the workers. [~sunjincheng121] can you explain a bit how you want to distribute the UDF? I see two options: 1. make the UDF a member of wrapping function. It might be a bit tricky to pass the reference into the code-gen'd function. 2. add a final byte[] field into the code-gen'd function that holds the serialized UDF object and deserialize during initialization. This will blow up the code-gen'd string but might work well. Best, Fabian > Improve “UDF/UDTF" to support constructor with parameter. > - > > Key: FLINK-5792 > URL: https://issues.apache.org/jira/browse/FLINK-5792 > Project: Flink > Issue Type: Improvement > Components: Table API & SQL >Reporter: sunjincheng >Assignee: sunjincheng > > Currently UDF/UDTF in the codegen phase using a nonparametric constructor to > create the instance, causing the user can not include the state value in the > UDF/UDTF. The UDF/UDTF's codegen phase can use a serialized mechanism so that > the UDTF can contain state values. > 1. UserDefinedFunction inherits Serializable. > 2. Modify CodeGenerator about UDF/UDTF part. > 3. Modify TableAPI about UDF/UDTF > 4. Add Test. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5792) Improve “UDF/UDTF" to support constructor with parameter.
[ https://issues.apache.org/jira/browse/FLINK-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15869029#comment-15869029 ] Jark Wu commented on FLINK-5792: +1 The serialization way will be more flexible. > Improve “UDF/UDTF" to support constructor with parameter. > - > > Key: FLINK-5792 > URL: https://issues.apache.org/jira/browse/FLINK-5792 > Project: Flink > Issue Type: Improvement > Components: Table API & SQL >Reporter: sunjincheng >Assignee: sunjincheng > > Currently UDF/UDTF in the codegen phase using a nonparametric constructor to > create the instance, causing the user can not include the state value in the > UDF/UDTF. The UDF/UDTF's codegen phase can use a serialized mechanism so that > the UDTF can contain state values. > 1. UserDefinedFunction inherits Serializable. > 2. Modify CodeGenerator about UDF/UDTF part. > 3. Modify TableAPI about UDF/UDTF > 4. Add Test. -- This message was sent by Atlassian JIRA (v6.3.15#6346)