Re: Flink SQL Async UDF

[Aitozi]

Hi Giannis:
I think this is due to the User Defined AsyncTableFunction have
not been supported yet. It has a little different in type inference.
I just opened a thread discuss about supporting this feature, you can
refer to: https://lists.apache.org/thread/7vk1799ryvrz4lsm5254q64ctm89mx2l

Thanks,
Aitozi.

Giannis Polyzos  于2023年5月8日周一 15:04写道：
>
> What im curious is about the completable future in the function signature 
> (eval)
> public final void eval(CompletableFuture> future, 
> Object... keys)
>
> is is injected automatically?
>
> Best
>
> On Mon, May 8, 2023 at 10:02 AM Giannis Polyzos  wrote:
>>
>> Hi Biao,
>> yeah, this means that since it's a Table Function UDF, I need to combine it 
>> with lateral correct?
>> I have tried both approaches to be honest. You can find attached the log 
>> output with verbose=true for both queries
>>
>> On Mon, May 8, 2023 at 9:49 AM Biao Geng  wrote:
>>>
>>> Hi Giannis,
>>> Would you mind executing SET 'sql-client.verbose' = 'true';. first in your 
>>> SQL client and then run the above example to show the error output in the 
>>> SQL client?
>>> For the complete example, you can refer to codes of 
>>> HBaseRowDataAsyncLookupFunction.
>>> Besides, in your example, you actually implement an async user-defiend 
>>> table function(see here for more details about UDTF usage), not a scalar 
>>> UDF. The usage could be different.
>>>
>>> Best,
>>> Biao Geng
>>>
>>> Giannis Polyzos  于2023年5月8日周一 14:27写道：

 just to illustrate with a simple example

 On Mon, May 8, 2023 at 7:20 AM Giannis Polyzos  
 wrote:
>
> Yes I have and the function is present in the catalog.
> And I have other udfs also registered, I’m just having issues with the 
> Async one and can’t really find any examples.
>
> Best
>
> On Mon, 8 May 2023 at 3:46 AM, Shammon FY  wrote:
>>
>> Hi Giannis,
>>
>> Have you use "CREATE FUNCTION asyncfn AS 'Your full class name of async 
>> function class'" or "CREATE TEMPORARY FUNCTION asyncfn AS 'Your full 
>> class name of async function class'" to create a customized function 
>> named "asyncfn" before it is used in your sql?
>>
>> The error message "No match found for function signature" usually 
>> indicates that the function does not exist or the parameters do not 
>> match.
>>
>> Best,
>> Shammon FY
>>
>> On Sun, May 7, 2023 at 2:55 PM Giannis Polyzos  
>> wrote:
>>>
>>> I can't really find any examples / docs for Flink's AsyncTableFunction 
>>> and I have a hard time getting it to work.
>>> Is there any example you can share that just takes as input a String 
>>> key and outputs lets say a record (returned by the lookup?)
>>> Also is not clear to me how the async happens internally.
>>> Is the future in the eval method signature used?
>>>
>>> I tried implementing eval methods like:
>>> public final void eval(CompletableFuture> future, 
>>> Object... keys)
>>>
>>> or
>>>
>>> public void eval(CompletableFuture> result, String 
>>> rowkey)
>>>
>>> but in both cases if I do something like
>>> SELECT asyncfn(accountId) from transactions;
>>> I get
>>> org.apache.calcite.sql.validate.SqlValidatorException: No match found 
>>> for function signature asyncfn()
>>>
>>> Not sure what I am missing
>>>
>>> Thanks,
>>> Giannis

Re: Flink SQL Async UDF

[Giannis Polyzos]

What im curious is about the completable future in the function signature
(eval)
public final void eval(CompletableFuture> future,
Object... keys)

is is injected automatically?

Best

On Mon, May 8, 2023 at 10:02 AM Giannis Polyzos 
wrote:

> Hi Biao,
> yeah, this means that since it's a Table Function UDF, I need to combine
> it with lateral correct?
> I have tried both approaches to be honest. You can find attached the log
> output with verbose=true for both queries
>
> On Mon, May 8, 2023 at 9:49 AM Biao Geng  wrote:
>
>> Hi Giannis,
>> Would you mind executing SET 'sql-client.verbose' = 'true';. first in
>> your SQL client and then run the above example to show the error output in
>> the SQL client?
>> For the complete example, you can refer to codes of
>> HBaseRowDataAsyncLookupFunction.
>> Besides, in your example, you actually implement an async user-defiend
>> table function(see here
>> 
>>  for
>> more details about UDTF usage), not a scalar UDF. The usage could be
>> different.
>>
>> Best,
>> Biao Geng
>>
>> Giannis Polyzos  于2023年5月8日周一 14:27写道：
>>
>>> just to illustrate with a simple example
>>>
>>> On Mon, May 8, 2023 at 7:20 AM Giannis Polyzos 
>>> wrote:
>>>
 Yes I have and the function is present in the catalog.
 And I have other udfs also registered, I’m just having issues with the
 Async one and can’t really find any examples.

 Best

 On Mon, 8 May 2023 at 3:46 AM, Shammon FY  wrote:

> Hi Giannis,
>
> Have you use "CREATE FUNCTION asyncfn AS 'Your full class name of
> async function class'" or "CREATE TEMPORARY FUNCTION asyncfn AS 'Your full
> class name of async function class'" to create a customized function named
> "asyncfn" before it is used in your sql?
>
> The error message "No match found for function signature" usually
> indicates that the function does not exist or the parameters do not match.
>
> Best,
> Shammon FY
>
> On Sun, May 7, 2023 at 2:55 PM Giannis Polyzos 
> wrote:
>
>> I can't really find any examples / docs for Flink's
>> AsyncTableFunction and I have a hard time getting it to work.
>> Is there any example you can share that just takes as input a String
>> key and outputs lets say a record (returned by the lookup?)
>> Also is not clear to me how the async happens internally.
>> Is the future in the eval method signature used?
>>
>> I tried implementing eval methods like:
>> *public final void eval(CompletableFuture>
>> future, Object... keys)*
>>
>> *or *
>>
>>
>> *public void eval(CompletableFuture> result, String
>> rowkey)*
>>
>> but in both cases if I do something like
>> *SELECT asyncfn(accountId) from transactions;*
>> I get
>> *org.apache.calcite.sql.validate.SqlValidatorException: No match
>> found for function signature asyncfn()*
>>
>> Not sure what I am missing
>>
>> Thanks,
>> Giannis
>>
>

Re: Flink SQL Async UDF

[Giannis Polyzos]

Hi Biao,
yeah, this means that since it's a Table Function UDF, I need to combine it
with lateral correct?
I have tried both approaches to be honest. You can find attached the log
output with verbose=true for both queries

On Mon, May 8, 2023 at 9:49 AM Biao Geng  wrote:

> Hi Giannis,
> Would you mind executing SET 'sql-client.verbose' = 'true';. first in
> your SQL client and then run the above example to show the error output in
> the SQL client?
> For the complete example, you can refer to codes of
> HBaseRowDataAsyncLookupFunction.
> Besides, in your example, you actually implement an async user-defiend
> table function(see here
> 
>  for
> more details about UDTF usage), not a scalar UDF. The usage could be
> different.
>
> Best,
> Biao Geng
>
> Giannis Polyzos  于2023年5月8日周一 14:27写道：
>
>> just to illustrate with a simple example
>>
>> On Mon, May 8, 2023 at 7:20 AM Giannis Polyzos 
>> wrote:
>>
>>> Yes I have and the function is present in the catalog.
>>> And I have other udfs also registered, I’m just having issues with the
>>> Async one and can’t really find any examples.
>>>
>>> Best
>>>
>>> On Mon, 8 May 2023 at 3:46 AM, Shammon FY  wrote:
>>>
 Hi Giannis,

 Have you use "CREATE FUNCTION asyncfn AS 'Your full class name of async
 function class'" or "CREATE TEMPORARY FUNCTION asyncfn AS 'Your full class
 name of async function class'" to create a customized function named
 "asyncfn" before it is used in your sql?

 The error message "No match found for function signature" usually
 indicates that the function does not exist or the parameters do not match.

 Best,
 Shammon FY

 On Sun, May 7, 2023 at 2:55 PM Giannis Polyzos 
 wrote:

> I can't really find any examples / docs for Flink's AsyncTableFunction
> and I have a hard time getting it to work.
> Is there any example you can share that just takes as input a String
> key and outputs lets say a record (returned by the lookup?)
> Also is not clear to me how the async happens internally.
> Is the future in the eval method signature used?
>
> I tried implementing eval methods like:
> *public final void eval(CompletableFuture> future,
> Object... keys)*
>
> *or *
>
>
> *public void eval(CompletableFuture> result, String
> rowkey)*
>
> but in both cases if I do something like
> *SELECT asyncfn(accountId) from transactions;*
> I get
> *org.apache.calcite.sql.validate.SqlValidatorException: No match found
> for function signature asyncfn()*
>
> Not sure what I am missing
>
> Thanks,
> Giannis
>

Flink SQL> SELECT customasync(accountId) from transactions;
[ERROR] Could not execute SQL statement. Reason:
org.apache.flink.runtime.rest.util.RestClientException: [Internal server 
error., )
at 
org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:187)
at 
org.apache.flink.table.planner.calcite.FlinkPlannerImpl.validate(FlinkPlannerImpl.scala:113)
at 
org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:281)
at 
org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:106)
at 
org.apache.flink.table.gateway.service.operation.OperationExecutor.executeStatement(OperationExecutor.java:191)
at 
org.apache.flink.table.gateway.service.SqlGatewayServiceImpl.lambda$executeStatement$1(SqlGatewayServiceImpl.java:212)
at 
org.apache.flink.table.gateway.service.operation.OperationManager.lambda$submitOperation$1(OperationManager.java:119)
at 
org.apache.flink.table.gateway.service.operation.OperationManager$Operation.lambda$run$0(OperationManager.java:258)
... 7 more
Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1, 
column 8 to line 1, column 29: No match found for function signature 
customasync()
at 
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
 Method)
at 
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown
 Source)
at 
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
 Source)
at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
at 
org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:505)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:932)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:917)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:5163)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1949)

Re: Flink SQL Async UDF

[Biao Geng]

Hi Giannis,
Would you mind executing SET 'sql-client.verbose' = 'true';. first in your
SQL client and then run the above example to show the error output in the
SQL client?
For the complete example, you can refer to codes of
HBaseRowDataAsyncLookupFunction.
Besides, in your example, you actually implement an async user-defiend
table function(see here

for
more details about UDTF usage), not a scalar UDF. The usage could be
different.

Best,
Biao Geng

Giannis Polyzos  于2023年5月8日周一 14:27写道：

> just to illustrate with a simple example
>
> On Mon, May 8, 2023 at 7:20 AM Giannis Polyzos 
> wrote:
>
>> Yes I have and the function is present in the catalog.
>> And I have other udfs also registered, I’m just having issues with the
>> Async one and can’t really find any examples.
>>
>> Best
>>
>> On Mon, 8 May 2023 at 3:46 AM, Shammon FY  wrote:
>>
>>> Hi Giannis,
>>>
>>> Have you use "CREATE FUNCTION asyncfn AS 'Your full class name of async
>>> function class'" or "CREATE TEMPORARY FUNCTION asyncfn AS 'Your full class
>>> name of async function class'" to create a customized function named
>>> "asyncfn" before it is used in your sql?
>>>
>>> The error message "No match found for function signature" usually
>>> indicates that the function does not exist or the parameters do not match.
>>>
>>> Best,
>>> Shammon FY
>>>
>>> On Sun, May 7, 2023 at 2:55 PM Giannis Polyzos 
>>> wrote:
>>>
 I can't really find any examples / docs for Flink's AsyncTableFunction
 and I have a hard time getting it to work.
 Is there any example you can share that just takes as input a String
 key and outputs lets say a record (returned by the lookup?)
 Also is not clear to me how the async happens internally.
 Is the future in the eval method signature used?

 I tried implementing eval methods like:
 *public final void eval(CompletableFuture> future,
 Object... keys)*

 *or *


 *public void eval(CompletableFuture> result, String
 rowkey)*

 but in both cases if I do something like
 *SELECT asyncfn(accountId) from transactions;*
 I get
 *org.apache.calcite.sql.validate.SqlValidatorException: No match found
 for function signature asyncfn()*

 Not sure what I am missing

 Thanks,
 Giannis

>>>

Re: Flink SQL Async UDF

[Giannis Polyzos]

Yes I have and the function is present in the catalog.
And I have other udfs also registered, I’m just having issues with the
Async one and can’t really find any examples.

Best

On Mon, 8 May 2023 at 3:46 AM, Shammon FY  wrote:

> Hi Giannis,
>
> Have you use "CREATE FUNCTION asyncfn AS 'Your full class name of async
> function class'" or "CREATE TEMPORARY FUNCTION asyncfn AS 'Your full class
> name of async function class'" to create a customized function named
> "asyncfn" before it is used in your sql?
>
> The error message "No match found for function signature" usually
> indicates that the function does not exist or the parameters do not match.
>
> Best,
> Shammon FY
>
> On Sun, May 7, 2023 at 2:55 PM Giannis Polyzos 
> wrote:
>
>> I can't really find any examples / docs for Flink's AsyncTableFunction
>> and I have a hard time getting it to work.
>> Is there any example you can share that just takes as input a String key
>> and outputs lets say a record (returned by the lookup?)
>> Also is not clear to me how the async happens internally.
>> Is the future in the eval method signature used?
>>
>> I tried implementing eval methods like:
>> *public final void eval(CompletableFuture> future,
>> Object... keys)*
>>
>> *or *
>>
>>
>> *public void eval(CompletableFuture> result, String
>> rowkey)*
>>
>> but in both cases if I do something like
>> *SELECT asyncfn(accountId) from transactions;*
>> I get
>> *org.apache.calcite.sql.validate.SqlValidatorException: No match found
>> for function signature asyncfn()*
>>
>> Not sure what I am missing
>>
>> Thanks,
>> Giannis
>>
>

Re: Flink SQL Async UDF

[Shammon FY]

Hi Giannis,

Have you use "CREATE FUNCTION asyncfn AS 'Your full class name of async
function class'" or "CREATE TEMPORARY FUNCTION asyncfn AS 'Your full class
name of async function class'" to create a customized function named
"asyncfn" before it is used in your sql?

The error message "No match found for function signature" usually indicates
that the function does not exist or the parameters do not match.

Best,
Shammon FY

On Sun, May 7, 2023 at 2:55 PM Giannis Polyzos 
wrote:

> I can't really find any examples / docs for Flink's AsyncTableFunction and
> I have a hard time getting it to work.
> Is there any example you can share that just takes as input a String key
> and outputs lets say a record (returned by the lookup?)
> Also is not clear to me how the async happens internally.
> Is the future in the eval method signature used?
>
> I tried implementing eval methods like:
> *public final void eval(CompletableFuture> future,
> Object... keys)*
>
> *or *
>
>
> *public void eval(CompletableFuture> result, String
> rowkey)*
>
> but in both cases if I do something like
> *SELECT asyncfn(accountId) from transactions;*
> I get
> *org.apache.calcite.sql.validate.SqlValidatorException: No match found for
> function signature asyncfn()*
>
> Not sure what I am missing
>
> Thanks,
> Giannis
>

Flink SQL Async UDF

[Giannis Polyzos]

I can't really find any examples / docs for Flink's AsyncTableFunction and
I have a hard time getting it to work.
Is there any example you can share that just takes as input a String key
and outputs lets say a record (returned by the lookup?)
Also is not clear to me how the async happens internally.
Is the future in the eval method signature used?

I tried implementing eval methods like:
*public final void eval(CompletableFuture> future,
Object... keys)*

*or *


*public void eval(CompletableFuture> result, String rowkey)*

but in both cases if I do something like
*SELECT asyncfn(accountId) from transactions;*
I get
*org.apache.calcite.sql.validate.SqlValidatorException: No match found for
function signature asyncfn()*

Not sure what I am missing

Thanks,
Giannis