Hello Barry,

I assume that by "we don't need another installation of Flink to manage the
stateful functions." You mean that you already have a running Flink cluster
and you would like to submit an additional Flink Job that executes a
Stateful functions application?

Then perhaps just try to submit [1] to the flink cluster. In addition you
would have to make the module.yaml available in the class path.
You can simply place it under your flink distribution /lib directory, or
alternatively:
1. create a jar-with-dependencies (uber jar / fat jar) that has [1] as a
dependency,
2. Add  the module.yaml definition under src/resources.
3. Make sure that the main class will
be org.apache.flink.statefun.flink.core.StatefulFunctionsJob
You should be able to submit it either the web interface, or by running:

./bin/flink run -c
org.apache.flink.statefun.flink.core.StatefulFunctionsJob
./statefun-example.jar

If this approach doesn't work for you, let me know and we will figure out
the DataStream integration approach.

All the best,
Igal.

[1]
https://mvnrepository.com/artifact/org.apache.flink/statefun-flink-distribution/3.1.0



On Thu, Sep 9, 2021 at 5:22 PM Barry Higgins <[email protected]>
wrote:

> Hi,
> I'm looing at using the DataStream API from a Flink application against a
> remote python stateful function deployed on another machine. I would like
> to investigate how feasible it is to have all of the state management being
> handled from the calling side meaning that we don't need another
> installation of Flink to manage the stateful functions.
>
> Unfortunately the example referenced in the documentation:
> https://ci.apache.org/projects/flink/flink-statefun-docs-release-3.0/docs/sdk/flink-datastream/
>
> is no longer in existence:
>
> https://github.com/apache/flink-statefun/blob/master/statefun-examples/statefun-flink-datastream-example/src/main/java/org/apache/flink/statefun/examples/datastream/Example.java
>
> There is an older version that is available here:
>
> https://github.com/apache/flink-statefun/tree/release-2.2/statefun-examples/statefun-flink-datastream-example
>
> and I have tried to work with this without much success
>
> The calling element of the code looks as follows:
>
>    StatefulFunctionEgressStreams out =
>         StatefulFunctionDataStreamBuilder.builder("example")
>             .withDataStreamAsIngress(names)
>             .withFunctionProvider(GREET, unused -> new MyFunction())
>             .withRequestReplyRemoteFunction(
>                 requestReplyFunctionBuilder(
>                         REMOTE_GREET, URI.create("
> http://localhost:5000/statefun";))
>                     .withPersistedState("seen_count")
>                     .withMaxRequestDuration(Duration.ofSeconds(15))
>                     .withMaxNumBatchRequests(500))
>             .withEgressId(GREETINGS)
>             .withConfiguration(statefunConfig)
>             .build(env);
>
> with a reference to a FunctionProvider that exists as an inner class in
> the same class. We would like this to be a remote call, where I guess I
> would replace http://localhost:5000/statefun with the remote address of
> the SF.
> However when I do make such a change the code is still referring to the
> inner function and any changes to the local MyFunction class are returned
> regardless of what is deployed remotely.
>
> If anyone has a working example of how to interact via DataStreams with a
> remotely deployed SF, I would be very grateful. I would be very happy to
> update the documentation if I can get this working.
> Cheers,
> Barry
>
>

Reply via email to