kennknowles opened a new issue, #19484:
URL: https://github.com/apache/beam/issues/19484
The main purpose of this JIRA is Turn `WindowedValue<T\>` into `T` for all
the interfaces of Data Plane.
The interface definitions for sending and receiving data in the code as
follows:
- org.apache.beam.runners.fnexecution.data.FnDataService
```
public interface FnDataService {
<T> InboundDataClient receive(LogicalEndpoint inputLocation,
Coder<WindowedValue<T>>
coder, FnDataReceiver<WindowedValue<T>> listener);
<T> CloseableFnDataReceiver<WindowedValue<T>>
send(
LogicalEndpoint outputLocation, Coder<WindowedValue<T>> coder);
}
```
- org.apache.beam.fn.harness.data.BeamFnDataClient
```
public interface BeamFnDataClient {
<T> InboundDataClient receive(ApiServiceDescriptor apiServiceDescriptor,
LogicalEndpoint inputLocation, Coder<WindowedValue<T>> coder,
FnDataReceiver<WindowedValue<T>> receiver);
<T> CloseableFnDataReceiver<WindowedValue<T>> send(BeamFnDataGrpcClient
Endpoints.ApiServiceDescriptor
apiServiceDescriptor, LogicalEndpoint outputLocation,
Coder<WindowedValue<T>> coder);
}
```
Both `Coder<WindowedValue<T\>\>` and `FnDataReceiver<WindowedValue<T\>\>`
use `WindowedValue` as the data structure that both sides of Runner and SDK
Harness know each other. Control Plane/Data Plane/State Plane/Logging is a
highly abstraction, such as Control Plane and Logging, these are common
requirements for all multi-language platforms. For example, the Flink community
is also discussing how to support Python UDF, as well as how to deal with
docker environment. how to data transfer, how to state access, how to logging
etc. If Beam can further abstract these service interfaces, i.e., interface
definitions are compatible with multiple engines, and finally provided to other
projects in the form of class libraries, it definitely will help other
platforms that want to support multiple languages. So we can further abstract
the interface definition of FnDataService's BeamFnDataClient, take the
FnDataService#receive interface as an example, and turn `WindowedValue<T\>`
into `T` so th
at other platforms can be extended arbitrarily, as follows:
```
<T> InboundDataClient receive(LogicalEndpoint inputLocation, Coder<T> coder,
FnDataReceiver<T>> listener);
```
For details, see the mail discussion list:
[https://lists.apache.org/[email protected]:lte=1M:%5BDISCUSS%5D%20Turn%20%60WindowedValue](https://lists.apache.org/[email protected]:lte=1M:%5BDISCUSS%5D%20Turn%20%60WindowedValue)
Imported from Jira
[BEAM-7120](https://issues.apache.org/jira/browse/BEAM-7120). Original Jira may
contain additional context.
Reported by: sunjincheng121.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
