This is an automated email from the ASF dual-hosted git repository.
pjfanning pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pekko-projection.git
The following commit(s) were added to refs/heads/main by this push:
new 01a3f11 add grpc docs (#468)
01a3f11 is described below
commit 01a3f11de046b30c1eef3ef9608ad4ec0378cc2a
Author: PJ Fanning <[email protected]>
AuthorDate: Fri May 8 10:49:58 2026 +0100
add grpc docs (#468)
* Add grpc.md with Pekko references and update index.md
Agent-Logs-Url:
https://github.com/pjfanning/incubator-pekko-projection/sessions/2e455f7c-97cc-447a-85d5-ad128cc892a5
Co-authored-by: pjfanning <[email protected]>
* Add grpc-overview.png image referenced by grpc.md
Agent-Logs-Url:
https://github.com/pjfanning/incubator-pekko-projection/sessions/d88fbdba-7808-42ef-ada7-15f05e7de296
Co-authored-by: pjfanning <[email protected]>
* refs
* remove mention of missing examples
* fix refs
---------
Co-authored-by: copilot-swe-agent[bot]
<[email protected]>
Co-authored-by: pjfanning <[email protected]>
---
docs/src/main/paradox/grpc.md | 156 +++++++++++++++++++++++++
docs/src/main/paradox/images/grpc-overview.png | Bin 0 -> 558448 bytes
docs/src/main/paradox/index.md | 1 +
project/project-info.conf | 4 +
4 files changed, 161 insertions(+)
diff --git a/docs/src/main/paradox/grpc.md b/docs/src/main/paradox/grpc.md
new file mode 100644
index 0000000..733a2be
--- /dev/null
+++ b/docs/src/main/paradox/grpc.md
@@ -0,0 +1,156 @@
+# Apache Pekko Projection gRPC
+
+Apache Pekko Projection gRPC can be used for implementing asynchronous event
based service-to-service communication.
+It provides an implementation of a Pekko Projection that uses
+[Pekko gRPC](https://pekko.apache.org/docs/pekko-grpc/current/index.html) as
underlying transport between event producer and consumer.
+
+@@@ warning
+
+This module is currently marked as [May
Change](https://pekko.apache.org/docs/pekko/current/common/may-change.html)
+in the sense that the API might be changed based on feedback from initial
usage.
+However, the module is ready for usage in production and we will not break
serialization format of
+messages or stored data.
+
+@@@
+
+## Overview
+
+
+
+1. An Entity stores events in its journal in service A.
+1. Consumer in service B starts a Pekko Projection which locally reads its
offset for service A's replication stream.
+1. Service B establishes a replication stream from service A.
+1. Events are read from the journal.
+1. Event is emitted to the replication stream.
+1. Event is handled.
+1. Offset is stored.
+1. Producer continues to read new events from the journal and emit to the
stream. As an optimization, events can also be published directly from the
entity to the producer.
+
+## Dependencies
+
+To use the gRPC module of Apache Pekko Projections add the following
dependency in your project:
+
+@@dependency [sbt,Maven,Gradle] {
+ group=org.apache.pekko
+ artifact=pekko-projection-grpc_$scala.binary.version$
+ version=$project.version$
+}
+
+Apache Pekko Projections require Pekko $pekko.version$ or later, see
@ref:[Pekko version](overview.md#pekko-version).
+
+@@project-info{ projectId="grpc" }
+
+
+### Transitive dependencies
+
+The table below shows `pekko-projection-grpc`'s direct dependencies, and the
second tab shows all libraries it depends on transitively.
+
+@@dependencies{ projectId="grpc" }
+
+## Consumer
+
+On the consumer side the `Projection` is an ordinary @ref:[SourceProvider for
eventsBySlices](eventsourced.md#sourceprovider-for-eventsbyslices)
+that is using `eventsBySlices` from the @apidoc[GrpcReadJournal].
+
+The Protobuf descriptors are defined when the @apidoc[GrpcReadJournal] is
created. The descriptors are used
+when deserializing the received events. @scala[The `protobufDescriptors` is a
list of the `javaDescriptor` for the used protobuf messages.
+It is defined in the ScalaPB generated `Proto` companion object.]
+Note that GrpcReadJournal should be created with the @apidoc[GrpcReadJournal$]
@scala[`apply`]@java[`create`] factory method
+and not from configuration via `GrpcReadJournalProvider` when using Protobuf
serialization.
+
+The gRPC connection to the producer is defined in the [consumer
configuration](#consumer-configuration).
+
+The
[R2dbcProjection](https://pekko.apache.org/docs/pekko-persistence-r2dbc/current/projection.html)
has support for storing the offset in a relational database using R2DBC.
+
+One approach is to use the
[ShardedDaemonProcess](https://pekko.apache.org/docs/pekko/current/typed/cluster-sharded-daemon-process.html)
to distribute the instances of the Projection across the cluster.
+There are alternative ways of running the `ProjectionBehavior` as described in
@ref:[Running a Projection](running.md)
+
+How to implement the `EventHandler` and choose between different processing
semantics is described in the [R2dbcProjection
documentation](https://pekko.apache.org/docs/pekko-persistence-r2dbc/current/projection.html).
+
+### gRPC client lifecycle
+
+When creating the @apidoc[GrpcReadJournal] a gRPC client is created for the
target producer. The same `GrpcReadJournal`
+instance and its gRPC client should be shared for the same target producer.
The code examples will share the instance
+between different Projection instances running in the same `ActorSystem`. The
gRPC clients will automatically be
+closed when the `ActorSystem` is terminated.
+
+If there is a need to close the gRPC client before `ActorSystem` termination
the `close()` method of the @apidoc[GrpcReadJournal]
+can be called. After closing the `GrpcReadJournal` instance cannot be used
again.
+
+## Producer
+
+Apache Pekko Projections gRPC provides the gRPC service implementation that is
used by the consumer side. It is created with the @apidoc[EventProducer$].
+
+Events can be transformed by application specific code on the producer side.
The purpose is to be able to have a
+different public representation from the internal representation (stored in
journal). The transformation functions
+are registered when creating the `EventProducer` service.
+
+To omit an event the transformation function can return
@scala[`None`]@java[`Optional.empty()`].
+
+That `EventProducer` service is started in a Pekko gRPC server.
+
+## Access control
+
+### From the consumer
+
+The consumer can pass metadata, such as auth headers, in each request to the
producer service by passing @apidoc[pekko.grpc.*.Metadata] to the
@apidoc[GrpcQuerySettings] when constructing the read journal.
+
+### In the producer
+
+Authentication and authorization for the producer can be done by implementing
a @apidoc[EventProducerInterceptor] and pass
+it to the `grpcServiceHandler` method during producer bootstrap. The
interceptor is invoked with the stream id and
+gRPC request metadata for each incoming request and can return a suitable
error through @apidoc[GrpcServiceException]
+
+## Performance considerations
+
+### Lower latency
+
+See [Publish events for lower latency of
eventsBySlices](https://pekko.apache.org/docs/pekko-persistence-r2dbc/current/query.html#publish-events-for-lower-latency-of-eventsbyslices)
+for low latency use cases.
+
+### Scalability limitations
+
+Each connected consumer will start a `eventsBySlices` query that will
periodically poll and read events from the journal.
+That means that the journal database will become a bottleneck, unless it can
be scaled out, when number of consumers increase.
+The producer service itself can easily be scaled out to more instances.
+
+For the case of many consumers of the same event stream a future improvement
to reduce the
+database load would be to share results of the queries across the different
consumers, since most of them are
+probably reading at the tail of the same event stream.
+
+## Configuration
+
+### Consumer configuration
+
+The `client` section in the configuration defines where the producer is
running. It is a [Pekko gRPC
configuration](https://pekko.apache.org/docs/pekko-grpc/current/client/configuration.html#by-configuration)
with several connection options.
+
+### Reference configuration
+
+The following can be overridden in your `application.conf` for the Projection
specific settings:
+
+@@snip [reference.conf](/grpc/src/main/resources/reference.conf) {}
+
+### Connecting to more than one producer
+
+If you have several Projections that are connecting to different producer
services they can be configured as separate
+@apidoc[GrpcReadJournal] configuration sections.
+
+```
+consumer1 = ${pekko.projection.grpc.consumer}
+consumer1 {
+ client {
+ host = "127.0.0.1"
+ port = 8101
+ }
+}
+
+consumer2 = ${pekko.projection.grpc.consumer}
+consumer2 {
+ client {
+ host = "127.0.0.1"
+ port = 8202
+ }
+}
+```
+
+The `GrpcReadJournal` plugin id is then `consumer1` and `consumer2` instead of
the default `pekko.projection.grpc.consumer`.
diff --git a/docs/src/main/paradox/images/grpc-overview.png
b/docs/src/main/paradox/images/grpc-overview.png
new file mode 100644
index 0000000..e29c5ba
Binary files /dev/null and b/docs/src/main/paradox/images/grpc-overview.png
differ
diff --git a/docs/src/main/paradox/index.md b/docs/src/main/paradox/index.md
index 5c9af16..c94425e 100644
--- a/docs/src/main/paradox/index.md
+++ b/docs/src/main/paradox/index.md
@@ -10,6 +10,7 @@
* [Event Sourced](eventsourced.md)
* [Durable State](durable-state.md)
* [Kafka](kafka.md)
+* [gRPC](grpc.md)
* [Cassandra](cassandra.md)
* [JDBC](jdbc.md)
* [Slick](slick.md)
diff --git a/project/project-info.conf b/project/project-info.conf
index 718e38d..852ea8f 100644
--- a/project/project-info.conf
+++ b/project/project-info.conf
@@ -67,6 +67,10 @@ project-info {
title: "Apache Pekko Projections Kafka"
jpms-name: "pekko.projection.kafka"
}
+ grpc: ${project-info.shared-info} {
+ title: "Apache Pekko Projections gRPC"
+ jpms-name: "pekko.projection.grpc"
+ }
testkit: ${project-info.shared-info} {
title: "Apache Pekko Projections TestKit"
jpms-name: "pekko.projection.testkit"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
