Hi Christian,
I guess you wanted to send this message on the kafka mailing list, right ?
Regards
JB
On 08/12/2021 09:31, Christian Schneider wrote:
We have a single tenant application that we deploy to a kubernetes
cluster in many instances.
Every customer has several environments of the application. Each
application lives in a separate namespace and should be isolated from
other applications.
We plan to use kafka to communicate inside an environment (between the
different pods).
As setting up one kafka cluster per such environment is a lot of
overhead and cost we would like to just use a single multi tenant kafka
cluster.
Let's assume we just have one topic with 10 partitions for simplicity.
We can now use the environment id as a key for the messages to make sure
the messages of each environment arrive in order while sharing the load
on the partitions.
Now we want each environment to only read the minimal number of messages
while consuming. Ideally we would like to to only consume its own
messages. Can we somehow filter to only
receive messages with a certain key? Can we maybe only listen to a
certain partition at least?
Additionally we ideally would like to have enforced isolation. So each
environment can only see its own messages even if it might receive
messages of other environments from the same partition.
I think in worst case we can make this happen by encrypting the messages
but it would be great if we could filter on broker side.
Christian
--
--
Christian Schneider
http://www.liquid-reality.de <http://www.liquid-reality.de>
Computer Scientist
http://www.adobe.com <http://www.adobe.com>
