Hi Olivier, Most of the time people uses Kafka not only for its technicalities and scalability but also simply for its ability to easily decouple organizations. From an engineering perspective, having a centralized, durable event hub means that teams can build, deploy, and scale their services completely independently. Even if the throughput is low, in practice this allows differents teams to work at their own pace, choose their own tech stacks, and isolate failures.
I find that we often under-estimate the importance of the human and organizational friction that point-to-point architectures create, and how minimizing that cognitive load directly impacts a company's overall agility. While engineering discussions tend to hyper-focus on hard metrics like latency, disk I/O, and message throughput, the true bottleneck in most scaling organizations is more frequent communication overhead than technology itself. >From my experience, the most common technical answer you will get is "processing events in real-time as they occur." But stepping back to look at the broader architectural and business value, it usually boils down to two major patterns: *Decoupling Ingestion from Processing :*Kafka acts as a massive, persistent buffer that safely separates the systems receiving requests from the systems doing the heavy lifting. Imagine a scenario where you suddenly receive a massive surge in user requests. Instead of routing that traffic directly to your backend services and risking a total system crash, you pipe those requests into a Kafka topic. Downstream consumers can then pull and process those requests one by one at their own optimal speed. It allows you to safely absorb exponential growth in demand without forcing your processing systems to scale identically in real-time. The system won't explode; the queue just gets a little longer until the consumers catch up. *Change Data Capture (CDC) and State Evolution:*Kafka is also quite useful for tracking how data evolves over time, rather than just looking at a static snapshot. Using CDC patterns, you can listen to a traditional relational database and instantly capture every single INSERT, UPDATE, or DELETE as an immutable event in a Kafka topic. Once those changes are flowing through Kafka, the possibilities open up quite nicely. You can easily "replay" the log to rollback or restore a database to a specific point in time. More commonly, you can use these event streams to automatically and reliably synchronize secondary data stores, like updating a KV store or keeping a search index (like Opensearch) up to date without writing fragile "dual-write" logic in your application code. Best Regards, Titouan, Staff Software Engineer Aiven On 2026/05/07 07:28:41 Olivier Dexheimer via users wrote: > Hi Kafka users, > > We are a 1000 people governemental french organization, willing to use Apache Kafka to integrate our applications. We currently use Talend ETL for that purpose. > We are developing a prototype on Apache Kafka 4.0, Kraft mode, which seems to fulfill our needs until now. > > Our request : we would like to have some feedbacks about using Apache Kafka in small organizations like ours - let's say, less than 5000 people : for what purpose it is used in your organization, what are the resources and tools needed for the run, main technical issues or tricky things to take care, etc. > > Best regards, > > Olivier Dexheimer > Architecture & Devs > Information Systems Management > GrandPalaisRmn > > >
