As an FYI. It's a misrepresentation to suggest that ES/CQRS requires each actor/aggregate root history to be materialised as a physically distinct stream. There may be specific implementations that require this, and I know there are others that do not.So regarding those references given, it's is not a requirement of this tech that we generate one stream in DL per actor for ES.It might for instance be necessary for Greg Young's event store to have one phys stream per aggregate sure to implementation choices. -------- Original message --------From: Poule Dodue <pouledo...@hotmail.com> Date: 28/10/2016 16:24 (GMT+00:00) To: dev@distributedlog.incubator.apache.org Subject: Re: hundreds of millions of streams? yes aka ES/CQRS
some links: https://msdn.microsoft.com/en-us/library/jj554200.aspx http://williamverdolini.github.io/2014/08/11/cqrses-architecture/ http://docs.geteventstore.com/introduction/3.9.0/event-sourcing-basics/ it needs lot of streams to basically replay events for any entity on a system. example: i could replay events for all changes that happened in 1 Cart of 1 User: (read events from stream "cart-of-user-233293111" ): 1- added item X 2- deleted item X 3- added item Y .... by replaying that stream, I can rebuild a user's cart state > Le 28 oct. 2016 à 10:13, Leigh Stewart <lstew...@twitter.com.INVALID> a écrit > : > > Poule- would you mind sharing some information on Event Sourcing? Are you > referring to something like > http://martinfowler.com/eaaDev/EventSourcing.html ? > > On Fri, Oct 28, 2016 at 7:11 AM, Leigh Stewart <lstew...@twitter.com> wrote: > >> DL is not able to handle 100s of millions of streams. 10^5-106 is probably >> ok. >> ZK is probably the biggest challenge (we are looking at ways to eliminate >> this as we would like to scale to 10^6-10^7 in the not too distant future), >> but 100s of millions is so far beyond what we've worked with there would >> likely be other scaling challenges on the way to that point. >> >> On Fri, Oct 28, 2016 at 5:56 AM, Poule Dodue <pouledo...@hotmail.com> >> wrote: >> >>> In Event Sourcing, we need to have 1 stream per entity/aggregate so for >>> a typical prod system it means we need hundreds of millions of streams. >>> >>> Is DL able to handle that or it is limited to, say, few hundreds >>> thousands of streams? >>> >>> >>> >>
