2010/2/22 Niclas Hedhman <[email protected]> > On Mon, Feb 22, 2010 at 6:07 PM, Jacek Sokulski <[email protected]> > wrote: > > > We have thought of several approaches: > > 1. replicating data store on our side through event sourcing, but due to > > database size (system is collecting up to 500GB data per day) it is > probable > > not feasible (there are also security issues). > > Is it possible to receive a stream of changes, which is pre-filtered > down to your needs? Even though the main system collects that kind of > amounts, it is often the case that your particular sub-domain is only > interested in a fraction of that. > we do not need access to all data but all the data needs pas through our system (at least "logically", as most of the data will not be used, although it is not possible to know in advance which one). We could separate slow changing data (it is bigger part of the domain model) and the frequently changing data keep remotely, and only sent events with marks which entities are outdated, although we have 1-2k transactions per second, so it will generate considerable traffic, but it seems to be doable. Anyway we have access only to changes at the database level what is a bit cumbersome to translate to domain events.
> > > 2. to have Qi4j-ish data store on our side but use it as a cache, it is a > > bit difficult as there are other systems changing the data as well, so > the > > data can be invalid very quickly. > > Well "invalid in time" is a interesting concept, since ALL data is > stale at some point in time when you might read it. So, what we do in > real-time systems (industrial automation) is figuring out the fastest > frequency the data is needed, and then sample twice that speed. So, in > an enterprise scenario; if it is a user at the other end, he can > probably only consume (depending on app) a small number of changes per > second, perhaps even only a minute per change, in which case you only > need to ensure 'staleness' better than half the given time. > that's true, as far as only data being used is considered > > > 3. develop custom Qi4j data store that maps entities to existing database > > schema and then use only transient mixins (ones that does not need to > > persist state) or use separate (possible local) storage for additional > > mixins state. > > This can work, but can become a complex issue to code up, depending on > the consistency of the datastore. Another approach is to create what > Eric Evans call an "anti-corruption layer", where the data from the > store is mapped 1:1 to a local domain model and then from that mapped > to the "ideal" domain model at class level. > > > 4. develop domain model without Qi4j and use only some features of Qi4j, > > like data validation, or simplified AOP. > > That is probably your best first shot. as the first step probable yes. > Don't bite off more than you > can chew. A lot will depend on how the existing services are > organized, but you can always import those external services into Qi4j > as Qi4j services and code up your thing in a SOA fashion instead of a > "Persistent" system. > > > I would be gratefull if anyone could share his or her experience or ideas > > I think "experience" at the current time is very limited... ;-) > so thanks for the ideas :) > > > Cheers > -- > Niclas Hedhman, Software Developer > http://www.qi4j.org - New Energy for Java > > I live here; http://tinyurl.com/2qq9er > I work here; http://tinyurl.com/2ymelc > I relax here; http://tinyurl.com/2cgsug > > _______________________________________________ > qi4j-dev mailing list > [email protected] > http://lists.ops4j.org/mailman/listinfo/qi4j-dev >
_______________________________________________ qi4j-dev mailing list [email protected] http://lists.ops4j.org/mailman/listinfo/qi4j-dev
