Well before patching a consistent solution needs to be found. Sorry to go back to LongAdder but the idea is to add instances but not as much as thread. Just what is needed. It means you decrease the number of instance to aggregate so the lock time.
The queue solution sounds worse since a queue is either empty or full ;) Le 5 nov. 2013 05:23, "Phil Steitz" <phil.ste...@gmail.com> a écrit : > On 11/4/13 3:44 PM, Ted Dunning wrote: > > The copy doesn't have to lock if you build the right data structure. > > The individual stats objects need to update multiple quantities > atomically when new values come in. Consistency in the copy > requires that you suppress updates while the copy is in progress > unless you implement some kind of update queue internally. What > exactly do you mean by "the right data structure?" > > This discussion is bringing me back to the view that we had before > we did anything to support synchronization - clients are better > situated to manage the supporting infrastructure. As long as we > support aggregation, clients should be able to do what they need to do. > > If the use case here, for example, is that lots of client threads > need to contribute to aggregate stats and you can't pause any of > them ever to aggregate, there is no free lunch, you are going to > have to queue somewhere. The primitive solution of queuing on the > sync lock for a single stats object is heavy and bad. Trying to > build more lightweight queueing into the stat object itself is > interesting, but I wonder if there is other stuff from these threads > that should also be queued, or in general there is always going to > be stuff about the application's execution environment unknown to us > that make whatever we do less than optimal. > > All that said, patches welcome for lighter weight locking or > explicit update queue management inside the stats objects. > > Phil > > > > The thread leak problem can be more serious. > > > > > > > > > > On Mon, Nov 4, 2013 at 2:47 PM, Phil Steitz <phil.ste...@gmail.com> > wrote: > > > >> On 11/4/13 2:31 PM, Romain Manni-Bucau wrote: > >>> The copy will lock too. > >> Right. That is why I asked exactly how things work. If you can't > >> lock during aggregation, we need something different. > >> > >>> And it doesnt solve leak issue of the one instance > >>> by thread solution, no? > >> Correct, again depends on the setup how big a problem that is / what > >> can be done to manage it. > >> > >> Phil > >>> Le 4 nov. 2013 23:27, "Phil Steitz" <phil.ste...@gmail.com> a écrit : > >>> > >>>> On 11/4/13 2:22 PM, Ted Dunning wrote: > >>>>> I still think that what you need is a thread-safe copy rather than a > >>>>> thread-safe mutate. > >>>> I was just thinking the same thing. Patches welcome. > >>>> > >>>> Phil > >>>>> Even if you force every thread to do the copy, the > >>>>> aggregation still still wins on complexity/correctness/performance > >> ideas. > >>>>> > >>>>> On Mon, Nov 4, 2013 at 12:58 PM, Romain Manni-Bucau > >>>>> <rmannibu...@gmail.com>wrote: > >>>>> > >>>>>> In sirona we collect (aggregate) data each N ms and we can still use > >>>> stats > >>>>>> during aggregation (worse case surely) > >>>>>> Le 4 nov. 2013 21:48, "Phil Steitz" <phil.ste...@gmail.com> a > écrit : > >>>>>> > >>>>>>> On 11/4/13 12:12 PM, Romain Manni-Bucau wrote: > >>>>>>>> But aggregation needs to lock so not a real solution. Lock is fine > >> on > >>>>>>> real > >>>>>>>> cases but not in simple/light ones. ThreadLocal leaks...so a trade > >> off > >>>>>>>> should be found > >>>>>>> Depends on the use case. If the use case is > >>>>>>> > >>>>>>> 0) launch a bunch of threads and let them gather stats individually > >>>>>>> 1) aggregate results > >>>>>>> > >>>>>>> Then the static aggregate method in AggregateSummaryStatistics that > >>>>>>> takes a collection as input will work with no locking required. > >>>>>>> > >>>>>>> Phil > >>>>>>>> Le 4 nov. 2013 18:42, "Phil Steitz" <phil.ste...@gmail.com> a > >> écrit : > >>>>>>>>> On 11/4/13 8:49 AM, Romain Manni-Bucau wrote: > >>>>>>>>>> Hi, > >>>>>>>>>> > >>>>>>>>>> ATM sirona (a java monitoring library in incubator) relies a lot > >> on > >>>>>>>>>> Summary stats object from [math3] but it needed a lock to ensure > >>>>>>>>>> consistency. I know there is a synchronized version but this one > >>>>>>>>>> scales less then the locked one. > >>>>>>>>>> > >>>>>>>>>> My question is quite simple then: will [math] add an > >> implementation > >>>>>>>>>> with thread safety guarantee and good performances? I think for > >>>>>>>>>> instance to the LongAdder of Doug Lea which could be used as a > >> good > >>>>>>>>>> base. > >>>>>>>>> The short answer is yes, patches welcome. > >>>>>>>>> > >>>>>>>>> Ted makes a good point, though; and there is already some support > >>>>>>>>> for aggregation in the stats classes in [math] (i.e., you can > >>>>>>>>> aggregate the results of per-thread stats by using, e.g. > >>>>>>>>> AggregateSummaryStatistics#aggregate). See MATH-1016 re > extending > >>>>>>>>> this to more stats. > >>>>>>>>> > >>>>>>>>> Phil > >>>>>>>>> > >>>>>>>>>> Romain Manni-Bucau > >>>>>>>>>> Twitter: @rmannibucau > >>>>>>>>>> Blog: http://rmannibucau.wordpress.com/ > >>>>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau > >>>>>>>>>> Github: https://github.com/rmannibucau > >>>>>>>>>> > >>>>>>>>>> > >>>> --------------------------------------------------------------------- > >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > >>>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org > >>>>>>>>>> > >>>>>>>>>> > >> --------------------------------------------------------------------- > >>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > >>>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org > >>>>>>>>> > >>>>>>>>> > >>>>>>> > --------------------------------------------------------------------- > >>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > >>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org > >>>>>>> > >>>>>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > >>>> For additional commands, e-mail: dev-h...@commons.apache.org > >>>> > >>>> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > >> For additional commands, e-mail: dev-h...@commons.apache.org > >> > >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
