Re: Theta sketch - concurrent union implementation

2019-07-02 Thread Eshcar Hillel
than a sketch? The only reason I can think of is being backward compatible with the current implementation, but this might be a good enough reason. On Sunday, June 30, 2019, 1:13:30 PM GMT+3, Eshcar Hillel wrote: Hi Everyone, As some of you may recall a year ago we had a conversation

Theta sketch - concurrent union implementation

2019-06-30 Thread Eshcar Hillel
Hi Everyone, As some of you may recall a year ago we had a conversation over the mailing list regarding the synchronization of sketches  https://lists.apache.org/thread.html/9899aa790a7eb561ab66f47b35c8f66ffe695432719251351339521a@%3Cdev.druid.apache.org%3E.Currently, the implementation of

Re: Question about sketches aggregation in druid

2018-07-24 Thread Eshcar Hillel
totally be able to remove SynchronizedUnion. -- Himanshu On Sun, Jul 22, 2018 at 9:10 AM, Eshcar Hillel wrote: >  I think part of my confusion stems from the gap in the level of > abstraction.Druid terminology focuses on aggregation.But in sketches there > are two levels of aggregation

Re: Question about sketches aggregation in druid

2018-07-22 Thread Eshcar Hillel
uses a sketch to aggregate the data?4) if it is the case then is it guaranteed that the merged sketches are immutable? otherwise wrapping the union with locks is not enough. I hope my questions make more sense now. Thanks,Eshcar On Sunday, July 22, 2018, 4:15:02 PM GMT+3, Eshcar Hillel

Re: Question about sketches aggregation in druid

2018-07-22 Thread Eshcar Hillel
et called simultaneously: https://github.com/apache/incubator-druid/blob/master/extensions-core/datasketches/src/main/java/io/druid/query/aggregation/datasketches/theta/SketchAggregator.java On Sun, Jul 15, 2018 at 12:11 AM Eshcar Hillel wrote: >  Apologies, I must be missing something ver

Re: Question about sketches aggregation in druid

2018-07-15 Thread Eshcar Hillel
ving concurrency it would probably end up in that area. > > On Tue, Jul 10, 2018 at 2:10 AM Eshcar Hillel > wrote: > > > Hi All, > > My name is Eshcar Hillel from Oath research. I'm currently working with > > Lee Rhodes on committing a new concurrent implementation

Re: Question about sketches aggregation in druid

2018-07-15 Thread Eshcar Hillel
t;aggregate" and a query thread will be calling "get" potentially simultaneously. On Wed, Jul 11, 2018 at 1:04 AM Eshcar Hillel wrote: >  Thanks Gian, > This is also my understanding.But even in a single-writer-single-reader > scenario removing the lock can increase the thr

Question about sketches aggregation in druid

2018-07-10 Thread Eshcar Hillel
Hi All, My name is Eshcar Hillel from Oath research. I'm currently working with Lee Rhodes on committing a new concurrent implementation of the theta sketch to the sketches-core library.I was wondering whether this implementation can help boost the union operation that is applied to multiple