I'm most interested in the frequency / cardinality tools as it could be used to help improve performance automatically for combiners by detecting the few keys case or automatically handle hot keys without needing users to specify the hints when they use a combiner.
On Thu, Aug 3, 2017 at 5:35 AM, Jean-Baptiste Onofré <[email protected]> wrote: > Nice work Arnaud ;) > > Happy to have been able to help. > > Let's see what the others will think about this. > > Regards > JB > > > On 08/03/2017 02:32 PM, Arnaud Fournier wrote: > >> Hello everyone, >> >> My name is Arnaud Fournier and I am a CS student. I am currently doing an >> internship at Talend. >> >> With the support of Jean-Baptiste Onofre and Ismaël Mejia, I have been >> working on statistical analysis of streams with Beam, using probabilistic >> data structures like HyperLogLog. >> >> I would like to share this work with the community, but I wanted first to >> show you my work in progress and ask you if this humble contribution could >> be interesting as an extension. >> >> I have made a little doc with more details about what I have done in case >> you are interested and want to give me some feedback : >> *https://docs.google.com/document/d/1Xy6g5RPBYX_HadpIr_2WrUe >> usiwL0Jo2ACI5PEOP1kc/edit* >> <https://docs.google.com/document/d/1Xy6g5RPBYX_HadpIr_2WrUe >> usiwL0Jo2ACI5PEOP1kc/edit> >> >> You can also find the current work implementation in progress here : >> >> https://github.com/ArnaudFnr/beam/tree/sketching/sdks/java/e >> xtensions/sketching >> >> >> <https://github.com/ArnaudFnr/beam/tree/sketching/sdks/java/ >> extensions/sketching> >> >> Thanks ! >> >> Arnaud >> >> > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com >
