SInce the extenson (module) will probably be in the form of new PTransforms to Beam, it is worth to take a look at: https://beam.apache.org/contribute/ptransform-style-guide/ and of course to: https://beam.apache.org/contribute/
On Wed, Oct 31, 2018 at 6:57 PM Alex <[email protected]> wrote: > > Great. Thank you. > > On Oct 31, 2018 6:32 PM, Kenneth Knowles <[email protected]> wrote: > > The word "extension" doesn't really mean anything in the case of Beam. It is > just a library. You can use the build set up of other libraries as examples. > > Kenn > > On Wed, Oct 31, 2018 at 10:23 AM Alejandro <[email protected]> wrote: > > Hello, > > I am going to get familiarized on how to write a Beam extension then, > although right now I am a little busy searching for a new job :-/. I > hope in a few weeks (Lets hope it doesn't take much longer to find a > job) I can get hands on it this and contribute with this preprocessing > extension to beam. > > Cheers. > > On 10/31/2018 02:13 PM, Ismaël Mejía wrote: > > Hello, > > > > I mentored Arnaud to contribute the sketching extension into Beam and > > from a quick look at Alex paper + implementation, I think this should > > be an independent extension. Sketching is a collection of transforms > > that rely on probabilistic data structures to give approximate results > > and correspond clearly to the data sketching category. > > > > Alex work is clearly a different area, it is more about data > > preprocessing and feature extraction, so I think it should be in a > > different module. > > Agree 100% that the best option is to do a rewrite on Java, this also > > has the advantage of easier maintainability. It would be really nice > > to have a new extension for this in Beam so don't hesitate to ask in > > the mailing list / slack if you have questions. > > > > Regards, > > Ismaël > > > > On Mon, Oct 29, 2018 at 10:38 AM Maximilian Michels <[email protected]> wrote: > >> > >> Hey Alex, > >> > >> No need to reimplement. Java is the best option, since we don't > >> currently have a Scala API in Beam. > >> > >> Cheers, > >> Max > >> > >> On 25.10.18 21:50, Alex wrote: > >>> Great! Right now there is a lot on that code I do not understand, hope in > >>> the next days I can document myself. > >>> > >>> Should I reimplement my algorithms in Scala? Or could I create a wrapper > >>> that interface with the sketching extension? > >>> > >>> Cheers.On Oct 24, 2018 15:00, Maximilian Michels <[email protected]> wrote: > >>>> > >>>> Welcome Alejandro! Interesting work. The sketching extension looks like > >>>> a good place for your algorithms. > >>>> > >>>> -Max > >>>> > >>>> On 23.10.18 19:05, Lukasz Cwik wrote: > >>>>> Arnoud Fournier ([email protected] <mailto:[email protected]>) > >>>>> started by adding a library to support sketching > >>>>> (https://github.com/apache/beam/tree/master/sdks/java/extensions/sketching), > >>>>> I feel as those some of these could be added there or possibly within > >>>>> another extension. > >>>>> > >>>>> On Tue, Oct 23, 2018 at 9:54 AM Austin Bennett > >>>>> <[email protected] <mailto:[email protected]>> > >>>>> wrote: > >>>>> > >>>>> Hi Beam Devs, > >>>>> > >>>>> Alejandro, copied, is an enthusiastic developer, who recently > >>>>> coded up: > >>>>> https://github.com/elbaulp/DPASF (associated paper found: > >>>>> https://arxiv.org/abs/1810.06021). > >>>>> > >>>>> He had been looking to contribute that code to FlinkML, at which > >>>>> point I found him and alerted him to Beam. He has been learning a > >>>>> bit on Beam recently. Would this data-preprocessing be a welcome > >>>>> contribution to the project. If yes, perhaps others better versed > >>>>> in internals (I'm not there yet -- though could follow along!) > >>>>> would > >>>>> be willing to provide feedback to shape this to be a suitable Beam > >>>>> contribution. > >>>>> > >>>>> Cheers, > >>>>> Austin > >>>>> > >>>>> > > -- > elbauldelprogramador.com
