be glad to go over the jira to make sure that we understand each other chaim
On Tue, Oct 17, 2017 at 5:00 PM, Jean-Baptiste Onofré <[email protected]> wrote: > Let me take a quick look and create a Jira if required. > > Thanks for the idea ! > > Regards > JB > > > On 10/17/2017 01:54 PM, Chaim Turkel wrote: >> >> I am fine if you can show me how to do the Create.of() on the >> MongoDbIO.read(). >> It would be nice also to have the status also on the MongoDbIO.write. >> >> again this is the reactive streams pattern that there always is a >> complete or error path >> >> chaim >> >> On Tue, Oct 17, 2017 at 2:51 PM, Jean-Baptiste Onofré <[email protected]> >> wrote: >>> >>> Good point but I was thinking more about a PTransform with Create.of() if >>> there's no data. >>> >>> Anyway, I can add a DoFn support in the write to update a status (not >>> sure >>> if it really makes sense). >>> >>> Regards >>> JB >>> >>> >>> On 10/17/2017 01:40 PM, Chaim Turkel wrote: >>>> >>>> >>>> but if there is no data then >>>> .apply(ParDo.of(new DoFn() { // check PCollection and set the status })) >>>> >>>> will not be called >>>> >>>> On Tue, Oct 17, 2017 at 8:33 AM, Jean-Baptiste Onofré <[email protected]> >>>> wrote: >>>>> >>>>> >>>>> I didn't mean on the read, I meant between the read and write. >>>>> >>>>> Basically, your pipeline could look like: >>>>> >>>>> pipeline.apply(MongoDbIO.read()...) >>>>> .apply(ParDo.of(new DoFn() { // check PCollection and set the >>>>> status >>>>> })) >>>>> .apply(MongoDbIO.write()...) >>>>> >>>>> Regards >>>>> JB >>>>> >>>>> >>>>> On 10/16/2017 09:42 PM, Chaim Turkel wrote: >>>>>> >>>>>> >>>>>> >>>>>> how to i add a ParDo on the MongoDbIO.read() if there are no records >>>>>> read? >>>>>> >>>>>> On Mon, Oct 16, 2017 at 6:53 PM, Jean-Baptiste Onofré >>>>>> <[email protected]> >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> You can always add your own ParDo(DoFn) where you write the status. >>>>>>> >>>>>>> Regards >>>>>>> JB >>>>>>> >>>>>>> >>>>>>> On 10/16/2017 04:24 PM, Chaim Turkel wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> that is the problem, i want to write a status that i tried and that >>>>>>>> there were no records >>>>>>>> >>>>>>>> On Mon, Oct 16, 2017 at 3:51 PM, Jean-Baptiste Onofré >>>>>>>> <[email protected]> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Chaim, >>>>>>>>> >>>>>>>>> So, you mean you call MongoDBIO.write() with an empty PCollection >>>>>>>>> (no >>>>>>>>> element in the collection) ? >>>>>>>>> >>>>>>>>> The write is basically a DoFn so, it won't do anything if the >>>>>>>>> PCollection >>>>>>>>> is >>>>>>>>> empty. >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> JB >>>>>>>>> >>>>>>>>> >>>>>>>>> On 10/16/2017 01:59 PM, Chaim Turkel wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> In the case that there are no records to read, is there a >>>>>>>>>> way >>>>>>>>>> to >>>>>>>>>> get >>>>>>>>>> called so that i can write the status? >>>>>>>>>> >>>>>>>>>> chaim >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Jean-Baptiste Onofré >>>>>>>>> [email protected] >>>>>>>>> http://blog.nanthrax.net >>>>>>>>> Talend - http://www.talend.com >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Jean-Baptiste Onofré >>>>>>> [email protected] >>>>>>> http://blog.nanthrax.net >>>>>>> Talend - http://www.talend.com >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Jean-Baptiste Onofré >>>>> [email protected] >>>>> http://blog.nanthrax.net >>>>> Talend - http://www.talend.com >>> >>> >>> >>> -- >>> Jean-Baptiste Onofré >>> [email protected] >>> http://blog.nanthrax.net >>> Talend - http://www.talend.com > > > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com
