Re: Problem scaling Extractors with data volume

2017-10-05 Thread Zhixiong Chen
5 PM To: dev@gobblin.incubator.apache.org; issac.buenros...@gmail.com Subject: Re: Problem scaling Extractors with data volume Issac, Do you have suggestions on this. I see you were engaged in conversation on the referred PR. Regards, Abhishek On Thu, Sep 28, 2017 at 5:26 PM, Gurjinder Singh Rathore < gurjinder.ra

Re: Problem scaling Extractors with data volume

2017-10-05 Thread Abhishek Tiwari
Issac, Do you have suggestions on this. I see you were engaged in conversation on the referred PR. Regards, Abhishek On Thu, Sep 28, 2017 at 5:26 PM, Gurjinder Singh Rathore < gurjinder.rath...@gmail.com> wrote: > Hi, > > I'm using gobblin (embedded) in my project to transfer loads of over 10

Problem scaling Extractors with data volume

2017-09-28 Thread Gurjinder Singh Rathore
Hi, I'm using gobblin (embedded) in my project to transfer loads of over 10 million+ rows at a time (single table). But when I run the load, gobbling starts giving me errors since looks like the number of Extractor instances grows almost linearly with the number of rows in my source table. This