Re: Carbon over-use cluster resources

2020-05-07 Thread David CaiQiang
Hi, manhua Now no_sort reuse the loading flow of local_sort. It is not a good solution and led to the situation which you have mentioned. In my opinion, we need to adjust the loading flow of no_sort, maybe like global_sort finally. In addition, the producer-consumer pattern in data encoding

Re: Carbon over-use cluster resources

2020-04-20 Thread Manhua Jiang
at 6:06 PM Liang Chen wrote: > > > OK, thank you feedbacked this issue, let us look into it. > > > > Regards > > Liang > > > > > > Manhua Jiang wrote > > > Hi All, > > > Recently, I found carbon over-use cluster resources. Gener

Re: Carbon over-use cluster resources

2020-04-20 Thread Manhua Jiang
> Ajantha > > > > On Tue, Apr 14, 2020 at 6:06 PM Liang Chen > > wrote: > > > > > OK, thank you feedbacked this issue, let us look into it. > > > > > > Regards > > > Liang > > > > > > > > > Manhua Jiang wro

Re: Carbon over-use cluster resources

2020-04-16 Thread ??????????
??Carbon?? ??Carbon?? --Original-- From:"chenliang6136"http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: Carbon over-use cluster resources

2020-04-16 Thread Kumar Vishal
ote: > > > OK, thank you feedbacked this issue, let us look into it. > > > > Regards > > Liang > > > > > > Manhua Jiang wrote > > > Hi All, > > > Recently, I found carbon over-use cluster resources. Generally the > design > >

Re: Carbon over-use cluster resources

2020-04-15 Thread Ajantha Bhat
ked this issue, let us look into it. > > Regards > Liang > > > Manhua Jiang wrote > > Hi All, > > Recently, I found carbon over-use cluster resources. Generally the design > > of carbon work flow does not act as common spark task which only do one > > small

Re: Carbon over-use cluster resources

2020-04-14 Thread Liang Chen
OK, thank you feedbacked this issue, let us look into it. Regards Liang Manhua Jiang wrote > Hi All, > Recently, I found carbon over-use cluster resources. Generally the design > of carbon work flow does not act as common spark task which only do one > small work in one thread, but

Carbon over-use cluster resources

2020-04-02 Thread Manhua Jiang
Hi All, Recently, I found carbon over-use cluster resources. Generally the design of carbon work flow does not act as common spark task which only do one small work in one thread, but the task has its mind/logic. For example, 1.launch carbon with --num-executors=1 but set