Eagle service should not load all the jobs because the filter should have been pushed down to HBase. One optimization is to create secondary index(Eagle provides application level secondary index), or maybe we can put user as most significant field in the row key if all of queries have user as filter.
Thanks Edward On Tue, May 9, 2017 at 9:42 PM, Tang Jijun(上海_技术部_数据平台_唐觊隽) < [email protected]> wrote: > It was quite inefficient to load all those jobs up into memory then > response to the client. StreamingOutput allows writing the data back to the > user as soon as they become available.I think that will improve the speed. > > -----邮件原件----- > 发件人: Edward Zhang [mailto:[email protected]] > 发送时间: 2017年5月10日 11:13 > 收件人: [email protected] > 主题: Re: EAGLE-1019 > > This is because Eagle scans the large table until it finds the first set > of jobs for this user. How StreamingOutput will improve the speed? > > Thanks > Edward > > On Tue, May 9, 2017 at 7:30 PM, Tang Jijun(上海_技术部_数据平台_唐觊隽) < > [email protected]> wrote: > > > About EAGLE-1019, when query job list by user and user job is sparse, > > query need much time. I think we can Streaming a Response using > > StreamingOutput to improve.Any idea? > > >
