One known issue that matches this condition was fixed on master, https://issues.apache.org/jira/browse/IMPALA-4765. It is not yet part of a release. The fix doesn't speed up the loading of the largest table, but it makes it more likely that loading of smaller tables can progress in parallel.
On Sun, Apr 9, 2017 at 12:30 AM, 吴朱华 <[email protected]> wrote: > The issue will occur when the new table has been created or impalad has > been just restarted. > > 2017-04-09 15:28 GMT+08:00 吴朱华 <[email protected]>: > >> During 10-15sec loading, from top, we only can see 2-3sec more than >> 100%-200% java usage, other than that there is nothing serious working, >> and we will attach profile once we have it, and this question has been >> haunted us for year. >> >> 2017-04-09 15:13 GMT+08:00 吴朱华 <[email protected]>: >> >>> Plus the schema we using is simple >>> >>> 2017-04-09 15:11 GMT+08:00 吴朱华 <[email protected]>: >>> >>>> The current version using early 2016, the metadata loading is >>>> 10-15sec. >>>> We haven't try the latest version, we hope the issue has been solved^_^ >>>> >>>> 2017-04-07 14:08 GMT+08:00 Alex Behm <[email protected]>: >>>> >>>>> What was your Impala version before and what is it now? >>>>> >>>>> How fast was it before and how fast is it now? >>>>> >>>>> On Thu, Apr 6, 2017 at 9:16 PM, 吴朱华 <[email protected]> wrote: >>>>> >>>>>> Hi guys: >>>>>> >>>>>> Is there any change on metadata loading? The metadata loading used to >>>>>> 10sec. >>>>>> >>>>>> 2017-03-23 12:42 GMT+08:00 吴朱华 <[email protected]>: >>>>>> >>>>>>> thx^_^ >>>>>>> >>>>>>> 2017-03-23 11:51 GMT+08:00 Henry Robinson <[email protected]>: >>>>>>> >>>>>>>> If your build does not have runtime filters (which are a little >>>>>>>> more than a year old), then the answer is yes: you should upgrade. >>>>>>>> >>>>>>>> Performance in general has been improving steadily over the last >>>>>>>> year. >>>>>>>> >>>>>>>> On 22 March 2017 at 20:36, 吴朱华 <[email protected]> wrote: >>>>>>>> >>>>>>>>> The question is there any big join perf related improvement has >>>>>>>>> implemented? Thx >>>>>>>>> >>>>>>>>> 2017-03-23 11:34 GMT+08:00 吴朱华 <[email protected]>: >>>>>>>>> >>>>>>>>> > Hi guys: >>>>>>>>> > >>>>>>>>> > I am currently using my one year old build (I guess 2016.3 >>>>>>>>> CDH5-Trunk), >>>>>>>>> > now I am facing a lot of join query situation(two big table >>>>>>>>> joins), do I >>>>>>>>> > need to upgrade my one year old build for potential join perf >>>>>>>>> improvement? >>>>>>>>> > Thx at advance^_^ >>>>>>>>> > >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
