I am guessing the configuration is different since some of these failing suites were added a while ago ( > 1 month)
On Wed, Dec 30, 2015 at 12:42 PM, Jacques Nadeau <[email protected]> wrote: > I'll let Steven answer your question directly. > > FYI, we are running a regression suite that was forked from the MapR repo a > month or so ago because we had to fix a bunch of things to make it work > with Apache Hadoop. (There was a thread about this back then and we haven't > yet figured out how to merge both suites.) It is possible that he had a > successful run but the failures are happening on items that you've recently > added to your suite. > > It is also possible (likely?) that the configuration settings for our > regression clusters are not the same. > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Wed, Dec 30, 2015 at 12:37 PM, Abdel Hakim Deneche < > [email protected] > > wrote: > > > Steven, > > > > were you able to successfully run the regression tests on the transfer > > patch ? I just tried and saw several queries running out of memory ! > > > > On Wed, Dec 30, 2015 at 11:46 AM, Abdel Hakim Deneche < > > [email protected] > > > wrote: > > > > > Created DRILL-4236 <https://issues.apache.org/jira/browse/DRILL-4236> > to > > > keep track of this improvement. > > > > > > On Wed, Dec 30, 2015 at 11:01 AM, Jacques Nadeau <[email protected]> > > > wrote: > > > > > >> Since the accounting changed (more accurate), the termination > condition > > >> for > > >> the sort operator will be different than before. In fact, this likely > > will > > >> be sooner since our accounting is much larger than previously (since > we > > >> correctly consider the entire allocation rather than simply the used > > >> allocation). > > >> > > >> Hakim, > > >> Steven and I were discussing the need to update the ExternalSort > > operator > > >> to use the new allocator functionality to better manage its memory > > >> envelope. Would you be interested in working on this since you seem to > > be > > >> working with that code the most? Basically, it used to be that there > was > > >> no > > >> way the sort operator would be able to correctly detect a memory > > condition > > >> and so it jumped through a bunch of hoops to try to figure out the > > >> termination condition.With the transfer accounting in place, this code > > can > > >> be greatly simplified to just use the current operator memory > > allocation. > > >> > > >> -- > > >> Jacques Nadeau > > >> CTO and Co-Founder, Dremio > > >> > > >> On Wed, Dec 30, 2015 at 10:48 AM, rahul challapalli < > > >> [email protected]> wrote: > > >> > > >> > I installed the latest master and ran this query. So > > >> > planner.memory.max_query_memory_per_node should have been the > default > > >> > value. I switched back to 1.4.0 branch and this query completed > > >> > successfully. > > >> > > > >> > On Wed, Dec 30, 2015 at 10:37 AM, Abdel Hakim Deneche < > > >> > [email protected] > > >> > > wrote: > > >> > > > >> > > Rahul, > > >> > > > > >> > > How much memory was assigned to the sort operator ( > > >> > > planner.memory.max_query_memory_per_node) ? > > >> > > > > >> > > On Wed, Dec 30, 2015 at 9:54 AM, rahul challapalli < > > >> > > [email protected]> wrote: > > >> > > > > >> > > > I am seeing an OOM error while executing a simple CTAS query. I > > >> raised > > >> > > > DRILL-4324 for this. The query mentioned in the JIRA used to > > >> complete > > >> > > > successfully without any issue prior to 1.5. Any idea what could > > >> have > > >> > > > caused the regression? > > >> > > > > > >> > > > - Rahul > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > -- > > >> > > > > >> > > Abdelhakim Deneche > > >> > > > > >> > > Software Engineer > > >> > > > > >> > > <http://www.mapr.com/> > > >> > > > > >> > > > > >> > > Now Available - Free Hadoop On-Demand Training > > >> > > < > > >> > > > > >> > > > >> > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > > > > > -- > > > > > > Abdelhakim Deneche > > > > > > Software Engineer > > > > > > <http://www.mapr.com/> > > > > > > > > > Now Available - Free Hadoop On-Demand Training > > > < > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > > > > > > > > > > -- > > > > Abdelhakim Deneche > > > > Software Engineer > > > > <http://www.mapr.com/> > > > > > > Now Available - Free Hadoop On-Demand Training > > < > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > >
