I'll let Steven answer your question directly. FYI, we are running a regression suite that was forked from the MapR repo a month or so ago because we had to fix a bunch of things to make it work with Apache Hadoop. (There was a thread about this back then and we haven't yet figured out how to merge both suites.) It is possible that he had a successful run but the failures are happening on items that you've recently added to your suite.
It is also possible (likely?) that the configuration settings for our regression clusters are not the same. -- Jacques Nadeau CTO and Co-Founder, Dremio On Wed, Dec 30, 2015 at 12:37 PM, Abdel Hakim Deneche <[email protected] > wrote: > Steven, > > were you able to successfully run the regression tests on the transfer > patch ? I just tried and saw several queries running out of memory ! > > On Wed, Dec 30, 2015 at 11:46 AM, Abdel Hakim Deneche < > [email protected] > > wrote: > > > Created DRILL-4236 <https://issues.apache.org/jira/browse/DRILL-4236> to > > keep track of this improvement. > > > > On Wed, Dec 30, 2015 at 11:01 AM, Jacques Nadeau <[email protected]> > > wrote: > > > >> Since the accounting changed (more accurate), the termination condition > >> for > >> the sort operator will be different than before. In fact, this likely > will > >> be sooner since our accounting is much larger than previously (since we > >> correctly consider the entire allocation rather than simply the used > >> allocation). > >> > >> Hakim, > >> Steven and I were discussing the need to update the ExternalSort > operator > >> to use the new allocator functionality to better manage its memory > >> envelope. Would you be interested in working on this since you seem to > be > >> working with that code the most? Basically, it used to be that there was > >> no > >> way the sort operator would be able to correctly detect a memory > condition > >> and so it jumped through a bunch of hoops to try to figure out the > >> termination condition.With the transfer accounting in place, this code > can > >> be greatly simplified to just use the current operator memory > allocation. > >> > >> -- > >> Jacques Nadeau > >> CTO and Co-Founder, Dremio > >> > >> On Wed, Dec 30, 2015 at 10:48 AM, rahul challapalli < > >> [email protected]> wrote: > >> > >> > I installed the latest master and ran this query. So > >> > planner.memory.max_query_memory_per_node should have been the default > >> > value. I switched back to 1.4.0 branch and this query completed > >> > successfully. > >> > > >> > On Wed, Dec 30, 2015 at 10:37 AM, Abdel Hakim Deneche < > >> > [email protected] > >> > > wrote: > >> > > >> > > Rahul, > >> > > > >> > > How much memory was assigned to the sort operator ( > >> > > planner.memory.max_query_memory_per_node) ? > >> > > > >> > > On Wed, Dec 30, 2015 at 9:54 AM, rahul challapalli < > >> > > [email protected]> wrote: > >> > > > >> > > > I am seeing an OOM error while executing a simple CTAS query. I > >> raised > >> > > > DRILL-4324 for this. The query mentioned in the JIRA used to > >> complete > >> > > > successfully without any issue prior to 1.5. Any idea what could > >> have > >> > > > caused the regression? > >> > > > > >> > > > - Rahul > >> > > > > >> > > > >> > > > >> > > > >> > > -- > >> > > > >> > > Abdelhakim Deneche > >> > > > >> > > Software Engineer > >> > > > >> > > <http://www.mapr.com/> > >> > > > >> > > > >> > > Now Available - Free Hadoop On-Demand Training > >> > > < > >> > > > >> > > >> > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > >> > > > > >> > > > >> > > >> > > > > > > > > -- > > > > Abdelhakim Deneche > > > > Software Engineer > > > > <http://www.mapr.com/> > > > > > > Now Available - Free Hadoop On-Demand Training > > < > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > > > > -- > > Abdelhakim Deneche > > Software Engineer > > <http://www.mapr.com/> > > > Now Available - Free Hadoop On-Demand Training > < > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > >
