Sorry, meant to say 4-6GB, not MB. Ram
On Mon, Dec 14, 2015 at 2:00 PM, Munagala Ramanath <[email protected]> wrote: > I'm OK with increasing it to 1280MB. > > I agree with Sandeep about new people trying out Apex, > often on limited environments like VMs limited to 4-6MB. With a default of > 0.5GB per Buffer Server > and 1GB per operator, things start to fail when the user has more than a > handful of operators. > > I'm guessing it's not just the number of the operators that puts memory > pressure on master memory > but the size of the jars that get pulled in by all the operator classes > and their dependencies. If this is > true, one could cause failure even with a single, sufficiently complex > operator. > > Ram > > On Mon, Dec 14, 2015 at 1:41 PM, Timothy Farkas <[email protected]> > wrote: > >> Hi Sandeep, >> >> I have seen the issue even with small apps like Twitter Demo. It happens >> less frequently in very small apps but the issue is still there. It >> happens >> frequently in medium size apps (10 containers). Could we increase the >> MASTER_MEMORY_MB to 1280 instead of 2048? That way we can get greater >> stability with the defaults settings without allocating too much >> additional >> memory? >> >> Thanks, >> Tim >> >> On Sun, Dec 13, 2015 at 1:22 AM, Sandeep Deshmukh < >> [email protected]> >> wrote: >> >> > With large number of containers, the problem becomes severe and I too >> have >> > faced it. >> > >> > Not sure if default should be increased as that should work in most of >> the >> > cases and even when new people are trying out Apex for the first time. >> > They may not have enough memory. >> > >> > Regards >> > Sandeep >> > On 13 Dec 2015 11:30, "Ganelin, Ilya" <[email protected]> >> wrote: >> > >> > > Tim - great point, we reduce the memory consumption for operators and >> do >> > > actually increase it for the app master, especially when we have more >> > > operators. I misread your original email. >> > > >> > > >> > > >> > > Thank you, >> > > Ilya Ganelin >> > > >> > > >> > > >> > > -----Original Message----- >> > > From: Timothy Farkas [[email protected]<mailto:[email protected] >> >] >> > > Sent: Saturday, December 12, 2015 11:30 PM Eastern Standard Time >> > > To: [email protected] >> > > Subject: Re: Default App Master Container Memory Size >> > > >> > > >> > > Hi Ilya, >> > > >> > > Dou you reduce the memory to 256 mb for operators only or for the App >> > > Master as well? >> > > >> > > Thanks, >> > > Tim >> > > >> > > On Sat, Dec 12, 2015 at 8:06 PM, Ganelin, Ilya < >> > > [email protected]> >> > > wrote: >> > > >> > > > Tim - in our applications we've actually tried to go with smaller >> > > > containers, as small as 256 MB, simply because many operators don't >> > > demand >> > > > any memory and this allows us to run more partitions. >> > > > >> > > > We haven't noticed performance degradation. >> > > > >> > > > >> > > > >> > > > Thank you, >> > > > Ilya Ganelin >> > > > >> > > > >> > > > >> > > > -----Original Message----- >> > > > From: Timothy Farkas [[email protected]<mailto: >> [email protected]>] >> > > > Sent: Saturday, December 12, 2015 11:02 PM Eastern Standard Time >> > > > To: [email protected] >> > > > Subject: Default App Master Container Memory Size >> > > > >> > > > >> > > > Hi All, >> > > > >> > > > The default value for MASTER_MEMORY_MB is 1024. I've observed in >> > multiple >> > > > demo apps and in the production apps of some of our users that 1024 >> MB >> > is >> > > > not enough memory, and the app master dies. This was observed on >> small >> > > > applications with 5 - 20 containers. I think the default app master >> > > memory >> > > > size should be 2048. >> > > > >> > > > Any feedback? >> > > > >> > > > Thanks, >> > > > Tim >> > > > ________________________________________________________ >> > > > >> > > > The information contained in this e-mail is confidential and/or >> > > > proprietary to Capital One and/or its affiliates and may only be >> used >> > > > solely in performance of work or services for Capital One. The >> > > information >> > > > transmitted herewith is intended only for use by the individual or >> > entity >> > > > to which it is addressed. If the reader of this message is not the >> > > intended >> > > > recipient, you are hereby notified that any review, retransmission, >> > > > dissemination, distribution, copying or other use of, or taking of >> any >> > > > action in reliance upon this information is strictly prohibited. If >> you >> > > > have received this communication in error, please contact the sender >> > and >> > > > delete the material from your computer. >> > > > >> > > ________________________________________________________ >> > > >> > > The information contained in this e-mail is confidential and/or >> > > proprietary to Capital One and/or its affiliates and may only be used >> > > solely in performance of work or services for Capital One. The >> > information >> > > transmitted herewith is intended only for use by the individual or >> entity >> > > to which it is addressed. If the reader of this message is not the >> > intended >> > > recipient, you are hereby notified that any review, retransmission, >> > > dissemination, distribution, copying or other use of, or taking of any >> > > action in reliance upon this information is strictly prohibited. If >> you >> > > have received this communication in error, please contact the sender >> and >> > > delete the material from your computer. >> > > >> > >> > >
