+1 HBASE-4047 On Friday, August 31, 2012, Gary Helmling wrote:
> Maybe we need to add a coprocessors section to the ref guide. I think > all the current documentation is in javadoc. And if all the > potentially destabilizing issues of in-process coprocessor usage are > not yet called out (memory usage, cpu, etc), we could more explicitly > detail that. > > In we want to really be able to support these cases, though, we > probably need to push HBASE-4047 (external coprocessor host) forward. > Without it there's not a lot we can do to prevent coprocessors from > taking down region servers. > > > On Thu, Aug 30, 2012 at 10:04 PM, lars hofhansl > <[email protected]<javascript:;>> > wrote: > > Maybe we should be better with marking improvement as such in jira, and > then have a special list in the release announcements highlighting these > (otherwise they just be in the noise of all the other bug fixes). > > > > -- Lars > > > > > > > > ________________________________ > > From: Ramkrishna.S.Vasudevan <[email protected]<javascript:;> > > > > To: [email protected] <javascript:;>; 'lars hofhansl' < > [email protected] <javascript:;>> > > Sent: Thursday, August 30, 2012 9:59 PM > > Subject: RE: Allocating more heap for endpoint coprocessors > > > > @Lars/@Gary > > > > Do we need to document such things. Recently someone was asking me a > > question like this, if my endpoint impl is so memory intensive it just > > affects a running cluster and already the RS > > has a huge memory heap associated with it. > > > > So its better we document saying if your endpoint consumes memory and > > because it runs along with RS based on your need add that extra amount of > > memory that will be taken up by the endpoint impl. > > > > Regards > > Ram > > > >> -----Original Message----- > >> From: lars hofhansl [mailto:[email protected] <javascript:;>] > >> Sent: Friday, August 31, 2012 3:04 AM > >> To: [email protected] <javascript:;> > >> Subject: Re: Allocating more heap for endpoint coprocessors > >> > >> In the upcoming 0.94.2 release will also have HBASE-6505, which allows > >> you to share state between RegionObservers (and Endpoints) within the > >> same RegionServer. > >> > >> -- Lars > >> > >> > >> > >> ________________________________ > >> From: Gary Helmling <[email protected] <javascript:;>> > >> To: [email protected] <javascript:;> > >> Sent: Thursday, August 30, 2012 1:59 PM > >> Subject: Re: Allocating more heap for endpoint coprocessors > >> > >> Endpoint coprocessors are loaded and run within the HBase RegionServer > >> process. Your endpoint coprocessors will be running on the region > >> servers hosting the regions for the table(s) on which the coprocessor > >> is configured. > >> > >> So the way to allocate more memory is by setting either HBASE_HEAPSIZE > >> or setting the max heap in HBASE_REGIONSERVER_OPTS in hbase-env.sh on > >> the region server. > >> > >> Note that a separate coprocessor instance is loaded for each table > >> region, so, say you want to allocate 10MB for your coprocessor, but > >> each region server hosts 20 regions, you would want to increase the > >> heap size by 200MB (20x10MB). > >> > >> --gh > >> > >> On Thu, Aug 30, 2012 at 1:45 PM, Young Kim > >> <[email protected]<javascript:;> > > > >> wrote: > >> > Hi, > >> > > >> > We have some memory intensive endpoint coprocessors running on our > >> RegionServers. As a result, we want to allocate more heap for the > >> coprocessors, but there doesn't seem to be much documentation on which > >> Hbase processes are directly responsible for coprocessors. Does anyone > >> happen to know or direct me to some resource that does? > >> > > >> > Thanks, > >> > Young Kim > >> > > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
