In production, I assume you have more than one server hosting the regions you want to process. Assuming load balancer does its job of spreading regions across servers, you should consider using batchCoprocessorService.
There're only so many handlers on each region server, parallel invocation of coprocessors of all relevant regions on individual server places burden on the server. FYI On Thu, Mar 2, 2017 at 8:22 AM, Daniel Połaczański <[email protected]> wrote: > thx, this is what I needed > > 2017-03-02 11:07 GMT+01:00 Ted Yu <[email protected]>: > > > Daniel: > > If you don't pass your ExecutorService to HTable ctor, the following > would > > be called: > > > > public static ThreadPoolExecutor getDefaultExecutor(Configuration > conf) > > { > > > > int maxThreads = conf.getInt("hbase.htable.threads.max", Integer. > > MAX_VALUE); > > > > Note the default value. > > > > When the number of regions involved is high, consider adjusting the above > > config. > > > > On Wed, Mar 1, 2017 at 11:04 PM, Daniel Połaczański < > > [email protected]> > > wrote: > > > > > Yes. I am using batch variant. I will try your variant > > > > > > 02.03.2017 7:53 AM "Jingcheng Du" <[email protected]> napisał(a): > > > > > > What APIs are you using to execute the coprocessor? > > > HTable#batchCoprocessorService? It executes the coprocessors regions > by > > > regions in each region server. > > > How about using coprocessorService? This API should run coprocessors in > > > parallel for different regions. > > > > > > 2017-03-02 14:32 GMT+08:00 Daniel Połaczański <[email protected] > >: > > > > > > > I invoke my business logic which is similar to map reduce paradigm. > > > > Everything works. Only performance is problem that regions are not > > > > processed parallely. > > > > I'm thinking about some hack like invoking 4 times rpc from client > > when i > > > > have 4 regions > > > > > > > > 02.03.2017 00:43 "Ted Yu" <[email protected]> napisał(a): > > > > > > > > > To my knowledge, there is no support for this type of combination > of > > > map > > > > > reduce and coprocessor. > > > > > > > > > > On Wed, Mar 1, 2017 at 2:55 PM, Daniel Połaczański < > > > > [email protected] > > > > > > > > > > > wrote: > > > > > > > > > > > It is something like map reduce processing. > > > > > > I want to run map and combine phase inside in a coprocessor for > > every > > > > > > region separately, return partial results to the client. On the > > > client > > > > > > invoke reduce phase and store result in another hbase table. > > > > > > > > > > > > 2017-03-01 23:26 GMT+01:00 Ted Yu <[email protected]>: > > > > > > > > > > > > > Can you describe your use case in more detail ? > > > > > > > > > > > > > > What type of custom coprocessor are you loading to the region > > > server > > > > ? > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > On Wed, Mar 1, 2017 at 2:24 PM, Daniel Połaczański < > > > > > > [email protected] > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > Let's assume that we have cluster consisting from one > > > RegionServer > > > > > and > > > > > > > the > > > > > > > > RegionsServer contains one table consisting from 3 regions. > > > > > > > > > > > > > > > > I would like to process regions in coprocessor parallely. Is > > it > > > > > > > possible? > > > > > > > > > > > > > > > > I observed that currenlty it invokes coproprocessor with all > > the > > > > > > regions > > > > > > > > one by one. So it process at the beginig region1, when > finished > > > > > process > > > > > > > > regions2 and ... > > > > > > > > > > > > > > > > Is it possible to change this behaviour? > > > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
