How many disks do you have? That can be bottle-necking throughput as the number of Xceivers is related to the number of resources (threads, sockets: http://blog.cloudera.com/blog/2012/03/hbase-hadoop-xceivers/) used at once to perform operations.
On Tue, Jan 1, 2013 at 6:45 PM, Eric Newton <[email protected]> wrote: > Hey Jeremy, > > Can you compare the ingest rate to the number of tablets, too? > > I've found, that if I have 20-80 tablets per server (on similar hardware) I > get the best performance. > > # of Xceivers == number of writers when ingest is the primary target. > > Also, is this 1.4 or trunk? > > On Tue, Jan 1, 2013 at 9:19 PM, Kepner, Jeremy - 1010 - MITLL < > [email protected]> wrote: > > > Accumulo Colleagues, > > I am trying to optimize my ingest into a single node Accumulo instance > > running on a 32 core node with 96 GB of RAM. I am seeing the follow > ingest > > variations as a I change the number of ingest processes (see attached): > > > > ------------------------------------- > > Ingestors, Ingest rate > > ------------------------------------- > > 1, 60K inserts/sec (stable) > > 2, 120K inserts/sec (stable) > > 3, 60K to 180K inserts/sec > > 4, 90K to 220K inserts/sec > > 8, 80K to 280K inserts/sec > > 12, 80K to 280K inserts/sec > > ------------------------------------- > > > > The only thing I can see that correlates with the ingest rate is the > > number of Xceivers. When the ingest rate is high the number of Xceivers > is > > usually low. Likewise, when the ingest rate drops, the number of > Xceivers > > usually increases significantly. > > > > Question: What role to Xceivers play in ingest? > > > > Request: It would be great to add a plot showing the number of Xceivers > > over time to the diagnostics. > > > > Regards. -Jeremy > > > > >
