Can you upload the graphs to some website ? I couldn't read them. Cheers
On Thu, Aug 15, 2013 at 7:21 PM, <[email protected]> wrote: > Hi,**** > > **** > > The table definition : **** > > {NAME => 'lbc_zte_1', COPROCESSOR$1 => > '/zsmar/zsmar.jar|com.zsmar.hbase.query.rowkey.RowKey3Endpoint|536870911', > DEFERRED_LOG_FLUSH => 'true', MAX_FILESIZE => '0', FAMILIES => [{NAME => > 'F', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'true'}]}**** > > {NAME => 'lbc_zte_1_imei_index', DEFERRED_LOG_FLUSH => 'true', > MAX_FILESIZE => '107374182400', FAMILIES => [{NAME => 'F', BLOOMFILTER => > 'ROW', IN_MEMORY => 'true'}]}**** > > Use HexStringSplit to pre-split :**** > > The start-key end-key :**** > > **- ** 051EB851**** > > 051EB851 0A3D70A2**** > > 0A3D70A2 0F5C28F3**** > > 0F5C28F3 147AE144**** > > . . . . . . **** > > F5C28F30 FAE14781**** > > FAE14781 -**** > > Totally 50 regions**** > > ** ** > > And The rowkey likes **** > > 0000175622867EBCF052CD83973A91C3000000**** > > 0000175622867EBCF052CD83973A91C3235959**** > > . . . . . . **** > > 00006C60F4538B20CF2248E5455D119E000000**** > > 00006C60F4538B20CF2248E5455D119E235959**** > > ** ** > > There is table regions and count after inserting data below :**** > > ** ** > > Table lbc_zte_1 regions :**** > > **** > > ** ** > > Table lbc_zte_1 region count :**** > > **** > > ** ** > > So the result is that regions increase automatically .and client throws > exceptions :**** > > ** ** > > 2013-08-14 10:05:26,228 INFO [pool-2-thread-7] put.PutTask > (PutTask.java:doTask(225)) - 现已装载:45300000**** > > 2013-08-14 10:05:46,493 WARN [pool-2-thread-11] > client.HConnectionManager$HConnectionImplementation > (HConnectionManager.java:processBatchCallback(1574)) - Failed all from > region=lbc_zte_1_imei_index,3333332A,1376379199776.b11321169ec6cdd765486773640523e3., > hostname=phd03.hadoop.audaque.com, port=60020**** > > java.util.concurrent.ExecutionException: java.io.IOException: Call to > phd03.hadoop.audaque.com/172.16.1.93:60020 failed on local exception: > java.io.IOException: Connection reset by peer**** > > at > java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)**** > > at java.util.concurrent.FutureTask.get(FutureTask.java:83)**** > > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1544) > **** > > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1396) > **** > > at > org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:918)**** > > at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:770)** > ** > > at org.apache.hadoop.hbase.client.HTable.put(HTable.java:757)**** > > at > org.apache.hadoop.hbase.client.HTablePool$PooledHTable.put(HTablePool.java:399) > **** > > at com.zsmulti.put.PutTask.processPut(PutTask.java:262)**** > > at com.zsmulti.put.PutTask.doTask(PutTask.java:212)**** > > at dev.toolkit.concurrent.Task2.run(Task2.java:61)**** > > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > **** > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > **** > > at java.lang.Thread.run(Thread.java:662)**** > > Caused by: java.io.IOException: Call to > phd03.hadoop.audaque.com/172.16.1.93:60020 failed on local exception: > java.io.IOException: Connection reset by peer**** > > at > org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:1056) > **** > > at > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1025)**** > > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150) > **** > > at $Proxy7.multi(Unknown Source)**** > > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1373) > **** > > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1371) > **** > > at > org.apache.hadoop.hbase.client.ServerCallable.withoutRetries(ServerCallable.java:210) > **** > > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1380) > **** > > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1368) > **** > > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)**** > > at java.util.concurrent.FutureTask.run(FutureTask.java:138)**** > > ... 3 more**** > > Caused by: java.io.IOException: Connection reset by peer**** > > at sun.nio.ch.FileDispatcher.read0(Native Method)**** > > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)**** > > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198)**** > > at sun.nio.ch.IOUtil.read(IOUtil.java:171)**** > > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:243)* > *** > > at > org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55) > **** > > at > org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142) > **** > > at > org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:159)** > ** > > at > org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:129)** > ** > > at java.io.FilterInputStream.read(FilterInputStream.java:116)**** > > at java.io.FilterInputStream.read(FilterInputStream.java:116)**** > > at > org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStream.read(HBaseClient.java:399) > **** > > at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) > **** > > at java.io.BufferedInputStream.read(BufferedInputStream.java:237) > **** > > at java.io.DataInputStream.readInt(DataInputStream.java:370)**** > > at > org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:672) > **** > > at > org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:606) > **** > > *发件人:* Jean-Marc Spaggiari [mailto:[email protected]] > *发送时间:* 2013年8月15日 星期四 20:42 > *收件人:* [email protected] > *主题:* Re: regionserver died when using Put to insert data**** > > ** ** > > Hi, > > What is you key and how have your splitted your regions? > > 2000 regions for 77GB of data mean 40MB regions... Default value is 10GB. > What's our table definition? There is something wrong here. You should have > get less than 15 regions at the end. > > Also, something you need to keep in mind. Here is an example. > > Let's say your keys are something like that: > > 000ABCDE > 000ABCDF > 000ABCDG > 000BBCDH > 000BBCDI > 000BBCDJ > 000CBCDK > 000CBCDE > 000CBCDF > 000DBCDG > 000DBCDH > 000DBCDI > 000EBCDJ > 000EBCDK > > And so on. I you pre-split your table with, let's say, 256 regions. > > starting from: > #00 to #01 > #01 to #02 > etc > 0 to 1 > 1 to 2 > 2 to 3 > etc > a to b > b to c > c to d > etc. > > All you keys are going to be on the same single region, even with the > pre-split. So this region will have to split many times, and others are > going to stay empty. > > So, to summarize: > 1) What is your key; > 2) How have you splited your regions; > 3) What is your table definition. > > JM**** > > 2013/8/15 <[email protected]>**** > > Hi jean-marc ;**** > > **** > > I know what you say , and I create the table with 50 empty regions first . > **** > > **** > > But during inserting data into hbase , the regions increase rapidly ; can > greater than 2000 regions ….**** > > **** > > I execute several times , it happens the same way . And soon after that > , the regionserver died .**** > > **** > > I don’t know why the regions increase so rapidly , When I restart the > regionserver and don’t insert data , the regions decrease to 50 over a > period of time .**** > > **** > > Best wishes .**** > > **** > > **** > > ** ** >
