Indeed Hector has a connection pool behind it, I think it uses 50 connectios per node. But also uses a node to discover the others, I assume that, as I saw connections from my app to nodes that I didn't configure in Hector.
So, you may check the fds in OS level to see if there is a bottleneck there. On Thu, Dec 16, 2010 at 2:39 PM, Amin Sakka, Novapost <amin.sa...@novapost.fr> wrote: > > I'm using a unique client instance (using Hector) and a unique connection to > cassandra. > For each insertion I'm using a new mutator and then I release it. > I have 473 sstable "Data.db", the average size of each is 30Mo. > > > > 2010/12/16 Ryan King <r...@twitter.com> >> >> Are you creating a new connection for each row you insert (and if so >> are you closing it)? >> >> -ryan >> >> On Wed, Dec 15, 2010 at 8:13 AM, Amin Sakka, Novapost >> <amin.sa...@novapost.fr> wrote: >> > Hello, >> > I'm using cassandra 0.7.0 rc1, a single node configuration, replication >> > factor 1, random partitioner, 2 GO heap size. >> > I ran my hector client to insert 5.000.000 rows but after a couple of >> > hours, >> > the following Exception occurs : >> > >> > WARN [main] 2010-12-15 16:38:53,335 CustomTThreadPoolServer.java (line >> > 104) >> > Transport error occurred during acceptance of message. >> > org.apache.thrift.transport.TTransportException: >> > java.net.SocketException: >> > Too many open files >> > at >> > >> > org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:124) >> > at >> > >> > org.apache.cassandra.thrift.TCustomServerSocket.acceptImpl(TCustomServerSocket.java:67) >> > at >> > >> > org.apache.cassandra.thrift.TCustomServerSocket.acceptImpl(TCustomServerSocket.java:38) >> > at >> > >> > org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31) >> > at >> > >> > org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:98) >> > at >> > >> > org.apache.cassandra.thrift.CassandraDaemon.start(CassandraDaemon.java:120) >> > at >> > >> > org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:229) >> > at >> > >> > org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:134) >> > Caused by: java.net.SocketException: Too many open files >> > at java.net.PlainSocketImpl.socketAccept(Native Method) >> > at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384) >> > at java.net.ServerSocket.implAccept(ServerSocket.java:453) >> > at java.net.ServerSocket.accept(ServerSocket.java:421) >> > at >> > >> > org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:119) >> > >> > When I try to restart Cassandra, I have the following exception : >> > >> > ERROR 16:42:26,573 Exception encountered during startup. >> > java.lang.ArithmeticException: / by zero >> > at >> > >> > org.apache.cassandra.io.sstable.SSTable.estimateRowsFromIndex(SSTable.java:233) >> > at >> > >> > org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:284) >> > at >> > >> > org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:200) >> > at >> > >> > org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:225) >> > at >> > >> > org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:449) >> > at >> > >> > org.apache.cassandra.db.ColumnFamilyStore.addIndex(ColumnFamilyStore.java:306) >> > at >> > >> > org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:246) >> > at >> > >> > org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:449) >> > at >> > >> > org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:437) >> > at org.apache.cassandra.db.Table.initCf(Table.java:341) >> > at org.apache.cassandra.db.Table.<init>(Table.java:283) >> > at org.apache.cassandra.db.Table.open(Table.java:114) >> > at >> > >> > org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:138) >> > at >> > >> > org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:55) >> > at >> > >> > org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:216) >> > at >> > >> > org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:134) >> > >> > I am looking for advice on how to debug this. >> > >> > Thanks, >> > -- >> > >> > Amin >> > >> > >> > >> > >> > > > > > -- > Amin > > > > -- //GK german.kond...@gmail.com // sites http://twitter.com/germanklf http://www.facebook.com/germanklf http://ar.linkedin.com/in/germankondolf
