I've reproduced the error and I've found this in the proxy logs:
2014-01-31 19:47:50,430 [server.THsHaServer] WARN : Got an
IOException in internalRead!
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at
org.apache.thrift.transport.TNonblockingSocket.read(TNonblockingSocket.java:141)
at
org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.internalRead(AbstractNonblockingServer.java:515)
at
org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.read(AbstractNonblockingServer.java:305)
at
org.apache.thrift.server.AbstractNonblockingServer$AbstractSelectThread.handleRead(AbstractNonblockingServer.java:202)
at
org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.select(TNonblockingServer.java:198)
at
org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.run(TNonblockingServer.java:154)
2014-01-31 19:51:13,185 [impl.ThriftTransportPool] WARN : Server
10.202.6.46:9997:9997 (30000) had 20 failures in a short time period,
will not complain anymore
A lot of this messages appear in all the proxies.
I tried the same stress tests agaisnt one proxy and I was able to
increase the load without getting any error.
Regards,
Diego
On Thu, Jan 30, 2014 at 2:47 PM, Keith Turner <[email protected]> wrote:
> Do you see more information in the proxy logs? "# exceptions 1" indicates
> an unexpected exception occured in the batch writer client code. The proxy
> uses this client code, so maybe there will be a more detailed stack trace in
> its logs.
>
>
> On Thu, Jan 30, 2014 at 9:46 AM, Diego Woitasen <[email protected]>
> wrote:
>>
>> Hi,
>> I'm testing with a ten node cluster with the proxy enabled in all the
>> nodes. I'm doing a stress test balancing the connection between the
>> proxies using round robin. When I increase the load (400 workers
>> writting) I get this error:
>>
>> AccumuloSecurityException:
>>
>> AccumuloSecurityException(msg='org.apache.accumulo.core.client.MutationsRejectedException:
>> # constraint violations : 0 security codes: [] # server errors 0 #
>> exceptions 1')
>>
>> The complete message is:
>>
>> AccumuloSecurityException:
>>
>> AccumuloSecurityException(msg='org.apache.accumulo.core.client.MutationsRejectedException:
>> # constraint violations : 0 security codes: [] # server errors 0 #
>> exceptions 1')
>> kvlayer-test client failed!
>> Traceback (most recent call last):
>> File "tests/kvlayer/test_accumulo_throughput.py", line 64, in __call__
>> self.client.put('t1', ((u,), self.one_mb))
>> File
>> "/home/ubuntu/kvlayer-env/local/lib/python2.7/site-packages/kvlayer-0.2.7-py2.7.egg/kvlayer/_decorators.py",
>> line 26, in wrapper
>> return method(*args, **kwargs)
>> File
>> "/home/ubuntu/kvlayer-env/local/lib/python2.7/site-packages/kvlayer-0.2.7-py2.7.egg/kvlayer/_accumulo.py",
>> line 154, in put
>> batch_writer.close()
>> File
>> "/home/ubuntu/kvlayer-env/local/lib/python2.7/site-packages/pyaccumulo_dev-1.5.0.2-py2.7.egg/pyaccumulo/__init__.py",
>> line 126, in close
>> self._conn.client.closeWriter(self._writer)
>> File
>> "/home/ubuntu/kvlayer-env/local/lib/python2.7/site-packages/pyaccumulo_dev-1.5.0.2-py2.7.egg/pyaccumulo/proxy/AccumuloProxy.py",
>> line 3149, in closeWriter
>> self.recv_closeWriter()
>> File
>> "/home/ubuntu/kvlayer-env/local/lib/python2.7/site-packages/pyaccumulo_dev-1.5.0.2-py2.7.egg/pyaccumulo/proxy/AccumuloProxy.py",
>> line 3172, in recv_closeWriter
>> raise result.ouch2
>>
>> I'm not sure if the errror is produced by the way I'm using the
>> cluster with multiple proxies, may be I should use one.
>>
>> Ideas are welcome.
>>
>> Regards,
>> Diego
>>
>> --
>> Diego Woitasen
>> VHGroup - Linux and Open Source solutions architect
>> www.vhgroup.net
>
>
--
Diego Woitasen
VHGroup - Linux and Open Source solutions architect
www.vhgroup.net
