You should probably put 100 instead of 1000 rather than playing with the timeouts.
-- Mike Stolz Principal Engineer, GemFire Product Manager Mobile: 631-835-4771 On Tue, Dec 20, 2016 at 11:23 AM, Dharam Thacker <[email protected]> wrote: > Yeah increasing read time out to 60 seconds worked out for me! > > I am putting 1000 objects with putAll with 7 functional indexes being > created synchronously! > > Thanks, > Dharam > > - Dharam Thacker > > On Tue, Dec 20, 2016 at 12:39 AM, Dan Smith <[email protected]> wrote: > >> You might consider upping the read-timeout on the client to see if that >> causes the issue to go away. The default read-timeout is 10 seconds. So if >> it's really just an issue of the server being slow and not responding in >> time a larger read timeout would help. >> >> How much data are you putting with this putAll? >> >> -Dan >> >> On Mon, Dec 19, 2016 at 10:02 AM, Thacker, Dharam < >> [email protected]> wrote: >> >>> Not sure if this is related with past bug as per link? There are few >>> more archive mails matching to same exception for jiras worked out in past! >>> >>> >>> >>> http://markmail.org/message/oefrls7bdof776t7?q=ServerConnect >>> ivityException&page=2 >>> >>> >>> >>> I have 7 functional indexes on server side for same region due to which >>> insertion happens slow, but that works fine if I mark region just as >>> REPLICATE. >>> >>> >>> >>> Regards, >>> >>> Dharam >>> >>> >>> >>> *From:* Thacker, Dharam >>> *Sent:* Monday, December 19, 2016 10:51 PM >>> >>> *To:* [email protected] >>> *Subject:* RE: Socket timeout after making regions persistent in async >>> way >>> >>> >>> >>> Yeah forgot to mention that it’s the only client connecting to server to >>> load region shown below! No high io/cpu consuming activity running on >>> server side! >>> >>> >>> >>> *From:* Thacker, Dharam >>> *Sent:* Monday, December 19, 2016 10:46 PM >>> *To:* [email protected] >>> *Subject:* RE: Socket timeout after making regions persistent in async >>> way >>> >>> >>> >>> Yeah sure! >>> >>> >>> >>> Client config looks simple. I have not enabled redundant copies. More >>> region is of type REPLICATE_PERSISTENT on server side. Server is just >>> initialized with –Xms = 20G and –Xmx = 30G with ConcMarkSweep GC along with >>> few more tuning parameters which works descent though! >>> >>> >>> >>> *Server logs:* >>> >>> >>> >>> [warn 2016/12/19 22:42:13.254 IST <ServerConnection on port 40404 Thread >>> 2> tid=0x5b] Server connection from >>> [identity(XXXXX(11700:loner):63652:3b290e18,connection=2; >>> port=63657]: Unexpected *IOException*: >>> >>> *java.net.SocketException*: Software caused connection abort: socket >>> write error >>> >>> at java.net.SocketOutputStream.socketWrite0(*Native Method*) >>> >>> at java.net.SocketOutputStream.socketWrite( >>> *SocketOutputStream.java:109*) >>> >>> at java.net.SocketOutputStream.write( >>> *SocketOutputStream.java:153*) >>> >>> at org.apache.geode.internal.cache.tier.sockets.Message.flushBu >>> ffer(*Message.java:649*) >>> >>> at org.apache.geode.internal.cache.tier.sockets.Message.sendByt >>> es(*Message.java:627*) >>> >>> at org.apache.geode.internal.cache.tier.sockets.ChunkedMessage. >>> sendChunk(*ChunkedMessage.java:314*) >>> >>> at org.apache.geode.internal.cache.tier.sockets.ChunkedMessage. >>> sendChunk(*ChunkedMessage.java:322*) >>> >>> at org.apache.geode.internal.cache.tier.sockets.command.PutAll8 >>> 0.writeReply(*PutAll80.java:388*) >>> >>> at org.apache.geode.internal.cache.tier.sockets.command.PutAll8 >>> 0.cmdExecute(*PutAll80.java:339*) >>> >>> at org.apache.geode.internal.cache.tier.sockets.BaseCommand.exe >>> cute(*BaseCommand.java:147*) >>> >>> at org.apache.geode.internal.cache.tier.sockets.ServerConnectio >>> n.doNormalMsg(*ServerConnection.java:783*) >>> >>> at org.apache.geode.internal.cache.tier.sockets.ServerConnectio >>> n.doOneMessage(*ServerConnection.java:913*) >>> >>> at org.apache.geode.internal.cache.tier.sockets.ServerConnectio >>> n.run(*ServerConnection.java:1180*) >>> >>> at java.util.concurrent.ThreadPoolExecutor.runWorker( >>> *ThreadPoolExecutor.java:1142*) >>> >>> at java.util.concurrent.ThreadPoolExecutor$Worker.run( >>> *ThreadPoolExecutor.java:617*) >>> >>> at org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$1$ >>> 1.run(*AcceptorImpl.java:546*) >>> >>> at java.lang.Thread.run(*Thread.java:745*) >>> >>> >>> >>> >>> >>> *Client config:* >>> >>> <gfe:pool id="event-geode-pool" subscription-enabled="true" >>> server-group="EVENT_GRP"> >>> >>> <gfe:localtor...> >>> >>> </gfe:pool> >>> >>> >>> >>> <gfe:client-region id="Event" >>> >>> cache-ref="event-cache" >>> >>> pool-name="event-geode-pool" >>> >>> shortcut="PROXY" >>> >>> key-constraint="java.lang.String" >>> >>> value-constraint="a.m.n.Event"> >>> >>> </gfe:client-region> >>> >>> >>> >>> >>> >>> <gfe:cq-listener-container id="cqListenerContainer" >>> cache="event-cache"> >>> >>> <gfe:listener ref="eventListener" >>> query="SELECT * FROM /Event WHERE status='PENDING'"/> >>> >>> </gfe:cq-listener-container> >>> >>> >>> >>> <bean id="eventListener" class="a.m.n.cqlisteners.MyEve >>> ntListener"/> >>> >>> >>> >>> Thanks, >>> >>> Dharam >>> >>> *From:* Udo Kohlmeyer [mailto:[email protected] >>> <[email protected]>] >>> *Sent:* Monday, December 19, 2016 10:26 PM >>> *To:* [email protected] >>> *Subject:* Re: Socket timeout after making regions persistent in async >>> way >>> >>> >>> >>> Hi there Dharam, >>> >>> It is possible that persistence has an effect on the performance of the >>> system. But generally socket timeouts occur due to other issues like GC or >>> system load. >>> >>> What is the server load around the time of the disconnection? Have you >>> enabled redundant copies for your cq? >>> >>> Could you please provide your client configuration for this problem. >>> >>> Maybe even a snippet of the server logs around the time of the client >>> disconnection. (a minute either side of the client disconnection is a good >>> start). >>> >>> --Udo >>> >>> >>> >>> On 12/19/16 05:22, Thacker, Dharam wrote: >>> >>> Hi, >>> >>> >>> >>> Has anyone faced below exceptions while cqlistener thread is working on? >>> It started coming once after I changes regions to be persistent. Am I >>> missing anything related to socket timeout? >>> >>> >>> >>> <geode:disk-store id=*"my_diskstore"* cache-ref=*"geodeCache"* >>> >>> compaction-threshold=*"50"* >>> >>> auto-compact=*"false"* >>> >>> allow-force-compaction=*"true"* >>> >>> max-oplog-size=*"512"* >>> >>> queue-size=*"10000"* >>> >>> time-interval=*"500"* >>> >>> write-buffer-size=*"65536"* >>> >>> disk-usage-warning-percentage=*"80"* >>> >>> disk-usage-critical-percentage=*"98"*> >>> >>> <geode:disk-dir location=*"mydata"* max-size=*"25600"*/> >>> >>> </geode:disk-store> >>> >>> >>> >>> It tells me that server is unreachable but bouncing client connects to >>> server again but after some interval of activity with server, again throws >>> back below exception! >>> >>> >>> >>> [warn 2016/12/19 18:43:22.608 IST <cqListenerContainer-1> tid=0x6c] Pool >>> unexpected socket timed out on client connection=Pooled Connection to >>> XXXX:40404: Connection[XXXX:40404]@1743344121). Server unreachable: >>> could not connect after 1 attempts >>> >>> >>> >>> [cqListenerContainer-1] ERROR o.s.d.g.l.a.ContinuousQueryListenerAdapter >>> Listener execution failed... >>> >>> *org.apache.geode.cache.client.ServerConnectivityException*: Pool >>> unexpected socket timed out on client connection=Pooled Connection to >>> XXXX:40404: Connection[XXXX:40404]@1743344121). Server unreachable: >>> could not connect after 1 attempts >>> >>> at org.apache.geode.cache.client.internal.OpExecutorImpl.handle >>> Exception(*OpExecutorImpl.java:819*) >>> >>> at org.apache.geode.cache.client.internal.OpExecutorImpl.handle >>> Exception(*OpExecutorImpl.java:603*) >>> >>> at org.apache.geode.cache.client.internal.OpExecutorImpl.execute( >>> *OpExecutorImpl.java:173*) >>> >>> at org.apache.geode.cache.client.internal.OpExecutorImpl.execute( >>> *OpExecutorImpl.java:110*) >>> >>> at org.apache.geode.cache.client.internal.PoolImpl.execute( >>> *PoolImpl.java:697*) >>> >>> at org.apache.geode.cache.client.internal.PutAllOp.execute( >>> *PutAllOp.java:110*) >>> >>> at org.apache.geode.cache.client.internal.ServerRegionProxy.put >>> All(*ServerRegionProxy.java:643*) >>> >>> at org.apache.geode.internal.cache.LocalRegion.basicPutAll( >>> *LocalRegion.java:10166*) >>> >>> at org.apache.geode.internal.cache.LocalRegion.putAll( >>> *LocalRegion.java:10101*) >>> >>> at org.apache.geode.internal.cache.LocalRegion.putAll( >>> *LocalRegion.java:10113*) >>> >>> at org.springframework.data.gemfire.GemfireTemplate.putAll( >>> *GemfireTemplate.java:201*) >>> >>> ... 25 common frames omitted >>> >>> >>> >>> Thanks, >>> >>> Dharam >>> >>> >>> >>> This email is confidential and subject to important disclaimers and >>> conditions including on offers for the purchase or sale of securities, >>> accuracy and completeness of information, viruses, confidentiality, legal >>> privilege, and legal entity disclaimers, available at >>> http://www.jpmorgan.com/pages/disclosures/email >>> >>> >>> >>> This email is confidential and subject to important disclaimers and >>> conditions including on offers for the purchase or sale of securities, >>> accuracy and completeness of information, viruses, confidentiality, legal >>> privilege, and legal entity disclaimers, available at >>> http://www.jpmorgan.com/pages/disclosures/email >>> >>> This email is confidential and subject to important disclaimers and >>> conditions including on offers for the purchase or sale of securities, >>> accuracy and completeness of information, viruses, confidentiality, legal >>> privilege, and legal entity disclaimers, available at >>> http://www.jpmorgan.com/pages/disclosures/email >>> >>> This email is confidential and subject to important disclaimers and >>> conditions including on offers for the purchase or sale of securities, >>> accuracy and completeness of information, viruses, confidentiality, legal >>> privilege, and legal entity disclaimers, available at >>> http://www.jpmorgan.com/pages/disclosures/email >>> >> >> >
