[Sorry I missed this earlier due to some extended travel.] Can you show us the exact code for your putBlob calls? Have you tried attaching a profiler like YourKit or tool like file-leak-detector to your application? Thread, socket, or some other file descriptor leaks can cause weird behavior with the GC and finalizer which can manifest in the application. We have one report of unbounded connection use in JCLOUDS_628:
https://issues.apache.org/jira/browse/JCLOUDS-628 On Mon, Nov 17, 2014 at 02:49:05PM -0500, Daniel Hsueh wrote: > Hello, > > I'm experiencing behavior I can't explain and I'm wondering if you have any > thoughts that may help. > > I'm running jclouds-1.8.0 using provider swift (auth-v1) writing to a swift > store. In long test runs of my program (after writing 20-30k+ blobs of > approx 4MiB each), I will run into a flurry of failed writes (retries are > attempted, and fail), with a SocketTimeoutException "Read timed out" > reported. There are no problems on the swift side. > > I can't explain why the PUTs are failing. > > The only correlation I can find is that the failures occur when the old > generation garbage collector runs. I've tested this with CMS, G1, > ParallelGC, and in every instance, the failure occurrences occur with the > old gen runs. Young gen runs don't seem to cause a problem, and each old > gen GC run doesn't take more than 1sec (and the timeouts I have configured > are all much longer than 1sec). > > The jclouds code runs in a tomcat7 container and uses one instance of a > org.jclouds.blobstore.BlobStore. I am using the strip-expect-header flag. > > Thank you very very much for any ideas you have, either with the actual > problem or how to identify a root cause. > > Aside: Number of JVM threads seems to go up staircase-wise with time, but > I'm not sure how to prove the threads are created by jclouds, and I'm not > sure if they are/will cause a problem or not. > > -- > Daniel Hsueh -- mailto:[email protected] <[email protected]> > tel:+1-905-287-2167 -- Andrew Gaul http://gaul.org/
