Hmm... looking at the stack trace of a stuck thread, I'd be inclined to guess that this problem was actually outside of Fedora, and relates either to the client consuming the resource, or the tcp connection itself.
Here's the last part of the stack trace: [1] java.net.SocketOutputStream.socketWrite0 (native method) [2] java.net.SocketOutputStream.socketWrite (SocketOutputStream.java:92) [3] java.net.SocketOutputStream.write (SocketOutputStream.java:136) To me, this implies that there are some bytes of content that trying to be sent, but for some reason cannot. As far as I understand, socketWrite0 will block until all bytes given in write() are successfully sent over the tcp connection. The bytes cannot be sent either because (a) the client cannot accept them for some fault of its own, and all buffers are filled or (b) something is wrong with the tcp connection. Digging around, I also found an assertion: "the blocking Java Socket API doesn't have a timeout on socketWrite invocations. This means, if a TCP session is dropped or terminated without RST or FIN packets, the operating system it left to eventually time out the session" http://mail-archives.apache.org/mod_mbox/activemq-dev/200811.mbox/% 3C1825990017.1225721586694.JavaMail.jira@brutus%3E As just about anybody in any web browser can reproduce this, I would guess that (a) is not a problem, leaving (b) something frequently is going wrong with tcp connections outside of the control of Tomcat or the browser. I could be wrong too... Say, could this be reproduced by trying to get the same image over and over again until it fails? -Aaron On Thu, 2011-04-28 at 12:41 -0400, Wolff, Robert wrote: > Our install.properties is attached. The only update is that we now have the > resource index enabled. > > The only additionally interesting thing about our setup is that we are > redirecting port 8080 to 80 at the kernel level using iptables. Obviously, > this will change if we use httpd in addition to Tomcat. > > Rob > > > -----Original Message----- > From: Stephen Bayliss [mailto:stephen.bayl...@acuityunlimited.net] > Sent: Thursday, April 28, 2011 12:24 PM > To: 'Support and info exchange list for Fedora users.' > Subject: Re: [fcrepo-user] Http response 200 but no payload? > > There's no migration utility currently, but there is a JIRA issue to provide > one - https://jira.duraspace.org/browse/FCREPO-769 > > The legacy low-level storage simply wraps a file in a FileInputStream, so I > think it is unlikely to be that. I believe (but haven't checked) that the > resulting InputStream would be passed all the way out. > > Is there anything else "interesting" about your configuration - maybe it > would be worth seeing a copy of your install.properties (with passwords > removed) so we can see what layers the response etc is likely to pass > through. > > Regards > Steve > > -----Original Message----- > From: Wolff, Robert [mailto:rob.wo...@unh.edu] > Sent: 28 April 2011 15:43 > To: Support and info exchange list for Fedora users. > Subject: Re: [fcrepo-user] Http response 200 but no payload? > > > Interesting. We're using the legacy low-level storage. Is there a > migration utility available for the switch to Akubra? > > Our current next step is to try putting Apache httpd in front of Tomcat and > see if its improved socket handling would help. > > Rob > > > > -----Original Message----- > From: Stephen Bayliss [mailto:stephen.bayl...@acuityunlimited.net] > Sent: Thursday, April 28, 2011 10:13 AM > To: 'Support and info exchange list for Fedora users.' > Subject: Re: [fcrepo-user] Http response 200 but no payload? > > I just hit a similar (but unrelated) issue, the underlying cause of which > was a PrintWriter on the HTTP response not being flushed - I got partial > content, but it was intermittent. > > So I wonder if this could be related? > > Are you using the legacy low-level storage implementation or Akubra? > > (And, if you have the opportunity to swap one for the other and see if the > problem persists then that may shed some more light.) > > Steve > > > > -----Original Message----- > From: Wolff, Robert [mailto:rob.wo...@unh.edu] > Sent: 22 April 2011 18:58 > To: Support and info exchange list for Fedora users. > Subject: Re: [fcrepo-user] Http response 200 but no payload? > > > I've tried the troubleshooting methods Aaron suggested. > > I should amend the problem to say that the server transmits some, but not > all, of the data for a requested datastream. I've used a better packet > trace (Wireshark instead of Fiddler) to confirm this. > > strace indicates the jvm is sending (sendto()) all of the data to the > socket. There doesn't appear to be any blocked system calls, unless I'm > missing something. > > jdb indicates that the thread(s) that get hung up are busy with > java.net.SocketOutputStream.socketWrite0. I've attached the full stack > trace. > > Does this sound like a Fedora issue? Are there relevant configuration > options we should check in Fedora, Tomcat, or the OS (RHEL5) that maybe we > hadn't considered? > > Thanks for any help. > > Rob > > > -----Original Message----- > From: Aaron Birkland [mailto:birkl...@cs.cornell.edu] > Sent: Wednesday, April 06, 2011 2:37 PM > To: Support and info exchange list for Fedora users. > Subject: Re: [fcrepo-user] Http response 200 but no payload? > > Hi Rob, > > > The problem is intermittent, unfortunately. (Or, at least, I have not > > been able to isolate the problem case.) > > Next time it happens, you could try running strace -f against the java vm to > to see if there any blocking system calls that are stuck waiting (like > read() or socket()). > > If you have the freedom to do so at this point, you could also have the java > vm run in -Xdebug mode so that you can jump in, pause execution, and examine > the state of the java threads whenever you see this happening. > > Between the two of these, I think you stand a fairly good chance of > narrowing down the basic problem area. > > By the way, what is the control group of these datastreams. Are they all > Managed? > > -Aaron > > > > > > We're having an issue wherein Fedora/Tomcat will respond to a > > REST API-A request with 200 and Transfer-Encoding: chunked but > > then sends no chunks. Here's a typical response header: > > > > HTTP/1.1 200 OK > > Server: Apache-Coyote/1.1 > > Pragma: No-cache > > Cache-Control: no-cache > > Expires: Wed, 31 Dec 1969 19:00:00 EST > > content-disposition: inline; filename="Annual report Town of > > Center Harbor New Hampshire 2008.jpg" > > Content-Type: image/jpeg > > Transfer-Encoding: chunked > > Date: Sun, 27 Mar 2011 01:46:23 GMT > > > > ...and that's it, no payload. > > > > > ---------------------------------------------------------------------------- > -- > Xperia(TM) PLAY > It's a major breakthrough. An authentic gaming > smartphone on the nation's most reliable network. > And it wants your games. > http://p.sf.net/sfu/verizon-sfdev > _______________________________________________ > Fedora-commons-users mailing list Fedora-commons-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/fedora-commons-users > > > ---------------------------------------------------------------------------- > -- > WhatsUp Gold - Download Free Network Management Software > The most intuitive, comprehensive, and cost-effective network > management toolset available today. Delivers lowest initial > acquisition cost and overall TCO of any competing solution. > http://p.sf.net/sfu/whatsupgold-sd > _______________________________________________ > Fedora-commons-users mailing list Fedora-commons-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/fedora-commons-users > > ---------------------------------------------------------------------------- > -- > WhatsUp Gold - Download Free Network Management Software > The most intuitive, comprehensive, and cost-effective network > management toolset available today. Delivers lowest initial > acquisition cost and overall TCO of any competing solution. > http://p.sf.net/sfu/whatsupgold-sd > _______________________________________________ > Fedora-commons-users mailing list Fedora-commons-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/fedora-commons-users > > > ------------------------------------------------------------------------------ > WhatsUp Gold - Download Free Network Management Software > The most intuitive, comprehensive, and cost-effective network > management toolset available today. Delivers lowest initial > acquisition cost and overall TCO of any competing solution. > http://p.sf.net/sfu/whatsupgold-sd > _______________________________________________ > Fedora-commons-users mailing list > Fedora-commons-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/fedora-commons-users ------------------------------------------------------------------------------ WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd _______________________________________________ Fedora-commons-users mailing list Fedora-commons-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
