I'm using the Apache HTTP Components to fetch data from Twitter and
I've run across something weird. I'm requesting
http://twitter.com/statuses/user_timeline.json for a given user,
specifying the since_id parameter.
The client starts recieving the data, but it stops at 2048
characters. As far as I can tell, it's really getting the first 2K of
the data, but then it's just quitting. For example:
[08/26/09 15:35:11:572] INFO Processing dcurtis (9395832)
[08/26/09 15:35:13:692] INFO content length: 14051 charset: utf-8
[08/26/09 15:35:13:693] SEVERE Error at 2049 of 14051 after 0.554
[08/26/09 15:35:13:694] SEVERE Error getting status list
java.net.SocketException: Socket closed
at java.net.SocketInputStream.socketRead0(Native Method)
This happens reliably and repeatable at the 2049th character. That
0.554 is the number of milliseconds that it was reading data from the
socket before it got the error (give or take a nanosecond, eh?) Other
API calls don't suffer from this problem, although I don't know if any
of them are generating more than 2K of data.
The weird thing is that I can use curl from the command line to fetch
the same URL and it seems to get the whole thing just fine. Is curl
perhaps recovering from an error that the Apache HTTP client should be
able to recover from as well (socket closed seems pretty un-
recoverable to me)?
I've been using the HTTP Components library for a while now for this
stuff, and this has just started being a problem in the last few days.
Any help would be appreciated!