As I said in previous response it is a difficult problem to track down. This is handled by the operation's team (not the API team) and will be looked at as resources allow. They know it's a problem but it's not the highest on their list. In the mean time, please make sure your application is robust, looks for truncated data and has retry logic when this error case is hit.
Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Tue, May 5, 2009 at 8:15 AM, atifzshaikh <[email protected]> wrote: > > Any updates on this issue? Received a few more cases today. Seems to > be caused by a timeout issue on twitter's end. > > Thanks, > > Atif > > On May 4, 10:35 am, atifzshaikh <[email protected]> wrote: > > Received 8 more cases over the weekend. Please let me know if you need > > more details. > > > > - Atif > > > > On Apr 30, 3:25 pm, atifzshaikh <[email protected]> wrote: > > > > > Just got 3 truncated XML requests from the users below, all of whom > > > have friend/follower sets > 1000. One thing to note about how my > > > application works. If the application is getting a user's friend/ > > > follower set and if the first GET request fails for whatever reason, > > > the application retries 2 more times. If after the 3 tries it still > > > fails to get a successful response from twitter it quits and assumes > > > twitter is over capacity or something else is wrong. I have detailed > > > these "re-tries" below for the 3 truncated XML requests I just > > > received. > > > > > =========================================== > > > First Failed Request > > > ----------------------------- > > > User: thevikings > > > Request: GET /statuses/friends.xml?page=14 > > > First Try: > > > - Response: Exception thrown! Message : The operation has timed > > > out. (I believe this is what caused the XML request to be truncated.) > > > - Exception Details: > > > - TargetSite: Int32 Read(Byte[], Int32, Int32) > > > - Stack Trace : at System.Net.ConnectStream.Read(Byte[] > > > buffer, Int32 offset, Int32 size) > > > at > > > System.IO.StreamReader.ReadBuffer() > > > at > > > System.IO.StreamReader.ReadToEnd() > > > Second Try: > > > - Response: Successful XML request received > > > > > =========================================== > > > Second Failed Request > > > ---------------------------------- > > > User: thevikings > > > Request: GET /statuses/friends.xml?page=18 > > > First Try: > > > - Response: Exception thrown! Message : The operation has timed > > > out. > > > - Exception Details: > > > - TargetSite: Int32 Read(Byte[], Int32, Int32) > > > - Stack Trace : at System.Net.ConnectStream.Read(Byte[] > > > buffer, Int32 offset, Int32 size) > > > at > > > System.IO.StreamReader.ReadBuffer() > > > at > > > System.IO.StreamReader.ReadToEnd() > > > Second Try: > > > - Response: The remote server returned an error: (502) Bad > > > Gateway. (don't know what caused this??) > > > - Exception Details: > > > - TargetSite: System.IO.Stream OpenRead(System.Uri) > > > - Stack Trace : at System.Net.WebClient.OpenRead(Uri > > > address) > > > at System.Net.WebClient.OpenRead > > > (String address) > > > Third Try: > > > - Response: Successful XML request received > > > > > =========================================== > > > Third Failed Request > > > ----------------------------- > > > User: ExciteCigi > > > Request: GET /statuses/friends.xml?page=9 > > > First Try: > > > - Response: Exception thrown! Message : The operation has timed > > > out. > > > - Exception Details: > > > - TargetSite: Int32 Read(Byte[], Int32, Int32) > > > - Stack Trace : at System.Net.ConnectStream.Read(Byte[] > > > buffer, Int32 offset, Int32 size) > > > at > > > System.IO.StreamReader.ReadBuffer() > > > at > > > System.IO.StreamReader.ReadToEnd() > > > Second Try: > > > - Response: Successful XML request received > > > =========================================== > > > > > As you can see the XML request gets truncated when the ReadToEnd() > > > operation times out. Hope this helps. > > > > > - Atif > > > > > On Apr 30, 2:16 pm, atifzshaikh <[email protected]> wrote: > > > > > > I have noticed that this issue occurs for users that have friend/ > > > > follower sets greater than 1000. For instance the three cases I > > > > mentioned in my previous post all had friend/follower sets in the > > > > thousands. There could be exceptions and maybe Dossy Shiobara has > come > > > > across cases where the sets were below a 1000, but for my application > > > > the sets were >= 1000. The users this happens to are pretty random > > > > and there is no one specific user that consistantly has their XML > > > > request truncated but I will let you know if I find any. > > > > > > Hope this helps. > > > > > > - Atif > > > > > > On Apr 30, 12:54 pm, Doug Williams <[email protected]> wrote: > > > > > > > How big are the friend/follower sets? Are they large? Is there a > user you > > > > > can consistently use to invoke this error? > > > > > > > Like I said, this is a difficult one to track do. Details and > > > > > reproducibility are helpful. > > > > > > > Thanks, > > > > > Doug > > > > > -- > > > > > > > Doug Williams > > > > > Twitter Platform Supporthttp://twitter.com/dougw > > > > > > > On Thu, Apr 30, 2009 at 8:05 AM, atifzshaikh < > [email protected]> wrote: > > > > > > > > Received two more similar issues yesterday and one this morning > at > > > > > > 10am. In all instances it was a GET request for either statuses/ > > > > > > friends or statuses/followers and the page parameter was always > > 1. > > > > > > If you need any more info please let me know. > > > > > > > > On Apr 29, 8:31 pm, Dossy Shiobara <[email protected]> wrote: > > > > > > > On 4/29/09 8:22 PM, Doug Williams wrote: > > > > > > > > > > Operations is going to look in to this. It is apparently a > known issue > > > > > > > > but very difficult to track down given the complexities of > our > > > > > > > > architecture so expect the fix to take a while. For now, > please make > > > > > > > > sure your application has logic to support this error case > gracefully. > > > > > > > > > Thanks, Matt. Anything I can do to help? Feel free to have > them > > > > > > > contact me directly if necessary. I'm fully versed in packet > capture > > > > > > > and analysis and I've been a sysadmin in various past lives. > > > > > > > > > -- > > > > > > > Dossy Shiobara | [email protected] | > http://dossy.org/ > > > > > > > Panoptic Computer Network |http://panoptic.com/ > > > > > > > "He realized the fastest way to change is to laugh at your > own > > > > > > > folly -- then you can let go and quickly move on." (p. 70) >
