Just got 3 truncated XML requests from the users below, all of whom
have friend/follower sets > 1000. One thing to note about how my
application works. If the application is getting a user's friend/
follower set and if the first GET request fails for whatever reason,
the application retries 2 more times. If after the 3 tries it still
fails to get a successful response from twitter it quits and assumes
twitter is over capacity or something else is wrong. I have detailed
these "re-tries" below for the 3 truncated XML requests I just
received.
===========================================
First Failed Request
-----------------------------
User: thevikings
Request: GET /statuses/friends.xml?page=14
First Try:
- Response: Exception thrown! Message : The operation has timed
out. (I believe this is what caused the XML request to be truncated.)
- Exception Details:
- TargetSite: Int32 Read(Byte[], Int32, Int32)
- Stack Trace : at System.Net.ConnectStream.Read(Byte[]
buffer, Int32 offset, Int32 size)
at
System.IO.StreamReader.ReadBuffer()
at
System.IO.StreamReader.ReadToEnd()
Second Try:
- Response: Successful XML request received
===========================================
Second Failed Request
----------------------------------
User: thevikings
Request: GET /statuses/friends.xml?page=18
First Try:
- Response: Exception thrown! Message : The operation has timed
out.
- Exception Details:
- TargetSite: Int32 Read(Byte[], Int32, Int32)
- Stack Trace : at System.Net.ConnectStream.Read(Byte[]
buffer, Int32 offset, Int32 size)
at
System.IO.StreamReader.ReadBuffer()
at
System.IO.StreamReader.ReadToEnd()
Second Try:
- Response: The remote server returned an error: (502) Bad
Gateway. (don't know what caused this??)
- Exception Details:
- TargetSite: System.IO.Stream OpenRead(System.Uri)
- Stack Trace : at System.Net.WebClient.OpenRead(Uri
address)
at System.Net.WebClient.OpenRead
(String address)
Third Try:
- Response: Successful XML request received
===========================================
Third Failed Request
-----------------------------
User: ExciteCigi
Request: GET /statuses/friends.xml?page=9
First Try:
- Response: Exception thrown! Message : The operation has timed
out.
- Exception Details:
- TargetSite: Int32 Read(Byte[], Int32, Int32)
- Stack Trace : at System.Net.ConnectStream.Read(Byte[]
buffer, Int32 offset, Int32 size)
at
System.IO.StreamReader.ReadBuffer()
at
System.IO.StreamReader.ReadToEnd()
Second Try:
- Response: Successful XML request received
===========================================
As you can see the XML request gets truncated when the ReadToEnd()
operation times out. Hope this helps.
- Atif
On Apr 30, 2:16 pm, atifzshaikh <[email protected]> wrote:
> I have noticed that this issue occurs for users that have friend/
> follower sets greater than 1000. For instance the three cases I
> mentioned in my previous post all had friend/follower sets in the
> thousands. There could be exceptions and maybe Dossy Shiobara has come
> across cases where the sets were below a 1000, but for my application
> the sets were >= 1000. The users this happens to are pretty random
> and there is no one specific user that consistantly has their XML
> request truncated but I will let you know if I find any.
>
> Hope this helps.
>
> - Atif
>
> On Apr 30, 12:54 pm, Doug Williams <[email protected]> wrote:
>
> > How big are the friend/follower sets? Are they large? Is there a user you
> > can consistently use to invoke this error?
>
> > Like I said, this is a difficult one to track do. Details and
> > reproducibility are helpful.
>
> > Thanks,
> > Doug
> > --
>
> > Doug Williams
> > Twitter Platform Supporthttp://twitter.com/dougw
>
> > On Thu, Apr 30, 2009 at 8:05 AM, atifzshaikh <[email protected]> wrote:
>
> > > Received two more similar issues yesterday and one this morning at
> > > 10am. In all instances it was a GET request for either statuses/
> > > friends or statuses/followers and the page parameter was always > 1.
> > > If you need any more info please let me know.
>
> > > On Apr 29, 8:31 pm, Dossy Shiobara <[email protected]> wrote:
> > > > On 4/29/09 8:22 PM, Doug Williams wrote:
>
> > > > > Operations is going to look in to this. It is apparently a known issue
> > > > > but very difficult to track down given the complexities of our
> > > > > architecture so expect the fix to take a while. For now, please make
> > > > > sure your application has logic to support this error case gracefully.
>
> > > > Thanks, Matt. Anything I can do to help? Feel free to have them
> > > > contact me directly if necessary. I'm fully versed in packet capture
> > > > and analysis and I've been a sysadmin in various past lives.
>
> > > > --
> > > > Dossy Shiobara | [email protected] |http://dossy.org/
> > > > Panoptic Computer Network |http://panoptic.com/
> > > > "He realized the fastest way to change is to laugh at your own
> > > > folly -- then you can let go and quickly move on." (p. 70)
