[twitter-dev] Can't get friends/followers list after page 101
Hi, I have been trying to get the friends/followers list using the REST API but I always get an empty users node after page 101. The GET request URL looks like this: http://www.twitter.com/statuses/followers.xml?screen_name=barackobamapage=102 I get the same result regardless of type, XML or JSON, and it happens when retrieving the friends list as well. All my requests are authenticated using OAuth, but even if I use the web browser to make an unauthenticated request and the put the above URL in I get the same result. At first I thought it was a rate limit issue but the IP address I am making the requests from is white listed and when I made the request I checked my rate limit and it was more than 10k at the time. We all know Barack Obama has more than 101 pages of followers so it can't be the last page. Any help or advice will be appreciated. Thanks
[twitter-dev] Re: Can't get friends/followers list after page 101
Seems to be working now, I guess it was a temporary thing. Would help if I knew what caused it :) thanks. On Jul 22, 3:55 pm, atifzshaikh atif.zsha...@gmail.com wrote: Hi, I have been trying to get the friends/followers list using the REST API but I always get an empty users node after page 101. The GET request URL looks like this: http://www.twitter.com/statuses/followers.xml?screen_name=barackobama... I get the same result regardless of type, XML or JSON, and it happens when retrieving the friends list as well. All my requests are authenticated using OAuth, but even if I use the web browser to make an unauthenticated request and the put the above URL in I get the same result. At first I thought it was a rate limit issue but the IP address I am making the requests from is white listed and when I made the request I checked my rate limit and it was more than 10k at the time. We all know Barack Obama has more than 101 pages of followers so it can't be the last page. Any help or advice will be appreciated. Thanks
[twitter-dev] Re: Seeing truncated XML responses again
Any updates on this issue? Received a few more cases today. Seems to be caused by a timeout issue on twitter's end. Thanks, Atif On May 4, 10:35 am, atifzshaikh atif.zsha...@gmail.com wrote: Received 8 more cases over the weekend. Please let me know if you need more details. - Atif On Apr 30, 3:25 pm, atifzshaikh atif.zsha...@gmail.com wrote: Just got 3 truncated XML requests from the users below, all of whom have friend/follower sets 1000. One thing to note about how my application works. If the application is getting a user's friend/ follower set and if the first GET request fails for whatever reason, the application retries 2 more times. If after the 3 tries it still fails to get a successful response from twitter it quits and assumes twitter is over capacity or something else is wrong. I have detailed these re-tries below for the 3 truncated XML requests I just received. === First Failed Request - User: thevikings Request: GET /statuses/friends.xml?page=14 First Try: - Response: Exception thrown! Message : The operation has timed out. (I believe this is what caused the XML request to be truncated.) - Exception Details: - TargetSite: Int32 Read(Byte[], Int32, Int32) - Stack Trace : at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.IO.StreamReader.ReadBuffer() at System.IO.StreamReader.ReadToEnd() Second Try: - Response: Successful XML request received === Second Failed Request -- User: thevikings Request: GET /statuses/friends.xml?page=18 First Try: - Response: Exception thrown! Message : The operation has timed out. - Exception Details: - TargetSite: Int32 Read(Byte[], Int32, Int32) - Stack Trace : at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.IO.StreamReader.ReadBuffer() at System.IO.StreamReader.ReadToEnd() Second Try: - Response: The remote server returned an error: (502) Bad Gateway. (don't know what caused this??) - Exception Details: - TargetSite: System.IO.Stream OpenRead(System.Uri) - Stack Trace : at System.Net.WebClient.OpenRead(Uri address) at System.Net.WebClient.OpenRead (String address) Third Try: - Response: Successful XML request received === Third Failed Request - User: ExciteCigi Request: GET /statuses/friends.xml?page=9 First Try: - Response: Exception thrown! Message : The operation has timed out. - Exception Details: - TargetSite: Int32 Read(Byte[], Int32, Int32) - Stack Trace : at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.IO.StreamReader.ReadBuffer() at System.IO.StreamReader.ReadToEnd() Second Try: - Response: Successful XML request received === As you can see the XML request gets truncated when the ReadToEnd() operation times out. Hope this helps. - Atif On Apr 30, 2:16 pm, atifzshaikh atif.zsha...@gmail.com wrote: I have noticed that this issue occurs for users that have friend/ follower sets greater than 1000. For instance the three cases I mentioned in my previous post all had friend/follower sets in the thousands. There could be exceptions and maybe Dossy Shiobara has come across cases where the sets were below a 1000, but for my application the sets were = 1000. The users this happens to are pretty random and there is no one specific user that consistantly has their XML request truncated but I will let you know if I find any. Hope this helps. - Atif On Apr 30, 12:54 pm, Doug Williams d...@twitter.com wrote: How big are the friend/follower sets? Are they large? Is there a user you can consistently use to invoke this error? Like I said, this is a difficult one to track do. Details and reproducibility are helpful. Thanks, Doug -- Doug Williams Twitter Platform Supporthttp://twitter.com/dougw On Thu, Apr 30, 2009 at 8:05 AM, atifzshaikh atif.zsha...@gmail.com wrote: Received two more similar issues yesterday and one this morning at 10am. In all instances it was a GET request for either statuses/ friends or statuses/followers and the page parameter was always 1. If you need any more info please let me know. On Apr
[twitter-dev] Re: Seeing truncated XML responses again
Received 8 more cases over the weekend. Please let me know if you need more details. - Atif On Apr 30, 3:25 pm, atifzshaikh atif.zsha...@gmail.com wrote: Just got 3 truncated XML requests from the users below, all of whom have friend/follower sets 1000. One thing to note about how my application works. If the application is getting a user's friend/ follower set and if the first GET request fails for whatever reason, the application retries 2 more times. If after the 3 tries it still fails to get a successful response from twitter it quits and assumes twitter is over capacity or something else is wrong. I have detailed these re-tries below for the 3 truncated XML requests I just received. === First Failed Request - User: thevikings Request: GET /statuses/friends.xml?page=14 First Try: - Response: Exception thrown! Message : The operation has timed out. (I believe this is what caused the XML request to be truncated.) - Exception Details: - TargetSite: Int32 Read(Byte[], Int32, Int32) - Stack Trace : at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.IO.StreamReader.ReadBuffer() at System.IO.StreamReader.ReadToEnd() Second Try: - Response: Successful XML request received === Second Failed Request -- User: thevikings Request: GET /statuses/friends.xml?page=18 First Try: - Response: Exception thrown! Message : The operation has timed out. - Exception Details: - TargetSite: Int32 Read(Byte[], Int32, Int32) - Stack Trace : at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.IO.StreamReader.ReadBuffer() at System.IO.StreamReader.ReadToEnd() Second Try: - Response: The remote server returned an error: (502) Bad Gateway. (don't know what caused this??) - Exception Details: - TargetSite: System.IO.Stream OpenRead(System.Uri) - Stack Trace : at System.Net.WebClient.OpenRead(Uri address) at System.Net.WebClient.OpenRead (String address) Third Try: - Response: Successful XML request received === Third Failed Request - User: ExciteCigi Request: GET /statuses/friends.xml?page=9 First Try: - Response: Exception thrown! Message : The operation has timed out. - Exception Details: - TargetSite: Int32 Read(Byte[], Int32, Int32) - Stack Trace : at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.IO.StreamReader.ReadBuffer() at System.IO.StreamReader.ReadToEnd() Second Try: - Response: Successful XML request received === As you can see the XML request gets truncated when the ReadToEnd() operation times out. Hope this helps. - Atif On Apr 30, 2:16 pm, atifzshaikh atif.zsha...@gmail.com wrote: I have noticed that this issue occurs for users that have friend/ follower sets greater than 1000. For instance the three cases I mentioned in my previous post all had friend/follower sets in the thousands. There could be exceptions and maybe Dossy Shiobara has come across cases where the sets were below a 1000, but for my application the sets were = 1000. The users this happens to are pretty random and there is no one specific user that consistantly has their XML request truncated but I will let you know if I find any. Hope this helps. - Atif On Apr 30, 12:54 pm, Doug Williams d...@twitter.com wrote: How big are the friend/follower sets? Are they large? Is there a user you can consistently use to invoke this error? Like I said, this is a difficult one to track do. Details and reproducibility are helpful. Thanks, Doug -- Doug Williams Twitter Platform Supporthttp://twitter.com/dougw On Thu, Apr 30, 2009 at 8:05 AM, atifzshaikh atif.zsha...@gmail.com wrote: Received two more similar issues yesterday and one this morning at 10am. In all instances it was a GET request for either statuses/ friends or statuses/followers and the page parameter was always 1. If you need any more info please let me know. On Apr 29, 8:31 pm, Dossy Shiobara do...@panoptic.com wrote: On 4/29/09 8:22 PM, Doug Williams wrote: Operations is going to look in to this. It is apparently a known issue but very difficult to track down given the complexities of our architecture so expect the fix to take a while
[twitter-dev] Re: Seeing truncated XML responses again
Received two more similar issues yesterday and one this morning at 10am. In all instances it was a GET request for either statuses/ friends or statuses/followers and the page parameter was always 1. If you need any more info please let me know. On Apr 29, 8:31 pm, Dossy Shiobara do...@panoptic.com wrote: On 4/29/09 8:22 PM, Doug Williams wrote: Operations is going to look in to this. It is apparently a known issue but very difficult to track down given the complexities of our architecture so expect the fix to take a while. For now, please make sure your application has logic to support this error case gracefully. Thanks, Matt. Anything I can do to help? Feel free to have them contact me directly if necessary. I'm fully versed in packet capture and analysis and I've been a sysadmin in various past lives. -- Dossy Shiobara | do...@panoptic.com |http://dossy.org/ Panoptic Computer Network |http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70)
[twitter-dev] Re: Seeing truncated XML responses again
I have noticed that this issue occurs for users that have friend/ follower sets greater than 1000. For instance the three cases I mentioned in my previous post all had friend/follower sets in the thousands. There could be exceptions and maybe Dossy Shiobara has come across cases where the sets were below a 1000, but for my application the sets were = 1000. The users this happens to are pretty random and there is no one specific user that consistantly has their XML request truncated but I will let you know if I find any. Hope this helps. - Atif On Apr 30, 12:54 pm, Doug Williams d...@twitter.com wrote: How big are the friend/follower sets? Are they large? Is there a user you can consistently use to invoke this error? Like I said, this is a difficult one to track do. Details and reproducibility are helpful. Thanks, Doug -- Doug Williams Twitter Platform Supporthttp://twitter.com/dougw On Thu, Apr 30, 2009 at 8:05 AM, atifzshaikh atif.zsha...@gmail.com wrote: Received two more similar issues yesterday and one this morning at 10am. In all instances it was a GET request for either statuses/ friends or statuses/followers and the page parameter was always 1. If you need any more info please let me know. On Apr 29, 8:31 pm, Dossy Shiobara do...@panoptic.com wrote: On 4/29/09 8:22 PM, Doug Williams wrote: Operations is going to look in to this. It is apparently a known issue but very difficult to track down given the complexities of our architecture so expect the fix to take a while. For now, please make sure your application has logic to support this error case gracefully. Thanks, Matt. Anything I can do to help? Feel free to have them contact me directly if necessary. I'm fully versed in packet capture and analysis and I've been a sysadmin in various past lives. -- Dossy Shiobara | do...@panoptic.com |http://dossy.org/ Panoptic Computer Network |http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70)
[twitter-dev] Re: Seeing truncated XML responses again
Just got 3 truncated XML requests from the users below, all of whom have friend/follower sets 1000. One thing to note about how my application works. If the application is getting a user's friend/ follower set and if the first GET request fails for whatever reason, the application retries 2 more times. If after the 3 tries it still fails to get a successful response from twitter it quits and assumes twitter is over capacity or something else is wrong. I have detailed these re-tries below for the 3 truncated XML requests I just received. === First Failed Request - User: thevikings Request: GET /statuses/friends.xml?page=14 First Try: - Response: Exception thrown! Message : The operation has timed out. (I believe this is what caused the XML request to be truncated.) - Exception Details: - TargetSite: Int32 Read(Byte[], Int32, Int32) - Stack Trace : at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.IO.StreamReader.ReadBuffer() at System.IO.StreamReader.ReadToEnd() Second Try: - Response: Successful XML request received === Second Failed Request -- User: thevikings Request: GET /statuses/friends.xml?page=18 First Try: - Response: Exception thrown! Message : The operation has timed out. - Exception Details: - TargetSite: Int32 Read(Byte[], Int32, Int32) - Stack Trace : at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.IO.StreamReader.ReadBuffer() at System.IO.StreamReader.ReadToEnd() Second Try: - Response: The remote server returned an error: (502) Bad Gateway. (don't know what caused this??) - Exception Details: - TargetSite: System.IO.Stream OpenRead(System.Uri) - Stack Trace : at System.Net.WebClient.OpenRead(Uri address) at System.Net.WebClient.OpenRead (String address) Third Try: - Response: Successful XML request received === Third Failed Request - User: ExciteCigi Request: GET /statuses/friends.xml?page=9 First Try: - Response: Exception thrown! Message : The operation has timed out. - Exception Details: - TargetSite: Int32 Read(Byte[], Int32, Int32) - Stack Trace : at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.IO.StreamReader.ReadBuffer() at System.IO.StreamReader.ReadToEnd() Second Try: - Response: Successful XML request received === As you can see the XML request gets truncated when the ReadToEnd() operation times out. Hope this helps. - Atif On Apr 30, 2:16 pm, atifzshaikh atif.zsha...@gmail.com wrote: I have noticed that this issue occurs for users that have friend/ follower sets greater than 1000. For instance the three cases I mentioned in my previous post all had friend/follower sets in the thousands. There could be exceptions and maybe Dossy Shiobara has come across cases where the sets were below a 1000, but for my application the sets were = 1000. The users this happens to are pretty random and there is no one specific user that consistantly has their XML request truncated but I will let you know if I find any. Hope this helps. - Atif On Apr 30, 12:54 pm, Doug Williams d...@twitter.com wrote: How big are the friend/follower sets? Are they large? Is there a user you can consistently use to invoke this error? Like I said, this is a difficult one to track do. Details and reproducibility are helpful. Thanks, Doug -- Doug Williams Twitter Platform Supporthttp://twitter.com/dougw On Thu, Apr 30, 2009 at 8:05 AM, atifzshaikh atif.zsha...@gmail.com wrote: Received two more similar issues yesterday and one this morning at 10am. In all instances it was a GET request for either statuses/ friends or statuses/followers and the page parameter was always 1. If you need any more info please let me know. On Apr 29, 8:31 pm, Dossy Shiobara do...@panoptic.com wrote: On 4/29/09 8:22 PM, Doug Williams wrote: Operations is going to look in to this. It is apparently a known issue but very difficult to track down given the complexities of our architecture so expect the fix to take a while. For now, please make sure your application has logic to support this error case gracefully. Thanks, Matt. Anything I can do to help? Feel free to have them contact me directly if necessary. I'm fully versed in packet capture and analysis and I've been
