[twitter-dev] Re: non json response

Raffi Krikorian Wed, 16 Sep 2009 10:51:01 -0700


hi naveen.


we are most certainly working on it.

the best way to mitigate the error case is to actually do what theresponse tells you to do -- in all cases that i've seen the "http 200error", there has been a refresh header ("Refresh: 0.1"). simply obeythe header, make a subsequent request in 0.1 seconds, and more likelythan not, you will receive your request. in general, "be strict inwhat you send out, and be lenient in what you accept" is a pretty goodphilosophy to follow. an additional strategy to try: if you receivean error, then try again with some form of exponential back-off.

if you do see the error, please either send me, personally, a tcpdumpof the issue, or attach it to the issue on http://code.google.com/p/twitter-api/issues/detail?id=1024.


thanks!

Is there a specific way we can construct our request to mitigate the
non-json response? I have used a few different twitter clients on the
same mobile device and some of them do not seem to be plagued with the
bad data like we are? Does including something in the header help get
us through whatever filter is returning the bad data?

Maybe the Twitter cookies that are returned back on each request?
Currently, we don't pass them back on subsequent requests because they
shouldn't be necessary, but if it will make some of the bad JSON
responses go away, I'll spend the time to implement it..

These bad json responses have been a problem for over a month now and
while I realize it is a difficult problem to track down, the fact
remains that the API is not functioning correctly.

A response from the twitter team would be greatly appreciated.


On Sep 13, 6:01 am, Rudifa <rudi.far...@gmail.com> wrote:

I just had one non-json response, in the middle of about 10 requests
made with curl -vvv (other responses were correct)

Below are 3 requests and the non-json response bracketted by 2 good
responses which contain the response time and other logging data.

HTH
Rudi

rudolf-farkass-macbook-pro:TwitterWeb rudifarkas$ curl 
-vvvhttp://twitter.com/users/show/rudifa.json
* About to connect() to twitter.com port 80 (#0)
*   Trying 168.143.161.20... connected

* Connected to twitter.com (168.143.161.20) port 80 (#0)> GET /users/show/rudifa.json HTTP/1.1

User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3OpenSSL/0.9.7l zlib/1.2.3
Host: twitter.com
Accept: */*


< HTTP/1.1 200 OK
< Date: Sun, 13 Sep 2009 09:45:23 GMT
< Server: hi
< X-RateLimit-Limit: 150
< X-Transaction: 1252835123-2408-31139
< Status: 200 OK
< ETag: "df090f6c8147e20ba7fe81315a66b9af"
< Last-Modified: Sun, 13 Sep 2009 09:45:23 GMT
< X-RateLimit-Remaining: 124
< Content-Type: application/json; charset=utf-8
< Pragma: no-cache
< Content-Length: 1176
< Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
< Expires: Tue, 31 Mar 1981 05:00:00 GMT
< X-Revision: a62881015b2c2fb6f795bf931bd56bd494f37254
< X-RateLimit-Reset: 1252836853
< Set-Cookie: lang=en; path=/
< Set-Cookie:

_twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWU5OGQyZmU3NWVkY2RhZjhkYTk5%250ANTBlNTA4OTk0MzhhIgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz%250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--66931156c75554797fc576876bdec52dc705 736e;

domain=.twitter.com; path=/
< Vary: Accept-Encoding
< Connection: close
<
* Closing connection #0

{"profile_sidebar_border_color":"BDDCAD","description":"Wrotefirmware

for world-class osciloscopes for many years. Now learning iPhone
programming tricks. Loves
skiing.","url":null,"screen_name":"rudifa","status":

{"in_reply_to_status_id":null,"favorited":false,"in_reply_to_user_id":null, "source":"<a

href=\"http://apiwiki.twitter.com/\"; rel=\"nofollow\">API</
a>","created_at":"Thu Sep 10 16:49:49 +0000
2009","in_reply_to_screen_name":null,"id":
3890997267,"truncated":false,"text":"De retour de la T\u00eate de

Parmelan"},"following":null,"verified":false,"profile_text_color":"333333","followers_count":

9,"profile_background_image_url":"http://a1.twimg.com/
profile_background_images/17762518/
DSC01211-63-2.jpeg","created_at":"Thu Apr 30 22:42:35 +0000
2009","notifications":null,"friends_count":

29,"profile_link_color":"0084B4","profile_background_tile":false,"favourite s_count":0,"profile_background_color":"9AE4E8","protected":false,"time_zone":"Bern","location":"Geneva","name":"Rudi

Farkas","profile_sidebar_fill_color":"DDFFCC","id":
36797542,"statuses_count":52,"utc_offset":
3600,"profile_image_url":"http://a1.twimg.com/profile_images/311858510/
me-bsp-6a_normal.png"}

rudolf-farkass-macbook-pro:TwitterWeb rudifarkas$ curl 
-vvvhttp://twitter.com/users/show/rudifa.json
* About to connect() to twitter.com port 80 (#0)
*   Trying 168.143.161.20... connected

* Connected to twitter.com (168.143.161.20) port 80 (#0)> GET /users/show/rudifa.json HTTP/1.1

User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3OpenSSL/0.9.7l zlib/1.2.3
Host: twitter.com
Accept: */*


< HTTP/1.0 200 OK
< Connection: Close
< Pragma: no-cache
< cache-control: no-cache
< Refresh: 0.1
< Content-Type: text/html; charset=iso-8859-1
<
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/
TR/1999/REC-html401-19991224/strict.dtd">
<!-- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd";> -->
<HTML>
<HEAD>
<META HTTP-EQUIV="Refresh" CONTENT="0.1">
<META HTTP-EQUIV="Pragma" CONTENT="no-cache">
<META HTTP-EQUIV="Expires" CONTENT="-1">
<TITLE></TITLE>
</HEAD>
<BODY><P></BODY>
</HTML>
* Closing connection #0

rudolf-farkass-macbook-pro:TwitterWeb rudifarkas$ curl 
-vvvhttp://twitter.com/users/show/rudifa.json
* About to connect() to twitter.com port 80 (#0)
*   Trying 168.143.161.20... connected

* Connected to twitter.com (168.143.161.20) port 80 (#0)> GET /users/show/rudifa.json HTTP/1.1

User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3OpenSSL/0.9.7l zlib/1.2.3
Host: twitter.com
Accept: */*


< HTTP/1.1 200 OK
< Date: Sun, 13 Sep 2009 09:45:35 GMT
< Server: hi
< X-RateLimit-Limit: 150
< X-Transaction: 1252835135-57410-29384
< Status: 200 OK
< ETag: "df090f6c8147e20ba7fe81315a66b9af"
< Last-Modified: Sun, 13 Sep 2009 09:45:35 GMT
< X-RateLimit-Remaining: 123
< Content-Type: application/json; charset=utf-8
< Pragma: no-cache
< Content-Length: 1176
< Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
< Expires: Tue, 31 Mar 1981 05:00:00 GMT
< X-Revision: a62881015b2c2fb6f795bf931bd56bd494f37254
< X-RateLimit-Reset: 1252836853
< Set-Cookie: lang=en; path=/
< Set-Cookie:

_twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWY0NDM0ODQ3YWI2NDk5OTg4OTY3%250AYWQ1ZjU5Njg3OWIwIgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz

%250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--
d454f0c630e2e82ace19b7056947734184aea7ed; domain=.twitter.com; path=/
< Vary: Accept-Encoding
< Connection: close
<
* Closing connection #0

{"profile_sidebar_border_color":"BDDCAD","description":"Wrotefirmware

for world-class osciloscopes for many years. Now learning iPhone
programming tricks. Loves
skiing.","url":null,"screen_name":"rudifa","status":

{"in_reply_to_status_id":null,"favorited":false,"in_reply_to_user_id":null, "source":"<a

href=\"http://apiwiki.twitter.com/\"; rel=\"nofollow\">API</
a>","created_at":"Thu Sep 10 16:49:49 +0000
2009","in_reply_to_screen_name":null,"id":
3890997267,"truncated":false,"text":"De retour de la T\u00eate de

Parmelan"},"following":null,"verified":false,"profile_text_color":"333333","followers_count":

9,"profile_background_image_url":"http://a1.twimg.com/
profile_background_images/17762518/
DSC01211-63-2.jpeg","created_at":"Thu Apr 30 22:42:35 +0000
2009","notifications":null,"friends_count":

29,"profile_link_color":"0084B4","profile_background_tile":false,"favourite s_count":0,"profile_background_color":"9AE4E8","protected":false,"time_zone":"Bern","location":"Geneva","name":"Rudi

Farkas","profile_sidebar_fill_color":"DDFFCC","id":
36797542,"statuses_count":52,"utc_offset":
3600,"profile_image_url":"http://a1.twimg.com/profile_images/311858510/
me-bsp-6a_normal.png"}

On Sep 12, 6:57 pm, Raffi Krikorian <ra...@twitter.com> wrote:

Hi all.

This is an extremely high priority problem for us, and for me
personally, to fix.

If you're having this problem, please free to reach out to me, but
please try to include:

* the IP address your request is coming from
* the request you're making
* the response you got back
* the time that the request/response was made/received

(or, a tcpdump or curl output when doing -vvv would be sufficientalso)

On Sep 10, 2009, at 10:05 PM, Matthew Ranney <m...@ranney.com>wrote:

I'm seeing tons of these as well.

However, I've found that if you follow the suggestion of the META
tag to simply "refresh" in 0.1 seconds if you get this bogus
response, you can hide most of this from users, especially if they
are on a fast network.

On Thu, Sep 10, 2009 at 2:09 PM, Monica Keller<monica.kel...@gmail.com
wrote:

We see this error 75% of the time. Have you guys made an progresson
resolving the issue ?

On Sep 6, 8:14 pm, archF6 <tylerjpeter...@gmail.com> wrote:

I am able to consistently reproduce this error.  I am making GET
requests via PHP from IP: 96.30.16.192. I receive the errorwithout
fail after periods of inactivity lasting 2 hours or more.  The

header

response code is 200.  Please let me know if I can provide any
additional info that might help you diagnose the problem, or ifyou
have suggestions about how best to handle.

Thanks.

On Sep 6, 3:35 pm, Rudifa <rudi.far...@gmail.com> wrote:

I have seen this same http page with empty body
<!DOCTYPEhtmlPUBLIC "-//W3C//DTDHTML4.01//EN" "http://www.w3.org/
TR/1999/REC-html401-19991224/strict.dtd">
<!-- <!DOCTYPEHTMLPUBLIC "-//W3C//DTDHTML4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd";> -->
<HTML>
<HEAD>
<META HTTP-EQUIV="Refresh" CONTENT="0.1">
<META HTTP-EQUIV="Pragma" CONTENT="no-cache">
<META HTTP-EQUIV="Expires" CONTENT="-1">
<TITLE></TITLE>
</HEAD>
<BODY><P></BODY>
</HTML>
a number of times in the last few days (but intermittently - a

good

response may come after several attempts),
in response tohttp://twitter.com/users/show/rudifa.json

The most recent one was on UTC time
2009-09-06 18:55:38.262
My IP is 84.227.186.88 as reported byhttp://www.whatismyip.com/

Could someone at twitter.com please tell us what does this mean?

Server

(s) overloaded?

On Aug 30, 1:20 pm, Steven Wilkin <iamthebisc...@gmail.com>wrote:

I'm consistently getting the same response when

accessinghttp://search.twitter.com/trends.jsonfrom209.40.204.183

Steve

On Aug 26, 5:27 pm, Ryan Sarver <rsar...@twitter.com> wrote:

Ben,

It's a known issue and we are trying to hunt it down. Can

you please

provide us with your source IP and an approximate time of

when you saw

it?

Thanks, RyanOn Wed, Aug 26, 2009 at 7:00 AM,

ben<ben.apperr...@googlemail.com> wrote:

Occassionally i get back a 200 statushtmlresponse from the

json

search api which look like this, most times the


...

read more »


--
Raffi Krikorian
Twitter Platform Team
ra...@twitter.com | @raffi

[twitter-dev] Re: non json response

Reply via email to