[twitter-dev] Re: 401 Unauthorized -- user_timeline -- using Numeric ID
If you try to pull a protected users' timeline w/o authentication then
it will throw a 401.
You can confirm whether a user id is valid by trying to call:
http://twitter.com/users/show.xml?id={insert_id_here}
users/show will throw a 404 on a non existent user.
-Chad
On Wed, Oct 21, 2009 at 1:43 AM, Ryan Rosario uclamath...@gmail.com wrote:
*bangs head on desk...again*
Now that I know what what my Twitter ID is, and I know that I am a
valid user, I tried it and it worked.
It's gone way past 15 users now without a problem (without error
checking).
I see what is happening now. After 15 users or so, I hit a
user that no longer exists. On a 404, I skip the user, but on a 401,
for the time being, I am retrying, and then quickly maxing out my
hourly
limit.
The other part was my overworked mind. I was pulling numbers that
looked like Twitter IDs from the site HTML that actually weren't.
And...then testing over and over with those bad numbers. Sigh.
I suppose it is safe to treat 401 as a 404? I don't think it would be
used for anything else in the REST API, right?
Is an HTTP error thrown if I try to pull a protected user's timeline?
To make a long story short, it does appear that invalid users are
returning 401 instead of 404.
Thanks,
Ryan
On Oct 20, 9:10 pm, Chad Etzel c...@twitter.com wrote:
Hi Ryan,
I think part of the problem is that user id 1255997062 does not exist,
so it should really be returning a 404 instead of a 401. Do you get
the same 401 result with a known valid user id?
-Chad
On Tue, Oct 20, 2009 at 3:13 PM, Ryan Rosario uclamath...@gmail.com wrote:
Correct. I thought that was weird, but didn't know what to make of it.
Same IP.
The unauthenticated call was from an IP that is not whitelisted, so
150/hr.
The second call was from the same IP, but authenticated as a
whitelisted account, so should be 2/hr.
I usually only work from the whitelisted IPs, but wanted to check an
IP that had never been used before for data mining.
On Oct 20, 11:36 am, Dewald Pretorius dpr...@gmail.com wrote:
Apart from the 401 issue, the following is also very weird:
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-RateLimit-Limit: 2
This is on the second authenticated call. On the unauthenticated call
X-RateLimit-Limit was 150. I assume both calls were made from the same
IP address?
Dewald
On Oct 20, 3:10 pm, Ryan Rosario uclamath...@gmail.com wrote:
Below are the responses. I tried various combinations of -
uusername:pass -u=username:pass -u username:pass and each time got an
authentication error.
Without authentication
[madhatter:~/Desktop] ryan% curl
-vvvhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.100... connected
* Connected to twitter.com (168.143.162.100) port 80 (#0) GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 132
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256065275
X-experimental-RLS-th: notreq
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:03:41 GMT
Server: hi
X-RateLimit-Limit: 150
X-Transaction: 1256061824-5820-4214
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:03:44 GMT
X-RateLimit-Remaining: 133
X-Runtime: 0.05350
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
Expires: Tue, 31 Mar 1981 05:00:00 GMT
X-Revision: 726d1f10cbe49a923e837d8c3fca3a3993a0f964
X-RateLimit-Reset: 1256065275
Set-Cookie: lang=en; path=/
Set-Cookie:
_twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWM5OTU5MWNkNmE5MjMwNzU0Nzhh
%250AMzdkMTA3NzE2Zjk5IgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz
%250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--73f950f824e3e048d3691a3eeec8a8fbbab2
1743;
domain=.twitter.com; path=/
Vary: Accept-Encoding
Content-Length: 118
Connection: close
* Closing connection #0
{request:/statuses/user_timeline.json?
user_id=1255997062count=100,error:This method requires
authentication.}
With authentication:
[madhatter:~/Desktop] ryan% curl -vvv
-uusername:passhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.68... connected
* Connected to twitter.com (168.143.162.68) port 80 (#0)
* Server auth using Basic with user ' ' GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
Authorization: Basic *redacted*
[twitter-dev] Re: 401 Unauthorized -- user_timeline -- using Numeric ID
Thanks for all of your help!
At some point, the 401 error for User Not Found should be changed to a
404. The fix would make this type of fetch much more efficient, and
save on the number of API calls.
I adapted my code for what you suggested above, but I had to check for
User Not Found which required another URL fetch, which required
another check for HTTP errors.
Thanks!
Ryan
On Oct 20, 11:08 pm, Chad Etzel c...@twitter.com wrote:
If you try to pull a protected users' timeline w/o authentication then
it will throw a 401.
You can confirm whether a user id is valid by trying to
call:http://twitter.com/users/show.xml?id={insert_id_here}
users/show will throw a 404 on a non existent user.
-Chad
On Wed, Oct 21, 2009 at 1:43 AM, Ryan Rosario uclamath...@gmail.com wrote:
*bangs head on desk...again*
Now that I know what what my Twitter ID is, and I know that I am a
valid user, I tried it and it worked.
It's gone way past 15 users now without a problem (without error
checking).
I see what is happening now. After 15 users or so, I hit a
user that no longer exists. On a 404, I skip the user, but on a 401,
for the time being, I am retrying, and then quickly maxing out my
hourly
limit.
The other part was my overworked mind. I was pulling numbers that
looked like Twitter IDs from the site HTML that actually weren't.
And...then testing over and over with those bad numbers. Sigh.
I suppose it is safe to treat 401 as a 404? I don't think it would be
used for anything else in the REST API, right?
Is an HTTP error thrown if I try to pull a protected user's timeline?
To make a long story short, it does appear that invalid users are
returning 401 instead of 404.
Thanks,
Ryan
On Oct 20, 9:10 pm, Chad Etzel c...@twitter.com wrote:
Hi Ryan,
I think part of the problem is that user id 1255997062 does not exist,
so it should really be returning a 404 instead of a 401. Do you get
the same 401 result with a known valid user id?
-Chad
On Tue, Oct 20, 2009 at 3:13 PM, Ryan Rosario uclamath...@gmail.com
wrote:
Correct. I thought that was weird, but didn't know what to make of it.
Same IP.
The unauthenticated call was from an IP that is not whitelisted, so
150/hr.
The second call was from the same IP, but authenticated as a
whitelisted account, so should be 2/hr.
I usually only work from the whitelisted IPs, but wanted to check an
IP that had never been used before for data mining.
On Oct 20, 11:36 am, Dewald Pretorius dpr...@gmail.com wrote:
Apart from the 401 issue, the following is also very weird:
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-RateLimit-Limit: 2
This is on the second authenticated call. On the unauthenticated call
X-RateLimit-Limit was 150. I assume both calls were made from the same
IP address?
Dewald
On Oct 20, 3:10 pm, Ryan Rosario uclamath...@gmail.com wrote:
Below are the responses. I tried various combinations of -
uusername:pass -u=username:pass -u username:pass and each time got an
authentication error.
Without authentication
[madhatter:~/Desktop] ryan% curl
-vvvhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.100... connected
* Connected to twitter.com (168.143.162.100) port 80 (#0) GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 132
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256065275
X-experimental-RLS-th: notreq
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:03:41 GMT
Server: hi
X-RateLimit-Limit: 150
X-Transaction: 1256061824-5820-4214
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:03:44 GMT
X-RateLimit-Remaining: 133
X-Runtime: 0.05350
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
Expires: Tue, 31 Mar 1981 05:00:00 GMT
X-Revision: 726d1f10cbe49a923e837d8c3fca3a3993a0f964
X-RateLimit-Reset: 1256065275
Set-Cookie: lang=en; path=/
Set-Cookie:
_twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWM5OTU5MWNkNmE5MjMwNzU0Nzhh
%250AMzdkMTA3NzE2Zjk5IgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz
%250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--73f950f824e3e048d3691a3eeec8a8fbbab2
1743;
domain=.twitter.com; path=/
Vary: Accept-Encoding
Content-Length: 118
Connection: close
* Closing connection #0
{request:/statuses/user_timeline.json?
[twitter-dev] Re: 401 Unauthorized -- user_timeline -- using Numeric ID
Hi Ryan, This sounds like a bug. Can you provide full HTTP request/response headers+body traces for some of these requests? (be sure to obscure the authentication header). Using curl -vvv would be good. -Chad On Tue, Oct 20, 2009 at 1:10 PM, Ryan Rosario uclamath...@gmail.com wrote: I have been running into a recurring problem that I have been facing for the past couple of days, on numerous machines. I am extracting 200 tweets for a large number of users, using the numeric user ID (user_id). After a small number of requests (between 10 and 50) I receive nothing but This method requires authentication. Authenticating with my username and password does not make the message go away. I have verified that the users I am trying to pull are not protected users. I have also verified that I am not hitting the rate limit. On my latest attempt, I used an IP I have not used in the past. I was able to extract the first 200 tweets for *12* users, then I started receiving 401s (This method requires authentication) on every single request of this type. After about 10 minutes, I tried again and it worked for about another 15 users, then I started getting the same message as well as other messages including 502. Is this behavior expected? Does this 401 message possibly mean something else?
[twitter-dev] Re: 401 Unauthorized -- user_timeline -- using Numeric ID
Below are the responses. I tried various combinations of -
uusername:pass -u=username:pass -u username:pass and each time got an
authentication error.
Without authentication
[madhatter:~/Desktop] ryan% curl -vvv
http://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.100... connected
* Connected to twitter.com (168.143.162.100) port 80 (#0)
GET /statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 132
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256065275
X-experimental-RLS-th: notreq
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:03:41 GMT
Server: hi
X-RateLimit-Limit: 150
X-Transaction: 1256061824-5820-4214
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:03:44 GMT
X-RateLimit-Remaining: 133
X-Runtime: 0.05350
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
Expires: Tue, 31 Mar 1981 05:00:00 GMT
X-Revision: 726d1f10cbe49a923e837d8c3fca3a3993a0f964
X-RateLimit-Reset: 1256065275
Set-Cookie: lang=en; path=/
Set-Cookie:
_twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWM5OTU5MWNkNmE5MjMwNzU0Nzhh
%250AMzdkMTA3NzE2Zjk5IgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz
%250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--73f950f824e3e048d3691a3eeec8a8fbbab21743;
domain=.twitter.com; path=/
Vary: Accept-Encoding
Content-Length: 118
Connection: close
* Closing connection #0
{request:/statuses/user_timeline.json?
user_id=1255997062count=100,error:This method requires
authentication.}
With authentication:
[madhatter:~/Desktop] ryan% curl -vvv -uusername:pass
http://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.68... connected
* Connected to twitter.com (168.143.162.68) port 80 (#0)
* Server auth using Basic with user ' '
GET /statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
Authorization: Basic *redacted*
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256064142
X-experimental-RLS-th: succ
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:04:33 GMT
Server: hi
X-RateLimit-Limit: 2
X-Transaction: 1256061882-45021-11545
* Authentication problem. Ignoring this.
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:04:42 GMT
X-RateLimit-Remaining: 19823
X-Runtime: 0.11979
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
Expires: Tue, 31 Mar 1981 05:00:00 GMT
X-Revision: 726d1f10cbe49a923e837d8c3fca3a3993a0f964
X-RateLimit-Reset: 1256064139
Set-Cookie: lang=en; path=/
Set-Cookie: lang=en; path=/
Set-Cookie:
_twitter_sess=BAh7CjoTcGFzc3dvcmRfdG9rZW4iLTg1ZTQ4OTU0ZjdmM2MyOGQzMjFhOGIy
%250ANjEyYjBhZDAzZjYzZjY3M2Y6EXRyYW5zX3Byb21wdDA6CXVzZXJpA57usDoH
%250AaWQiJTgxZTNhMGNlYmE0YjRlNWI2NWRlZjdhZjU3ZDk3MmM2IgpmbGFzaElD
%250AOidBY3Rpb25Db250cm9sbGVyOjpGbGFzaDo6Rmxhc2hIYXNoewAGOgpAdXNl
%250AZHsA--26052e0c4f20f2f852537f173106f426be9df437;
domain=.twitter.com; path=/
Vary: Accept-Encoding
Content-Length: 118
Connection: close
* Closing connection #0
On Oct 20, 10:25 am, Chad Etzel c...@twitter.com wrote:
Hi Ryan,
This sounds like a bug. Can you provide full HTTP request/response
headers+body traces for some of these requests? (be sure to obscure
the authentication header). Using curl -vvv would be good.
-Chad
On Tue, Oct 20, 2009 at 1:10 PM, Ryan Rosario uclamath...@gmail.com wrote:
I have been running into a recurring problem that I have been facing
for the past couple of days, on numerous machines.
I am extracting 200 tweets for a large number of users, using the
numeric user ID (user_id). After a small number of requests (between
10 and 50) I receive nothing but This method requires
authentication. Authenticating with my username and password does not
make the message go away. I have verified that the users I am trying
to pull are not protected users. I have also verified that I am not
hitting the rate limit.
On my latest attempt, I used an IP I have not used in the past. I was
able to extract the first 200 tweets for *12* users, then I started
receiving 401s (This method requires authentication) on every single
request of this type.
After about 10 minutes, I tried again and it worked for about another
15 users, then I
[twitter-dev] Re: 401 Unauthorized -- user_timeline -- using Numeric ID
Apart from the 401 issue, the following is also very weird:
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-RateLimit-Limit: 2
This is on the second authenticated call. On the unauthenticated call
X-RateLimit-Limit was 150. I assume both calls were made from the same
IP address?
Dewald
On Oct 20, 3:10 pm, Ryan Rosario uclamath...@gmail.com wrote:
Below are the responses. I tried various combinations of -
uusername:pass -u=username:pass -u username:pass and each time got an
authentication error.
Without authentication
[madhatter:~/Desktop] ryan% curl
-vvvhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.100... connected
* Connected to twitter.com (168.143.162.100) port 80 (#0) GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 132
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256065275
X-experimental-RLS-th: notreq
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:03:41 GMT
Server: hi
X-RateLimit-Limit: 150
X-Transaction: 1256061824-5820-4214
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:03:44 GMT
X-RateLimit-Remaining: 133
X-Runtime: 0.05350
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
Expires: Tue, 31 Mar 1981 05:00:00 GMT
X-Revision: 726d1f10cbe49a923e837d8c3fca3a3993a0f964
X-RateLimit-Reset: 1256065275
Set-Cookie: lang=en; path=/
Set-Cookie:
_twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWM5OTU5MWNkNmE5MjMwNzU0Nzhh
%250AMzdkMTA3NzE2Zjk5IgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz
%250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--73f950f824e3e048d3691a3eeec8a8fbbab21743;
domain=.twitter.com; path=/
Vary: Accept-Encoding
Content-Length: 118
Connection: close
* Closing connection #0
{request:/statuses/user_timeline.json?
user_id=1255997062count=100,error:This method requires
authentication.}
With authentication:
[madhatter:~/Desktop] ryan% curl -vvv
-uusername:passhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.68... connected
* Connected to twitter.com (168.143.162.68) port 80 (#0)
* Server auth using Basic with user ' ' GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
Authorization: Basic *redacted*
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256064142
X-experimental-RLS-th: succ
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:04:33 GMT
Server: hi
X-RateLimit-Limit: 2
X-Transaction: 1256061882-45021-11545
* Authentication problem. Ignoring this.
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:04:42 GMT
X-RateLimit-Remaining: 19823
X-Runtime: 0.11979
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
Expires: Tue, 31 Mar 1981 05:00:00 GMT
X-Revision: 726d1f10cbe49a923e837d8c3fca3a3993a0f964
X-RateLimit-Reset: 1256064139
Set-Cookie: lang=en; path=/
Set-Cookie: lang=en; path=/
Set-Cookie:
_twitter_sess=BAh7CjoTcGFzc3dvcmRfdG9rZW4iLTg1ZTQ4OTU0ZjdmM2MyOGQzMjFhOGIy
%250ANjEyYjBhZDAzZjYzZjY3M2Y6EXRyYW5zX3Byb21wdDA6CXVzZXJpA57usDoH
%250AaWQiJTgxZTNhMGNlYmE0YjRlNWI2NWRlZjdhZjU3ZDk3MmM2IgpmbGFzaElD
%250AOidBY3Rpb25Db250cm9sbGVyOjpGbGFzaDo6Rmxhc2hIYXNoewAGOgpAdXNl
%250AZHsA--26052e0c4f20f2f852537f173106f426be9df437;
domain=.twitter.com; path=/
Vary: Accept-Encoding
Content-Length: 118
Connection: close
* Closing connection #0
On Oct 20, 10:25 am, Chad Etzel c...@twitter.com wrote:
Hi Ryan,
This sounds like a bug. Can you provide full HTTP request/response
headers+body traces for some of these requests? (be sure to obscure
the authentication header). Using curl -vvv would be good.
-Chad
On Tue, Oct 20, 2009 at 1:10 PM, Ryan Rosario uclamath...@gmail.com wrote:
I have been running into a recurring problem that I have been facing
for the past couple of days, on numerous machines.
I am extracting 200 tweets for a large number of users, using the
numeric user ID (user_id). After a small number of requests (between
10 and 50) I receive nothing but This method requires
authentication. Authenticating with my username and password does not
make the
[twitter-dev] Re: 401 Unauthorized -- user_timeline -- using Numeric ID
Correct. I thought that was weird, but didn't know what to make of it.
Same IP.
The unauthenticated call was from an IP that is not whitelisted, so
150/hr.
The second call was from the same IP, but authenticated as a
whitelisted account, so should be 2/hr.
I usually only work from the whitelisted IPs, but wanted to check an
IP that had never been used before for data mining.
On Oct 20, 11:36 am, Dewald Pretorius dpr...@gmail.com wrote:
Apart from the 401 issue, the following is also very weird:
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-RateLimit-Limit: 2
This is on the second authenticated call. On the unauthenticated call
X-RateLimit-Limit was 150. I assume both calls were made from the same
IP address?
Dewald
On Oct 20, 3:10 pm, Ryan Rosario uclamath...@gmail.com wrote:
Below are the responses. I tried various combinations of -
uusername:pass -u=username:pass -u username:pass and each time got an
authentication error.
Without authentication
[madhatter:~/Desktop] ryan% curl
-vvvhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.100... connected
* Connected to twitter.com (168.143.162.100) port 80 (#0) GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 132
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256065275
X-experimental-RLS-th: notreq
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:03:41 GMT
Server: hi
X-RateLimit-Limit: 150
X-Transaction: 1256061824-5820-4214
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:03:44 GMT
X-RateLimit-Remaining: 133
X-Runtime: 0.05350
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
Expires: Tue, 31 Mar 1981 05:00:00 GMT
X-Revision: 726d1f10cbe49a923e837d8c3fca3a3993a0f964
X-RateLimit-Reset: 1256065275
Set-Cookie: lang=en; path=/
Set-Cookie:
_twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWM5OTU5MWNkNmE5MjMwNzU0Nzhh
%250AMzdkMTA3NzE2Zjk5IgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz
%250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--73f950f824e3e048d3691a3eeec8a8fbbab2
1743;
domain=.twitter.com; path=/
Vary: Accept-Encoding
Content-Length: 118
Connection: close
* Closing connection #0
{request:/statuses/user_timeline.json?
user_id=1255997062count=100,error:This method requires
authentication.}
With authentication:
[madhatter:~/Desktop] ryan% curl -vvv
-uusername:passhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.68... connected
* Connected to twitter.com (168.143.162.68) port 80 (#0)
* Server auth using Basic with user ' ' GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
Authorization: Basic *redacted*
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256064142
X-experimental-RLS-th: succ
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:04:33 GMT
Server: hi
X-RateLimit-Limit: 2
X-Transaction: 1256061882-45021-11545
* Authentication problem. Ignoring this.
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:04:42 GMT
X-RateLimit-Remaining: 19823
X-Runtime: 0.11979
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
Expires: Tue, 31 Mar 1981 05:00:00 GMT
X-Revision: 726d1f10cbe49a923e837d8c3fca3a3993a0f964
X-RateLimit-Reset: 1256064139
Set-Cookie: lang=en; path=/
Set-Cookie: lang=en; path=/
Set-Cookie:
_twitter_sess=BAh7CjoTcGFzc3dvcmRfdG9rZW4iLTg1ZTQ4OTU0ZjdmM2MyOGQzMjFhOGIy
%250ANjEyYjBhZDAzZjYzZjY3M2Y6EXRyYW5zX3Byb21wdDA6CXVzZXJpA57usDoH
%250AaWQiJTgxZTNhMGNlYmE0YjRlNWI2NWRlZjdhZjU3ZDk3MmM2IgpmbGFzaElD
%250AOidBY3Rpb25Db250cm9sbGVyOjpGbGFzaDo6Rmxhc2hIYXNoewAGOgpAdXNl
%250AZHsA--26052e0c4f20f2f852537f173106f426be9df437;
domain=.twitter.com; path=/
Vary: Accept-Encoding
Content-Length: 118
Connection: close
* Closing connection #0
On Oct 20, 10:25 am, Chad Etzel c...@twitter.com wrote:
Hi Ryan,
This sounds like a bug. Can you provide full HTTP request/response
headers+body traces for some of these requests? (be sure
[twitter-dev] Re: 401 Unauthorized -- user_timeline -- using Numeric ID
Hi Ryan,
I think part of the problem is that user id 1255997062 does not exist,
so it should really be returning a 404 instead of a 401. Do you get
the same 401 result with a known valid user id?
-Chad
On Tue, Oct 20, 2009 at 3:13 PM, Ryan Rosario uclamath...@gmail.com wrote:
Correct. I thought that was weird, but didn't know what to make of it.
Same IP.
The unauthenticated call was from an IP that is not whitelisted, so
150/hr.
The second call was from the same IP, but authenticated as a
whitelisted account, so should be 2/hr.
I usually only work from the whitelisted IPs, but wanted to check an
IP that had never been used before for data mining.
On Oct 20, 11:36 am, Dewald Pretorius dpr...@gmail.com wrote:
Apart from the 401 issue, the following is also very weird:
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-RateLimit-Limit: 2
This is on the second authenticated call. On the unauthenticated call
X-RateLimit-Limit was 150. I assume both calls were made from the same
IP address?
Dewald
On Oct 20, 3:10 pm, Ryan Rosario uclamath...@gmail.com wrote:
Below are the responses. I tried various combinations of -
uusername:pass -u=username:pass -u username:pass and each time got an
authentication error.
Without authentication
[madhatter:~/Desktop] ryan% curl
-vvvhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.100... connected
* Connected to twitter.com (168.143.162.100) port 80 (#0) GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 132
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256065275
X-experimental-RLS-th: notreq
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:03:41 GMT
Server: hi
X-RateLimit-Limit: 150
X-Transaction: 1256061824-5820-4214
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:03:44 GMT
X-RateLimit-Remaining: 133
X-Runtime: 0.05350
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
Expires: Tue, 31 Mar 1981 05:00:00 GMT
X-Revision: 726d1f10cbe49a923e837d8c3fca3a3993a0f964
X-RateLimit-Reset: 1256065275
Set-Cookie: lang=en; path=/
Set-Cookie:
_twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWM5OTU5MWNkNmE5MjMwNzU0Nzhh
%250AMzdkMTA3NzE2Zjk5IgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz
%250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--73f950f824e3e048d3691a3eeec8a8fbbab2
1743;
domain=.twitter.com; path=/
Vary: Accept-Encoding
Content-Length: 118
Connection: close
* Closing connection #0
{request:/statuses/user_timeline.json?
user_id=1255997062count=100,error:This method requires
authentication.}
With authentication:
[madhatter:~/Desktop] ryan% curl -vvv
-uusername:passhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.68... connected
* Connected to twitter.com (168.143.162.68) port 80 (#0)
* Server auth using Basic with user ' ' GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
Authorization: Basic *redacted*
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256064142
X-experimental-RLS-th: succ
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:04:33 GMT
Server: hi
X-RateLimit-Limit: 2
X-Transaction: 1256061882-45021-11545
* Authentication problem. Ignoring this.
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:04:42 GMT
X-RateLimit-Remaining: 19823
X-Runtime: 0.11979
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
Expires: Tue, 31 Mar 1981 05:00:00 GMT
X-Revision: 726d1f10cbe49a923e837d8c3fca3a3993a0f964
X-RateLimit-Reset: 1256064139
Set-Cookie: lang=en; path=/
Set-Cookie: lang=en; path=/
Set-Cookie:
_twitter_sess=BAh7CjoTcGFzc3dvcmRfdG9rZW4iLTg1ZTQ4OTU0ZjdmM2MyOGQzMjFhOGIy
%250ANjEyYjBhZDAzZjYzZjY3M2Y6EXRyYW5zX3Byb21wdDA6CXVzZXJpA57usDoH
%250AaWQiJTgxZTNhMGNlYmE0YjRlNWI2NWRlZjdhZjU3ZDk3MmM2IgpmbGFzaElD
%250AOidBY3Rpb25Db250cm9sbGVyOjpGbGFzaDo6Rmxhc2hIYXNoewAGOgpAdXNl
%250AZHsA--26052e0c4f20f2f852537f173106f426be9df437;
domain=.twitter.com; path=/
Vary:
[twitter-dev] Re: 401 Unauthorized -- user_timeline -- using Numeric ID
I think I see what is happening now. After 15 users or so, I hit a
user that no longer exists. On a 404, I skip the user, but on a 401,
for the time being, I am retrying, and then maxing out my hourly
limit.
The other part was my overworked mind. I was pulling numbers that
looked like Twitter IDs from the site HTML that actually weren't.
And...then testing over and over with those bad numbers. Sigh.
I suppose it is safe to treat 401 as a 404? I don't think it would be
used for anything else in the REST API, right?
Is an HTTP error thrown if I try to pull a protected user's timeline?
Thanks,
Ryan
On Oct 20, 9:10 pm, Chad Etzel c...@twitter.com wrote:
Hi Ryan,
I think part of the problem is that user id 1255997062 does not exist,
so it should really be returning a 404 instead of a 401. Do you get
the same 401 result with a known valid user id?
-Chad
On Tue, Oct 20, 2009 at 3:13 PM, Ryan Rosario uclamath...@gmail.com wrote:
Correct. I thought that was weird, but didn't know what to make of it.
Same IP.
The unauthenticated call was from an IP that is not whitelisted, so
150/hr.
The second call was from the same IP, but authenticated as a
whitelisted account, so should be 2/hr.
I usually only work from the whitelisted IPs, but wanted to check an
IP that had never been used before for data mining.
On Oct 20, 11:36 am, Dewald Pretorius dpr...@gmail.com wrote:
Apart from the 401 issue, the following is also very weird:
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-RateLimit-Limit: 2
This is on the second authenticated call. On the unauthenticated call
X-RateLimit-Limit was 150. I assume both calls were made from the same
IP address?
Dewald
On Oct 20, 3:10 pm, Ryan Rosario uclamath...@gmail.com wrote:
Below are the responses. I tried various combinations of -
uusername:pass -u=username:pass -u username:pass and each time got an
authentication error.
Without authentication
[madhatter:~/Desktop] ryan% curl
-vvvhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.100... connected
* Connected to twitter.com (168.143.162.100) port 80 (#0) GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 132
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256065275
X-experimental-RLS-th: notreq
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:03:41 GMT
Server: hi
X-RateLimit-Limit: 150
X-Transaction: 1256061824-5820-4214
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:03:44 GMT
X-RateLimit-Remaining: 133
X-Runtime: 0.05350
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
Expires: Tue, 31 Mar 1981 05:00:00 GMT
X-Revision: 726d1f10cbe49a923e837d8c3fca3a3993a0f964
X-RateLimit-Reset: 1256065275
Set-Cookie: lang=en; path=/
Set-Cookie:
_twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWM5OTU5MWNkNmE5MjMwNzU0Nzhh
%250AMzdkMTA3NzE2Zjk5IgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz
%250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--73f950f824e3e048d3691a3eeec8a8fbbab2
1743;
domain=.twitter.com; path=/
Vary: Accept-Encoding
Content-Length: 118
Connection: close
* Closing connection #0
{request:/statuses/user_timeline.json?
user_id=1255997062count=100,error:This method requires
authentication.}
With authentication:
[madhatter:~/Desktop] ryan% curl -vvv
-uusername:passhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.68... connected
* Connected to twitter.com (168.143.162.68) port 80 (#0)
* Server auth using Basic with user ' ' GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
Authorization: Basic *redacted*
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256064142
X-experimental-RLS-th: succ
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:04:33 GMT
Server: hi
X-RateLimit-Limit: 2
X-Transaction: 1256061882-45021-11545
* Authentication problem. Ignoring this.
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:04:42 GMT
X-RateLimit-Remaining:
[twitter-dev] Re: 401 Unauthorized -- user_timeline -- using Numeric ID
*bangs head on desk...again*
Now that I know what what my Twitter ID is, and I know that I am a
valid user, I tried it and it worked.
It's gone way past 15 users now without a problem (without error
checking).
I see what is happening now. After 15 users or so, I hit a
user that no longer exists. On a 404, I skip the user, but on a 401,
for the time being, I am retrying, and then quickly maxing out my
hourly
limit.
The other part was my overworked mind. I was pulling numbers that
looked like Twitter IDs from the site HTML that actually weren't.
And...then testing over and over with those bad numbers. Sigh.
I suppose it is safe to treat 401 as a 404? I don't think it would be
used for anything else in the REST API, right?
Is an HTTP error thrown if I try to pull a protected user's timeline?
To make a long story short, it does appear that invalid users are
returning 401 instead of 404.
Thanks,
Ryan
On Oct 20, 9:10 pm, Chad Etzel c...@twitter.com wrote:
Hi Ryan,
I think part of the problem is that user id 1255997062 does not exist,
so it should really be returning a 404 instead of a 401. Do you get
the same 401 result with a known valid user id?
-Chad
On Tue, Oct 20, 2009 at 3:13 PM, Ryan Rosario uclamath...@gmail.com wrote:
Correct. I thought that was weird, but didn't know what to make of it.
Same IP.
The unauthenticated call was from an IP that is not whitelisted, so
150/hr.
The second call was from the same IP, but authenticated as a
whitelisted account, so should be 2/hr.
I usually only work from the whitelisted IPs, but wanted to check an
IP that had never been used before for data mining.
On Oct 20, 11:36 am, Dewald Pretorius dpr...@gmail.com wrote:
Apart from the 401 issue, the following is also very weird:
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-RateLimit-Limit: 2
This is on the second authenticated call. On the unauthenticated call
X-RateLimit-Limit was 150. I assume both calls were made from the same
IP address?
Dewald
On Oct 20, 3:10 pm, Ryan Rosario uclamath...@gmail.com wrote:
Below are the responses. I tried various combinations of -
uusername:pass -u=username:pass -u username:pass and each time got an
authentication error.
Without authentication
[madhatter:~/Desktop] ryan% curl
-vvvhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.100... connected
* Connected to twitter.com (168.143.162.100) port 80 (#0) GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 132
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256065275
X-experimental-RLS-th: notreq
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date: Tue, 20 Oct 2009 18:03:41 GMT
Server: hi
X-RateLimit-Limit: 150
X-Transaction: 1256061824-5820-4214
WWW-Authenticate: Basic realm=Twitter API
Status: 401 Unauthorized
Last-Modified: Tue, 20 Oct 2009 18:03:44 GMT
X-RateLimit-Remaining: 133
X-Runtime: 0.05350
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0,
post-check=0
Expires: Tue, 31 Mar 1981 05:00:00 GMT
X-Revision: 726d1f10cbe49a923e837d8c3fca3a3993a0f964
X-RateLimit-Reset: 1256065275
Set-Cookie: lang=en; path=/
Set-Cookie:
_twitter_sess=BAh7CDoRdHJhbnNfcHJvbXB0MDoHaWQiJWM5OTU5MWNkNmE5MjMwNzU0Nzhh
%250AMzdkMTA3NzE2Zjk5IgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVyOjpGbGFz
%250AaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--73f950f824e3e048d3691a3eeec8a8fbbab2
1743;
domain=.twitter.com; path=/
Vary: Accept-Encoding
Content-Length: 118
Connection: close
* Closing connection #0
{request:/statuses/user_timeline.json?
user_id=1255997062count=100,error:This method requires
authentication.}
With authentication:
[madhatter:~/Desktop] ryan% curl -vvv
-uusername:passhttp://twitter.com/statuses/user_timeline.json\?user_id=1255997062\count=100
* About to connect() to twitter.com port 80 (#0)
* Trying 168.143.162.68... connected
* Connected to twitter.com (168.143.162.68) port 80 (#0)
* Server auth using Basic with user ' ' GET
/statuses/user_timeline.json?user_id=1255997062count=100 HTTP/1.1
Authorization: Basic *redacted*
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4
OpenSSL/0.9.8k zlib/1.2.3
Host: twitter.com
Accept: */*
HTTP/1.1 401 Unauthorized
X-experimental-RLS-remaining: 0
X-experimental-RLS-maxvalue: 150
X-experimental-RLS-reset: 1256064142
X-experimental-RLS-th: succ
X-RLS-id: sjc1c032
Via: Cachet/0.91
Date:
