Hi everyone:

I am going to crawl the user relationship information, like, followers
and friends, from Twitter Inc. for my research work.
While, I am confronted with some problems, can anybody give me some
My problem is as follow:
Problem A: I try Twitter4j to fetch the relationship in Twitter .
But I always limit by the rate for not owning a white-list key and
secret, and sometimes I also got some errors, like HTTP 414 Error,
denoting request-URI too large.
Thus, is there any other method to escape the rate limitation with
Twitter API?

Problem B:
For this limit, I attempt another method to get the relationship.
I notice that if I input the url http://twitter.com/!#/dannygao/followers
, I could get the followers of dannygao in browser.
If I input the url http://twitter.com/dannygao/followers, I need to
login first, and then, get the followers in browser.
Thus, I use the AsyncHttpClient to crawl this relationship just like
the process of browser.
The code is as follows:
AsyncHttpClient client = new AsyncHttpClient();
Response response = client.prepareGet(this.url).execute().get();
if (response.getStatusCode() == 302) {
    System.out.print("Redirecting to... ");
    String redirect = response.getHeader("Location");
    this.url = redirect;
    response = client.prepareGet(this.url).execute().get();
return response.getResponseBody("UTF-8");
If I input the url http://twitter.com/!#/dannygao/followers, I got
following output.
         Connecting url http://twitter.com/!#/dannygao/followers
If I input the url http://twitter.com/dannygao/followers, I get
following output.
Connecting url http://twitter.com/dannygao/followers
Redirecting to... 

Neither of the scripts I crawled contains the information about user
Thus, 1) Can anybody explain what is the difference between these two
         2) If I want to use this method to crawl the twitter use
relationship, can you provide me some suggestions?

Many thanks in advance.


Twitter developer documentation and resources: https://dev.twitter.com/doc
API updates via Twitter: https://twitter.com/twitterapi
Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list
Change your membership to this group: 

Reply via email to