I haven't tried to test what you are doing, but I do know NYTimes sometimes will request you for login after certain number of continuous requests. Also see this tweet: http://tweetree.com/posts/1629269063 . But it's odd to see 301 as a returned status code.
BTW, what are you trying to achieve by fetching the article? If you don't need fulltext, you can look into the NYTimes Article api: http://developer.nytimes.com/docs/article_search_api Charlie On Apr 30, 6:18 pm, James <[email protected]> wrote: > I've discovered that when you urlfetch() a site fromhttp://www.nytimes.com, > the returned document has status code 301, but when follow_redirects > is true, it returns a 'Too many repeated redirects' error. > > example call: > > from google.appengine.api import urlfetch > fetch_page = urlfetch.fetch("http://www.nytimes.com/2009/04/19/jobs/ > 19pre.html?_r=1", follow_redirects=True) > > I'm guessing this might be because of the request user-agent or > something. Has anyone run into this sort of problem? --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
