I fixed it..
It was related to http://code.google.com/p/googleappengine/issues/detail?id=739
i didnt notice the Cache stuff in the headers.. My requests were
hitting google cache.
It got fixed when i tried the solution suggested in the issue..
fetch_headers = {'Cache-Control':'no-cache,max-age=0', 'Pragma':'no-
cache'}
f_result = urlfetch.fetch(feed_url, None, urlfetch.GET, fetch_headers)
Thanx anyway!!
On Feb 1, 2:16 pm, z33m <[email protected]> wrote:
> im trying to make something like a feed reader. Im using feedparser.py
> to parse the feeds like this
>
> def fetch_feed(url, modified = None, etag = None):
> import feedparser
> headers = {}
> if modified:
> headers['If-Modified-Since'] = modified
> if etag:
> headers['If-None-Match'] = etag
>
> response = urlfetch.fetch(url, headers = headers)
> feed = feedparser.parse(response.content)
> feed.modified = response.headers['Last-Modified'] if 'Last-
> Modified' in response.headers else None
> feed.etag = response.headers['ETag'] if 'ETag' in
> response.headers else None
> feed.status = response.status_code
> return feed
>
> but the If-Modified-Since and If-None-Match headers dont seem to work
> on some feeds.. so i tried a test onhttp://shell.appspot.com/
>
> >>> from google.appengine.api import urlfetch
> >>> r = urlfetch.fetch('http://www.scripting.com/rss.xml')
> >>> r.headers
>
> {'Content-Length': '48035', 'Via': 'HTTP/1.1 GWA (remote cache hit)',
> 'Accept-Ranges': 'bytes', 'X-Google-Cache-Control': 'remote-cache-
> hit', 'Server': 'Apache/2.2.4 (Win32)', 'Last-Modified': 'Fri, 30 Jan
> 2009 03:38:05 GMT', 'ETag': '"1dd8-bba3-eec4f564"', 'Date': 'Sun, 01
> Feb 2009 04:46:09 GMT', 'Content-Type': 'application/xml', 'Age': '
> 14856'}>>> h = r.headers
> >>> r = urlfetch.fetch('http://www.scripting.com/rss.xml', headers =
> >>> {'If-Modified-Since': h['Last-Modified'], 'If-None-Match': h['ETag']})
> >>> r.status_code
>
> 200L
>
> but if i do curl for the same i get a 304 http code as expected..
>
> $ curl --headhttp://www.scripting.com/rss.xml
> HTTP/1.1 200 OK
> Date: Sun, 01 Feb 2009 08:58:24 GMT
> Server: Apache/2.2.4 (Win32)
> Last-Modified: Fri, 30 Jan 2009 03:38:05 GMT
> ETag: "1dd8-bba3-eec4f564"
> Accept-Ranges: bytes
> Content-Length: 48035
> Content-Type: application/xml
>
> $ curl --head --header "If-Modified-Since: Fri, 30 Jan 2009 03:38:05
> GMT" --header "If-None-Match:
> \"1dd8-bba3-eec4f564\""http://www.scripting.com/rss.xml
> HTTP/1.1 304 Not Modified
> Date: Sun, 01 Feb 2009 08:58:16 GMT
> Server: Apache/2.2.4 (Win32)
> ETag: "1dd8-bba3-eec4f564"
>
> Can some one help me with this.. am i doing anything wrong?
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
