It doesn't seem like a hack at all, actually. The problem, I think, is that many web servers do NOT support it. Do you have a list of services that do?
Otherwise, nice find! On Tue, Jul 27, 2010 at 12:34 PM, hector rovira <[email protected]>wrote: > I found a way to get around the 1MB limit on the URLFetchService, but > it depends on the HTTP Range header... the web service you are calling > must support that header. My problem was that I needed to request > some data from a bucket on Google Storage for Developers (which > supports Range header queries). > > So let's say my original request was for 20MB worth of data, which > hits the 1MB limit throwing a ResponseTooLargeException. I "split" my > original request into 20 separate requests each with a different range > (0-1MB, 1MB-2MB, etc...). Then I submitted the 20 requests in > parallel using URLFetchService.fetchAsynch. Each request would take > less than 10secs and come back with less than 1MB. Finally, I > reassembled the responses into a single ByteArrayOutputStream and > proceeded as if I had made a single request. > > This was a simple way for me to process a request for an analysis that > ran on GAE, but where the data was stored externally. I agree that if > you are trying to serve files you should offer the Blobstore API, or > simply send a redirect to the actual service holding the data. > > If this smells like a hack, it probably is. > > On Jul 27, 11:45 am, "Ikai L (Google)" <[email protected]> wrote: > > 10 second limit for URLfetch, I mean. You can serve larger data sizes > using > > the Blobstore API: > > > > http://code.google.com/appengine/docs/python/blobstore/ > > > > On Tue, Jul 27, 2010 at 11:45 AM, Ikai L (Google) <[email protected]> > wrote: > > > > > > > > > No, that's the limit. Also be aware that there's a 10 second limit. > > > > > On Tue, Jul 27, 2010 at 2:02 AM, Ahmed Emir KAMMOUN < > > > [email protected]> wrote: > > > > >> Hello, > > > > >> I want to know if there is a possibility to have a more big amount of > > >> request and response size. > > >> Actually, I find that it is limited to 1 MB. > > > > >>http://code.google.com/intl/fr-FR/appengine/docs/java/urlfetch/overvi. > .. > > > > >> Regards, > > > > >> -- > > >> You received this message because you are subscribed to the Google > Groups > > >> "Google App Engine" group. > > >> To post to this group, send email to > [email protected]. > > >> To unsubscribe from this group, send email to > > >> [email protected]<google-appengine%[email protected]> > <google-appengine%[email protected]<google-appengine%[email protected]> > > > > >> . > > >> For more options, visit this group at > > >>http://groups.google.com/group/google-appengine?hl=en. > > > > > -- > > > Ikai Lan > > > Developer Programs Engineer, Google App Engine > > > Blog:http://googleappengine.blogspot.com > > > Twitter:http://twitter.com/app_engine > > > Reddit:http://www.reddit.com/r/appengine > > > > -- > > Ikai Lan > > Developer Programs Engineer, Google App Engine > > Blog:http://googleappengine.blogspot.com > > Twitter:http://twitter.com/app_engine > > Reddit:http://www.reddit.com/r/appengine > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
