Not much? With crawler I'd rather "fan in" in MapReduce Framework's MutationPool style. It's a waste of instance time to make whole thread wait for just a single website.
On Tue, Jan 8, 2013 at 10:34 PM, alex <[email protected]> wrote: > looks like it's a crawler so there's not much you can do when external > sites respond slowly, except for fanning out. > > On Tue, Jan 8, 2013 at 3:30 PM, Jeff Schnitzer <[email protected]> wrote: >> On Mon, Jan 7, 2013 at 1:54 PM, Kristopher Giesing <[email protected]> >> wrote: >>> >>> 16s average latency though... >> >> >> ...and 294 instances at an average 0.175 qps. Ouch. >> >> Jeff >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
