I'm working on a fix. I got it to go fast on py2.7 now by patching _is_gce to always return False, but py2.6 is still slow. Not sure why yet, and I can't seem to attach the debugger to the running py2.6 tests (maybe pycharm debugger doesn't work with 2.6?).
I have a theory that the async_request stuff is timing out because it doesn't appear to be mocked out and the GoogleBaseConnection class inherits from PollingConnection, but I didn't have a chance to follow the idea all the way before I had to leave for the day. I'll keep looking tomorrow and hopefully get a PR up. Greg From: Scott Crunkleton <[email protected]<mailto:[email protected]>> Date: Monday, November 30, 2015 at 3:54 PM To: Greg <[email protected]<mailto:[email protected]>> Cc: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: COMMERCIAL:Re: [dev] Google Storage tests really slow? Ok, cool. Good debug. Looking into it now. On Mon, Nov 30, 2015 at 1:05 PM, Greg Hill <[email protected]<mailto:[email protected]>> wrote: Yep, it's attempting to contact the google API: GoogleBaseConnection.__init___ calls GoogleAuthType.guess_type because 'auth_type' was not provided. GoogleAuthType.guess_type calls _is_gce _is_gce calls _get_gce_metadata _get_gce_metadata does a GET http://metadata/computeMetadata/v1/ That URL apparently takes no time to fail on Linux but takes a long time to time out on a Mac. 'metadata' does not resolve on either of my machines. Anyway, the unit tests should never be attempting to GET anything over HTTP, so we need to mock something out in the google compute tests. I'll see if I can figure it out. Greg On 11/30/15, 2:43 PM, "Greg Hill" <[email protected]<mailto:[email protected]>> wrote: >I do my dev on a local macbook. Those appear to be the same tests causing >me grief. Since you mentioned that you weren't having issues on another >server, I ran the tests on linux box running ubuntu 12.04 and the tests >were as performant as can be expected. So I guess it's something specific >to Macs? So bizarre. I'll try to run it through the debugger and see if >I can pinpoint what it's doing that's taking up so much time. > >Greg > >On 11/30/15, 2:33 PM, "Eric Johnson" ><[email protected]<mailto:[email protected]>> wrote: > >>Greg, can you describe your dev environment? >> >>I do all my development on a GCE instance and don't see any issues >>either. >>However, on a whim, I tried running them on my Mac and I saw significant >>slowdowns on, libcloud.test.common.GoogleAuthTypeTest and >>libcloud.test.storage.GoogleStorage* >> >>Since I've never done development on my Mac, I don't fully trust that I >>have it set up correctly. But if you're using a Mac too, it probably >>merits >>a closer look into the differences. >> >>GCE instance: Ran 5889 tests in 31.906s >>Local Mac: ?? I killed it after 16 minutes >> >> >>On Mon, Nov 30, 2015 at 11:52 AM, Scott Crunkleton >><[email protected]<mailto:[email protected]>> >>wrote: >> >>> Hmm, that is really weird. I noticed that the GCS tests took a moment, >>>but >>> no longer than ~5-10 secs. >>> >>> On Mon, Nov 30, 2015 at 11:46 AM, Scott Crunkleton >>><[email protected]<mailto:[email protected]>> >>> wrote: >>> >>>> Is there an issue tracking this in Jira? >>>> >>>> On Mon, Nov 30, 2015 at 10:51 AM, Eric Johnson >>>> <[email protected]<mailto:[email protected]>> >>>> wrote: >>>> >>>>> +scott since he was recently doing some work on the GCS driver. >>>>> >>>>> On Mon, Nov 30, 2015 at 10:05 AM, Greg Hill >>>>> <[email protected]<mailto:[email protected]>> >>>>> wrote: >>>>> >>>>>> Is this a known issue? The google storage tests take a really >>>>>>really >>>>>> long time to complete. Are they waiting on a timeout or something? >>>>>> >>>>>> Greg >>>>>> >>>>>> >>>>> >>>> >>> >
