Alex, just to clarify, my PR is not going to change the current behavior, GCE will still use the key as a String but I have introduced a facility for the BaseGoogleComputeEngineApiLiveTest [1] to simplify the test/debugging from IDE, as for example Intellij doens't like particularly the multiline system properties [2]
Best, Andrea [1]: https://github.com/andreaturli/jclouds-labs-google/blob/fix/firewalls/google-compute-engine/src/test/java/org/jclouds/googlecomputeengine/internal/BaseGoogleComputeEngineApiLiveTest.java#L78LiveTests [2]; https://twitter.com/abayer/status/409125947737010176 > 1) For the folks from Google (Rick looking at you) -- is there a shorter > secret which could be used? (GCE is just about the only cloud that needs > an entire key here so just about everything else passes the value.) > > 2) What do folks think about autodetecting, ie if it's a key (ie starts > with the ----- BEGIN PRIVATE KEY ----- token) use it as a key, otherwise > try to use it as a file? > > --A > > > On 13/03/2014 07:33, Andrea Turli wrote: > >> Hi, >> >> yes in my PR I've replaced the way we pass the creadential, now it needs >> to >> point to your filesystem, something like /home/$USER/my/key >> >> HTH, >> Andrea >> >> >> On Thu, Mar 13, 2014 at 7:04 AM, Sathyasuriya Priya >> <sathyasur...@gmail.com>wrote: >> >> hi Andrea, >>> >>> I am facing some issues testing your PR >>> https://github.com/jclouds/jclouds-labs-google/pull/20 >>> The patch expects test.google-compute-engine.credential property to >>> point >>> to a file. >>> But earlier I was using this property to have the entire private key as >>> string. >>> >>> So some test is failing one way or the other. >>> >>> May be something is wrong with the way I applied the PR. I had to look >>> into >>> that. >>> >>> I am running late for my gsoc proposal. I will test it again after a >>> week. >>> If you could let me know how you store the private key, that would be >>> good. >>> >>> Richard, >>> Yes, I see the link to increase quota in 'quotas' page. >>> I will request for it after testing Andrea's PR. >>> >>> Thanks >>> >>> >>> On Tue, Mar 11, 2014 at 3:59 AM, Richard Downer < >>> richard.dow...@cloudsoftcorp.com> wrote: >>> >>> Suriya, Andrea, >>>> >>>> It's also possible to request a quota increase from Google - there is a >>>> form to do this somewhere inside the web console. I've had to go through >>>> this process a couple of times, it was easy to fill in the form and both >>>> times I have had my request approved in less than a day. >>>> >>>> Richard. >>>> >>>> >>>> On 11 March 2014 09:00, Andrea Turli <andrea.tu...@gmail.com> wrote: >>>> >>>> Suriya, >>>>> >>>>> unfortunately I can see your limit is pretty low (5 networks) and I >>>>> >>>> suspect >>>> >>>>> the LiveTests are going to create more than 5. I've issued PR that >>>>> >>>> proposes >>>> >>>>> a fix for this high number of networks created by jclouds ( >>>>> https://github.com/jclouds/jclouds-labs-google/pull/20) but it is >>>>> >>>> still >>> >>>> unmerged. >>>>> My suggestion is to either test my PR or, another "hack", you can try >>>>> >>>> is >>> >>>> by >>>> >>>>> specifying a property like >>>>> "-Djclouds.google-compute-engine.networkName=my-network" (untested!) >>>>> >>>> that >>> >>>> should force jclouds to re-use always the same network. >>>>> >>>>> HTH, >>>>> Andrea >>>>> >>>>> >>>>> On Tue, Mar 11, 2014 at 9:51 AM, Ignasi Barrera < >>>>> >>>> ignasi.barr...@gmail.com >>>> >>>>> wrote: >>>>>> Hi, >>>>>> >>>>>> If you don't specify any template details, jclouds will try to find >>>>>> >>>>> an >>> >>>> image known to work in the provider. The predicate you are seeing is >>>>>> the default one defined for the GCE provider [1], but it seems that >>>>>> it's not the appropriate one. That predicate should always find a >>>>>> usable template (at least one with SSH enabled). Mind opening a JIRA >>>>>> [2] issue to fix this? >>>>>> >>>>>> You can build the example by providing custom values to the template >>>>>> builder. The Chef examples do that [3]. You don't have to stick to >>>>>> >>>>> the >>> >>>> provider's default template. You can build the examples by >>>>>> >>>>> instructing >>> >>>> the templateBuilder to look for a template that will work nice with >>>>>> the example. >>>>>> >>>>>> >>>>>> >>>>>> HTH! >>>>>> >>>>>> Ignasi >>>>>> >>>>>> >>>>>> [1] >>>>>> >>>>>> https://github.com/jclouds/jclouds-labs-google/blob/ >>> master/google-compute-engine/src/main/java/org/jclouds/ >>> googlecomputeengine/GoogleComputeEngineApiMetadata.java#L68-L69 >>> >>>> [2] https://issues.apache.org/jira/browse/JCLOUDS >>>>>> [3] >>>>>> >>>>>> https://github.com/jclouds/jclouds-examples/blob/master/ >>> chef-basics/src/main/java/org/apache/jclouds/examples/chef/ >>> basics/MainApp.java#L154 >>> >>>> On 11 March 2014 09:35, Sathyasuriya Priya <sathyasur...@gmail.com> >>>>>> >>>>> wrote: >>>>> >>>>>> hi Ignasi, >>>>>>> >>>>>>> Just an update and clarification on creating an example project for >>>>>>> >>>>>> GCE. >>>>> >>>>>> I started with the compute-basics example project. One important >>>>>>> >>>>>> change >>>> >>>>> needed was to change the dependency versions to 1.7.1 in pom. >>>>>>> >>>>>>> I see that compute.listImages(), compute.listNodes() are working >>>>>>> >>>>>> fine. >>>> >>>>> However, compute.createNodesInGroup() is failing with below error. >>>>>>> >>>>>>> adding node to group mygroup >>>>>>>>> >>>>>>>> error: no image matched predicate: >>>>>>> >>>>>>> And(nullEqualToIsParentOrIsGrandparentOfCurrentLocation(),And( >>> osFamily(gcel),osVersion(1[012].[01][04]))) >>> >>>> I was thinking the issue is with compute.templateBuilder(). Any way >>>>>>> >>>>>> to >>>> >>>>> relax the predicate constraint ? >>>>>>> I also tried to look into how live tests are able to create nodes >>>>>>> >>>>>> in >>> >>>> BaseComputeServiceLiveTest and see very similar code is being used. >>>>>>> Any help/directions to look for is appreciated. >>>>>>> >>>>>>> [1] >>>>>>> >>>>>>> https://github.com/jclouds/jclouds-examples/blob/master/ >>> compute-basics/pom.xml >>> >>>> Thanks >>>>>>> Suriya >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Mar 10, 2014 at 4:30 AM, Andrea Turli < >>>>>>> >>>>>> andrea.tu...@gmail.com >>>> >>>>> wrote: >>>>>>> >>>>>>> Hi Suriya, >>>>>>>> >>>>>>>> for "network operations quota exceeded" please look at your ' >>>>>>>> https://console.developers.google.com/project/ >>>>>>>> >>>>>>> <PROJECT_ID>/compute/quotas' >>>>>> >>>>>>> and try to clean up networks before running the liveTests. >>>>>>>> >>>>>>>> Best, >>>>>>>> Andrea >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Mar 10, 2014 at 12:22 PM, Ignasi Barrera >>>>>>>> <ignasi.barr...@gmail.com>wrote: >>>>>>>> >>>>>>>> Thanks for checking! >>>>>>>>> >>>>>>>>> If you are going to write something in you blog, that would be >>>>>>>>> >>>>>>>> excelent. >>>>>> >>>>>>> Also, if you want to contribute your code examples to the >>>>>>>>> jclouds-examples repo [1] or add a GCE starter guide to the main >>>>>>>>> jclouds site [2], you're very welcome too! :) >>>>>>>>> >>>>>>>>> Just let us know if we can help! >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> I. >>>>>>>>> >>>>>>>>> [1] https://github.com/jclouds/jclouds-examples >>>>>>>>> [2] https://github.com/jclouds/jclouds-site/tree/master/guides >>>>>>>>> >>>>>>>>> On 10 March 2014 12:11, Sathyasuriya Priya < >>>>>>>>> >>>>>>>> sathyasur...@gmail.com> >>>> >>>>> wrote: >>>>>>>> >>>>>>>>> Thanks Ignasi. >>>>>>>>>> Yes, you are right. I don't need any changes to start using. >>>>>>>>>> >>>>>>>>>> I am able to run the live tests following below instructions >>>>>>>>>> >>>>>>>>> [1], >>>> >>>>> [2]. >>>>>> >>>>>>> I feel some steps are not clear(for a new jclouds/gce user), >>>>>>>>>> >>>>>>>>> and >>> >>>> would >>>>>> >>>>>>> like >>>>>>>>> >>>>>>>>>> to post my steps in a bit more detailed manner(may be in my >>>>>>>>>> >>>>>>>>> blog). >>>> >>>>> I will start looking at how to use in a separate example >>>>>>>>>> >>>>>>>>> project(following >>>>>>>>> >>>>>>>>>> your suggestion [3]), and try to post them as well. >>>>>>>>>> >>>>>>>>>> In case if anybody is interested on result of my live tests: >>>>>>>>>> Single OAuth live test passed: >>>>>>>>>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time >>>>>>>>>> >>>>>>>>> elapsed: >>> >>>> 3.573 >>>>>> >>>>>>> sec >>>>>>>> >>>>>>>>> Some GCE live tests failed: >>>>>>>>>> Tests run: 86, Failures: 2, Errors: 0, Skipped: 11 >>>>>>>>>> >>>>>>>>>> Looks to me failures are due to network operations quota >>>>>>>>>> >>>>>>>>> exceeded. >>>> >>>>> statusCode=403, message=FORBIDDEN, headers={}}, >>>>>>>>>> >>>>>>>>> operationType=insert, >>>>>> >>>>>>> errors=[Error{code=QUOTA_EXCEEDED, message=Quota 'NETWORKS' >>>>>>>>>> >>>>>>>>> exceeded. >>>>>> >>>>>>> Limit: 5.0}]} >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> >>>>>>>>>> https://github.com/jclouds/jclouds-labs-google/tree/ >>> master/google-compute-engine >>> >>>> [2] >>>>>>>>>> >>>>>>>>> https://github.com/jclouds/jclouds-labs-google/tree/master/oauth >>>>>> >>>>>>> [3] >>>>>>>>>> >>>>>>>>> http://www.mail-archive.com/dev@jclouds.apache.org/msg04369.html >>>>>> >>>>>>> On Mon, Mar 10, 2014 at 1:36 AM, Ignasi Barrera < >>>>>>>>>> >>>>>>>>> ignasi.barr...@gmail.com>wrote: >>>>>>>>> >>>>>>>>>> Hi Suriya, >>>>>>>>>>> >>>>>>>>>>> Despite being in a different "labs" repository, the GCE >>>>>>>>>>> >>>>>>>>>> provider >>>> >>>>> is >>>>> >>>>>> also released as part of the regular release process. You'll >>>>>>>>>>> >>>>>>>>>> find >>>> >>>>> the >>>>>> >>>>>>> release artifacts in Maven Central [1], so you download or >>>>>>>>>>> >>>>>>>>>> include >>>>> >>>>>> them as a dependency where needed. Moving the provider to the >>>>>>>>>>> >>>>>>>>>> main >>>>> >>>>>> repository should not be a blocker for downstream projects, >>>>>>>>>>> >>>>>>>>>> as >>> >>>> long as >>>>>> >>>>>>> they can use the released artifacts. >>>>>>>>>>> >>>>>>>>>>> Creating the provider metadata class should be ideal, but I'm >>>>>>>>>>> >>>>>>>>>> not >>>> >>>>> sure >>>>>> >>>>>>> if it would be a must. A "provider" in jclouds is just an >>>>>>>>>>> >>>>>>>>>> "api", >>>> >>>>> exposed in a concrete location, with a set of defaults (check >>>>>>>>>>> >>>>>>>>>> this >>>>> >>>>>> [2] >>>>>> >>>>>>> core concept definition page). Many providers have common >>>>>>>>>>> >>>>>>>>>> APIs >>> >>>> (think >>>>>> >>>>>>> about OpenStack providers). While the API they expose is the >>>>>>>>>>> >>>>>>>>>> same >>>> >>>>> (nova, swift, etc), the endpoints where they expose the api >>>>>>>>>>> >>>>>>>>>> and >>> >>>> some >>>>>> >>>>>>> defaults change. That is what the provider metadata >>>>>>>>>>> >>>>>>>>>> describes. >>> >>>> Even though if there is no provider metadata describing a >>>>>>>>>>> >>>>>>>>>> "concrete >>>>> >>>>>> deployment of the api", you can still instanciate the >>>>>>>>>>> >>>>>>>>>> provider, >>> >>>> using >>>>>> >>>>>>> the ContextBuilder normally. >>>>>>>>>>> You can use "google-compute-engine" as the provider id, and >>>>>>>>>>> >>>>>>>>>> use >>> >>>> the >>>>> >>>>>> default endpoint, defined by the >>>>>>>>>>> >>>>>>>>>> GoogleComputeEngineApiMetadata >>> >>>> [3]. >>>>>> >>>>>>> If you have the provider dependency in your classpath, >>>>>>>>>>> >>>>>>>>>> jclouds >>> >>>> will >>>>> >>>>>> find it and pick it up. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> So, taking into account these things (if I am not missing >>>>>>>>>>> >>>>>>>>>> something) >>>>>> >>>>>>> you should be already able to use the GCE provider without >>>>>>>>>>> >>>>>>>>>> having >>>> >>>>> to >>>>>> >>>>>>> wait for those changes in the jclouds side. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> HTH! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Ignasi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> [1] >>>>>>>>>>> >>>>>>>>>>> http://search.maven.org/#artifactdetails%7Corg.apache. >>> jclouds.labs%7Cgoogle-compute-engine%7C1.7.1%7Cjar >>> >>>> [2] http://wiki.apache.org/jclouds/Core%20Concepts >>>>>>>>>>> [3] >>>>>>>>>>> >>>>>>>>>>> https://github.com/jclouds/jclouds-labs-google/blob/ >>> master/google-compute-engine/src/main/java/org/jclouds/ >>> googlecomputeengine/GoogleComputeEngineApiMetadata.java#L78-L85 >>> >>>> On 10 March 2014 09:18, Sathyasuriya Priya < >>>>>>>>>>> >>>>>>>>>> sathyasur...@gmail.com >>>>> >>>>>> wrote: >>>>>>>>> >>>>>>>>>> Dear Jclouds developers, >>>>>>>>>>>> >>>>>>>>>>>> I am Suriyapriya Veluchamy, currently an MS CS student at >>>>>>>>>>>> >>>>>>>>>>> NPU, >>>> >>>>> Fremont, >>>>>>>>> >>>>>>>>>> CA, >>>>>>>>>>> >>>>>>>>>>>> USA. >>>>>>>>>>>> I am interested in the following GSOC project in Stratos. >>>>>>>>>>>> https://issues.apache.org/jira/browse/STRATOS-480 >>>>>>>>>>>> >>>>>>>>>>>> I have some queries on the GCE support in jclouds, since >>>>>>>>>>>> >>>>>>>>>>> this >>> >>>> is >>>>> >>>>>> needed >>>>>>>>> >>>>>>>>>> for >>>>>>>>>>> >>>>>>>>>>>> Stratos to integrate with GCE. >>>>>>>>>>>> https://github.com/jclouds/jclouds-labs-google >>>>>>>>>>>> >>>>>>>>>>>> I looked at a relevant email thread as well. >>>>>>>>>>>> >>>>>>>>>>>> http://www.mail-archive.com/dev@jclouds.apache.org/ >>>>> msg04369.html >>>>> >>>>>> I understand the following JIRA issue is for moving the >>>>>>>>>>>> >>>>>>>>>>> separate >>>>> >>>>>> GCE >>>>>> >>>>>>> project to the core project. >>>>>>>>>>>> https://issues.apache.org/jira/browse/JCLOUDS-172 >>>>>>>>>>>> >>>>>>>>>>>> I would like to understand the complexity of this work, >>>>>>>>>>>> >>>>>>>>>>> based >>> >>>> on >>>>> >>>>>> which I >>>>>>>>> >>>>>>>>>> would decide whether it could be part of my gsoc work. I >>>>>>>>>>>> >>>>>>>>>>> have >>> >>>> come >>>>>> >>>>>>> up >>>>>>>> >>>>>>>>> with >>>>>>>>>>> >>>>>>>>>>>> list of todo items, based on my analysis. Pls correct me or >>>>>>>>>>>> >>>>>>>>>>> add >>>> >>>>> other >>>>>>>> >>>>>>>>> items >>>>>>>>>>> >>>>>>>>>>>> needed as well. >>>>>>>>>>>> >>>>>>>>>>>> 1. Create GCEProviderMetadata extending >>>>>>>>>>>> >>>>>>>>>>> BaseProviderMetadata >>> >>>> (similar >>>>>>>> >>>>>>>>> to >>>>>>>>> >>>>>>>>>> AWSEC2ProviderMetadata) - this is needed for >>>>>>>>>>>> ContextBuilder.newBuilder(provider) to pick up GCE ? >>>>>>>>>>>> 2. Move all the files (oauth, google-compute-engine) to >>>>>>>>>>>> jclouds-providers-project. >>>>>>>>>>>> 3. Hopefully, jclouds should pick up GCE as provider. Fix >>>>>>>>>>>> >>>>>>>>>>> any >>> >>>> issues >>>>>> >>>>>>> that >>>>>>>>> >>>>>>>>>> come up. >>>>>>>>>>>> 4. Andrew Bayer's criticisms of OAuth code - I couldn't >>>>>>>>>>>> >>>>>>>>>>> find >>> >>>> the >>>>> >>>>>> relevant >>>>>>>>> >>>>>>>>>> discussion/email related to this. Any idea if this is a >>>>>>>>>>>> >>>>>>>>>>> critical >>>>> >>>>>> piece >>>>>>>> >>>>>>>>> and >>>>>>>>>>> >>>>>>>>>>>> how much effort is needed to fix it. >>>>>>>>>>>> >>>>>>>>>>>> I am assuming other jclouds components are provider >>>>>>>>>>>> >>>>>>>>>>> agnostic >>> >>>> and >>>>> >>>>>> shouldn't >>>>>>>>>>> >>>>>>>>>>>> break. Do you aware of any other code changes needed in >>>>>>>>>>>> >>>>>>>>>>> other >>> >>>> components. >>>>>>>>> >>>>>>>>>> Also, I came across the GCE API java library from google. >>>>>>>>>>>> >>>>>>>>>>> Any >>> >>>> idea >>>>>> >>>>>>> why we >>>>>>>>> >>>>>>>>>> don't use it directly (instead rely on REST apis). May be >>>>>>>>>>>> >>>>>>>>>>> for >>> >>>> OAuth >>>>>> >>>>>>> support >>>>>>>>>>> >>>>>>>>>>>> alone we can make use of this ? >>>>>>>>>>>> >>>>>>>>>>>> https://developers.google.com/compute/docs/api/api-lib/java >>> >>>> I was able to build jclouds-labs-google and run unittests >>>>>>>>>>>> >>>>>>>>>>> successfully. >>>>>>>>> >>>>>>>>>> I will try to work on an example to make use of this >>>>>>>>>>>> >>>>>>>>>>> project >>> >>>> and >>>>> >>>>>> will >>>>>>>> >>>>>>>>> update. >>>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> Suriya >>>>>>>>>>>> >>>>>>>>>>> >>>> >>>> -- >>>> Richard Downer * Principal Engineer * Cloudsoft Corporation * >>>> http://www.cloudsoftcorp.com >>>> GitHub richardcloudsoft * Twitter @FrontierTown >>>> >>>> >
