Ignasi <[email protected]> writes: > It is not intentional if there are some missing apis. We added the > environment API time ago [1], but not all endpoints were added then, > and are still missing. > > Mind opening an Improvement in JIRA to add the missing ones? > > Also, if you want to contribute them [2], that would be very welcome! > :) I plan to merge JCLOUDS-246 as it might block contributions (due to > the change of the project structure). >
Created https://issues.apache.org/jira/browse/JCLOUDS-256 But I am not able to assign it to myself. So I left a comment saying that I am already working on this. Thanks and Regards Noorul > > Ignasi > > [1] https://github.com/jclouds/legacy-jclouds-chef/pull/61 > [2] https://wiki.apache.org/jclouds/How%20to%20Contribute > > On 28 August 2013 11:03, Noorul Islam K M <[email protected]> wrote: >> Ignasi <[email protected]> writes: >> >>> Hi Noorul, >>> >>> Finally, I've fixed the issue! :) Here is the commit [1] that fixes >>> it. Basically we don't have to encode the '+' and ' ' characters in >>> the Signature query parameter. With this fix applied, all live tests >>> are passing for Chef 0.10, Chef 11 and Enterprise Chef. >>> >>> Feel free to checkout my branch [2] and validate the live tests >>> (specially the testDownloadCookbooks). >>> >> >> Thanks a lot! >> >> I see that some of the APIs are not exposed, for example >> >> http://docs.opscode.com/api_chef_server_environments_recipe.html >> >> Is this something that consciously avoided? >> >> Thanks and Regards >> Noorul >> >>> >>> Ignasi >>> >>> >>> P.S. I'll submit the PR to have it merged in 1.7.0-SNAPSHOT as soon as >>> JCLOUDS-246 [2] gets merged. >>> >>> >>> [1] >>> https://github.com/nacx/jclouds-chef/commit/67c1545aab3b742e9508f8bd3837b57de25c47e5 >>> [2] https://github.com/nacx/jclouds-chef/tree/245-live-tests >>> [3] https://issues.apache.org/jira/browse/JCLOUDS-246 >>> >>> On 27 August 2013 13:18, Noorul Islam K M <[email protected]> wrote: >>>> Ignasi <[email protected]> writes: >>>> >>>>> Hi Noorul, >>>>> >>>>> Just a follow up. I've been able to isolate (I think) the issue and >>>>> I've opened the corresponding JIRA [1]. I asked the chef-dev mailing >>>>> list [2] to see if someone can give us the details of how to properly >>>>> sign the URLs that are failing. It seems that some change may be >>>>> required in the signing filter [3]. >>>>> >>>>> Let's see if we can get this fixed! >>>>> >>>> >>>> Thank you! >>>> >>>> Thanks and Regards >>>> Noorul >>>> >>>>> >>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-254 >>>>> [2] http://lists.opscode.com/sympa/arc/chef-dev/2013-08/msg00034.html >>>>> [3] >>>>> https://github.com/jclouds/jclouds-chef/blob/master/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java >>>>> >>>>> On 26 August 2013 16:54, Ignasi <[email protected]> wrote: >>>>>> Ok, I just added a few cookbooks more to my Hosted Chef account and >>>>>> the download is failing the same way. >>>>>> I still haven't figured out what can be failing in the signature (it >>>>>> shouldn't have changed since Chef 0.10), so any help would be >>>>>> appreciated! >>>>>> >>>>>> On 26 August 2013 15:55, Ignasi <[email protected]> wrote: >>>>>>> Hi again, >>>>>>> >>>>>>> The problem appears in the "getResourceContents" api call. After >>>>>>> debugging for a while I haven't found an explanation. It fails >>>>>>> randomly with random resources, and there is no pattern for that :( >>>>>>> Sometimes it fails to download a root file resource, sometimes not. >>>>>>> Sometimes it fails downloading a recipe file, sometimes it just works. >>>>>>> >>>>>>> I executed the live test suite against a Chef 0.10.8, and against >>>>>>> Enterprise Chef and all tests are passing, so I assume it might be an >>>>>>> issue in Chef 11; otherwise I can not explain why the same resource >>>>>>> sometimes is downloaded and sometimes it is not. The authentication >>>>>>> mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise >>>>>>> Chef, it should work for Chef 11 too (or at least have some >>>>>>> deterministic behavior). >>>>>>> >>>>>>> >>>>>>> To properly reflect that in the live tests, I've separated the >>>>>>> "testListCookbooks" in two different tests: "testListCookbooks" and >>>>>>> "testDownloadCookbooks". If you update my branch, you should get those >>>>>>> changes. >>>>>>> >>>>>>> I'll search if there is any open issue and file a new one if >>>>>>> appropriate. >>>>>>> >>>>>>> Anyway, let me know if you find something that could help getting rid >>>>>>> of this! >>>>>>> >>>>>>> >>>>>>> Ignasi >>>>>>> >>>>>>> >>>>>>> On 26 August 2013 14:29, Noorul Islam K M <[email protected]> wrote: >>>>>>>> Ignasi <[email protected]> writes: >>>>>>>> >>>>>>>>> Thanks a lot for your feedback Noorul! >>>>>>>>> >>>>>>>>> You can run the 'core' live tests as follows: >>>>>>>>> >>>>>>>>> mvn clean install -pl core -Plive \ >>>>>>>>> -Dtest.chef.endpoint=<you chef server url> \ >>>>>>>>> -Dtest.chef.identity=<identity> \ >>>>>>>>> -Dtest.chef.credential=<path to the PEM private key> >>>>>>>>> >>>>>>>> >>>>>>>> I think, I get the same as you >>>>>>>> >>>>>>>> >>>>>>>> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1. >>>>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: >>>>>>>> 187.134 sec <<< FAILURE! >>>>>>>> >>>>>>>> Results : >>>>>>>> >>>>>>>> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): >>>>>>>> <?xml version="1.0" >>>>>>>> encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch >>>>>>>> </Code><Message>The request signature we calculated does not match the >>>>>>>> signature you provided. Check your key and signing method.</Message><S >>>>>>>> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu >>>>>>>> reProvided>hRkpN1RzN >>>>>>>> /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..) >>>>>>>> >>>>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1 >>>>>>>> >>>>>>>> Thanks and Regards >>>>>>>> Noorul >>>>>>>> >>>>>>>>> And if you have a hosted chef account, you can run them like this: >>>>>>>>> >>>>>>>>> mvn clean install -pl enterprise -Plive \ >>>>>>>>> -Dtest.enterprisechef.org=<you organization name> \ >>>>>>>>> -Dtest.enterprisechef.identity=<identity> \ >>>>>>>>> -Dtest.enterprisechef.credential=<path to the PEM private key> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 26 August 2013 14:10, Noorul Islam K M <[email protected]> wrote: >>>>>>>>>> Ignasi <[email protected]> writes: >>>>>>>>>> >>>>>>>>>>> Noorul, I've just uploaded to my branch [1] a change that registers >>>>>>>>>>> a >>>>>>>>>>> custom Map deserializer to allow repeated keys. It just keeps the >>>>>>>>>>> last >>>>>>>>>>> one, which seems to be the one with the right content. You can take >>>>>>>>>>> a >>>>>>>>>>> look at the commit [2] if you want to know the details. >>>>>>>>>>> >>>>>>>>>>> I've tried the api call that was failing to you, and now it works >>>>>>>>>>> and >>>>>>>>>>> the cookbook is properly deserialized. However, the live test to >>>>>>>>>>> list >>>>>>>>>>> all cookbooks is still failing due to a signature error (which is >>>>>>>>>>> weird, because the signature is generated in a generic filter [3] >>>>>>>>>>> that >>>>>>>>>>> applies to all requests). >>>>>>>>>>> >>>>>>>>>>> I think it might be related to my Chef 11 installation. To make >>>>>>>>>>> sure, >>>>>>>>>>> could you update the branch and run the live tests, or try to list >>>>>>>>>>> the >>>>>>>>>>> cookbooks using the ChefApi and ChefServer, and share the results? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I tested these methods after pulling latest from your branch. >>>>>>>>>> >>>>>>>>>> getCookbook() >>>>>>>>>> listCookbooks() >>>>>>>>>> listEnvironmentCookbooks() >>>>>>>>>> >>>>>>>>>> Everything seems to be working fine. >>>>>>>>>> >>>>>>>>>> How do I run live tests? Is this documented somewhere? >>>>>>>>>> >>>>>>>>>> Thanks and Regards >>>>>>>>>> Noorul >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Ignasi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests >>>>>>>>>>> [2] >>>>>>>>>>> https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8 >>>>>>>>>>> [3] >>>>>>>>>>> https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 26 August 2013 12:24, Ignasi <[email protected]> wrote: >>>>>>>>>>>> As soon as possible. >>>>>>>>>>>> >>>>>>>>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not >>>>>>>>>>>> backward compatible (just a massive package and artifactId >>>>>>>>>>>> renaming), >>>>>>>>>>>> I sent a mail last week to let users express an opinion on the >>>>>>>>>>>> change >>>>>>>>>>>> [2]. I plan to merge it this week, since there are no opinions >>>>>>>>>>>> against, and as soon as it is merged, I'll merge my current branch. >>>>>>>>>>>> >>>>>>>>>>>> You could expect to have it merged by the end of the week, if I >>>>>>>>>>>> finish >>>>>>>>>>>> the fix by then (which I hope :)). I plan to upload to my branch >>>>>>>>>>>> the >>>>>>>>>>>> fix this afternoon; I'll let you know when I have something usable >>>>>>>>>>>> so >>>>>>>>>>>> you can test and validate it. >>>>>>>>>>>> >>>>>>>>>>>> Thanks for your help! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Ignasi >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246 >>>>>>>>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h >>>>>>>>>>>> >>>>>>>>>>>> On 26 August 2013 12:02, Noorul Islam K M <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> Ignasi <[email protected]> writes: >>>>>>>>>>>>> >>>>>>>>>>>>>> Just tested the openssl cookbook and found that the returned >>>>>>>>>>>>>> Metadata >>>>>>>>>>>>>> has a duplicate key in the recipes list. I've pasted the json >>>>>>>>>>>>>> here [1] >>>>>>>>>>>>>> so you can have a look. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'll ping Chef people to see if it is an issue in their side, >>>>>>>>>>>>>> and try >>>>>>>>>>>>>> to build a workaround in the meanwhile. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I would like to know, usually how long it takes something to get >>>>>>>>>>>>> into >>>>>>>>>>>>> upstream from the branch that you are working on? >>>>>>>>>>>>> >>>>>>>>>>>>> I can provide any help needed to make this working but need to be >>>>>>>>>>>>> sure >>>>>>>>>>>>> that I can use this in production as soon as possible. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks and Regards >>>>>>>>>>>>> Noorul >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] http://pastie.org/8270391 >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 26 August 2013 08:28, Ignasi <[email protected]> wrote: >>>>>>>>>>>>>>> Hi Noorul! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> There is definitely the intention to support Chef 11 as soon as >>>>>>>>>>>>>>> possible. I >>>>>>>>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise >>>>>>>>>>>>>>> Chef live >>>>>>>>>>>>>>> tests and also made all live tests pass against a Chef 11 >>>>>>>>>>>>>>> except the >>>>>>>>>>>>>>> "listCookbooks" one. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I get the exact same error than you, and it happens when >>>>>>>>>>>>>>> deserializing the >>>>>>>>>>>>>>> json for the cookbook into the CookbookVersion object. It seems >>>>>>>>>>>>>>> that the >>>>>>>>>>>>>>> Chef Server is returning a duplicate key in the json structure >>>>>>>>>>>>>>> (although it >>>>>>>>>>>>>>> shouldn't because we are forcing by default a 0.10 version >>>>>>>>>>>>>>> header to make it >>>>>>>>>>>>>>> return the same responses than in previous versions), and the >>>>>>>>>>>>>>> json parser >>>>>>>>>>>>>>> fails to build the corresponding map. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm currently working on this but it's taking longer than >>>>>>>>>>>>>>> expected, so any >>>>>>>>>>>>>>> help would be much appreciated. If you could build my branch >>>>>>>>>>>>>>> and enable >>>>>>>>>>>>>>> jclouds wire logging [2], you would be able to see the json >>>>>>>>>>>>>>> returned for >>>>>>>>>>>>>>> your cookbook. Could you share that json? Also, any help >>>>>>>>>>>>>>> tracking/isolating >>>>>>>>>>>>>>> the error (or patch :)) is appreciated too! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Ignasi >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests >>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <[email protected]> >>>>>>>>>>>>>>> escribió: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We are planning to use jclouds-chef library to talk to >>>>>>>>>>>>>>>> community Chef >>>>>>>>>>>>>>>> Server 11. In my initial testings, I get the following >>>>>>>>>>>>>>>> exception when I >>>>>>>>>>>>>>>> try to get cookbook details using >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> api.getCookbook("openssl", "1.0.2") >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172) >>>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795) >>>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761) >>>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> README.md says that >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard >>>>>>>>>>>>>>>> Chef server >>>>>>>>>>>>>>>> apis, and an initial and very basic (still in progress) >>>>>>>>>>>>>>>> implementation >>>>>>>>>>>>>>>> of the user and organization api of the Hosted and Private Chef >>>>>>>>>>>>>>>> flavours. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think that the issue I am facing is because I am using Chef >>>>>>>>>>>>>>>> server 11. >>>>>>>>>>>>>>>> Is there any plans to officially support Chef Server 11 in >>>>>>>>>>>>>>>> near future? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks and Regards >>>>>>>>>>>>>>>> Noorul
