Ignasi <[email protected]> writes: > Hi Noorul, > > Finally, I've fixed the issue! :) Here is the commit [1] that fixes > it. Basically we don't have to encode the '+' and ' ' characters in > the Signature query parameter. With this fix applied, all live tests > are passing for Chef 0.10, Chef 11 and Enterprise Chef. > > Feel free to checkout my branch [2] and validate the live tests > (specially the testDownloadCookbooks). >
Thanks a lot! I see that some of the APIs are not exposed, for example http://docs.opscode.com/api_chef_server_environments_recipe.html Is this something that consciously avoided? Thanks and Regards Noorul > > Ignasi > > > P.S. I'll submit the PR to have it merged in 1.7.0-SNAPSHOT as soon as > JCLOUDS-246 [2] gets merged. > > > [1] > https://github.com/nacx/jclouds-chef/commit/67c1545aab3b742e9508f8bd3837b57de25c47e5 > [2] https://github.com/nacx/jclouds-chef/tree/245-live-tests > [3] https://issues.apache.org/jira/browse/JCLOUDS-246 > > On 27 August 2013 13:18, Noorul Islam K M <[email protected]> wrote: >> Ignasi <[email protected]> writes: >> >>> Hi Noorul, >>> >>> Just a follow up. I've been able to isolate (I think) the issue and >>> I've opened the corresponding JIRA [1]. I asked the chef-dev mailing >>> list [2] to see if someone can give us the details of how to properly >>> sign the URLs that are failing. It seems that some change may be >>> required in the signing filter [3]. >>> >>> Let's see if we can get this fixed! >>> >> >> Thank you! >> >> Thanks and Regards >> Noorul >> >>> >>> [1] https://issues.apache.org/jira/browse/JCLOUDS-254 >>> [2] http://lists.opscode.com/sympa/arc/chef-dev/2013-08/msg00034.html >>> [3] >>> https://github.com/jclouds/jclouds-chef/blob/master/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java >>> >>> On 26 August 2013 16:54, Ignasi <[email protected]> wrote: >>>> Ok, I just added a few cookbooks more to my Hosted Chef account and >>>> the download is failing the same way. >>>> I still haven't figured out what can be failing in the signature (it >>>> shouldn't have changed since Chef 0.10), so any help would be >>>> appreciated! >>>> >>>> On 26 August 2013 15:55, Ignasi <[email protected]> wrote: >>>>> Hi again, >>>>> >>>>> The problem appears in the "getResourceContents" api call. After >>>>> debugging for a while I haven't found an explanation. It fails >>>>> randomly with random resources, and there is no pattern for that :( >>>>> Sometimes it fails to download a root file resource, sometimes not. >>>>> Sometimes it fails downloading a recipe file, sometimes it just works. >>>>> >>>>> I executed the live test suite against a Chef 0.10.8, and against >>>>> Enterprise Chef and all tests are passing, so I assume it might be an >>>>> issue in Chef 11; otherwise I can not explain why the same resource >>>>> sometimes is downloaded and sometimes it is not. The authentication >>>>> mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise >>>>> Chef, it should work for Chef 11 too (or at least have some >>>>> deterministic behavior). >>>>> >>>>> >>>>> To properly reflect that in the live tests, I've separated the >>>>> "testListCookbooks" in two different tests: "testListCookbooks" and >>>>> "testDownloadCookbooks". If you update my branch, you should get those >>>>> changes. >>>>> >>>>> I'll search if there is any open issue and file a new one if appropriate. >>>>> >>>>> Anyway, let me know if you find something that could help getting rid of >>>>> this! >>>>> >>>>> >>>>> Ignasi >>>>> >>>>> >>>>> On 26 August 2013 14:29, Noorul Islam K M <[email protected]> wrote: >>>>>> Ignasi <[email protected]> writes: >>>>>> >>>>>>> Thanks a lot for your feedback Noorul! >>>>>>> >>>>>>> You can run the 'core' live tests as follows: >>>>>>> >>>>>>> mvn clean install -pl core -Plive \ >>>>>>> -Dtest.chef.endpoint=<you chef server url> \ >>>>>>> -Dtest.chef.identity=<identity> \ >>>>>>> -Dtest.chef.credential=<path to the PEM private key> >>>>>>> >>>>>> >>>>>> I think, I get the same as you >>>>>> >>>>>> >>>>>> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1. >>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 >>>>>> sec <<< FAILURE! >>>>>> >>>>>> Results : >>>>>> >>>>>> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): >>>>>> <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch >>>>>> </Code><Message>The request signature we calculated does not match the >>>>>> signature you provided. Check your key and signing method.</Message><S >>>>>> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu >>>>>> reProvided>hRkpN1RzN >>>>>> /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..) >>>>>> >>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1 >>>>>> >>>>>> Thanks and Regards >>>>>> Noorul >>>>>> >>>>>>> And if you have a hosted chef account, you can run them like this: >>>>>>> >>>>>>> mvn clean install -pl enterprise -Plive \ >>>>>>> -Dtest.enterprisechef.org=<you organization name> \ >>>>>>> -Dtest.enterprisechef.identity=<identity> \ >>>>>>> -Dtest.enterprisechef.credential=<path to the PEM private key> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 26 August 2013 14:10, Noorul Islam K M <[email protected]> wrote: >>>>>>>> Ignasi <[email protected]> writes: >>>>>>>> >>>>>>>>> Noorul, I've just uploaded to my branch [1] a change that registers a >>>>>>>>> custom Map deserializer to allow repeated keys. It just keeps the last >>>>>>>>> one, which seems to be the one with the right content. You can take a >>>>>>>>> look at the commit [2] if you want to know the details. >>>>>>>>> >>>>>>>>> I've tried the api call that was failing to you, and now it works and >>>>>>>>> the cookbook is properly deserialized. However, the live test to list >>>>>>>>> all cookbooks is still failing due to a signature error (which is >>>>>>>>> weird, because the signature is generated in a generic filter [3] that >>>>>>>>> applies to all requests). >>>>>>>>> >>>>>>>>> I think it might be related to my Chef 11 installation. To make sure, >>>>>>>>> could you update the branch and run the live tests, or try to list the >>>>>>>>> cookbooks using the ChefApi and ChefServer, and share the results? >>>>>>>>> >>>>>>>> >>>>>>>> I tested these methods after pulling latest from your branch. >>>>>>>> >>>>>>>> getCookbook() >>>>>>>> listCookbooks() >>>>>>>> listEnvironmentCookbooks() >>>>>>>> >>>>>>>> Everything seems to be working fine. >>>>>>>> >>>>>>>> How do I run live tests? Is this documented somewhere? >>>>>>>> >>>>>>>> Thanks and Regards >>>>>>>> Noorul >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> >>>>>>>>> Ignasi >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests >>>>>>>>> [2] >>>>>>>>> https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8 >>>>>>>>> [3] >>>>>>>>> https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 26 August 2013 12:24, Ignasi <[email protected]> wrote: >>>>>>>>>> As soon as possible. >>>>>>>>>> >>>>>>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not >>>>>>>>>> backward compatible (just a massive package and artifactId renaming), >>>>>>>>>> I sent a mail last week to let users express an opinion on the change >>>>>>>>>> [2]. I plan to merge it this week, since there are no opinions >>>>>>>>>> against, and as soon as it is merged, I'll merge my current branch. >>>>>>>>>> >>>>>>>>>> You could expect to have it merged by the end of the week, if I >>>>>>>>>> finish >>>>>>>>>> the fix by then (which I hope :)). I plan to upload to my branch the >>>>>>>>>> fix this afternoon; I'll let you know when I have something usable so >>>>>>>>>> you can test and validate it. >>>>>>>>>> >>>>>>>>>> Thanks for your help! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Ignasi >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246 >>>>>>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h >>>>>>>>>> >>>>>>>>>> On 26 August 2013 12:02, Noorul Islam K M <[email protected]> wrote: >>>>>>>>>>> Ignasi <[email protected]> writes: >>>>>>>>>>> >>>>>>>>>>>> Just tested the openssl cookbook and found that the returned >>>>>>>>>>>> Metadata >>>>>>>>>>>> has a duplicate key in the recipes list. I've pasted the json here >>>>>>>>>>>> [1] >>>>>>>>>>>> so you can have a look. >>>>>>>>>>>> >>>>>>>>>>>> I'll ping Chef people to see if it is an issue in their side, and >>>>>>>>>>>> try >>>>>>>>>>>> to build a workaround in the meanwhile. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I would like to know, usually how long it takes something to get >>>>>>>>>>> into >>>>>>>>>>> upstream from the branch that you are working on? >>>>>>>>>>> >>>>>>>>>>> I can provide any help needed to make this working but need to be >>>>>>>>>>> sure >>>>>>>>>>> that I can use this in production as soon as possible. >>>>>>>>>>> >>>>>>>>>>> Thanks and Regards >>>>>>>>>>> Noorul >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> [1] http://pastie.org/8270391 >>>>>>>>>>>> >>>>>>>>>>>> On 26 August 2013 08:28, Ignasi <[email protected]> wrote: >>>>>>>>>>>>> Hi Noorul! >>>>>>>>>>>>> >>>>>>>>>>>>> There is definitely the intention to support Chef 11 as soon as >>>>>>>>>>>>> possible. I >>>>>>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise >>>>>>>>>>>>> Chef live >>>>>>>>>>>>> tests and also made all live tests pass against a Chef 11 except >>>>>>>>>>>>> the >>>>>>>>>>>>> "listCookbooks" one. >>>>>>>>>>>>> >>>>>>>>>>>>> I get the exact same error than you, and it happens when >>>>>>>>>>>>> deserializing the >>>>>>>>>>>>> json for the cookbook into the CookbookVersion object. It seems >>>>>>>>>>>>> that the >>>>>>>>>>>>> Chef Server is returning a duplicate key in the json structure >>>>>>>>>>>>> (although it >>>>>>>>>>>>> shouldn't because we are forcing by default a 0.10 version header >>>>>>>>>>>>> to make it >>>>>>>>>>>>> return the same responses than in previous versions), and the >>>>>>>>>>>>> json parser >>>>>>>>>>>>> fails to build the corresponding map. >>>>>>>>>>>>> >>>>>>>>>>>>> I'm currently working on this but it's taking longer than >>>>>>>>>>>>> expected, so any >>>>>>>>>>>>> help would be much appreciated. If you could build my branch and >>>>>>>>>>>>> enable >>>>>>>>>>>>> jclouds wire logging [2], you would be able to see the json >>>>>>>>>>>>> returned for >>>>>>>>>>>>> your cookbook. Could you share that json? Also, any help >>>>>>>>>>>>> tracking/isolating >>>>>>>>>>>>> the error (or patch :)) is appreciated too! >>>>>>>>>>>>> >>>>>>>>>>>>> Ignasi >>>>>>>>>>>>> >>>>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests >>>>>>>>>>>>> [2] >>>>>>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/ >>>>>>>>>>>>> >>>>>>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <[email protected]> >>>>>>>>>>>>> escribió: >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> We are planning to use jclouds-chef library to talk to community >>>>>>>>>>>>>> Chef >>>>>>>>>>>>>> Server 11. In my initial testings, I get the following exception >>>>>>>>>>>>>> when I >>>>>>>>>>>>>> try to get cookbook details using >>>>>>>>>>>>>> >>>>>>>>>>>>>> api.getCookbook("openssl", "1.0.2") >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl >>>>>>>>>>>>>> at >>>>>>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172) >>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795) >>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761) >>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86) >>>>>>>>>>>>>> >>>>>>>>>>>>>> README.md says that >>>>>>>>>>>>>> >>>>>>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef >>>>>>>>>>>>>> server >>>>>>>>>>>>>> apis, and an initial and very basic (still in progress) >>>>>>>>>>>>>> implementation >>>>>>>>>>>>>> of the user and organization api of the Hosted and Private Chef >>>>>>>>>>>>>> flavours. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think that the issue I am facing is because I am using Chef >>>>>>>>>>>>>> server 11. >>>>>>>>>>>>>> Is there any plans to officially support Chef Server 11 in near >>>>>>>>>>>>>> future? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks and Regards >>>>>>>>>>>>>> Noorul
