Ignasi <[email protected]> writes: > Hi Noorul, > > Just a follow up. I've been able to isolate (I think) the issue and > I've opened the corresponding JIRA [1]. I asked the chef-dev mailing > list [2] to see if someone can give us the details of how to properly > sign the URLs that are failing. It seems that some change may be > required in the signing filter [3]. > > Let's see if we can get this fixed! >
Thank you! Thanks and Regards Noorul > > [1] https://issues.apache.org/jira/browse/JCLOUDS-254 > [2] http://lists.opscode.com/sympa/arc/chef-dev/2013-08/msg00034.html > [3] > https://github.com/jclouds/jclouds-chef/blob/master/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java > > On 26 August 2013 16:54, Ignasi <[email protected]> wrote: >> Ok, I just added a few cookbooks more to my Hosted Chef account and >> the download is failing the same way. >> I still haven't figured out what can be failing in the signature (it >> shouldn't have changed since Chef 0.10), so any help would be >> appreciated! >> >> On 26 August 2013 15:55, Ignasi <[email protected]> wrote: >>> Hi again, >>> >>> The problem appears in the "getResourceContents" api call. After >>> debugging for a while I haven't found an explanation. It fails >>> randomly with random resources, and there is no pattern for that :( >>> Sometimes it fails to download a root file resource, sometimes not. >>> Sometimes it fails downloading a recipe file, sometimes it just works. >>> >>> I executed the live test suite against a Chef 0.10.8, and against >>> Enterprise Chef and all tests are passing, so I assume it might be an >>> issue in Chef 11; otherwise I can not explain why the same resource >>> sometimes is downloaded and sometimes it is not. The authentication >>> mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise >>> Chef, it should work for Chef 11 too (or at least have some >>> deterministic behavior). >>> >>> >>> To properly reflect that in the live tests, I've separated the >>> "testListCookbooks" in two different tests: "testListCookbooks" and >>> "testDownloadCookbooks". If you update my branch, you should get those >>> changes. >>> >>> I'll search if there is any open issue and file a new one if appropriate. >>> >>> Anyway, let me know if you find something that could help getting rid of >>> this! >>> >>> >>> Ignasi >>> >>> >>> On 26 August 2013 14:29, Noorul Islam K M <[email protected]> wrote: >>>> Ignasi <[email protected]> writes: >>>> >>>>> Thanks a lot for your feedback Noorul! >>>>> >>>>> You can run the 'core' live tests as follows: >>>>> >>>>> mvn clean install -pl core -Plive \ >>>>> -Dtest.chef.endpoint=<you chef server url> \ >>>>> -Dtest.chef.identity=<identity> \ >>>>> -Dtest.chef.credential=<path to the PEM private key> >>>>> >>>> >>>> I think, I get the same as you >>>> >>>> >>>> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1. >>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 >>>> sec <<< FAILURE! >>>> >>>> Results : >>>> >>>> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): <?xml >>>> version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch >>>> </Code><Message>The request signature we calculated does not match the >>>> signature you provided. Check your key and signing method.</Message><S >>>> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu >>>> reProvided>hRkpN1RzN >>>> /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..) >>>> >>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1 >>>> >>>> Thanks and Regards >>>> Noorul >>>> >>>>> And if you have a hosted chef account, you can run them like this: >>>>> >>>>> mvn clean install -pl enterprise -Plive \ >>>>> -Dtest.enterprisechef.org=<you organization name> \ >>>>> -Dtest.enterprisechef.identity=<identity> \ >>>>> -Dtest.enterprisechef.credential=<path to the PEM private key> >>>>> >>>>> >>>>> >>>>> On 26 August 2013 14:10, Noorul Islam K M <[email protected]> wrote: >>>>>> Ignasi <[email protected]> writes: >>>>>> >>>>>>> Noorul, I've just uploaded to my branch [1] a change that registers a >>>>>>> custom Map deserializer to allow repeated keys. It just keeps the last >>>>>>> one, which seems to be the one with the right content. You can take a >>>>>>> look at the commit [2] if you want to know the details. >>>>>>> >>>>>>> I've tried the api call that was failing to you, and now it works and >>>>>>> the cookbook is properly deserialized. However, the live test to list >>>>>>> all cookbooks is still failing due to a signature error (which is >>>>>>> weird, because the signature is generated in a generic filter [3] that >>>>>>> applies to all requests). >>>>>>> >>>>>>> I think it might be related to my Chef 11 installation. To make sure, >>>>>>> could you update the branch and run the live tests, or try to list the >>>>>>> cookbooks using the ChefApi and ChefServer, and share the results? >>>>>>> >>>>>> >>>>>> I tested these methods after pulling latest from your branch. >>>>>> >>>>>> getCookbook() >>>>>> listCookbooks() >>>>>> listEnvironmentCookbooks() >>>>>> >>>>>> Everything seems to be working fine. >>>>>> >>>>>> How do I run live tests? Is this documented somewhere? >>>>>> >>>>>> Thanks and Regards >>>>>> Noorul >>>>>> >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> >>>>>>> Ignasi >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests >>>>>>> [2] >>>>>>> https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8 >>>>>>> [3] >>>>>>> https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 26 August 2013 12:24, Ignasi <[email protected]> wrote: >>>>>>>> As soon as possible. >>>>>>>> >>>>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not >>>>>>>> backward compatible (just a massive package and artifactId renaming), >>>>>>>> I sent a mail last week to let users express an opinion on the change >>>>>>>> [2]. I plan to merge it this week, since there are no opinions >>>>>>>> against, and as soon as it is merged, I'll merge my current branch. >>>>>>>> >>>>>>>> You could expect to have it merged by the end of the week, if I finish >>>>>>>> the fix by then (which I hope :)). I plan to upload to my branch the >>>>>>>> fix this afternoon; I'll let you know when I have something usable so >>>>>>>> you can test and validate it. >>>>>>>> >>>>>>>> Thanks for your help! >>>>>>>> >>>>>>>> >>>>>>>> Ignasi >>>>>>>> >>>>>>>> >>>>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246 >>>>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h >>>>>>>> >>>>>>>> On 26 August 2013 12:02, Noorul Islam K M <[email protected]> wrote: >>>>>>>>> Ignasi <[email protected]> writes: >>>>>>>>> >>>>>>>>>> Just tested the openssl cookbook and found that the returned Metadata >>>>>>>>>> has a duplicate key in the recipes list. I've pasted the json here >>>>>>>>>> [1] >>>>>>>>>> so you can have a look. >>>>>>>>>> >>>>>>>>>> I'll ping Chef people to see if it is an issue in their side, and try >>>>>>>>>> to build a workaround in the meanwhile. >>>>>>>>>> >>>>>>>>> >>>>>>>>> I would like to know, usually how long it takes something to get into >>>>>>>>> upstream from the branch that you are working on? >>>>>>>>> >>>>>>>>> I can provide any help needed to make this working but need to be sure >>>>>>>>> that I can use this in production as soon as possible. >>>>>>>>> >>>>>>>>> Thanks and Regards >>>>>>>>> Noorul >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [1] http://pastie.org/8270391 >>>>>>>>>> >>>>>>>>>> On 26 August 2013 08:28, Ignasi <[email protected]> wrote: >>>>>>>>>>> Hi Noorul! >>>>>>>>>>> >>>>>>>>>>> There is definitely the intention to support Chef 11 as soon as >>>>>>>>>>> possible. I >>>>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise >>>>>>>>>>> Chef live >>>>>>>>>>> tests and also made all live tests pass against a Chef 11 except the >>>>>>>>>>> "listCookbooks" one. >>>>>>>>>>> >>>>>>>>>>> I get the exact same error than you, and it happens when >>>>>>>>>>> deserializing the >>>>>>>>>>> json for the cookbook into the CookbookVersion object. It seems >>>>>>>>>>> that the >>>>>>>>>>> Chef Server is returning a duplicate key in the json structure >>>>>>>>>>> (although it >>>>>>>>>>> shouldn't because we are forcing by default a 0.10 version header >>>>>>>>>>> to make it >>>>>>>>>>> return the same responses than in previous versions), and the json >>>>>>>>>>> parser >>>>>>>>>>> fails to build the corresponding map. >>>>>>>>>>> >>>>>>>>>>> I'm currently working on this but it's taking longer than expected, >>>>>>>>>>> so any >>>>>>>>>>> help would be much appreciated. If you could build my branch and >>>>>>>>>>> enable >>>>>>>>>>> jclouds wire logging [2], you would be able to see the json >>>>>>>>>>> returned for >>>>>>>>>>> your cookbook. Could you share that json? Also, any help >>>>>>>>>>> tracking/isolating >>>>>>>>>>> the error (or patch :)) is appreciated too! >>>>>>>>>>> >>>>>>>>>>> Ignasi >>>>>>>>>>> >>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests >>>>>>>>>>> [2] >>>>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/ >>>>>>>>>>> >>>>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <[email protected]> >>>>>>>>>>> escribió: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> We are planning to use jclouds-chef library to talk to community >>>>>>>>>>>> Chef >>>>>>>>>>>> Server 11. In my initial testings, I get the following exception >>>>>>>>>>>> when I >>>>>>>>>>>> try to get cookbook details using >>>>>>>>>>>> >>>>>>>>>>>> api.getCookbook("openssl", "1.0.2") >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl >>>>>>>>>>>> at >>>>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119) >>>>>>>>>>>> at >>>>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67) >>>>>>>>>>>> at >>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249) >>>>>>>>>>>> at >>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235) >>>>>>>>>>>> at >>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313) >>>>>>>>>>>> at >>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278) >>>>>>>>>>>> at >>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93) >>>>>>>>>>>> at >>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172) >>>>>>>>>>>> at >>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93) >>>>>>>>>>>> at >>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172) >>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795) >>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761) >>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710) >>>>>>>>>>>> at >>>>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47) >>>>>>>>>>>> at >>>>>>>>>>>> org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86) >>>>>>>>>>>> >>>>>>>>>>>> README.md says that >>>>>>>>>>>> >>>>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef >>>>>>>>>>>> server >>>>>>>>>>>> apis, and an initial and very basic (still in progress) >>>>>>>>>>>> implementation >>>>>>>>>>>> of the user and organization api of the Hosted and Private Chef >>>>>>>>>>>> flavours. >>>>>>>>>>>> >>>>>>>>>>>> I think that the issue I am facing is because I am using Chef >>>>>>>>>>>> server 11. >>>>>>>>>>>> Is there any plans to officially support Chef Server 11 in near >>>>>>>>>>>> future? >>>>>>>>>>>> >>>>>>>>>>>> Thanks and Regards >>>>>>>>>>>> Noorul
