Ok, I just added a few cookbooks more to my Hosted Chef account and the download is failing the same way. I still haven't figured out what can be failing in the signature (it shouldn't have changed since Chef 0.10), so any help would be appreciated!
On 26 August 2013 15:55, Ignasi <[email protected]> wrote: > Hi again, > > The problem appears in the "getResourceContents" api call. After > debugging for a while I haven't found an explanation. It fails > randomly with random resources, and there is no pattern for that :( > Sometimes it fails to download a root file resource, sometimes not. > Sometimes it fails downloading a recipe file, sometimes it just works. > > I executed the live test suite against a Chef 0.10.8, and against > Enterprise Chef and all tests are passing, so I assume it might be an > issue in Chef 11; otherwise I can not explain why the same resource > sometimes is downloaded and sometimes it is not. The authentication > mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise > Chef, it should work for Chef 11 too (or at least have some > deterministic behavior). > > > To properly reflect that in the live tests, I've separated the > "testListCookbooks" in two different tests: "testListCookbooks" and > "testDownloadCookbooks". If you update my branch, you should get those > changes. > > I'll search if there is any open issue and file a new one if appropriate. > > Anyway, let me know if you find something that could help getting rid of this! > > > Ignasi > > > On 26 August 2013 14:29, Noorul Islam K M <[email protected]> wrote: >> Ignasi <[email protected]> writes: >> >>> Thanks a lot for your feedback Noorul! >>> >>> You can run the 'core' live tests as follows: >>> >>> mvn clean install -pl core -Plive \ >>> -Dtest.chef.endpoint=<you chef server url> \ >>> -Dtest.chef.identity=<identity> \ >>> -Dtest.chef.credential=<path to the PEM private key> >>> >> >> I think, I get the same as you >> >> >> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1. >> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 sec >> <<< FAILURE! >> >> Results : >> >> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): <?xml >> version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch >> </Code><Message>The request signature we calculated does not match the >> signature you provided. Check your key and signing method.</Message><S >> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu >> reProvided>hRkpN1RzN >> /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..) >> >> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1 >> >> Thanks and Regards >> Noorul >> >>> And if you have a hosted chef account, you can run them like this: >>> >>> mvn clean install -pl enterprise -Plive \ >>> -Dtest.enterprisechef.org=<you organization name> \ >>> -Dtest.enterprisechef.identity=<identity> \ >>> -Dtest.enterprisechef.credential=<path to the PEM private key> >>> >>> >>> >>> On 26 August 2013 14:10, Noorul Islam K M <[email protected]> wrote: >>>> Ignasi <[email protected]> writes: >>>> >>>>> Noorul, I've just uploaded to my branch [1] a change that registers a >>>>> custom Map deserializer to allow repeated keys. It just keeps the last >>>>> one, which seems to be the one with the right content. You can take a >>>>> look at the commit [2] if you want to know the details. >>>>> >>>>> I've tried the api call that was failing to you, and now it works and >>>>> the cookbook is properly deserialized. However, the live test to list >>>>> all cookbooks is still failing due to a signature error (which is >>>>> weird, because the signature is generated in a generic filter [3] that >>>>> applies to all requests). >>>>> >>>>> I think it might be related to my Chef 11 installation. To make sure, >>>>> could you update the branch and run the live tests, or try to list the >>>>> cookbooks using the ChefApi and ChefServer, and share the results? >>>>> >>>> >>>> I tested these methods after pulling latest from your branch. >>>> >>>> getCookbook() >>>> listCookbooks() >>>> listEnvironmentCookbooks() >>>> >>>> Everything seems to be working fine. >>>> >>>> How do I run live tests? Is this documented somewhere? >>>> >>>> Thanks and Regards >>>> Noorul >>>> >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> Ignasi >>>>> >>>>> >>>>> >>>>> >>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests >>>>> [2] >>>>> https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8 >>>>> [3] >>>>> https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java >>>>> >>>>> >>>>> >>>>> On 26 August 2013 12:24, Ignasi <[email protected]> wrote: >>>>>> As soon as possible. >>>>>> >>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not >>>>>> backward compatible (just a massive package and artifactId renaming), >>>>>> I sent a mail last week to let users express an opinion on the change >>>>>> [2]. I plan to merge it this week, since there are no opinions >>>>>> against, and as soon as it is merged, I'll merge my current branch. >>>>>> >>>>>> You could expect to have it merged by the end of the week, if I finish >>>>>> the fix by then (which I hope :)). I plan to upload to my branch the >>>>>> fix this afternoon; I'll let you know when I have something usable so >>>>>> you can test and validate it. >>>>>> >>>>>> Thanks for your help! >>>>>> >>>>>> >>>>>> Ignasi >>>>>> >>>>>> >>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246 >>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h >>>>>> >>>>>> On 26 August 2013 12:02, Noorul Islam K M <[email protected]> wrote: >>>>>>> Ignasi <[email protected]> writes: >>>>>>> >>>>>>>> Just tested the openssl cookbook and found that the returned Metadata >>>>>>>> has a duplicate key in the recipes list. I've pasted the json here [1] >>>>>>>> so you can have a look. >>>>>>>> >>>>>>>> I'll ping Chef people to see if it is an issue in their side, and try >>>>>>>> to build a workaround in the meanwhile. >>>>>>>> >>>>>>> >>>>>>> I would like to know, usually how long it takes something to get into >>>>>>> upstream from the branch that you are working on? >>>>>>> >>>>>>> I can provide any help needed to make this working but need to be sure >>>>>>> that I can use this in production as soon as possible. >>>>>>> >>>>>>> Thanks and Regards >>>>>>> Noorul >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> [1] http://pastie.org/8270391 >>>>>>>> >>>>>>>> On 26 August 2013 08:28, Ignasi <[email protected]> wrote: >>>>>>>>> Hi Noorul! >>>>>>>>> >>>>>>>>> There is definitely the intention to support Chef 11 as soon as >>>>>>>>> possible. I >>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef >>>>>>>>> live >>>>>>>>> tests and also made all live tests pass against a Chef 11 except the >>>>>>>>> "listCookbooks" one. >>>>>>>>> >>>>>>>>> I get the exact same error than you, and it happens when >>>>>>>>> deserializing the >>>>>>>>> json for the cookbook into the CookbookVersion object. It seems that >>>>>>>>> the >>>>>>>>> Chef Server is returning a duplicate key in the json structure >>>>>>>>> (although it >>>>>>>>> shouldn't because we are forcing by default a 0.10 version header to >>>>>>>>> make it >>>>>>>>> return the same responses than in previous versions), and the json >>>>>>>>> parser >>>>>>>>> fails to build the corresponding map. >>>>>>>>> >>>>>>>>> I'm currently working on this but it's taking longer than expected, >>>>>>>>> so any >>>>>>>>> help would be much appreciated. If you could build my branch and >>>>>>>>> enable >>>>>>>>> jclouds wire logging [2], you would be able to see the json returned >>>>>>>>> for >>>>>>>>> your cookbook. Could you share that json? Also, any help >>>>>>>>> tracking/isolating >>>>>>>>> the error (or patch :)) is appreciated too! >>>>>>>>> >>>>>>>>> Ignasi >>>>>>>>> >>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests >>>>>>>>> [2] >>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/ >>>>>>>>> >>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <[email protected]> escribió: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> We are planning to use jclouds-chef library to talk to community Chef >>>>>>>>>> Server 11. In my initial testings, I get the following exception >>>>>>>>>> when I >>>>>>>>>> try to get cookbook details using >>>>>>>>>> >>>>>>>>>> api.getCookbook("openssl", "1.0.2") >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl >>>>>>>>>> at >>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119) >>>>>>>>>> at >>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67) >>>>>>>>>> at >>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249) >>>>>>>>>> at >>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235) >>>>>>>>>> at >>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313) >>>>>>>>>> at >>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278) >>>>>>>>>> at >>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93) >>>>>>>>>> at >>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172) >>>>>>>>>> at >>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93) >>>>>>>>>> at >>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172) >>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795) >>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761) >>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710) >>>>>>>>>> at >>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47) >>>>>>>>>> at >>>>>>>>>> org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86) >>>>>>>>>> >>>>>>>>>> README.md says that >>>>>>>>>> >>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef >>>>>>>>>> server >>>>>>>>>> apis, and an initial and very basic (still in progress) >>>>>>>>>> implementation >>>>>>>>>> of the user and organization api of the Hosted and Private Chef >>>>>>>>>> flavours. >>>>>>>>>> >>>>>>>>>> I think that the issue I am facing is because I am using Chef server >>>>>>>>>> 11. >>>>>>>>>> Is there any plans to officially support Chef Server 11 in near >>>>>>>>>> future? >>>>>>>>>> >>>>>>>>>> Thanks and Regards >>>>>>>>>> Noorul
