On Mon, Aug 25, 2008 at 2:05 AM, Chris Chabot <[EMAIL PROTECTED]> wrote:
> When you load up the reference test suite in the java sample container the > result is: > 94 Passed > 47 Failed > 2 Warnings > 22 Unverified > > now 47 is a bit steep, especially considering that this uses java's > json-rpc interface, so instruction ordering shouldn't be an issue here. > > A fair bit of those are silly errors (like the empty proxy string), or > errors like "expected 'gadgets.Tab', got " 'gadgets.[object > Object],<spam>'". and some errors that make no sense to me "[PPL005.1] > Nonsupported Field - familyName: FAILED: (got 'Doe'), expected 'undefined'" > (name is supported, so why complain you got a familyName?) > > However that doesn't account for all 47 errors, there's a few real ones in > there too, and it's currently quite hard to separate the real failures from > the ones that don't really matter. > > I'm slightly concerned that with such a volume of errors (wether they are > real errors or not), the tool looses it's usefulness. I mean if someone > checks out shindig, implements the basic services and runs the test suite to > see if they did that correctly .... How would one not completely familiar > with the complete opensocial stack be able to diagnose what is or isn't the > fault of their own code? A needle and haystack come to mind :) I agree, which is why we should talk to the people writing the compliance gadget and get rid of the unnecessary stuff first, then we can look for real problems. > > > -- Chris > > > On Aug 25, 2008, at 4:58 AM, Cassie wrote: > > Ahh - disclaimer for my last statement - I was only talking about the >> opensocial related tests... I don't usually check the non-social ones >> :) >> >> - Cassie >> >> >> On Sat, Aug 23, 2008 at 4:46 PM, Chris Chabot <[EMAIL PROTECTED]> wrote: >> >>> Don't suppose these are easy to fix on the javascript side? >>> >>> gadgets.io.* TestSuite: >>> >>> Description> Tests if we can get the proxy URL with given URL as proxy >>> [GIO101.0] gadgets.io.getProxyUrl(String) - With valid URL.: PASS: (got >>> 'proxy?refresh=3600&url=http%3A%2F%2Fwww%2F~user') >>> [GIO101.1] gadgets.io.getProxyUrl(String) - With valid URL.: PASS: (got >>> 'proxy?refresh=3600&url=http%3A%2F%2Fwww%2F~user') >>> [GIO102] [P2 ]:: gadgets.io.getProxyUrl() - With no parameters.: FAILED >>> [458 >>> ms] >>> Description> Tests if we can call getProxyUrl API without any parameter >>> and >>> it returns the proxy URL with empty proxy >>> [GIO102.0] gadgets.io.getProxyUrl() - With no parameters.: PASS: (got >>> 'proxy?refresh=3600&url=undefined') >>> [GIO102.1] gadgets.io.getProxyUrl() - With no parameters.: FAILED: (got >>> 'proxy?refresh=3600&url=undefined'), expected 'url=' >>> [GIO103] [P2 ]:: gadgets.io.getProxyUrl(null) - With null parameter.: >>> FAILED >>> [462 ms] >>> Description> Tests if we can call getProxyUrl API with null as parameter >>> and >>> it returns the proxy URL with empty proxy >>> [GIO103.0] gadgets.io.getProxyUrl(null) - With null parameter.: PASS: >>> (got >>> 'proxy?refresh=3600&url=null') >>> [GIO103.1] gadgets.io.getProxyUrl(null) - With null parameter.: FAILED: >>> (got >>> 'proxy?refresh=3600&url=null'), expected 'url=' >>> [GIO104] [P2 ]:: gadgets.io.getProxyUrl(undefined) - With undefined >>> parameter.: FAILED [466 ms] >>> Description> Tests if we can call getProxyUrl API with undefined as >>> parameter and it returns the proxy URL with empty proxy >>> [GIO104.0] gadgets.io.getProxyUrl(undefined) - With undefined parameter.: >>> PASS: (got 'proxy?refresh=3600&url=undefined') >>> [GIO104.1] gadgets.io.getProxyUrl(undefined) - With undefined parameter.: >>> FAILED: (got 'proxy?refresh=3600&url=undefined'), expected 'url=' >>> >>> I'm just a sucker for seeing green boxes is all :) >>> >>> -- Chris >>> >>> On Aug 22, 2008, at 11:04 PM, Cassie wrote: >>> >>> I check the compliance tests regularly for the actual deployment of >>>> Shindig >>>> that I work on at work. We are failing more now only because the tests >>>> are >>>> getting much more thorough. (The tests are also very active so sometimes >>>> they have bugs too although it is usually our code that's wrong :) >>>> >>>> I haven't found many issues with Shindig's actual js layer though - its >>>> usually been in the server layer and most often in the service >>>> implementations that are container specific. >>>> >>>> The non-rpc based container definitely has some issues though because it >>>> sending requests to the server in a json map format... which doesn't >>>> preserve order. So, some of the compliance tests would fail simply >>>> because >>>> they were fetching app data before it was updated and so forth. >>>> >>>> So... hopefully someone out there can get a patch to switch the php to >>>> rpc >>>> batching going :) >>>> >>>> - Cassie >>>> >>>> >>>> On Fri, Aug 22, 2008 at 12:28 PM, Dan Peterson <[EMAIL PROTECTED]> >>>> wrote: >>>> >>>> By the way, the docs for the compliance test suite are at: >>>>> http://code.google.com/p/opensocial-resources/wiki/ComplianceTests >>>>> >>>>> -Dan >>>>> On Fri, Aug 22, 2008 at 12:22 PM, Louis Ryan <[EMAIL PROTECTED]> wrote: >>>>> >>>>> Im seeing some similar issues. One thing I noticed is that lookingFor >>>>>> is >>>>>> now >>>>>> an Enum in JS but its not the Java datamodel. Im going to fix that >>>>>> one. >>>>>> >>>>>> On Fri, Aug 22, 2008 at 11:54 AM, Chris Chabot <[EMAIL PROTECTED]> >>>>>> >>>>> >>>>> wrote: >>>>> >>>>>> >>>>>> When running the compliance test suite: >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> http://opensocial-resources.googlecode.com/svn/tests/trunk/suites/0.7/compliance/reference/reference.xml >>>>> >>>>>> >>>>>>> I get 28 failed on my live version of partuza+php shindig ( >>>>>>> >>>>>> >>>>>> www.partuza.nl is >>>>>> >>>>>>> >>>>>>> running a checkout that is about 1.5 weeks old), while the latest >>>>>>> code >>>>>>> locally gives me 42 errors. >>>>>>> >>>>>>> To rule out that it wasn't the php code, i updated just shindig/php >>>>>>> on >>>>>>> >>>>>> >>>>>> the >>>>>> >>>>>>> >>>>>>> live server, and the error count didn't change, so it's probably some >>>>>>> shindig//features/* changes that cause this. >>>>>>> >>>>>>> Anyone checking if our JS code is 'compliant' ? And/or working on >>>>>>> >>>>>> >>>>> fixing >>>>> >>>>>> >>>>>>> it? Seems right now it's only getting less so :) >>>>>>> >>>>>>> -- Chris >>>>>>> >>>>>>> >>>>>> >>>>> >>> >>> >
